Skip to content

feat(docs): generate llms.txt and llms-full.txt #6430

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

okineadev
Copy link

📑 Summary

This PR adds the vitepress-plugin-llms plugin to the VitePress configuration, which automatically generates LLM-Friendly documentation for the website

It generates llms.txt, llms-full.txt and an LLM-Friendly version of each page in .md format

See https://llmstxt.org

  • 📖 have read the contribution guidelines
  • 💻 have added necessary unit/e2e tests.
  • 📓 have added documentation. Make sure MERMAID_RELEASE_VERSION is used for all new features.
  • 🦋 If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

If you accept this PR - I suggest you post a news story on your X that the site https://mermaid.js.org/ now has documentation for LLMs and mention my plugin and me (@okineadev)

Copy link

changeset-bot bot commented Mar 28, 2025

⚠️ No Changeset found

Latest commit: 1e8b476

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

netlify bot commented Mar 28, 2025

Deploy Preview for mermaid-js ready!

Name Link
🔨 Latest commit 1e8b476
🔍 Latest deploy log https://app.netlify.com/sites/mermaid-js/deploys/67f5e9c973828c0008d2f65d
😎 Deploy Preview https://deploy-preview-6430--mermaid-js.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

pkg-pr-new bot commented Mar 28, 2025

Open in StackBlitz

npm i https://pkg.pr.new/mermaid-js/mermaid@6430
npm i https://pkg.pr.new/mermaid-js/mermaid/@mermaid-js/mermaid-zenuml@6430
npm i https://pkg.pr.new/mermaid-js/mermaid/@mermaid-js/layout-elk@6430
npm i https://pkg.pr.new/mermaid-js/mermaid/@mermaid-js/parser@6430

commit: 1e8b476

Copy link

argos-ci bot commented Mar 28, 2025

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build Status Details Updated (UTC)
default (Inspect) ✅ No changes detected - Apr 9, 2025, 3:40 AM

@knsv knsv assigned knsv and aloisklink and unassigned knsv Mar 31, 2025
Copy link
Member

@aloisklink aloisklink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the idea of adding llms.txt and llms-full.txt to our docs site! I think it would make AIs so much smarter when working with Mermaid!

But I am a bit worried about using a new library that doesn't have many users, and as a widely uses open-source project, we're a bit sensitive to supply-chain attacks.

Since you're the author of this lib, and you already seem to be doing all the good security practices (e.g. signed provenance statements on NPM), that's good to hear!

It will help to replace tsup with plain old typescript, so that the dist/index.js file is easier to manually review when you make updates (and for such a small lib, minifying it won't reduce package sizes by much)!

I am also a bit worried about the markdown-title dependency, since it has pretty much no downloads except for vite-plugin-llms and was last updated 6 years ago. Could we maybe replace it with @mdit-vue/plugin-title, which is what Vitepress uses?

I wonder if it makes sense to remove gray-matter too and replace it with what Vitepress uses, see https://github.com/vuejs/vitepress/blob/35f36a96bb1cb34f7b2e1f1f3b26d26a9038ef8a/src/node/markdown/markdown.ts#L5-L8, just to make sure parsing is consistent with Vitepress.

It looks like you could even do import { createMarkdownRenderer } from 'vitepress' and const md = await createMarkdownRenderer(); to use the exact markdown parser that Vitepress uses too!

Edit: I fixed the ESLint issues too, so hopefully this PR gets a ✔️

@okineadev
Copy link
Author

I'm minifying anyway

@okineadev
Copy link
Author

I also won't replace tsup with typescript, tsup is very good

@okineadev
Copy link
Author

I am also a bit worried about the markdown-title dependency, since it has pretty much no downloads except for vite-plugin-llms and was last updated 6 years ago. Could we maybe replace it with @mdit-vue/plugin-title, which is what Vitepress uses?

I wonder if it makes sense to remove gray-matter too and replace it with what Vitepress uses, see https://github.com/vuejs/vitepress/blob/35f36a96bb1cb34f7b2e1f1f3b26d26a9038ef8a/src/node/markdown/markdown.ts#L5-L8, just to make sure parsing is consistent with Vitepress.

It looks like you could even do import { createMarkdownRenderer } from 'vitepress' and const md = await createMarkdownRenderer(); to use the exact markdown parser that Vitepress uses too!

Just yesterday I figured out how to process markdown, I thought my project was finally finished (since almost everything is ready), but now I seem to understand that I will have to rewrite the entire mechanism

@okineadev
Copy link
Author

I just tried switching to markdown-it (which uses VitePress), but it can't convert Markdown to Markdown, it can only Markdown to HTML

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants