Skip to content

feat(web): add web generator #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat(web): add web generator #285

wants to merge 5 commits into from

Conversation

avivkeller
Copy link
Member

@avivkeller avivkeller commented May 28, 2025

Fixes #7.

This PR adds the web generator.

Tasks / Issues

P1 – Must Complete Before Merge

  • Add more items (anyone can do this as they review1)

P2 – Must complete before migration

P3 – Can Be Done in a Follow-up

  • Remove mustache dependency
  • Add more items (anyone can do this as they review1)

Get a preview

node bin/cli.mjs generate -t orama-db -i "/node/doc/api/*.md" -o "./out" # If you want search functionality
node bin/cli.mjs generate -t web -i "/node/doc/api/*.md" -o "./out" --index "/node/doc/api/index.md" # Specifying `--index` is optional, but recommended
npx serve out # You can serve the output, or just open one of the files in your browser. Serving is required for using search.

Footnotes

  1. Add things as they appear, or leave review comments. 2 3

@codecov-commenter
Copy link

codecov-commenter commented May 28, 2025

Codecov Report

Attention: Patch coverage is 73.04688% with 414 lines in your changes missing coverage. Please review.

Project coverage is 72.15%. Comparing base (385f0c3) to head (2a191b4).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...nerators/jsx-ast/utils/createSignatureElements.mjs 23.86% 201 Missing ⚠️
src/generators/jsx-ast/utils/buildContent.mjs 47.30% 88 Missing ⚠️
src/generators/jsx-ast/index.mjs 10.52% 34 Missing ⚠️
src/generators/web/index.mjs 79.03% 26 Missing ⚠️
src/generators/orama-db/index.mjs 54.34% 21 Missing ⚠️
src/utils/queries/index.mjs 71.73% 10 Missing and 3 partials ⚠️
src/generators/web/build/bundle.mjs 91.02% 7 Missing ⚠️
src/generators/jsx-ast/utils/buildBarProps.mjs 50.00% 5 Missing and 1 partial ⚠️
src/generators/web/build/plugins.mjs 95.18% 4 Missing ⚠️
src/utils/remark.mjs 50.00% 4 Missing ⚠️
... and 8 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #285      +/-   ##
==========================================
+ Coverage   71.51%   72.15%   +0.64%     
==========================================
  Files         117      131      +14     
  Lines        9721    10969    +1248     
  Branches      590      676      +86     
==========================================
+ Hits         6952     7915     +963     
- Misses       2766     3047     +281     
- Partials        3        7       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@avivkeller avivkeller marked this pull request as ready for review May 28, 2025 22:13
@avivkeller avivkeller requested a review from a team as a code owner May 28, 2025 22:13
@avivkeller avivkeller marked this pull request as draft May 29, 2025 16:36
const language = matches?.groups?.language ?? '';
const [copyText, setCopyText] = useState('Copy to clipboard');

const handleCopy = async text => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not including useCopyToClipord in ui lib ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC I didn't include it since it didn't fall under the "UI Component" category, it's a hook, however, if you disagree, I won't block.

"estree-util-value-to-estree": "^3.4.0",
"estree-util-visit": "^2.0.0",
"github-slugger": "^2.0.0",
"glob": "^11.0.2",
"hast-util-to-string": "^3.0.1",
"hastscript": "^9.0.1",
"html-minifier-terser": "^7.2.0",
"mustache": "^4.2.0",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to use __<VARIABLE>__ like in the other generators, however, the JavaScript I was trying to add to the HTML broke the structure of the file. Using a library like Mustache properly escapes anything injected.

Copy link
Member

@AugustinMauroy AugustinMauroy May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

export const TEMPLATE = ({...props}) => dedent`
<html>
 ${props.foo}
</html>
`; 

this can work

Copy link
Member Author

@avivkeller avivkeller May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if I wasn’t clear, the compiled JavaScript broke the HTML structure because it contained characters that need to be escaped to be in an HTML script.

For example, in your scenario, if props.foo contains a </html>, it’ll escape the sandbox.

Mustache performs all the needed escaping.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like that we're using yet another layer of whatever freakery here. Mustache is great, but it feels like you're throing a nuke at an ant's sized problem. Not to mention adding yet another engine here will make things even more complex and costly-performance wise. Let's try to avoid throwing npm packages to any problem we encounter, shall we?

@avivkeller
Copy link
Member Author

avivkeller commented May 30, 2025

🎉 The code now dehydrates to the client so it can render without JavaScript!

@AugustinMauroy
Copy link
Member

🎉 The code now dehydrates to the client so it run without JavaScript!

Wow 😵‍💫 and what about codetab

@avivkeller
Copy link
Member Author

avivkeller commented May 30, 2025

It rehydrates and runs with JS, but if you don't have JS, you can still view the docs. I used React's SSRing

@avivkeller
Copy link
Member Author

@AugustinMauroy and I got search to finally work 🎉
image

@avivkeller avivkeller force-pushed the feat/web/gen branch 2 times, most recently from 024bbea to dbfe55d Compare June 3, 2025 21:40
@avivkeller
Copy link
Member Author

@nodejs/nodejs-website @nodejs/web-infra ChangeHistory and SideBar aren't implemented yet, so this is still a draft, but it's ready for you to take a look, so feel free to review :-)

@avivkeller avivkeller linked an issue Jun 6, 2025 that may be closed by this pull request
@ovflowd
Copy link
Member

ovflowd commented Jun 25, 2025

To be honest, I like this design:

image

But it doesn't speak our design language and it is a bit complex visually speaking. The one from the example of Next.js docs is simple and easy to visualize. With variation we can adopt many different paths and experiment.

@avivkeller
Copy link
Member Author

Okay, I'll program the table, and we'll see how it looks.

@avivkeller
Copy link
Member Author

@ovflowd (Note: This is a first draft to show what a table would look it more-or-less, i.e. Property will be changed to Parameter, etc)

image
image

@ovflowd
Copy link
Member

ovflowd commented Jun 25, 2025

Nice, I still think we need to add that snippet of method signature that I mentioned. This would allow us to rename the header section to be simply the method name or class name. It also helps on Search Indexing

@avivkeller
Copy link
Member Author

Nice, I still think we need to add that snippet of method signature that I mentioned. This would allow us to rename the header section to be simply the method name or class name. It also helps on Search Indexing

Yes, the draft was just for the table, but I plan to use parseSignature (or a variation of) for that.

@avivkeller
Copy link
Member Author

avivkeller commented Jun 27, 2025

Tables:
image


Combined signatures:
image

@avivkeller
Copy link
Member Author

image

@avivkeller
Copy link
Member Author

avivkeller commented Jun 27, 2025

With the speed improvements we've made, this generator takes roughly a minute and a half. It's not amazing, but it's better than the 2 minutes we were seeing earlier. The hottest code path is PostCSS, so once we preprocess the CSS/JSX on the ui-components side, it'll speed up more.

All-in-all, the slowest code, in order of speed (slowest 1, fastest 4):

  1. PostCSS (by a large margin)
  2. Dehydration
  3. Terser
  4. ESTree to Babel

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

Let's prioritze the publishing of ui-components; Is @Harkunwar working on it? Regarding Terser, let's see what we can move to ESBuild or Rollup.

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

Tables: image

Combined signatures: image

  • Can we add syntax highlighting on these entries.
  • Can we update the table so that the <code> tags are not in bold
  • I also feel the heading sections don't need to be a <code> tag anymore as they're just a method name.

@avivkeller
Copy link
Member Author

avivkeller commented Jun 28, 2025

Regarding Terser, let's see what we can move to ESBuild or Rollup.

Terser is specifically for HTML minification, everything else (CSS/JS) is minified by ESBuild.

I wonder if we can configure preact to minify during generation 🤔?

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

  • Class Constructors should it be "new ClassName" or just "ClassName" Constructor?

image

  • We should document what C, M, E (and the different versions of "C") mean somewhere. Either by a hover tooltip or "about this documentation"
  • Otherwise let's rename the Pills to be like "Class", "Nethod", "Event", "Class Constructor"
    • And instead of a rounded pill it can be a rectangular one

image


EDIT:

  • Maybe also add a <hr /> between top level things (classes, idk)

image

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

Regarding Terser, let's see what we can move to ESBuild or Rollup.

Terser is specifically for HTML minification, everything else (CSS/JS) is minified by ESBuild.

I wonder if we can configure preact to minify during generation 🤔?

Hm? How?

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

Ah and some stuff seems to not be using the new table yet:

image

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

For Orama DB generation:

image

I feel we need to add "tags" labels or something; as it upsets me so much "write file" and none ofn the fs: write file methods are even showing up on these results.

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

Another example of tables not being generated:

image

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

Also regarding header
image

It'd be great if we could also have a navbar 🤔 but maybe not (and not a blocker can be a folow-up)

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

image

Select shows major + minor; But options show major with .x; Shouldn't select options label show the full version?

@avivkeller
Copy link
Member Author

Another example of tables not being generated:

image

That specifically is an issue with the table being generated, but the original content not being removed (which is it's own bug)

@avivkeller
Copy link
Member Author

I also need to fix several edge cases with table generation like:
Screenshot 2025-06-28 at 17 51 12

@ovflowd
Copy link
Member

ovflowd commented Jun 28, 2025

I also need to fix several edge cases with table generation like:
Screenshot 2025-06-28 at 17 51 12

Is it just me or is the font size changing for code tags within the table? And yes, feels like the styling is a bit broken not gonna lie.

@avivkeller
Copy link
Member Author

Screenshot 2025-06-28 at 17 51 12

How should we render it in this case? Should the functions' properties be a sub-table in Description?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add react-web generator Write React Components
6 participants