Skip to content

Add npm package using napi-rs #182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 61 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
6901039
feat(napi): add napi bindings and test
remorses Jun 18, 2025
9974eab
works on mac
remorses Jun 18, 2025
da4c5d2
adding mdx support
remorses Jun 18, 2025
250df48
adding napi release process
remorses Jun 18, 2025
ddbf3ae
Update package.json
remorses Jun 18, 2025
00d9dad
Create .node-version
remorses Jun 18, 2025
6f96c73
Update release-napi.yml
remorses Jun 18, 2025
eeb9d07
use setup node from actions
remorses Jun 18, 2025
b163178
package.json
remorses Jun 18, 2025
c67a78a
Update release-napi.yml
remorses Jun 18, 2025
0923d3e
Update release-napi.yml
remorses Jun 18, 2025
ee92349
Update release-napi.yml
remorses Jun 18, 2025
de77211
package
remorses Jun 18, 2025
5c78406
Update package.json
remorses Jun 18, 2025
368f47b
package
remorses Jun 18, 2025
d2d1689
Update pnpm-workspace.yaml
remorses Jun 18, 2025
c43301d
disable windows for now
remorses Jun 18, 2025
0c24e1f
fix root package.json
remorses Jun 18, 2025
cc68c55
workaround napi-rs bug with -
remorses Jun 18, 2025
b14cf0c
lock to older version
remorses Jun 18, 2025
2acdf12
run build once
remorses Jun 18, 2025
db0ff41
remove -
remorses Jun 18, 2025
9d48fa7
add publishConfig
remorses Jun 18, 2025
1201c14
fix wrong readme.md
remorses Jun 18, 2025
d17d1e9
fix mdx test
remorses Jun 18, 2025
1c4a8fd
adding more options in the exports
remorses Jun 18, 2025
92adec7
new version
remorses Jun 18, 2025
0096e70
added more options
remorses Jun 18, 2025
2e8ff1e
esm works
remorses Jun 18, 2025
6f2e20f
using napi package.json
remorses Jun 18, 2025
984fe39
export a single parse function
remorses Jun 18, 2025
1a0f8a2
added split_into_sections
remorses Jun 18, 2025
080f502
add dep
remorses Jun 18, 2025
a5bd835
nn
remorses Jun 18, 2025
aec84a5
nn
remorses Jun 18, 2025
4901324
publishConfig
remorses Jun 18, 2025
81c38ee
nn
remorses Jun 18, 2025
7aa0b53
fix targets
remorses Jun 18, 2025
911945a
remove freebsd
remorses Jun 18, 2025
efaa769
adding benchmark, does not work tho
remorses Jun 18, 2025
2561138
add support for fronmatter
remorses Jun 18, 2025
48971b8
meta strings
remorses Jun 18, 2025
8d11d99
heading handling
remorses Jun 18, 2025
f5977e4
add gfm option
remorses Jun 18, 2025
a5fe72a
remove positions from tests, simpler split into sections
remorses Jun 18, 2025
845ac34
smaller npm package
remorses Jun 18, 2025
6beb6b3
package
remorses Jun 18, 2025
6dc1af1
shorter readme
remorses Jun 20, 2025
27b14ef
remoe splitIntoSections function
remorses Jun 20, 2025
e78532f
re enable windows
remorses Jun 20, 2025
d02b6ad
rename to tests
remorses Jun 20, 2025
b30edd9
add benchmark, add toHtml function
remorses Jun 20, 2025
871037f
better benchmark, use the zod markdown long doc
remorses Jun 20, 2025
af708ca
Update package.json
remorses Jun 20, 2025
758f944
fix version check
remorses Jun 20, 2025
e5a6bb6
Update release-napi.yml
remorses Jun 20, 2025
b72e973
Update package.json
remorses Jun 20, 2025
53000fe
add missing generated files
remorses Jun 20, 2025
61aeda7
update generated files
remorses Jun 20, 2025
cad5c32
scripts in root are wrong
remorses Jun 20, 2025
229f7df
fix artifacts step in ci
remorses Jun 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[target.x86_64-apple-darwin]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]

[target.aarch64-apple-darwin]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]
8 changes: 8 additions & 0 deletions .github/contribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ It’s probably a good idea to first post a question or open an issue to report
bug or suggest a new feature before creating a pull request.
See [Project][] for more info.

## `napi` folder

This is the package that handles NAPI bindings for markdown-rs. It also supports WASM. It uses napi-rs to generate the npm packages published on npm.

Tests use vitest, expect().toMatchInlineSnapshot() mostly. Most tests should be run with `pnpm vitest --run -u` to update snapshots, then read the test file again and make sure they have the expected result.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend npm first over requiring one of the many alternative package managers (yarn, pnpm, jsr, etc).

Copy link
Author

@remorses remorses Jun 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most popular projects use pnpm now, it's become the standard for workspaces and a must as soon as you hit one of the many bugs in npm, which is barely maintained now

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most popular projects use pnpm now

That is objectively false.
None of the top 10 packages by download use pnpm.
And most in the top 1000 don't either.
https://github.com/wooorm/npm-high-impact

More important, unified, which this is broadly part of does not https://github.com/unifiedjs/unified

soon as you hit one of the many bugs in npm

I hit far more pnpm bugs than npm

barely maintained now

Last release less than a week ago https://www.npmjs.com/package/npm?activeTab=versions
And they come at a regular cadence.


When making changes in rust run `pnpm build:debug` to make sure the Rust code compiles.

## Submitting an issue

* the issue tracker is for issues, discussions are for questions
Expand Down
183 changes: 183 additions & 0 deletions .github/workflows/release-napi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
name: Release NAPI

permissions: {}

on:
workflow_dispatch:
push:
paths:
- napi/package.json # make new release on version change of napi package
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Start with being able to release locally.
Before adding automation, I dont think we would want every merge to main to become a release.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately to release a cross platform native binary you need to use the platform with the same architecture, GitHub actions is perfect for this. This code was taken for the most part from the oxc-resolver which is state of the art regarding this kind of publishing from Rust to npm

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again see my main comment.
I don't think we should release platform specific versions yet.
WASM is universal.

# TODO enable the release napi workflow only on main
# branches:
# - main

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
DEBUG: "napi:*"

jobs:
check:
name: Check version
runs-on: ubuntu-latest
outputs:
version_changed: ${{ steps.version.outputs.changed }}
steps:
- uses: taiki-e/checkout-action@b13d20b7cda4e2f325ef19895128f7ff735c0b3d # v1.3.1

- name: Check version changes
uses: EndBug/version-check@36ff30f37c7deabe56a30caa043d127be658c425 # v2.1.5
id: version
with:
static-checking: localIsNew
file-url: https://unpkg.com/@xmorse/markdown-rs@latest/package.json
file-name: napi/package.json

- name: Set version name
if: steps.version.outputs.changed == 'true'
env:
version: ${{ steps.version.outputs.version }}
run: echo "version=${version}"

build:
needs: check
if: needs.check.outputs.version_changed == 'true'
strategy:
fail-fast: false
matrix:
include:
- os: windows-latest
target: x86_64-pc-windows-msvc
build: |
pnpm build --target x86_64-pc-windows-msvc

- os: windows-latest
target: aarch64-pc-windows-msvc
build: |
pnpm build --target aarch64-pc-windows-msvc

- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
build: |
pnpm build --target x86_64-unknown-linux-gnu --use-napi-cross

- os: ubuntu-latest
target: x86_64-unknown-linux-musl
build: |
pnpm build --target x86_64-unknown-linux-musl -x

- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
build: |
pnpm build --target aarch64-unknown-linux-gnu --use-napi-cross

- os: ubuntu-latest
target: aarch64-unknown-linux-musl
build: |
pnpm build --target aarch64-unknown-linux-musl -x

- os: ubuntu-latest
target: armv7-unknown-linux-gnueabihf
build: |
pnpm build --target armv7-unknown-linux-gnueabihf --use-napi-cross

- os: macos-latest
target: x86_64-apple-darwin
build: |
pnpm build --target x86_64-apple-darwin

- os: macos-latest
target: aarch64-apple-darwin
build: |
pnpm build --target aarch64-apple-darwin

- os: ubuntu-latest
target: wasm32-wasip1-threads
build: |
pnpm build --target wasm32-wasip1-threads

- os: ubuntu-latest
target: s390x-unknown-linux-gnu
build: |
export CFLAGS="-fuse-ld=lld"
pnpm build --target s390x-unknown-linux-gnu --use-napi-cross

- os: ubuntu-latest
target: riscv64gc-unknown-linux-gnu
build: |
sudo apt-get update
sudo apt-get install gcc-riscv64-linux-gnu g++-riscv64-linux-gnu -y
export CC=riscv64-linux-gnu-gcc
export CXX=riscv64-linux-gnu-g++
pnpm build --target riscv64gc-unknown-linux-gnu

name: Build ${{ matrix.target }}
runs-on: ${{ matrix.os }}
steps:
- uses: taiki-e/checkout-action@b13d20b7cda4e2f325ef19895128f7ff735c0b3d # v1.3.1
- uses: actions/setup-node@v4
with:
node-version: 22
- uses: pnpm/action-setup@v4
- run: pnpm i --frozen-lockfile=false

- run: rustup target add ${{ matrix.target }}

- uses: goto-bus-stop/setup-zig@abea47f85e598557f500fa1fd2ab7464fcb39406 # v2.2.1
if: ${{ contains(matrix.target, 'musl') }}
with:
version: 0.13.0

- name: Build
run: ${{ matrix.build }}
shell: bash
working-directory: napi
env:
CC: clang # for mimalloc

- name: Upload artifacts
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
if-no-files-found: "error"
name: bindings-${{ matrix.target }}
path: |
napi/*.node
napi/*.wasm

publish:
name: Publish NAPI
runs-on: ubuntu-latest
permissions:
id-token: write # for `npm publish --provenance`
needs:
- build
steps:
- uses: taiki-e/checkout-action@b13d20b7cda4e2f325ef19895128f7ff735c0b3d # v1.3.1

- uses: actions/setup-node@v4
with:
node-version: 22
- uses: pnpm/action-setup@v4
- run: pnpm i --frozen-lockfile=false

- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
path: artifacts

- run: pnpm napi create-npm-dirs --package-json-path napi/package.json

- run: pnpm napi artifacts --package-json-path package.json --npm-dir ../npm --build-output-dir .
working-directory: napi

- name: Publish npm packages as latest
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" >> ~/.npmrc
pnpm napi pre-publish --no-gh-release --tagstyle npm --package-json-path napi/package.json --npm-dir npm

# Publish root package
npm publish napi/ --tag latest --provenance --access public
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,8 @@ fuzz/corpus
fuzz/artifacts
fuzz/hfuzz_target
fuzz/hfuzz_workspace
*.node
*.wasm
node_modules
/npm
target/
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ rust-version = "1.56"
version = "1.0.0"

[workspace]
members = ["generate", "mdast_util_to_markdown"]
members = ["generate", "mdast_util_to_markdown", "napi"]

[workspace.dependencies]
pretty_assertions = "1"
5 changes: 5 additions & 0 deletions napi/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[target.x86_64-apple-darwin]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]

[target.aarch64-apple-darwin]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]
20 changes: 20 additions & 0 deletions napi/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[package]
name = "markdown-napi"
version = "0.1.0"
edition = "2021"
publish = false

[lib]
crate-type = ["cdylib", "lib"]


[dependencies]
napi = { version = "3.0.0-beta.8", default-features = false, features = ["napi3", "serde-json"] }
napi-derive = { version = "3.0.0-beta.8" }
markdown = { path = "..", package = "markdown", version = "1.0.0", features = ["serde"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"


[build-dependencies]
napi-build = "2.2.1"
3 changes: 3 additions & 0 deletions napi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# markdown-rs

CommonMark compliant markdown parser in Rust with ASTs and extensions. Packaged for npm with full WASM support.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The readme would need to be fairly complete, a lot could probably be lifted from https://github.com/remarkjs/remark or https://github.com/micromark/micromark
Which this is based off

Copy link
Author

@remorses remorses Jun 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I think this can be done in a follow-up PR after the first version is released on npm

1 change: 1 addition & 0 deletions napi/browser.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from '@xmorse/markdown-rs-binding-wasm32-wasi'
54 changes: 54 additions & 0 deletions napi/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/* auto-generated by NAPI-RS */
/* eslint-disable */
export interface CompileOptions {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid keeping a separate..d.ts that needs to be maintained separate from the source.
Use JSDoc mode for TypeScript https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html

Which can both type check and generate a definitions file.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is generated by NAPI, based on the Rust lib.rs source. It needs to be in the source code, not sure exactly why, all projects using NAPI do this.

/** Whether to allow (dangerous) HTML */
allowDangerousHtml?: boolean
/** Whether to allow dangerous protocols in links and images */
allowDangerousProtocol?: boolean
/** Whether to allow all values in images */
allowAnyImgSrc?: boolean
/** Default line ending to use when compiling to HTML */
defaultLineEnding?: string
/** Textual label to describe the backreference back to footnote calls */
gfmFootnoteBackLabel?: string
/** Prefix to use before the `id` attribute on footnotes */
gfmFootnoteClobberPrefix?: string
/** Textual label to use for the footnotes section */
gfmFootnoteLabel?: string
/** Attributes to use on the footnote label */
gfmFootnoteLabelAttributes?: string
/** HTML tag name to use for the footnote label element */
gfmFootnoteLabelTagName?: string
/** Whether or not GFM task list html `<input>` items are enabled */
gfmTaskListItemCheckable?: boolean
/** Whether to support the GFM tagfilter */
gfmTagfilter?: boolean
}

export interface HtmlOptions {
/** Configuration that describes how to parse from markdown */
parse?: ParseOptions
/** Configuration that describes how to compile to HTML */
compile?: CompileOptions
}

export declare function parse(input: string, options?: ParseOptions | undefined | null): any

export interface ParseOptions {
/** Whether to parse as MDX */
mdx?: boolean
/** Whether to enable GitHub Flavored Markdown (GFM) constructs */
gfm?: boolean
/** Whether to support GFM strikethrough with a single tilde */
gfmStrikethroughSingleTilde?: boolean
/** Whether to support math (text) with a single dollar */
mathTextSingleDollar?: boolean
/** Whether to enable basic MDX expression parsing */
mdxExpressionParse?: boolean
/** Whether to enable basic MDX ESM parsing */
mdxEsmParse?: boolean
/** Whether to support frontmatter */
frontmatter?: boolean
}

export declare function toHtml(input: string, options?: HtmlOptions | undefined | null): string
Loading