Add lightweight entry points and dynamic data loading#12
Merged
Conversation
…on size The full library bundles 6+ MB of Tanach data into a single entry point, causing edge functions that only need preferred verse lookups to exceed Vercel's 1 MB size limit (2.69 MB). This adds a separate subpath export at @shafeh/tanach/preferred (122 KB) with pre-embedded verse text, so consumers can import only what they need. Changes: - Add src/preferred-verses-data.ts with 457 preferred verses + embedded text - Add src/preferred.ts as lightweight entry point (no data.ts dependency) - Add scripts/generate-preferred-with-text.ts to regenerate enriched data - Update vite.config.ts with multi-entry build (index + preferred) - Add ./preferred subpath export in package.json - Fix final-form letter normalization bug in preferred verse map lookup - Fix build order: vite build first, then tsc --emitDeclarationOnly https://claude.ai/code/session_01XDYfksN1mSwXwrduTMtrtQ
…ions
The full library bundles 6+ MB of Tanach data, exceeding Vercel's 1 MB
edge function limit. The new @shafeh/tanach/dynamic entry point (8 KB)
fetches per-book JSON files at runtime instead of bundling them, giving
access to the full Tanach data without any size constraints.
Changes:
- Add src/dynamic.ts: async API that fetches per-book data on demand
- All data functions (tanach, findPesukimByName, etc.) are async
- Books fetched in parallel and cached in memory
- getPreferredPasukForName remains sync (uses embedded data)
- configure({ baseUrl }) sets where to fetch book JSON files
- clearCache() frees memory after use
- Add scripts/generate-book-data.ts: splits data.ts into 39 JSON files
(largest book: Yirmiyahu at 214 KB, smallest: Ovadiah at 3 KB)
- Add ./dynamic subpath export in package.json
- Build now generates dist/books/*.json alongside bundles
- Add comprehensive tests with mocked fetch (131 total tests pass)
Usage in edge function:
import { configure, findPesukimByName } from '@shafeh/tanach/dynamic';
configure({ baseUrl: 'https://myapp.com/data/tanach/books' });
const verses = await findPesukimByName("ד", "ד"); // no maxResults cap
https://claude.ai/code/session_01XDYfksN1mSwXwrduTMtrtQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces two new entry points to the
@shafeh/tanachpackage to support size-constrained environments (edge functions, serverless, etc.):/preferred– A lightweight module (~50 KB) for preferred verse lookups without importing the full Tanach data/dynamic– An async module that fetches per-book JSON files on-demand instead of bundling the entire 6+ MB datasetKey Changes
package.json: Added exports for./preferredand./dynamicalongside the main entry pointvite.config.tsto build three separate entry points with proper file naming (index.cjs/index.js,preferred.cjs/preferred.js,dynamic.cjs/dynamic.js)scripts/generate-preferred-with-text.ts– Generatessrc/preferred-verses-data.tswith 457 preferred verses and their embedded textscripts/generate-book-data.ts– Splits the full Tanach data into 24 per-book JSON files (5–215 KB each) for dynamic loadingsrc/preferred.ts– Synchronous preferred verse lookups with embedded text (no data import needed)src/dynamic.ts– Async data functions that fetch books on-demand from a configurable base URLsrc/preferred.test.ts– Tests for the lightweight preferred verses modulesrc/dynamic.test.ts– Tests for dynamic loading with mocked fetchsrc/preferred-verses.tsto normalize final-form Hebrew letters (ם→מ, ך→כ, etc.) in lookup keys for consistencyImplementation Details
preferredVersesWithTextarray is auto-generated and includes full verse text, Hebrew/English book names, and metadata—enabling zero-dependency preferred verse lookupsconfigure()function to set the base URL for book JSON files. All data functions are async and fetch books in parallelnormalizeHebrewLetter,extractHebrewLetters,getAllLetterForms) are available in both/preferredand/dynamicmodulesSize Benefits
https://claude.ai/code/session_01XDYfksN1mSwXwrduTMtrtQ