Free pipeline: arxiv OAI-PMH harvest + PDF download + pymupdf4llm convert + push to HuggingFace Hub. Fresh post-2024 corpora in common-pile schema.
-
Updated
Apr 28, 2026 - Python
#
corpus-building
Here are 3 public repositories matching this topic...
CLI for downloading Project Gutenberg e-texts - orderly, efficient, and vaguely Teutonic
text-processing ebook-downloader project-gutenberg gutendexapi gutendex gutendex-api corpus-building
-
Updated
Feb 12, 2026 - Python
Turn a folder of PDFs into an analysis-ready text corpus. Wizard + skills + scripts for Claude Code and Codex.
text-mining ocr hpc text-analysis digital-humanities leiden-university vision-language-model claude-code corpus-building
-
Updated
Apr 26, 2026 - JavaScript
Improve this page
Add a description, image, and links to the corpus-building topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the corpus-building topic, visit your repo's landing page and select "manage topics."