Document Introductory ENSRainbow Topics #1513
Open
+246
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced `convert-csv` command for converting CSV files to .ensrainbow format. - Added support for single and two-column CSV formats. - Implemented error handling for invalid CSV data. - Created tests for various CSV scenarios, including special characters and invalid formats. - Updated package dependencies to include `csv-simple-parser` for CSV parsing.
- Introduced `--existing-db-path` option to filter out existing labels from an ENSRainbow database during CSV conversion. - Enhanced conversion process to skip duplicate labels within the same CSV file. - Updated logging to include statistics on filtered labels. - Added comprehensive tests for filtering functionality and updated documentation to reflect new features.
- Added new command-line options for CSV conversion: `--silent`, `--disable-dedup`, `--cache-size`, `--use-bloom-filter`, and `--bloom-filter-size`. - Implemented a deduplication database using ClassicLevel with optional Bloom filter for faster processing. - Updated the conversion process to support deduplication and improved memory management. - Enhanced logging for large file processing and added tests for new deduplication features.
- Added a function to estimate memory usage of Maps for better tracking. - Reduced default cache size in DeduplicationDB from 10000 to 1000. - Enhanced backpressure handling during CSV writing to prevent memory overflow. - Updated logging to include output backpressure events and improved performance for large files. - Streamlined the CSV processing to operate in a completely sequential manner.
- Removed unused command-line options for deduplication and Bloom filter from the CLI interface. - Updated default progress interval from 10000 to 50000 records for improved performance. - Enhanced logging for file processing and memory management during CSV conversion. - Cleaned up code for better readability and maintainability.
…s, adding references to the glossary for clarity. Update environment variable descriptions in `.env.local.example` files and improve comments in scripts to ensure consistent understanding of label set concepts.
…ipts to enhance clarity on label sets and versions. Add glossary references for better understanding across multiple files.
…criptions in Terraform variables for clarity on label sets and versions. Add references to the glossary for improved understanding.
…sh and add references to the ENSRainbow Glossary for comprehensive understanding.
…l" in the ENSRainbow overview and API client comments, improving clarity and user understanding.
…rd" in multiple files, improving clarity and user understanding of the data model and related concepts.
…VERSION references and enhancing clarity in environment variable definitions. Update examples for consistency and improve overall readability.
…set reference links, ensuring consistency and clarity in documentation. Enhance user understanding of environment variables related to label sets.
Updated the URL for the labelhash glossary reference.
…creating `.ensrainbow` files from both SQL and CSV sources. Update the description of the `searchlight` label set to reflect its availability and improved dataset features. Revise performance metrics for the `searchlight` dataset and include usage examples for downloading the extended discovery dataset.
…sed dependencies from package.json
7 tasks
🦋 Changeset detectedLatest commit: ea5399a The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive documentation explaining the fundamental problem of unknown labels in ENS and how ENSRainbow solves it.
Changes:
- Added a new "Unknown Labels" concept documentation page that explains what unknown labels are, why they exist, their impact on indexing and applications, and how ENSRainbow's healing service addresses them
- Updated the main ENSRainbow overview page to include a link to the new unknown labels documentation
- Updated the concepts index page to include the unknown labels page in the learning path and navigation
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| docs/ensnode.io/src/content/docs/ensrainbow/index.mdx | Added a reference link to the new unknown labels documentation page for comprehensive explanation |
| docs/ensnode.io/src/content/docs/ensrainbow/concepts/unknown-labels.mdx | New comprehensive documentation explaining unknown labels, their causes, impacts, and how ENSRainbow's healing service works to mitigate them |
| docs/ensnode.io/src/content/docs/ensrainbow/concepts/index.mdx | Added the unknown labels page to the concepts navigation and inserted it as step 2 in the recommended learning path |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
djstrong
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Lite PR
Summary
Why
This documentation fills a gap in explaining the core problem that ENSRainbow addresses. The unknown labels concept is fundamental to understanding ENSRainbow's purpose, but was previously only briefly mentioned without comprehensive explanation. This new page provides detailed coverage of what unknown labels are, why they exist, how they impact indexing and applications, and how ENSRainbow mitigates the problem.
Testing
Notes for Reviewer (Optional)
This is documentation-only; no code or runtime behavior changes.
Checklist