From 69aad040438d73d08ce3cd387c6bb53417f89b2e Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 15 Jul 2025 10:49:28 +0000 Subject: [PATCH 1/3] Initial plan From a6b2d09336043fdec27ff343846a6bceb6dc6961 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 15 Jul 2025 10:58:57 +0000 Subject: [PATCH 2/3] Restructure documentation according to documentation instructions Co-authored-by: kozo2 <12192+kozo2@users.noreply.github.com> --- docs/articles/appendices/glossary.md | 64 +++++ docs/articles/devs/Contributing.md | 63 +++++ docs/articles/devs/DevelopmentSetup.md | 145 ++++++++++ docs/articles/devs/RepositoryStructure.md | 133 +++++++++ docs/articles/devs/SchemaVersioning.md | 162 +++++++++++ docs/articles/table_of_contents.md | 39 +++ docs/articles/users/GettingStarted.md | 69 +++++ docs/articles/users/PackageSchema/Examples.md | 130 +++++++++ docs/articles/users/PackageSchema/Fields.md | 260 ++++++++++++++++++ docs/articles/users/PackageSchema/Overview.md | 96 +++++++ docs/articles/users/ProgrammaticUsage.md | 174 ++++++++++++ docs/articles/users/RegistrySchema/Fields.md | 210 ++++++++++++++ .../articles/users/RegistrySchema/Overview.md | 89 ++++++ docs/articles/users/SchemaAccess.md | 90 ++++++ docs/articles/users/SchemaValidation.md | 116 ++++++++ docs/index.md | 42 ++- 16 files changed, 1874 insertions(+), 8 deletions(-) create mode 100644 docs/articles/appendices/glossary.md create mode 100644 docs/articles/devs/Contributing.md create mode 100644 docs/articles/devs/DevelopmentSetup.md create mode 100644 docs/articles/devs/RepositoryStructure.md create mode 100644 docs/articles/devs/SchemaVersioning.md create mode 100644 docs/articles/table_of_contents.md create mode 100644 docs/articles/users/GettingStarted.md create mode 100644 docs/articles/users/PackageSchema/Examples.md create mode 100644 docs/articles/users/PackageSchema/Fields.md create mode 100644 docs/articles/users/PackageSchema/Overview.md create mode 100644 docs/articles/users/ProgrammaticUsage.md create mode 100644 docs/articles/users/RegistrySchema/Fields.md create mode 100644 docs/articles/users/RegistrySchema/Overview.md create mode 100644 docs/articles/users/SchemaAccess.md create mode 100644 docs/articles/users/SchemaValidation.md diff --git a/docs/articles/appendices/glossary.md b/docs/articles/appendices/glossary.md new file mode 100644 index 0000000..f8361cb --- /dev/null +++ b/docs/articles/appendices/glossary.md @@ -0,0 +1,64 @@ +# Glossary + +This article is about: +- Key terms and concepts used in Hatch Schemas documentation +- Definitions for technical terminology + +You will learn about: +- The meaning of important terms in the Hatch ecosystem +- Context for technical concepts used throughout the documentation + +## Core Concepts + +**Hatch Schema** +: A JSON schema that defines the structure and validation rules for metadata in the CrackingShells package ecosystem. + +**JSON Schema** +: A vocabulary that allows you to annotate and validate JSON documents. Uses JSON format to describe the structure, data types, and validation rules. + +**Package Metadata** +: Structured information about a package including name, version, dependencies, author information, and other descriptive data. + +**Registry** +: The central catalog of all available packages in the Hatch ecosystem, maintained as a searchable database. + +## Schema Types + +**Package Schema** +: Schema for validating individual package metadata files (`hatch_pkg_metadata_schema.json`). + +**Registry Schema** +: Schema for validating the central package registry structure (`hatch_all_pkg_metadata_schema.json`). + +## Validation Terms + +**Schema Validation** +: The process of checking that a JSON document conforms to the rules defined in a JSON schema. + +**Version Constraint** +: A specification defining which versions of a dependency are acceptable (e.g., `>=1.0.0`, `^2.1.0`). + +**Entry Point** +: The main executable file or script that serves as the primary interface for a package. + +## Technical Terms + +**URI** +: Uniform Resource Identifier - a string that identifies a resource, typically a URL. + +**Semantic Versioning** +: A versioning scheme using three numbers (major.minor.patch) to indicate backwards compatibility. + +**Dependency** +: An external package or component that a package requires to function properly. + +## Ecosystem Terms + +**CrackingShells** +: The GitHub organization that maintains the Hatch package ecosystem. + +**Hatchling** +: The build system and package manager for the Hatch ecosystem. + +**Package Manager** +: A tool for installing, updating, and managing software packages (e.g., pip, apt, npm). \ No newline at end of file diff --git a/docs/articles/devs/Contributing.md b/docs/articles/devs/Contributing.md new file mode 100644 index 0000000..e7ad38f --- /dev/null +++ b/docs/articles/devs/Contributing.md @@ -0,0 +1,63 @@ +# Contributing Guidelines + +This article is about: +- How to contribute to the Hatch Schemas project +- Development workflow and standards +- Issue reporting and pull request processes + +You will learn about: +- How to propose schema changes +- Development setup and contribution workflow +- Quality standards and review processes + +Contributions to Hatch Schemas help improve the standardization and reliability of the Hatch package ecosystem. + +## How to Contribute + +### Proposing Schema Changes + +To propose revisions to the schemas: + +1. **Open an Issue**: Create a detailed issue describing the proposed changes +2. **Provide Rationale**: Explain why the change is needed and how it improves the ecosystem +3. **Include Examples**: Show how the change would affect existing schemas +4. **Consider Backwards Compatibility**: Discuss migration strategies for existing packages + +### Development Workflow + +1. **Fork the Repository**: Create your own fork of the project +2. **Create a Feature Branch**: Use descriptive branch names +3. **Make Changes**: Follow the project's coding standards +4. **Test Changes**: Ensure all validation tests pass +5. **Submit Pull Request**: Include detailed description of changes + +### Code Standards + +- **JSON Schema Format**: Follow JSON Schema Draft 7 specification +- **Versioning**: Use semantic versioning for schema releases +- **Documentation**: Update documentation for any schema changes +- **Testing**: Include validation tests for new schema features + +## Issue Reporting + +When reporting issues: + +- Use clear, descriptive titles +- Provide schema validation examples +- Include expected vs actual behavior +- Reference relevant schema versions + +## Review Process + +All contributions go through: + +1. **Automated Testing**: Schema validation and documentation checks +2. **Peer Review**: Community review of proposed changes +3. **Maintainer Approval**: Final review by project maintainers +4. **Release Process**: Integration into versioned releases + +## See Also + +- [Development Setup](DevelopmentSetup.md) +- [Repository Structure](RepositoryStructure.md) +- [Schema Versioning](SchemaVersioning.md) \ No newline at end of file diff --git a/docs/articles/devs/DevelopmentSetup.md b/docs/articles/devs/DevelopmentSetup.md new file mode 100644 index 0000000..c18dc6c --- /dev/null +++ b/docs/articles/devs/DevelopmentSetup.md @@ -0,0 +1,145 @@ +# Development Setup + +This article is about: +- Setting up a development environment for Hatch Schemas +- Required tools and dependencies +- Testing and validation workflows + +You will learn about: +- How to set up your development environment +- How to run tests and validation +- How to test schema changes locally + +Setting up a proper development environment ensures you can contribute effectively to the Hatch Schemas project. + +## Prerequisites + +- **Git**: For version control +- **Python 3.8+**: For running validation scripts +- **Node.js** (optional): For additional JSON schema tools + +## Setup Steps + +### 1. Clone the Repository + +```bash +git clone https://github.com/CrackingShells/Hatch-Schemas.git +cd Hatch-Schemas +``` + +### 2. Install Python Dependencies + +```bash +# Install required packages for validation +pip install jsonschema requests +``` + +### 3. Verify Setup + +```bash +# Test schema loading +python examples/schema_updater.py +``` + +## Development Workflow + +### Schema Validation + +Test your schema changes: + +```bash +# Validate package schema +python -c " +import jsonschema +import json + +# Load schema +with open('package/v1.2.0/hatch_pkg_metadata_schema.json') as f: + schema = json.load(f) + +# Test with example data +test_data = { + 'package_schema_version': '1.2.0', + 'name': 'test_package', + 'version': '1.0.0', + 'description': 'Test package', + 'tags': ['test'], + 'author': {'name': 'Test User'}, + 'license': {'name': 'MIT'}, + 'entry_point': 'main.py' +} + +jsonschema.validate(test_data, schema) +print('Schema validation passed!') +" +``` + +### Testing Changes + +Before submitting changes: + +1. **Validate All Schemas**: Ensure JSON schema syntax is correct +2. **Test Examples**: Verify example data validates against schemas +3. **Update Documentation**: Keep documentation in sync with schema changes +4. **Check CI**: Ensure automated tests pass + +### Local Testing + +Create test files to validate your changes: + +```bash +# Create test package metadata +cat > test_package.json << 'EOF' +{ + "$schema": "./package/v1.2.0/hatch_pkg_metadata_schema.json", + "package_schema_version": "1.2.0", + "name": "test_package", + "version": "1.0.0", + "description": "Test package for development", + "tags": ["test", "development"], + "author": { + "name": "Developer", + "email": "dev@example.com" + }, + "license": { + "name": "MIT" + }, + "entry_point": "main.py" +} +EOF + +# Validate with jsonschema +python -c " +import jsonschema +import json + +with open('test_package.json') as f: + data = json.load(f) +with open('package/v1.2.0/hatch_pkg_metadata_schema.json') as f: + schema = json.load(f) + +jsonschema.validate(data, schema) +print('Validation successful!') +" +``` + +## CI/CD Pipeline + +The project uses GitHub Actions for: + +- **Schema Validation**: Ensuring all schemas are valid JSON Schema documents +- **Documentation Building**: Verifying documentation builds correctly +- **Release Automation**: Publishing versioned releases + +## Debugging Tips + +- Use online JSON Schema validators for quick testing +- Check schema syntax with JSON linters +- Validate against multiple example files +- Test both valid and invalid data + +## See Also + +- [Contributing Guidelines](Contributing.md) +- [Repository Structure](RepositoryStructure.md) +- [Schema Versioning](SchemaVersioning.md) \ No newline at end of file diff --git a/docs/articles/devs/RepositoryStructure.md b/docs/articles/devs/RepositoryStructure.md new file mode 100644 index 0000000..01c98ce --- /dev/null +++ b/docs/articles/devs/RepositoryStructure.md @@ -0,0 +1,133 @@ +# Repository Structure + +This article is about: +- Organization of files and directories in the Hatch Schemas repository +- Purpose and contents of each major directory +- Relationship between schemas, documentation, and examples + +You will learn about: +- How the repository is organized +- Where to find specific types of files +- How schemas are versioned and distributed + +Understanding the repository structure helps you navigate the codebase and contribute effectively. + +## Directory Overview + +``` +Hatch-Schemas/ +├── .github/ # GitHub Actions workflows +│ └── workflows/ +│ ├── schema-deployment.yml +│ ├── schema-validation.yml +│ └── manual-schema-test.yml +├── docs/ # Documentation +│ └── articles/ # Structured documentation +│ ├── users/ # User-facing documentation +│ ├── devs/ # Developer documentation +│ └── appendices/ # Supporting information +├── examples/ # Example scripts and utilities +│ └── schema_updater.py # Schema loading utility +├── package/ # Package schema versions +│ ├── v1/ # Legacy version +│ ├── v1.1.0/ # Previous version +│ └── v1.2.0/ # Current version +│ └── hatch_pkg_metadata_schema.json +├── registry/ # Registry schema versions +│ ├── v1/ # Legacy version +│ ├── v1.1.0/ # Previous version +│ └── v1.2.0/ # Current version +│ └── hatch_all_pkg_metadata_schema.json +├── LICENSE # Project license +└── README.md # Project overview +``` + +## Schema Organization + +### Package Schemas + +Located in `package/` directory: + +- **Current Version**: `package/v1.2.0/hatch_pkg_metadata_schema.json` +- **Previous Versions**: Maintained for backward compatibility +- **Deprecated Versions**: Marked but kept for reference + +### Registry Schemas + +Located in `registry/` directory: + +- **Current Version**: `registry/v1.2.0/hatch_all_pkg_metadata_schema.json` +- **Previous Versions**: Maintained for backward compatibility +- **Deprecated Versions**: Marked but kept for reference + +## Documentation Structure + +The `docs/` directory follows a standardized structure: + +### User Documentation (`docs/articles/users/`) + +- **Getting Started**: Introduction and quick start guides +- **Schema Usage**: Access, validation, and programmatic usage +- **Schema References**: Detailed field documentation for each schema type + +### Developer Documentation (`docs/articles/devs/`) + +- **Contributing**: Guidelines for contributing to the project +- **Development Setup**: Environment setup and testing procedures +- **Architecture**: Repository structure and design decisions + +### Appendices (`docs/articles/appendices/`) + +- **Glossary**: Definitions of key terms and concepts +- **Reference Materials**: Supporting documentation + +## CI/CD Infrastructure + +### GitHub Actions (`.github/workflows/`) + +- **schema-deployment.yml**: Automated release deployment +- **schema-validation.yml**: Schema validation and testing +- **manual-schema-test.yml**: Manual testing workflows + +### Release Process + +1. **Schema Changes**: Modifications to schema files trigger validation +2. **Automated Testing**: All schemas are validated against test data +3. **Release Creation**: Successful changes create GitHub releases +4. **Tag Format**: `schemas-{type}-{version}` (e.g., `schemas-package-v1.2.0`) + +## Examples and Utilities + +### Schema Updater (`examples/schema_updater.py`) + +Provides programmatic access to schemas: + +- **Schema Loading**: Automatic download and caching +- **Version Management**: Support for specific version requests +- **API Integration**: GitHub API integration for release discovery + +## Design Patterns + +### Versioning Strategy + +- **Semantic Versioning**: Major.minor.patch format +- **Backward Compatibility**: Previous versions maintained +- **Deprecation Path**: Clear migration guidelines + +### Distribution Method + +- **GitHub Releases**: Primary distribution channel +- **Direct Access**: Raw file access via GitHub +- **API Discovery**: Programmatic version detection + +## File Naming Conventions + +- **Schema Files**: Descriptive names with clear purpose +- **Version Directories**: Semantic version format (v1.2.0) +- **Documentation**: Clear, hierarchical organization + +## See Also + +- [Contributing Guidelines](Contributing.md) +- [Development Setup](DevelopmentSetup.md) +- [Schema Versioning](SchemaVersioning.md) \ No newline at end of file diff --git a/docs/articles/devs/SchemaVersioning.md b/docs/articles/devs/SchemaVersioning.md new file mode 100644 index 0000000..c4b32f6 --- /dev/null +++ b/docs/articles/devs/SchemaVersioning.md @@ -0,0 +1,162 @@ +# Schema Versioning + +This article is about: +- Versioning strategy for Hatch Schemas +- Release processes and compatibility management +- Migration guidelines for schema updates + +You will learn about: +- How schema versions are managed +- How to handle schema migrations +- Release automation and distribution processes + +Proper schema versioning ensures stability and provides clear upgrade paths for the Hatch ecosystem. + +## Versioning Strategy + +### Semantic Versioning + +Hatch Schemas follow semantic versioning (semver) principles: + +- **Major Version** (X.0.0): Breaking changes that require migration +- **Minor Version** (0.X.0): New features that maintain backward compatibility +- **Patch Version** (0.0.X): Bug fixes and clarifications + +### Version Format + +Schema versions use the format: `vX.Y.Z` + +Examples: +- `v1.2.0` - Current package and registry schema version +- `v1.1.0` - Previous minor version +- `v1.0.0` - Initial major version + +## Release Process + +### Automated Releases + +Schema releases are automated through GitHub Actions: + +1. **Change Detection**: Commits to versioned schema folders trigger releases +2. **Validation**: All schemas are validated before release +3. **Tag Creation**: Releases are tagged with format `schemas-{type}-{version}` +4. **Asset Publication**: Schema files are attached to GitHub releases + +### Release Tags + +- **Package Schema**: `schemas-package-v1.2.0` +- **Registry Schema**: `schemas-registry-v1.2.0` + +### Distribution Channels + +Released schemas are available through: + +1. **GitHub Releases**: Direct file downloads +2. **Raw URLs**: Direct access via raw.githubusercontent.com +3. **API Discovery**: Programmatic version detection + +## Compatibility Management + +### Backward Compatibility + +- **Minor Versions**: Must be backward compatible +- **Optional Fields**: New fields should be optional when possible +- **Deprecation**: Clear deprecation warnings before removal + +### Breaking Changes + +When breaking changes are necessary: + +1. **Major Version Bump**: Increment major version number +2. **Migration Guide**: Provide clear migration instructions +3. **Legacy Support**: Maintain previous version for transition period + +### Deprecation Process + +1. **Mark as Deprecated**: Add deprecation notices +2. **Provide Alternatives**: Document replacement approaches +3. **Sunset Timeline**: Communicate removal schedule +4. **Remove After Grace Period**: Remove after reasonable transition time + +## Migration Guidelines + +### Schema Updates + +When migrating to a new schema version: + +1. **Review Changes**: Check release notes for breaking changes +2. **Update References**: Update `$schema` references in JSON files +3. **Validate Data**: Ensure existing data validates against new schema +4. **Test Applications**: Verify application compatibility + +### Example Migration + +Migrating from v1.1.0 to v1.2.0: + +```json +// Old schema reference +{ + "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.1.0/hatch_pkg_metadata_schema.json", + // ... package data +} + +// New schema reference +{ + "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", + // ... package data (may need updates for new requirements) +} +``` + +## Version Discovery + +### Programmatic Discovery + +Use the GitHub API to discover latest versions: + +```python +import requests + +def get_latest_schema_version(schema_type): + """Get the latest version for a schema type.""" + api_url = "https://api.github.com/repos/crackingshells/Hatch-Schemas/releases" + response = requests.get(api_url) + releases = response.json() + + for release in releases: + tag = release['tag_name'] + if tag.startswith(f'schemas-{schema_type}-'): + return tag.replace(f'schemas-{schema_type}-', '') + + return None + +# Get latest package schema version +latest_package_version = get_latest_schema_version("package") +print(f"Latest package schema: {latest_package_version}") +``` + +### Manual Discovery + +Visit the releases page to see all available versions: +https://github.com/CrackingShells/Hatch-Schemas/releases + +## Best Practices + +### For Schema Maintainers + +- **Test Thoroughly**: Validate against real-world data +- **Document Changes**: Provide clear release notes +- **Gradual Rollouts**: Consider phased releases for major changes +- **Community Input**: Gather feedback before breaking changes + +### For Schema Consumers + +- **Pin Versions**: Use specific versions in production +- **Monitor Releases**: Stay informed about new versions +- **Test Updates**: Validate compatibility before upgrading +- **Provide Feedback**: Report issues and suggestions + +## See Also + +- [Contributing Guidelines](Contributing.md) +- [Development Setup](DevelopmentSetup.md) +- [Repository Structure](RepositoryStructure.md) \ No newline at end of file diff --git a/docs/articles/table_of_contents.md b/docs/articles/table_of_contents.md new file mode 100644 index 0000000..675cc98 --- /dev/null +++ b/docs/articles/table_of_contents.md @@ -0,0 +1,39 @@ +# Table of Contents + +This article is about: +- Navigation structure for all Hatch Schemas documentation + +You will learn about: +- How to find specific documentation topics +- The organization of user and developer resources + +## User Documentation + +### Getting Started +- [Getting Started with Hatch Schemas](users/GettingStarted.md) + +### Schema Usage +- [Schema Access Guide](users/SchemaAccess.md) +- [Schema Validation Guide](users/SchemaValidation.md) +- [Programmatic Usage Guide](users/ProgrammaticUsage.md) + +### Schema References +- [Package Schema Overview](users/PackageSchema/Overview.md) +- [Package Schema Field Reference](users/PackageSchema/Fields.md) +- [Package Schema Examples](users/PackageSchema/Examples.md) +- [Registry Schema Overview](users/RegistrySchema/Overview.md) +- [Registry Schema Field Reference](users/RegistrySchema/Fields.md) + +## Developer Documentation + +### Contributing +- [Contributing Guidelines](devs/Contributing.md) +- [Development Setup](devs/DevelopmentSetup.md) + +### Architecture +- [Repository Structure](devs/RepositoryStructure.md) +- [Schema Versioning](devs/SchemaVersioning.md) + +## Appendices + +- [Glossary](appendices/glossary.md) \ No newline at end of file diff --git a/docs/articles/users/GettingStarted.md b/docs/articles/users/GettingStarted.md new file mode 100644 index 0000000..a0b20ee --- /dev/null +++ b/docs/articles/users/GettingStarted.md @@ -0,0 +1,69 @@ +# Getting Started with Hatch Schemas + +This article is about: +- Introduction to Hatch Schemas +- Basic concepts and usage patterns +- First steps for new users + +You will learn about: +- What Hatch Schemas are and their purpose +- How to access and use schemas in your projects +- Basic validation workflows + +Hatch Schemas provide standardized JSON schemas for validating metadata in the CrackingShells package ecosystem. + +## What are Hatch Schemas? + +Hatch Schemas define the structure and validation rules for two key components: + +- **Package Schema**: Validates individual package metadata files +- **Registry Schema**: Validates the central package registry + +## Quick Start + +### 1. Choose Your Schema + +Determine which schema you need: +- Use the **Package Schema** when creating or validating individual package metadata +- Use the **Registry Schema** when working with the central package registry + +### 2. Access the Schema + +The simplest way to access schemas is through direct URLs: + +```bash +# Package Schema (latest) +https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json + +# Registry Schema (latest) +https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/registry/v1.2.0/hatch_all_pkg_metadata_schema.json +``` + +### 3. Reference in Your JSON + +Add a `$schema` reference to your JSON files: + +```json +{ + "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", + "package_schema_version": "1.2.0", + "name": "my_package", + "version": "1.0.0", + "description": "My awesome package", + "tags": ["example"], + "author": {"name": "John Doe"}, + "license": {"name": "MIT"}, + "entry_point": "server.py" +} +``` + +## Next Steps + +- Learn more about [Schema Access](SchemaAccess.md) methods +- Understand [Schema Validation](SchemaValidation.md) techniques +- Explore [Programmatic Usage](ProgrammaticUsage.md) patterns +- Review detailed schema documentation for [Package Schema](PackageSchema/Overview.md) or [Registry Schema](RegistrySchema/Overview.md) + +## Need Help? + +Check the [Glossary](../appendices/glossary.md) for definitions of key terms. \ No newline at end of file diff --git a/docs/articles/users/PackageSchema/Examples.md b/docs/articles/users/PackageSchema/Examples.md new file mode 100644 index 0000000..dc1bf28 --- /dev/null +++ b/docs/articles/users/PackageSchema/Examples.md @@ -0,0 +1,130 @@ +# Package Schema Examples + +This article is about: +- Real-world examples of valid package metadata files +- Common patterns and use cases for package schemas +- Best practices for structuring package metadata + +You will learn about: +- How to create valid package metadata files +- Examples ranging from simple to comprehensive packages +- Common field combinations and patterns + +This document provides examples of valid package metadata files compliant with the Hatch Package Schema. + +## Basic Example + +```json +{ + "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", + "package_schema_version": "1.2.0", + "name": "simple_package", + "version": "1.0.0", + "description": "A simple example package", + "tags": ["example", "simple"], + "author": { + "name": "John Doe", + "email": "john.doe@example.com" + }, + "license": { + "name": "MIT" + }, + "entry_point": "server.py" +} +``` + +## Complete Example + +```json +{ + "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", + "package_schema_version": "1.2.0", + "name": "complete_package", + "version": "1.2.0", + "description": "A comprehensive example with all possible fields", + "tags": ["example", "complete", "reference"], + "author": { + "name": "Jane Smith", + "email": "jane.smith@example.com" + }, + "contributors": [ + { + "name": "John Doe", + "email": "john.doe@example.com" + }, + { + "name": "Alice Johnson" + } + ], + "license": { + "name": "MIT", + "uri": "https://opensource.org/licenses/MIT" + }, + "repository": "https://github.com/crackingshells/example-package", + "documentation": "https://docs.example.com/package", + "dependencies": { + "hatch": [ + { + "name": "base_package", + "version_constraint": ">=1.0.0" + }, + { + "name": "utils_package", + "version_constraint": "==1.2.0" + } + ], + "python": [ + { + "name": "numpy", + "version_constraint": ">=1.20.0" + }, + { + "name": "pandas", + "version_constraint": ">=1.3.0" + }, + { + "name": "scikit-learn", + "version_constraint": ">=1.0.0" + } + ], + "system": [ + { + "name": "libopencv-dev", + "version_constraint": ">=4.0.0", + "package_manager": "apt" + } + ], + "docker": [ + { + "name": "postgres", + "version_constraint": ">=13.0.0", + "registry": "dockerhub" + } + ] + }, + "compatibility": { + "hatchling": ">=0.1.0", + "python": ">=3.8" + }, + "entry_point": "server.py", + "tools": [ + { + "name": "data_preprocessor", + "description": "Preprocesses input data for the model" + }, + { + "name": "model_evaluator", + "description": "Evaluates model performance on test data" + } + ], + "citations": { + "origin": "Smith, J. et al. (2023). Advanced machine learning techniques. Journal of AI Research, 45(2), 123-145.", + "mcp": "Doe, J. (2023). MCP implementation of Smith's algorithms. https://doi.org/10.xxxx/xxxxx" + } +} +``` + +## See Also + +- [Package Schema Field Reference](Fields.md) +- [Package Schema Overview](Overview.md) \ No newline at end of file diff --git a/docs/articles/users/PackageSchema/Fields.md b/docs/articles/users/PackageSchema/Fields.md new file mode 100644 index 0000000..a80e52f --- /dev/null +++ b/docs/articles/users/PackageSchema/Fields.md @@ -0,0 +1,260 @@ +# Package Schema Field Reference + +This article is about: +- Detailed specifications for each field in the Package Schema +- Data types, formats, and validation rules +- Examples and usage patterns for schema fields + +You will learn about: +- Required and optional field definitions +- Proper formatting for each field type +- How to structure complex fields like dependencies + +This document provides detailed information about each field in the Package Schema. + +## Required Fields + +### package_schema_version + +- **Type**: String +- **Pattern**: `^\d+\.\d+\.\d+$` +- **Description**: Version of the schema used for this package metadata +- **Example**: `"1.2.0"` + +### name + +- **Type**: String +- **Pattern**: `^[a-z0-9_]+$` +- **Description**: Package identifier. Must be lowercase alphanumeric with underscores. +- **Example**: `"my_package"` or `"image_classifier"` + +### version + +- **Type**: String +- **Pattern**: `^\d+(\.\d+)*$` +- **Description**: Semantic version of the package +- **Example**: `"1.0.0"` or `"2.1.3"` + +### description + +- **Type**: String +- **Description**: Human-readable description of the package +- **Example**: `"A package for image classification using deep learning"` + +### tags + +- **Type**: Array of strings +- **Description**: Keywords for package discovery +- **Example**: `["machine learning", "image", "classification"]` + +### author + +- **Type**: Object +- **Description**: Information about the package author +- **Required Properties**: + - **name** (String): The name of the author +- **Optional Properties**: + - **email** (String): The email address of the author (format: email) +- **Example**: + ```json + "author": { + "name": "John Doe", + "email": "john.doe@example.com" + } + ``` + +### license + +- **Type**: Object +- **Description**: License information for the package +- **Required Properties**: + - **name** (String): The name of the license +- **Optional Properties**: + - **uri** (String): URL to the license text (format: URI) +- **Example**: + ```json + "license": { + "name": "MIT", + "uri": "https://opensource.org/licenses/MIT" + } + ``` + +### entry_point + +- **Type**: String +- **Description**: Primary entry point for the package +- **Example**: `"server.py"` or `"main.py"` + +## Optional Fields + +### contributors + +- **Type**: Array of objects +- **Description**: List of additional contributors +- **Each Contributor**: + - **Required Properties**: + - **name** (String): The name of the contributor + - **Optional Properties**: + - **email** (String): The email address of the contributor (format: email) +- **Example**: + ```json + "contributors": [ + { + "name": "Jane Smith", + "email": "jane.smith@example.com" + }, + { + "name": "Bob Johnson" + } + ] + ``` + +### repository + +- **Type**: String +- **Format**: URI +- **Description**: URL to the source code repository +- **Example**: `"https://github.com/username/repository"` + +### documentation + +- **Type**: String +- **Format**: URI +- **Description**: URL to the package documentation +- **Example**: `"https://docs.example.com/my_package"` + +### dependencies + +- **Type**: Object +- **Description**: Dependencies required by the package +- **Properties**: + - **hatch** (Array): Hatch package dependencies + - **python** (Array): Python package dependencies + - **system** (Array): System package dependencies + - **docker** (Array): Docker image dependencies + +#### hatch dependencies + +- **Type**: Array of objects +- **Description**: Hatch packages required by this package +- **Each Dependency**: + - **name** (String): Name of the Hatch package + - **version_constraint** (String): Version constraint (e.g., ">=1.0.0") +- **Example**: + ```json + "hatch": [ + { + "name": "base_package", + "version_constraint": ">=1.0.0" + } + ] + ``` + +#### python dependencies + +- **Type**: Array of objects +- **Description**: Python packages required by this package +- **Each Dependency**: + - **name** (String): Name of the Python package + - **version_constraint** (String): Version constraint (e.g., ">=1.0.0") + - **package_manager** (String, default: "pip"): Package manager to use +- **Example**: + ```json + "python": [ + { + "name": "numpy", + "version_constraint": ">=1.20.0" + }, + { + "name": "pandas", + "version_constraint": ">=1.3.0" + } + ] + ``` + +#### system dependencies + +- **Type**: Array of objects +- **Description**: System packages required by this package +- **Each Dependency**: + - **name** (String): Name of the system package + - **version_constraint** (String): Version constraint + - **package_manager** (String, default: "apt"): Package manager to use +- **Example**: + ```json + "system": [ + { + "name": "libopencv-dev", + "version_constraint": ">=4.0.0" + } + ] + ``` + +#### docker dependencies + +- **Type**: Array of objects +- **Description**: Docker images required by this package +- **Each Dependency**: + - **name** (String): Name of the Docker image + - **version_constraint** (String): Version constraint + - **registry** (String, default: "dockerhub"): Registry to pull from +- **Example**: + ```json + "docker": [ + { + "name": "postgres", + "version_constraint": ">=13.0.0" + } + ] + ``` + +### compatibility + +- **Type**: Object +- **Description**: Compatibility requirements +- **Properties**: + - **hatchling** (String): Version constraint for Hatchling + - **python** (String): Version constraint for Python +- **Example**: + ```json + "compatibility": { + "hatchling": ">=0.1.0", + "python": ">=3.8" + } + ``` + +### tools + +- **Type**: Array of objects +- **Description**: Additional tools provided by the package +- **Each Tool**: + - **name** (String): Name of the tool + - **description** (String): Description of the tool +- **Example**: + ```json + "tools": [ + { + "name": "data_preprocessor", + "description": "Preprocesses input data for the model" + }, + { + "name": "model_evaluator", + "description": "Evaluates model performance on test data" + } + ] + ``` + +### citations + +- **Type**: Object +- **Description**: Citation information for the package. +- **Properties**: + - **origin** (String): Citation for the original work. Free format. + - **mcp** (String): Citation for the MCP implementation. Free format. +- **Example**: + ```json + "citations": { + "origin": "Doe, J. et al. (2023). Novel image classification approach. Journal of AI Research, 45(2), 123-145.", + "mcp": "Smith, J. (2023). MCP implementation of Doe's image classifier. https://doi.org/10.xxxx/xxxxx" + } + ``` \ No newline at end of file diff --git a/docs/articles/users/PackageSchema/Overview.md b/docs/articles/users/PackageSchema/Overview.md new file mode 100644 index 0000000..57b32a6 --- /dev/null +++ b/docs/articles/users/PackageSchema/Overview.md @@ -0,0 +1,96 @@ +# Package Schema Overview + +This article is about: +- Structure and purpose of the Package Schema +- Core components and validation rules +- Usage patterns for package metadata + +You will learn about: +- How to structure valid package metadata +- Required and optional fields in the schema +- Examples of compliant package definitions + +The Package Schema (`hatch_pkg_metadata_schema.json`) defines the structure for individual package metadata files in the Hatch ecosystem. + +## Current Version + +The current version of the Package Schema is **v1.2.0**. + +## Schema Structure + +The Package Schema includes the following major sections: + +- **Basic Package Information**: Schema version, name, version, description, and tags +- **Author Information**: Author details and optional contributors list +- **License Information**: License name and optional URI +- **Package Links**: Repository and documentation URLs +- **Dependencies**: Hatch, Python, system, and Docker dependencies +- **Compatibility Requirements**: Hatchling and Python version constraints +- **Entry Points and Tools**: Primary entry point and additional tools + +For detailed field-by-field documentation including types, formats, and examples, see the [Package Schema Field Reference](Fields.md). + +### Compatibility + +- **compatibility**: Object defining compatibility requirements (optional) + - **hatchling**: Version constraint for Hatchling compatibility + - **python**: Version constraint for Python compatibility + +### Entry Points and Tools + +- **entry_point**: Primary entry point for the package (required) +- **tools**: List of additional tools provided by the package (optional) + +### Citations + +- **citations**: Citation information for the package (optional) + - **origin**: Citation for the original work + - **mcp**: Citation for the MCP implementation + +## Example + +```json +{ + "package_schema_version": "1.2.0", + "name": "example_package", + "version": "1.0.0", + "description": "An example Hatch package", + "tags": ["example", "demo"], + "author": { + "name": "John Doe", + "email": "john.doe@example.com" + }, + "license": { + "name": "MIT", + "uri": "https://opensource.org/licenses/MIT" + }, + "entry_point": "server.py", + "dependencies": { + "hatch": [ + { + "name": "base_package", + "version_constraint": ">=1.0.0" + } + ], + "python": [ + { + "name": "numpy", + "version_constraint": ">=1.20.0" + }, + { + "name": "pandas", + "version_constraint": ">=1.3.0" + } + ] + }, + "compatibility": { + "python": ">=3.8" + } +} +``` + +## See Also + +- [Package Schema Field Reference](Fields.md) +- [Registry Schema Overview](../RegistrySchema/Overview.md) +- [Schema Validation Guide](../SchemaValidation.md) \ No newline at end of file diff --git a/docs/articles/users/ProgrammaticUsage.md b/docs/articles/users/ProgrammaticUsage.md new file mode 100644 index 0000000..8b469e0 --- /dev/null +++ b/docs/articles/users/ProgrammaticUsage.md @@ -0,0 +1,174 @@ +# Programmatic Usage Guide + +This article is about: +- Using Hatch schemas programmatically in applications +- Schema loading, caching, and retrieval techniques +- Integration patterns for automated workflows + +You will learn about: +- How to load schemas programmatically +- Implementing schema caching for performance +- Building validation into your applications + +This guide covers integrating Hatch schemas into your applications and automated workflows. + +## Simple Schema Loading + +For basic usage, you can use the provided example utility: + +```python +from examples.schema_updater import load_schema, configure_logger +import logging + +# Optional: Configure logging +configure_logger(level=logging.INFO) + +# Load the latest schemas (automatically downloads if needed) +package_schema = load_schema("package") # Latest package schema +registry_schema = load_schema("registry") # Latest registry schema + +# Load specific version +package_v1_2_0 = load_schema("package", "v1.2.0") # Specific version +``` + +## Manual Schema Retrieval + +For more control over schema retrieval, you can implement your own functions: + +```python +import requests +import json + +def get_latest_schema_info(): + """Get latest schema version information from GitHub API.""" + api_url = "https://api.github.com/repos/crackingshells/Hatch-Schemas/releases" + response = requests.get(api_url) + releases = response.json() + + # Find latest package and registry schema releases + latest_schemas = {} + for release in releases: + tag = release['tag_name'] + if tag.startswith('schemas-package-'): + if 'package' not in latest_schemas: + version = tag.replace('schemas-package-', '') + latest_schemas['package'] = { + 'version': version, + 'url': f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-package-{version}/hatch_pkg_metadata_schema.json" + } + elif tag.startswith('schemas-registry-'): + if 'registry' not in latest_schemas: + version = tag.replace('schemas-registry-', '') + latest_schemas['registry'] = { + 'version': version, + 'url': f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-registry-{version}/hatch_all_pkg_metadata_schema.json" + } + + return latest_schemas + +def load_schema(schema_type, version=None): + """Load a specific schema or the latest version.""" + if version is None: + # Get latest version + latest_info = get_latest_schema_info() + schema_url = latest_info[schema_type]['url'] + else: + # Use specific version from release + if schema_type == 'package': + schema_url = f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-package-{version}/hatch_pkg_metadata_schema.json" + else: + schema_url = f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-registry-{version}/hatch_all_pkg_metadata_schema.json" + + response = requests.get(schema_url) + return response.json() +``` + +## Basic Validation + +```python +import jsonschema + +# Load schema +schema = load_schema("package") + +# Your package data +package_data = { + "package_schema_version": "1.2.0", + "name": "my_package", + "version": "1.0.0", + "description": "My awesome package", + "tags": ["example"], + "author": {"name": "John Doe"}, + "license": {"name": "MIT"}, + "entry_point": "server.py" +} + +# Validate +try: + jsonschema.validate(package_data, schema) + print("Package metadata is valid") +except jsonschema.ValidationError as e: + print(f"Validation error: {e.message}") +``` + +## Advanced Use Cases + +An implementation example is also [available](../../examples/schema_updater.py) + +### Schema Caching + +For performance and offline availability, implement a schema caching mechanism: + +```python +import os +import json +from pathlib import Path + +CACHE_DIR = Path.home() / ".cache" / "hatch-schemas" + +def cache_schema(schema_type, version, schema_data): + """Cache a schema locally.""" + # Ensure cache directory exists + os.makedirs(CACHE_DIR, exist_ok=True) + + # Write schema to cache + cache_file = CACHE_DIR / f"{schema_type}-{version}.json" + with open(cache_file, 'w') as f: + json.dump(schema_data, f) + + return cache_file + +def get_cached_schema(schema_type, version): + """Retrieve a schema from the local cache.""" + cache_file = CACHE_DIR / f"{schema_type}-{version}.json" + + if cache_file.exists(): + with open(cache_file, 'r') as f: + return json.load(f) + + return None + +def load_schema_with_cache(schema_type, version=None): + """Load schema with caching support.""" + if version is None: + # Get latest version info + latest_info = get_latest_schema_info() + version = latest_info[schema_type]['version'] + + # Try to load from cache + cached_schema = get_cached_schema(schema_type, version) + if cached_schema: + return cached_schema + + # If not in cache, download and cache + schema_data = load_schema(schema_type, version) + if schema_data: + cache_schema(schema_type, version, schema_data) + + return schema_data +``` + +## See Also + +- [Schema Access Guide](SchemaAccess.md) +- [Schema Validation Guide](SchemaValidation.md) \ No newline at end of file diff --git a/docs/articles/users/RegistrySchema/Fields.md b/docs/articles/users/RegistrySchema/Fields.md new file mode 100644 index 0000000..e08a526 --- /dev/null +++ b/docs/articles/users/RegistrySchema/Fields.md @@ -0,0 +1,210 @@ +# Registry Schema Field Reference + +This article is about: +- Detailed specifications for each field in the Registry Schema +- Data types, formats, and validation rules for registry objects +- Structure of repositories, packages, and version information + +You will learn about: +- Required and optional field definitions for the registry +- Proper formatting for registry-specific field types +- How to structure complex objects like package versions and verification data + +This document provides detailed information about each field in the Registry Schema. + +## Required Fields + +### registry_schema_version + +- **Type**: String +- **Pattern**: `^\d+(\.\d+)*$` +- **Description**: Version of this registry schema +- **Example**: `"1.2.0"` + +### last_updated + +- **Type**: String +- **Format**: date-time +- **Description**: Timestamp of when the registry was last updated +- **Example**: `"2024-06-01T12:00:00Z"` + +### repositories + +- **Type**: Array of objects +- **Description**: List of repositories containing packages +- **Each Repository**: See [Repository Object](#repository-object) + +### stats + +- **Type**: Object +- **Description**: Registry-wide statistics +- **Properties**: + - **total_packages** (Integer): Total number of unique packages + - **total_versions** (Integer): Total number of package versions +- **Example**: + ```json + "stats": { + "total_packages": 10, + "total_versions": 25 + } + ``` + +## Object Definitions + +### Repository Object + +- **Type**: Object +- **Description**: Repository containing packages +- **Required Properties**: + - **name** (String): Repository name (e.g., 'Hatch-Dev') + - **url** (String, format: URI): URL to the repository + - **packages** (Array): List of packages in this repository (see [Package Object](#package-object)) + - **last_indexed** (String, format: date-time): When this repository was last indexed +- **Example**: + ```json + { + "name": "Hatch-Dev", + "url": "https://github.com/crackingshells/Hatch-Dev", + "last_indexed": "2024-06-01T12:00:00Z", + "packages": [...] + } + ``` + +### Package Object + +- **Type**: Object +- **Description**: Package within a repository +- **Required Properties**: + - **name** (String): Package identifier (pattern: `^[a-z0-9_]+$`) + - **description** (String): Human-readable description of the package + - **tags** (Array of strings): Keywords for discovery + - **versions** (Array of objects): Available versions of this package (see [Version Object](#version-object)) + - **latest_version** (String): Latest version of the package (pattern: `^\d+(\.\d+)*$`) +- **Example**: + ```json + { + "name": "example_package", + "description": "An example Hatch package", + "tags": ["example", "demo"], + "latest_version": "1.2.0", + "versions": [...] + } + ``` + +### Version Object + +- **Type**: Object +- **Description**: Version of a package +- **Required Properties**: + - **version** (String): Semantic version of the package (pattern: `^\d+(\.\d+)*$`) + - **author** (Object): Author of the submission of this version + - **GitHubID** (String): GitHub username of the author + - **email** (String, format: email): Email of the author + - **release_uri** (String, format: URI): URI to the release page for this version + - **added_date** (String, format: date-time): When this version was added to the registry +- **Optional Properties**: + - **base_version** (String): Version this differential is based on (null for first version) + - **dependency_changes** (Object): Changes to dependencies since base_version + - **compatibility_changes** (Object): Changed compatibility constraints since base_version + - **verification** (Object): Verification status and metadata +- **Example**: + ```json + { + "version": "1.2.0", + "author": { + "GitHubID": "johndoe", + "email": "john.doe@example.com" + }, + "release_uri": "https://github.com/crackingshells/Hatch-Dev/releases/tag/example_package-v1.2.0", + "base_version": "1.1.0", + "added_date": "2024-05-15T10:30:00Z", + "verification": { + "status": "verified", + "timestamp": "2024-05-16T14:20:00Z", + "verifier": { + "GitHubID": "packagereviewer", + "email": "reviewer@example.com" + }, + "notes": "All tests passed" + } + } + ``` + +### Dependency Changes Object + +- **Type**: Object +- **Description**: Changes to dependencies since base_version +- **Properties**: + - **hatch** (Object): Changes to Hatch dependencies + - **python** (Object): Changes to Python dependencies + - **system** (Object): Changes to system dependencies + - **docker** (Object): Changes to Docker dependencies +- **Each Dependency Type**: + - **added** (Array): Dependencies added since base_version + - **removed** (Array): Names of dependencies removed since base_version + - **modified** (Array): Dependencies with modified version constraints +- **Example**: + ```json + "dependency_changes": { + "hatch": { + "added": [ + { + "name": "new_dependency", + "version_constraint": ">=1.0.0" + } + ], + "removed": ["old_dependency"], + "modified": [ + { + "name": "existing_dependency", + "old_version_constraint": ">=1.0.0", + "new_version_constraint": ">=1.2.0" + } + ] + } + } + ``` + +### Compatibility Changes Object + +- **Type**: Object +- **Description**: Changed compatibility constraints since base_version +- **Properties**: + - **hatchling** (String): Changed version constraint for Hatchling compatibility + - **python** (String): Changed version constraint for Python compatibility +- **Example**: + ```json + "compatibility_changes": { + "hatchling": ">=0.2.0", + "python": ">=3.9" + } + ``` + +### Verification Object + +- **Type**: Object +- **Description**: Verification metadata for a package version +- **Required Properties**: + - **status** (String, enum): Verification status + - Values: "unverified", "validated", "reviewed", "verified", "deprecated" + - Default: "unverified" +- **Optional Properties**: + - **timestamp** (String, format: date-time): When this verification status was last updated + - **verifier** (Object): Entity that performed the verification + - **GitHubID** (String): GitHub username of the verifier + - **email** (String, format: email): Email of the verifier + - **name** (String): Name of the verifier + - **notes** (String): Additional information about verification status +- **Example**: + ```json + "verification": { + "status": "verified", + "timestamp": "2024-05-16T14:20:00Z", + "verifier": { + "GitHubID": "packagereviewer", + "email": "reviewer@example.com", + "name": "Package Reviewer" + }, + "notes": "All tests passed, code review completed" + } + ``` \ No newline at end of file diff --git a/docs/articles/users/RegistrySchema/Overview.md b/docs/articles/users/RegistrySchema/Overview.md new file mode 100644 index 0000000..57edfb5 --- /dev/null +++ b/docs/articles/users/RegistrySchema/Overview.md @@ -0,0 +1,89 @@ +# Registry Schema Overview + +This article is about: +- Structure and purpose of the Registry Schema +- Central catalog organization and validation +- Registry maintenance and update processes + +You will learn about: +- How the registry maintains package catalogs +- Required fields for registry compliance +- Package versioning and verification workflows + +The Registry Schema (`hatch_all_pkg_metadata_schema.json`) defines the structure for the central package registry in the Hatch ecosystem. + +> [!Note] +> The registry is being maintained internally. It is modified by specific functions, and updated during package submission process. In principle, there **MUST NOT** be any manual modification of the registry to add a package. Although, highly discouraged, there **MAY** be manual modification by core members of the organization in case of errors. + +## Current Version + +The current version of the Registry Schema is **v1.2.0**. + +## Schema Structure + +The Registry Schema includes the following major sections: + +- **Registry Metadata**: Schema version, last updated timestamp, and global statistics +- **Repositories**: List of repositories with their metadata and packages +- **Packages**: Package information including name, description, tags, and versions +- **Version Information**: Details about each version of a package, including author, release location, and verification status + +For detailed field-by-field documentation including types, formats, and examples, see the [Registry Schema Field Reference](Fields.md). +- **verification**: Verification status and metadata +- **dependency_changes**: Changes to dependencies since the base version +- **compatibility_changes**: Changes to compatibility requirements + +## Example + +```json +{ + "registry_schema_version": "1.2.0", + "last_updated": "2024-06-01T12:00:00Z", + "repositories": [ + { + "name": "Hatch-Dev", + "url": "https://github.com/crackingshells/Hatch-Dev", + "last_indexed": "2024-06-01T12:00:00Z", + "packages": [ + { + "name": "example_package", + "description": "An example Hatch package", + "tags": ["example", "demo"], + "latest_version": "1.2.0", + "versions": [ + { + "version": "1.2.0", + "author": { + "GitHubID": "johndoe", + "email": "john.doe@example.com" + }, + "release_uri": "https://github.com/crackingshells/Hatch-Dev/releases/tag/example_package-v1.2.0", + "base_version": "1.1.0", + "added_date": "2024-05-15T10:30:00Z", + "verification": { + "status": "verified", + "timestamp": "2024-05-16T14:20:00Z", + "verifier": { + "GitHubID": "packagereviewer", + "email": "reviewer@example.com" + }, + "notes": "All tests passed" + } + } + ] + } + ] + } + ], + "stats": { + "total_packages": 1, + "total_versions": 1 + } +} +``` + +## See Also + +- [Registry Schema Field Reference](Fields.md) +- [Package Schema Overview](../PackageSchema/Overview.md) +- [Schema Validation Guide](../SchemaValidation.md) \ No newline at end of file diff --git a/docs/articles/users/SchemaAccess.md b/docs/articles/users/SchemaAccess.md new file mode 100644 index 0000000..56b3a33 --- /dev/null +++ b/docs/articles/users/SchemaAccess.md @@ -0,0 +1,90 @@ +# Schema Access Guide + +This article is about: +- Methods for accessing Hatch Schemas +- Distribution channels and usage patterns +- Integration approaches for different scenarios + +You will learn about: +- How to access schemas via GitHub, releases, and API +- How to reference schemas in your JSON files +- How to implement local caching strategies + +Hatch Schemas are distributed through multiple channels to support various access patterns and use cases. + +## Distribution Methods + +Hatch Schemas are available through: + +1. **GitHub Repository** - Direct access to schema files via raw.githubusercontent.com +2. **GitHub Releases** - Versioned releases with metadata and direct downloads +3. **GitHub API** - Programmatic discovery of latest versions and releases + +## Accessing Schemas + +### Manual Discovery + +Visit the release page: https://github.com/CrackingShells/Hatch-Schemas/releases + +### Release Downloads + +Download schema files from specific GitHub releases: + +```bash +# Download schema files from specific releases +https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-package-v1.2.0/hatch_pkg_metadata_schema.json +https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-registry-v1.2.0/hatch_all_pkg_metadata_schema.json +``` + +### Direct Schema Access + +Access schema files directly from the GitHub repository: + +```bash +# Direct access to schema files (always current from main branch) +https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json +https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/registry/v1.2.0/hatch_all_pkg_metadata_schema.json +``` + +## Using Schemas in Your Project + +### Referencing in JSON Files + +Reference schemas in your JSON files using the `$schema` property: + +```json +{ + "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", + "name": "my_package", + "version": "1.0.0", + "description": "My awesome package", + "tags": ["example"], + "author": {"name": "John Doe"}, + "license": {"name": "MIT"}, + "entry_point": "server.py" +} +``` + +This reference: +1. Provides editors with schema information for autocompletion and validation +2. Documents which schema version the file adheres to +3. Creates a clear contract for validation tools + +### Local Cache Management + +Consider implementing local caching of schemas. The `examples/schema_updater.py` utility provides this functionality: + +```python +from examples.schema_updater import configure_cache, update_schemas + +# Configure local cache location (optional) +configure_cache(cache_dir="./schemas_cache") + +# Update local schema cache +update_schemas() +``` + +## See Also + +- [Schema Validation](SchemaValidation.md) +- [Programmatic Usage](ProgrammaticUsage.md) \ No newline at end of file diff --git a/docs/articles/users/SchemaValidation.md b/docs/articles/users/SchemaValidation.md new file mode 100644 index 0000000..6e7a9f7 --- /dev/null +++ b/docs/articles/users/SchemaValidation.md @@ -0,0 +1,116 @@ +# Schema Validation Guide + +This article is about: +- Methods for validating JSON data against Hatch schemas +- Common validation tools and techniques +- Troubleshooting validation errors + +You will learn about: +- How to validate metadata files for compliance +- Using Python, command-line, and online validation tools +- Resolving common validation issues + +Schema validation ensures that your metadata files comply with the expected structure and catches errors early. + +## Validation Tools + +### Using Python's jsonschema + +The `jsonschema` package is a popular choice for schema validation in Python: + +```python +import jsonschema +import json +from examples.schema_updater import load_schema + +# Load your data +with open('my_package_metadata.json') as f: + data = json.load(f) + +# Load the schema +schema = load_schema("package") + +# Validate +try: + jsonschema.validate(data, schema) + print("Package metadata is valid") +except jsonschema.ValidationError as e: + print(f"Validation error: {e.message}") + print(f"At path: {' > '.join([str(x) for x in e.path])}") +``` + +### Using Hatch Validator + +The [Hatch-Validator](https://github.com/CrackingShells/Hatch-Validator) project provides a dedicated tool for validating Hatch metadata: + +```bash +# Install directly from GitHub +pip install git+https://github.com/CrackingShells/Hatch-Validator.git + +# Validate a package +hatch-validator validate-package path/to/hatch_metadata.json + +# Validate a registry +hatch-validator validate-registry path/to/registry.json +``` + +### Using Online Validation + +You can also use online JSON Schema validators like: + +1. [JSONSchemaValidator.net](https://www.jsonschemavalidator.net/) +2. [JSON Schema Validator by Newtonsoft](https://jsonschema.net/) + +Simply paste your schema and data to validate. + +## Common Validation Issues + +### Required Fields Missing + +All required fields must be present. For package metadata, these include: +- package_schema_version +- name +- version +- entry_point +- description +- tags +- author +- license + +### Format Errors + +Some fields require specific formats: +- `name` must match pattern `^[a-z0-9_]+$` (lowercase alphanumeric + underscore) +- `version` must match pattern `^\d+(\.\d+)*$` (semantic versioning) +- URLs must be valid URIs +- Emails must be valid email addresses + +### Dependency Constraints + +Dependency version constraints must follow the specified pattern, e.g., `>=1.0.0` + +## Advanced Validation + +### Custom Validation Logic + +For more complex validation requirements beyond JSON Schema: + +```python +def validate_dependencies(data): + """Custom validation for dependencies.""" + if 'dependencies' not in data: + return True + + # Check for circular dependencies + deps = data.get('dependencies', {}).get('hatch', []) + dep_names = [d['name'] for d in deps] + + # Add your custom checks here + + return True +``` + +## See Also + +- [Schema Access Guide](SchemaAccess.md) +- [Programmatic Usage Guide](ProgrammaticUsage.md) \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 2914c48..59ffb24 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,7 +1,23 @@ # Hatch Schemas Documentation +This article is about: +- Overview of Hatch Schemas documentation structure +- Navigation to user and developer resources +- Quick access to schema references and guides + +You will learn about: +- How to navigate the documentation +- Where to find specific information for your role +- Available resources for different use cases + Welcome to the documentation for Hatch Schemas, the JSON schema library for the CrackingShells package ecosystem. +## Quick Navigation + +- **[Table of Contents](articles/table_of_contents.md)** - Complete documentation index +- **[Getting Started](articles/users/GettingStarted.md)** - New user introduction +- **[Contributing](articles/devs/Contributing.md)** - Developer contribution guide + ## Overview The Hatch Schemas project provides standardized JSON schemas for validating metadata in the Hatch ecosystem: @@ -9,18 +25,28 @@ The Hatch Schemas project provides standardized JSON schemas for validating meta - **Package Schema**: For validating individual package metadata files - **Registry Schema**: For validating the central package registry -## Documentation Contents +## Documentation Structure -### Schema Documentation +### User Documentation +- [Getting Started](articles/users/GettingStarted.md) - Introduction and quick start +- [Schema Access](articles/users/SchemaAccess.md) - How to access and reference schemas +- [Schema Validation](articles/users/SchemaValidation.md) - Validation tools and techniques +- [Programmatic Usage](articles/users/ProgrammaticUsage.md) - Integration patterns -- [Package Schema Documentation](package/overview.md) -- [Registry Schema Documentation](registry/overview.md) +### Schema References +- [Package Schema Overview](articles/users/PackageSchema/Overview.md) +- [Package Schema Fields](articles/users/PackageSchema/Fields.md) +- [Registry Schema Overview](articles/users/RegistrySchema/Overview.md) +- [Registry Schema Fields](articles/users/RegistrySchema/Fields.md) -### Usage Guides +### Developer Documentation +- [Contributing Guidelines](articles/devs/Contributing.md) +- [Development Setup](articles/devs/DevelopmentSetup.md) +- [Repository Structure](articles/devs/RepositoryStructure.md) +- [Schema Versioning](articles/devs/SchemaVersioning.md) -- [Schema Access](usage/access.md) -- [Schema Validation](usage/validation.md) -- [Programmatic Usage](usage/programmatic.md) +### Supporting Information +- [Glossary](articles/appendices/glossary.md) - Key terms and definitions ## Current Schema Versions From 86d0333963a3bfbb0264d26be186cf6ab8c10463 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 15 Jul 2025 11:00:59 +0000 Subject: [PATCH 3/3] Remove old documentation structure after migration Co-authored-by: kozo2 <12192+kozo2@users.noreply.github.com> --- docs/package/examples.md | 120 ------------------ docs/package/fields.md | 250 ------------------------------------- docs/package/overview.md | 87 ------------- docs/registry/fields.md | 200 ----------------------------- docs/registry/overview.md | 80 ------------ docs/usage/access.md | 81 ------------ docs/usage/index.md | 39 ------ docs/usage/programmatic.md | 164 ------------------------ docs/usage/validation.md | 110 ---------------- 9 files changed, 1131 deletions(-) delete mode 100644 docs/package/examples.md delete mode 100644 docs/package/fields.md delete mode 100644 docs/package/overview.md delete mode 100644 docs/registry/fields.md delete mode 100644 docs/registry/overview.md delete mode 100644 docs/usage/access.md delete mode 100644 docs/usage/index.md delete mode 100644 docs/usage/programmatic.md delete mode 100644 docs/usage/validation.md diff --git a/docs/package/examples.md b/docs/package/examples.md deleted file mode 100644 index e8a88fe..0000000 --- a/docs/package/examples.md +++ /dev/null @@ -1,120 +0,0 @@ -# Package Schema Examples - -This document provides examples of valid package metadata files compliant with the Hatch Package Schema. - -## Basic Example - -```json -{ - "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", - "package_schema_version": "1.2.0", - "name": "simple_package", - "version": "1.0.0", - "description": "A simple example package", - "tags": ["example", "simple"], - "author": { - "name": "John Doe", - "email": "john.doe@example.com" - }, - "license": { - "name": "MIT" - }, - "entry_point": "server.py" -} -``` - -## Complete Example - -```json -{ - "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", - "package_schema_version": "1.2.0", - "name": "complete_package", - "version": "1.2.0", - "description": "A comprehensive example with all possible fields", - "tags": ["example", "complete", "reference"], - "author": { - "name": "Jane Smith", - "email": "jane.smith@example.com" - }, - "contributors": [ - { - "name": "John Doe", - "email": "john.doe@example.com" - }, - { - "name": "Alice Johnson" - } - ], - "license": { - "name": "MIT", - "uri": "https://opensource.org/licenses/MIT" - }, - "repository": "https://github.com/crackingshells/example-package", - "documentation": "https://docs.example.com/package", - "dependencies": { - "hatch": [ - { - "name": "base_package", - "version_constraint": ">=1.0.0" - }, - { - "name": "utils_package", - "version_constraint": "==1.2.0" - } - ], - "python": [ - { - "name": "numpy", - "version_constraint": ">=1.20.0" - }, - { - "name": "pandas", - "version_constraint": ">=1.3.0" - }, - { - "name": "scikit-learn", - "version_constraint": ">=1.0.0" - } - ], - "system": [ - { - "name": "libopencv-dev", - "version_constraint": ">=4.0.0", - "package_manager": "apt" - } - ], - "docker": [ - { - "name": "postgres", - "version_constraint": ">=13.0.0", - "registry": "dockerhub" - } - ] - }, - "compatibility": { - "hatchling": ">=0.1.0", - "python": ">=3.8" - }, - "entry_point": "server.py", - "tools": [ - { - "name": "data_preprocessor", - "description": "Preprocesses input data for the model" - }, - { - "name": "model_evaluator", - "description": "Evaluates model performance on test data" - } - ], - "citations": { - "origin": "Smith, J. et al. (2023). Advanced machine learning techniques. Journal of AI Research, 45(2), 123-145.", - "mcp": "Doe, J. (2023). MCP implementation of Smith's algorithms. https://doi.org/10.xxxx/xxxxx" - } -} -``` - -## See Also - -- [Package Schema Field Reference](fields.md) -- [Registry Schema Examples](../registry/examples.md) diff --git a/docs/package/fields.md b/docs/package/fields.md deleted file mode 100644 index 5e55992..0000000 --- a/docs/package/fields.md +++ /dev/null @@ -1,250 +0,0 @@ -# Package Schema Field Reference - -This document provides detailed information about each field in the Package Schema. - -## Required Fields - -### package_schema_version - -- **Type**: String -- **Pattern**: `^\d+\.\d+\.\d+$` -- **Description**: Version of the schema used for this package metadata -- **Example**: `"1.2.0"` - -### name - -- **Type**: String -- **Pattern**: `^[a-z0-9_]+$` -- **Description**: Package identifier. Must be lowercase alphanumeric with underscores. -- **Example**: `"my_package"` or `"image_classifier"` - -### version - -- **Type**: String -- **Pattern**: `^\d+(\.\d+)*$` -- **Description**: Semantic version of the package -- **Example**: `"1.0.0"` or `"2.1.3"` - -### description - -- **Type**: String -- **Description**: Human-readable description of the package -- **Example**: `"A package for image classification using deep learning"` - -### tags - -- **Type**: Array of strings -- **Description**: Keywords for package discovery -- **Example**: `["machine learning", "image", "classification"]` - -### author - -- **Type**: Object -- **Description**: Information about the package author -- **Required Properties**: - - **name** (String): The name of the author -- **Optional Properties**: - - **email** (String): The email address of the author (format: email) -- **Example**: - ```json - "author": { - "name": "John Doe", - "email": "john.doe@example.com" - } - ``` - -### license - -- **Type**: Object -- **Description**: License information for the package -- **Required Properties**: - - **name** (String): The name of the license -- **Optional Properties**: - - **uri** (String): URL to the license text (format: URI) -- **Example**: - ```json - "license": { - "name": "MIT", - "uri": "https://opensource.org/licenses/MIT" - } - ``` - -### entry_point - -- **Type**: String -- **Description**: Primary entry point for the package -- **Example**: `"server.py"` or `"main.py"` - -## Optional Fields - -### contributors - -- **Type**: Array of objects -- **Description**: List of additional contributors -- **Each Contributor**: - - **Required Properties**: - - **name** (String): The name of the contributor - - **Optional Properties**: - - **email** (String): The email address of the contributor (format: email) -- **Example**: - ```json - "contributors": [ - { - "name": "Jane Smith", - "email": "jane.smith@example.com" - }, - { - "name": "Bob Johnson" - } - ] - ``` - -### repository - -- **Type**: String -- **Format**: URI -- **Description**: URL to the source code repository -- **Example**: `"https://github.com/username/repository"` - -### documentation - -- **Type**: String -- **Format**: URI -- **Description**: URL to the package documentation -- **Example**: `"https://docs.example.com/my_package"` - -### dependencies - -- **Type**: Object -- **Description**: Dependencies required by the package -- **Properties**: - - **hatch** (Array): Hatch package dependencies - - **python** (Array): Python package dependencies - - **system** (Array): System package dependencies - - **docker** (Array): Docker image dependencies - -#### hatch dependencies - -- **Type**: Array of objects -- **Description**: Hatch packages required by this package -- **Each Dependency**: - - **name** (String): Name of the Hatch package - - **version_constraint** (String): Version constraint (e.g., ">=1.0.0") -- **Example**: - ```json - "hatch": [ - { - "name": "base_package", - "version_constraint": ">=1.0.0" - } - ] - ``` - -#### python dependencies - -- **Type**: Array of objects -- **Description**: Python packages required by this package -- **Each Dependency**: - - **name** (String): Name of the Python package - - **version_constraint** (String): Version constraint (e.g., ">=1.0.0") - - **package_manager** (String, default: "pip"): Package manager to use -- **Example**: - ```json - "python": [ - { - "name": "numpy", - "version_constraint": ">=1.20.0" - }, - { - "name": "pandas", - "version_constraint": ">=1.3.0" - } - ] - ``` - -#### system dependencies - -- **Type**: Array of objects -- **Description**: System packages required by this package -- **Each Dependency**: - - **name** (String): Name of the system package - - **version_constraint** (String): Version constraint - - **package_manager** (String, default: "apt"): Package manager to use -- **Example**: - ```json - "system": [ - { - "name": "libopencv-dev", - "version_constraint": ">=4.0.0" - } - ] - ``` - -#### docker dependencies - -- **Type**: Array of objects -- **Description**: Docker images required by this package -- **Each Dependency**: - - **name** (String): Name of the Docker image - - **version_constraint** (String): Version constraint - - **registry** (String, default: "dockerhub"): Registry to pull from -- **Example**: - ```json - "docker": [ - { - "name": "postgres", - "version_constraint": ">=13.0.0" - } - ] - ``` - -### compatibility - -- **Type**: Object -- **Description**: Compatibility requirements -- **Properties**: - - **hatchling** (String): Version constraint for Hatchling - - **python** (String): Version constraint for Python -- **Example**: - ```json - "compatibility": { - "hatchling": ">=0.1.0", - "python": ">=3.8" - } - ``` - -### tools - -- **Type**: Array of objects -- **Description**: Additional tools provided by the package -- **Each Tool**: - - **name** (String): Name of the tool - - **description** (String): Description of the tool -- **Example**: - ```json - "tools": [ - { - "name": "data_preprocessor", - "description": "Preprocesses input data for the model" - }, - { - "name": "model_evaluator", - "description": "Evaluates model performance on test data" - } - ] - ``` - -### citations - -- **Type**: Object -- **Description**: Citation information for the package. -- **Properties**: - - **origin** (String): Citation for the original work. Free format. - - **mcp** (String): Citation for the MCP implementation. Free format. -- **Example**: - ```json - "citations": { - "origin": "Doe, J. et al. (2023). Novel image classification approach. Journal of AI Research, 45(2), 123-145.", - "mcp": "Smith, J. (2023). MCP implementation of Doe's image classifier. https://doi.org/10.xxxx/xxxxx" - } - ``` diff --git a/docs/package/overview.md b/docs/package/overview.md deleted file mode 100644 index d57148d..0000000 --- a/docs/package/overview.md +++ /dev/null @@ -1,87 +0,0 @@ -# Package Schema Documentation - -## Overview - -The Package Schema (`hatch_pkg_metadata_schema.json`) defines the structure for individual package metadata files in the Hatch ecosystem. This schema ensures consistency and validity of package metadata across the ecosystem. - -## Current Version - -The current version of the Package Schema is **v1.2.0**. - -## Schema Structure - -The Package Schema includes the following major sections: - -- **Basic Package Information**: Schema version, name, version, description, and tags -- **Author Information**: Author details and optional contributors list -- **License Information**: License name and optional URI -- **Package Links**: Repository and documentation URLs -- **Dependencies**: Hatch, Python, system, and Docker dependencies -- **Compatibility Requirements**: Hatchling and Python version constraints -- **Entry Points and Tools**: Primary entry point and additional tools - -For detailed field-by-field documentation including types, formats, and examples, see the [Package Schema Field Reference](fields.md). - -### Compatibility - -- **compatibility**: Object defining compatibility requirements (optional) - - **hatchling**: Version constraint for Hatchling compatibility - - **python**: Version constraint for Python compatibility - -### Entry Points and Tools - -- **entry_point**: Primary entry point for the package (required) -- **tools**: List of additional tools provided by the package (optional) - -### Citations - -- **citations**: Citation information for the package (optional) - - **origin**: Citation for the original work - - **mcp**: Citation for the MCP implementation - -## Example - -```json -{ - "package_schema_version": "1.2.0", - "name": "example_package", - "version": "1.0.0", - "description": "An example Hatch package", - "tags": ["example", "demo"], - "author": { - "name": "John Doe", - "email": "john.doe@example.com" - }, - "license": { - "name": "MIT", - "uri": "https://opensource.org/licenses/MIT" - }, - "entry_point": "server.py", - "dependencies": { - "hatch": [ - { - "name": "base_package", - "version_constraint": ">=1.0.0" - } - ], - "python": [ - { - "name": "numpy", - "version_constraint": ">=1.20.0" - }, - { - "name": "pandas", - "version_constraint": ">=1.3.0" - } - ] - }, - "compatibility": { - "python": ">=3.8" - } -} -``` - -## See Also - -- [Registry Schema](../registry/overview.md) -- [Schema Validation Guide](../usage/validation.md) diff --git a/docs/registry/fields.md b/docs/registry/fields.md deleted file mode 100644 index e776996..0000000 --- a/docs/registry/fields.md +++ /dev/null @@ -1,200 +0,0 @@ -# Registry Schema Field Reference - -This document provides detailed information about each field in the Registry Schema. - -## Required Fields - -### registry_schema_version - -- **Type**: String -- **Pattern**: `^\d+(\.\d+)*$` -- **Description**: Version of this registry schema -- **Example**: `"1.2.0"` - -### last_updated - -- **Type**: String -- **Format**: date-time -- **Description**: Timestamp of when the registry was last updated -- **Example**: `"2024-06-01T12:00:00Z"` - -### repositories - -- **Type**: Array of objects -- **Description**: List of repositories containing packages -- **Each Repository**: See [Repository Object](#repository-object) - -### stats - -- **Type**: Object -- **Description**: Registry-wide statistics -- **Properties**: - - **total_packages** (Integer): Total number of unique packages - - **total_versions** (Integer): Total number of package versions -- **Example**: - ```json - "stats": { - "total_packages": 10, - "total_versions": 25 - } - ``` - -## Object Definitions - -### Repository Object - -- **Type**: Object -- **Description**: Repository containing packages -- **Required Properties**: - - **name** (String): Repository name (e.g., 'Hatch-Dev') - - **url** (String, format: URI): URL to the repository - - **packages** (Array): List of packages in this repository (see [Package Object](#package-object)) - - **last_indexed** (String, format: date-time): When this repository was last indexed -- **Example**: - ```json - { - "name": "Hatch-Dev", - "url": "https://github.com/crackingshells/Hatch-Dev", - "last_indexed": "2024-06-01T12:00:00Z", - "packages": [...] - } - ``` - -### Package Object - -- **Type**: Object -- **Description**: Package within a repository -- **Required Properties**: - - **name** (String): Package identifier (pattern: `^[a-z0-9_]+$`) - - **description** (String): Human-readable description of the package - - **tags** (Array of strings): Keywords for discovery - - **versions** (Array of objects): Available versions of this package (see [Version Object](#version-object)) - - **latest_version** (String): Latest version of the package (pattern: `^\d+(\.\d+)*$`) -- **Example**: - ```json - { - "name": "example_package", - "description": "An example Hatch package", - "tags": ["example", "demo"], - "latest_version": "1.2.0", - "versions": [...] - } - ``` - -### Version Object - -- **Type**: Object -- **Description**: Version of a package -- **Required Properties**: - - **version** (String): Semantic version of the package (pattern: `^\d+(\.\d+)*$`) - - **author** (Object): Author of the submission of this version - - **GitHubID** (String): GitHub username of the author - - **email** (String, format: email): Email of the author - - **release_uri** (String, format: URI): URI to the release page for this version - - **added_date** (String, format: date-time): When this version was added to the registry -- **Optional Properties**: - - **base_version** (String): Version this differential is based on (null for first version) - - **dependency_changes** (Object): Changes to dependencies since base_version - - **compatibility_changes** (Object): Changed compatibility constraints since base_version - - **verification** (Object): Verification status and metadata -- **Example**: - ```json - { - "version": "1.2.0", - "author": { - "GitHubID": "johndoe", - "email": "john.doe@example.com" - }, - "release_uri": "https://github.com/crackingshells/Hatch-Dev/releases/tag/example_package-v1.2.0", - "base_version": "1.1.0", - "added_date": "2024-05-15T10:30:00Z", - "verification": { - "status": "verified", - "timestamp": "2024-05-16T14:20:00Z", - "verifier": { - "GitHubID": "packagereviewer", - "email": "reviewer@example.com" - }, - "notes": "All tests passed" - } - } - ``` - -### Dependency Changes Object - -- **Type**: Object -- **Description**: Changes to dependencies since base_version -- **Properties**: - - **hatch** (Object): Changes to Hatch dependencies - - **python** (Object): Changes to Python dependencies - - **system** (Object): Changes to system dependencies - - **docker** (Object): Changes to Docker dependencies -- **Each Dependency Type**: - - **added** (Array): Dependencies added since base_version - - **removed** (Array): Names of dependencies removed since base_version - - **modified** (Array): Dependencies with modified version constraints -- **Example**: - ```json - "dependency_changes": { - "hatch": { - "added": [ - { - "name": "new_dependency", - "version_constraint": ">=1.0.0" - } - ], - "removed": ["old_dependency"], - "modified": [ - { - "name": "existing_dependency", - "old_version_constraint": ">=1.0.0", - "new_version_constraint": ">=1.2.0" - } - ] - } - } - ``` - -### Compatibility Changes Object - -- **Type**: Object -- **Description**: Changed compatibility constraints since base_version -- **Properties**: - - **hatchling** (String): Changed version constraint for Hatchling compatibility - - **python** (String): Changed version constraint for Python compatibility -- **Example**: - ```json - "compatibility_changes": { - "hatchling": ">=0.2.0", - "python": ">=3.9" - } - ``` - -### Verification Object - -- **Type**: Object -- **Description**: Verification metadata for a package version -- **Required Properties**: - - **status** (String, enum): Verification status - - Values: "unverified", "validated", "reviewed", "verified", "deprecated" - - Default: "unverified" -- **Optional Properties**: - - **timestamp** (String, format: date-time): When this verification status was last updated - - **verifier** (Object): Entity that performed the verification - - **GitHubID** (String): GitHub username of the verifier - - **email** (String, format: email): Email of the verifier - - **name** (String): Name of the verifier - - **notes** (String): Additional information about verification status -- **Example**: - ```json - "verification": { - "status": "verified", - "timestamp": "2024-05-16T14:20:00Z", - "verifier": { - "GitHubID": "packagereviewer", - "email": "reviewer@example.com", - "name": "Package Reviewer" - }, - "notes": "All tests passed, code review completed" - } - ``` diff --git a/docs/registry/overview.md b/docs/registry/overview.md deleted file mode 100644 index 7b010b1..0000000 --- a/docs/registry/overview.md +++ /dev/null @@ -1,80 +0,0 @@ -# Registry Schema Documentation - -## Overview - -The Registry Schema (`hatch_all_pkg_metadata_schema.json`) defines the structure for the central package registry in the Hatch ecosystem. This schema is used to maintain a comprehensive, searchable catalog of all available packages across CrackingShells repositories. - -> [!Note] -> The registry is being maintained internally. It is modified by specific functions, and updated during package submission process. In principle, there **MUST NOT** be any manual modification of the registry to add a package. Although, highly discouraged, there **MAY** be manual modification by core members of the organization in case of errors. - -## Current Version - -The current version of the Registry Schema is **v1.2.0**. - -## Schema Structure - -The Registry Schema includes the following major sections: - -- **Registry Metadata**: Schema version, last updated timestamp, and global statistics -- **Repositories**: List of repositories with their metadata and packages -- **Packages**: Package information including name, description, tags, and versions -- **Version Information**: Details about each version of a package, including author, release location, and verification status - -For detailed field-by-field documentation including types, formats, and examples, see the [Registry Schema Field Reference](fields.md). -- **verification**: Verification status and metadata -- **dependency_changes**: Changes to dependencies since the base version -- **compatibility_changes**: Changes to compatibility requirements - -## Example - -```json -{ - "registry_schema_version": "1.2.0", - "last_updated": "2024-06-01T12:00:00Z", - "repositories": [ - { - "name": "Hatch-Dev", - "url": "https://github.com/crackingshells/Hatch-Dev", - "last_indexed": "2024-06-01T12:00:00Z", - "packages": [ - { - "name": "example_package", - "description": "An example Hatch package", - "tags": ["example", "demo"], - "latest_version": "1.2.0", - "versions": [ - { - "version": "1.2.0", - "author": { - "GitHubID": "johndoe", - "email": "john.doe@example.com" - }, - "release_uri": "https://github.com/crackingshells/Hatch-Dev/releases/tag/example_package-v1.2.0", - "base_version": "1.1.0", - "added_date": "2024-05-15T10:30:00Z", - "verification": { - "status": "verified", - "timestamp": "2024-05-16T14:20:00Z", - "verifier": { - "GitHubID": "packagereviewer", - "email": "reviewer@example.com" - }, - "notes": "All tests passed" - } - } - ] - } - ] - } - ], - "stats": { - "total_packages": 1, - "total_versions": 1 - } -} -``` - -## See Also - -- [Package Schema](../package/overview.md) -- [Schema Validation Guide](../usage/validation.md) diff --git a/docs/usage/access.md b/docs/usage/access.md deleted file mode 100644 index 6b02afd..0000000 --- a/docs/usage/access.md +++ /dev/null @@ -1,81 +0,0 @@ -# Schema Access Guide - -This guide explains how to access and use the Hatch Schemas in your projects. - -## Distribution Methods - -Hatch Schemas are distributed through multiple channels: - -1. **GitHub Repository** - Direct access to schema files via raw.githubusercontent.com -2. **GitHub Releases** - Versioned releases with metadata and direct downloads -3. **GitHub API** - Programmatic discovery of latest versions and releases - -## Accessing Schemas - -### Manual Discovery - -You can visit the release page: https://github.com/CrackingShells/Hatch-Schemas/releases - -### Release Downloads - -Download schema files from specific GitHub releases: - -```bash -# Download schema files from specific releases -https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-package-v1.2.0/hatch_pkg_metadata_schema.json -https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-registry-v1.2.0/hatch_all_pkg_metadata_schema.json -``` - -### Direct Schema Access - -Access schema files directly from the GitHub repository: - -```bash -# Direct access to schema files (always current from main branch) -https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json -https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/registry/v1.2.0/hatch_all_pkg_metadata_schema.json -``` - - -## Using Schemas in Your Project - -### Referencing in JSON Files - -You can reference these schemas in your JSON files using the `$schema` property: - -```json -{ - "$schema": "https://raw.githubusercontent.com/crackingshells/Hatch-Schemas/main/package/v1.2.0/hatch_pkg_metadata_schema.json", - "name": "my_package", - "version": "1.0.0", - "description": "My awesome package", - "tags": ["example"], - "author": {"name": "John Doe"}, - "license": {"name": "MIT"}, - "entry_point": "server.py" -} -``` - -This reference: -1. Provides editors with schema information for autocompletion and validation -2. Documents which schema version the file adheres to -3. Creates a clear contract for validation tools - -### Local Cache Management - -Consider implementing local caching of schemas. The `examples/schema_updater.py` utility provides this functionality: - -```python -from examples.schema_updater import configure_cache, update_schemas - -# Configure local cache location (optional) -configure_cache(cache_dir="./schemas_cache") - -# Update local schema cache -update_schemas() -``` - -## See Also - -- [Schema Validation](validation.md) -- [Programmatic Usage](programmatic.md) diff --git a/docs/usage/index.md b/docs/usage/index.md deleted file mode 100644 index c4a0f91..0000000 --- a/docs/usage/index.md +++ /dev/null @@ -1,39 +0,0 @@ -# Usage Guides - -This section provides comprehensive guides on how to use the Hatch schemas in your projects. - -## Available Guides - -### [Schema Access](access.md) - -Learn how to access and retrieve Hatch schemas using various methods: -- Direct Raw Access via GitHub -- GitHub Releases Download -- GitHub API Discovery -- Local Caching - -### [Schema Validation](validation.md) - -Understand how to validate your JSON data against Hatch schemas: -- Using Python's jsonschema library -- Using Hatch Validator -- Online validation tools -- Common validation issues and solutions - -### [Programmatic Usage](programmatic.md) - -Discover advanced techniques for integrating Hatch schemas into your applications: -- Simple schema loading with the provided utilities -- Manual schema retrieval -- Schema caching for offline use - -## Getting Started - -If you're new to using Hatch schemas, we recommend starting with the [Schema Access Guide](access.md), followed by the [Schema Validation Guide](validation.md). - -For developers integrating Hatch schemas into applications, the [Programmatic Usage Guide](programmatic.md) provides detailed examples and best practices. - -## Additional Resources - -- [Package Schema Reference](../package/fields.md) -- [Registry Schema Reference](../registry/fields.md) diff --git a/docs/usage/programmatic.md b/docs/usage/programmatic.md deleted file mode 100644 index 8493128..0000000 --- a/docs/usage/programmatic.md +++ /dev/null @@ -1,164 +0,0 @@ -# Programmatic Usage Guide - -This guide explains how to use Hatch schemas programmatically in your applications. - -## Simple Schema Loading - -For basic usage, you can use the provided example utility: - -```python -from examples.schema_updater import load_schema, configure_logger -import logging - -# Optional: Configure logging -configure_logger(level=logging.INFO) - -# Load the latest schemas (automatically downloads if needed) -package_schema = load_schema("package") # Latest package schema -registry_schema = load_schema("registry") # Latest registry schema - -# Load specific version -package_v1_2_0 = load_schema("package", "v1.2.0") # Specific version -``` - -## Manual Schema Retrieval - -For more control over schema retrieval, you can implement your own functions: - -```python -import requests -import json - -def get_latest_schema_info(): - """Get latest schema version information from GitHub API.""" - api_url = "https://api.github.com/repos/crackingshells/Hatch-Schemas/releases" - response = requests.get(api_url) - releases = response.json() - - # Find latest package and registry schema releases - latest_schemas = {} - for release in releases: - tag = release['tag_name'] - if tag.startswith('schemas-package-'): - if 'package' not in latest_schemas: - version = tag.replace('schemas-package-', '') - latest_schemas['package'] = { - 'version': version, - 'url': f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-package-{version}/hatch_pkg_metadata_schema.json" - } - elif tag.startswith('schemas-registry-'): - if 'registry' not in latest_schemas: - version = tag.replace('schemas-registry-', '') - latest_schemas['registry'] = { - 'version': version, - 'url': f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-registry-{version}/hatch_all_pkg_metadata_schema.json" - } - - return latest_schemas - -def load_schema(schema_type, version=None): - """Load a specific schema or the latest version.""" - if version is None: - # Get latest version - latest_info = get_latest_schema_info() - schema_url = latest_info[schema_type]['url'] - else: - # Use specific version from release - if schema_type == 'package': - schema_url = f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-package-{version}/hatch_pkg_metadata_schema.json" - else: - schema_url = f"https://github.com/crackingshells/Hatch-Schemas/releases/download/schemas-registry-{version}/hatch_all_pkg_metadata_schema.json" - - response = requests.get(schema_url) - return response.json() -``` - -## Basic Validation - -```python -import jsonschema - -# Load schema -schema = load_schema("package") - -# Your package data -package_data = { - "package_schema_version": "1.2.0", - "name": "my_package", - "version": "1.0.0", - "description": "My awesome package", - "tags": ["example"], - "author": {"name": "John Doe"}, - "license": {"name": "MIT"}, - "entry_point": "server.py" -} - -# Validate -try: - jsonschema.validate(package_data, schema) - print("Package metadata is valid") -except jsonschema.ValidationError as e: - print(f"Validation error: {e.message}") -``` - -## Advanced Use Cases - -An, implementation example is also [available](../../examples/schema_updater.py) - -### Schema Caching - -For performance and offline availability, implement a schema caching mechanism: - -```python -import os -import json -from pathlib import Path - -CACHE_DIR = Path.home() / ".cache" / "hatch-schemas" - -def cache_schema(schema_type, version, schema_data): - """Cache a schema locally.""" - # Ensure cache directory exists - os.makedirs(CACHE_DIR, exist_ok=True) - - # Write schema to cache - cache_file = CACHE_DIR / f"{schema_type}-{version}.json" - with open(cache_file, 'w') as f: - json.dump(schema_data, f) - - return cache_file - -def get_cached_schema(schema_type, version): - """Retrieve a schema from the local cache.""" - cache_file = CACHE_DIR / f"{schema_type}-{version}.json" - - if cache_file.exists(): - with open(cache_file, 'r') as f: - return json.load(f) - - return None - -def load_schema_with_cache(schema_type, version=None): - """Load schema with caching support.""" - if version is None: - # Get latest version info - latest_info = get_latest_schema_info() - version = latest_info[schema_type]['version'] - - # Try to load from cache - cached_schema = get_cached_schema(schema_type, version) - if cached_schema: - return cached_schema - - # If not in cache, download and cache - schema_data = load_schema(schema_type, version) - if schema_data: - cache_schema(schema_type, version, schema_data) - - return schema_data -``` - -## See Also - -- [Schema Access Guide](access.md) -- [Schema Validation Guide](validation.md) diff --git a/docs/usage/validation.md b/docs/usage/validation.md deleted file mode 100644 index 49a020f..0000000 --- a/docs/usage/validation.md +++ /dev/null @@ -1,110 +0,0 @@ -# Schema Validation Guide - -This document explains how to validate your JSON data against Hatch schemas. - -## Validation Overview - -Schema validation ensures that your metadata files comply with the expected structure. This helps catch errors early and ensures compatibility across the ecosystem. - -## Validation Tools - -### Using Python's jsonschema - -The `jsonschema` package is a popular choice for schema validation in Python: - -```python -import jsonschema -import json -from examples.schema_updater import load_schema - -# Load your data -with open('my_package_metadata.json') as f: - data = json.load(f) - -# Load the schema -schema = load_schema("package") - -# Validate -try: - jsonschema.validate(data, schema) - print("Package metadata is valid") -except jsonschema.ValidationError as e: - print(f"Validation error: {e.message}") - print(f"At path: {' > '.join([str(x) for x in e.path])}") -``` - -### Using Hatch Validator - -The [Hatch-Validator](https://github.com/CrackingShells/Hatch-Validator) project provides a dedicated tool for validating Hatch metadata: - -```bash -# Install directly from GitHub -pip install git+https://github.com/CrackingShells/Hatch-Validator.git - -# Validate a package -hatch-validator validate-package path/to/hatch_metadata.json - -# Validate a registry -hatch-validator validate-registry path/to/registry.json -``` - -### Using Online Validation - -You can also use online JSON Schema validators like: - -1. [JSONSchemaValidator.net](https://www.jsonschemavalidator.net/) -2. [JSON Schema Validator by Newtonsoft](https://jsonschema.net/) - -Simply paste your schema and data to validate. - -## Common Validation Issues - -### Required Fields Missing - -All required fields must be present. For package metadata, these include: -- package_schema_version -- name -- version -- entry_point -- description -- tags -- author -- license - -### Format Errors - -Some fields require specific formats: -- `name` must match pattern `^[a-z0-9_]+$` (lowercase alphanumeric + underscore) -- `version` must match pattern `^\d+(\.\d+)*$` (semantic versioning) -- URLs must be valid URIs -- Emails must be valid email addresses - -### Dependency Constraints - -Dependency version constraints must follow the specified pattern, e.g., `>=1.0.0` - -## Advanced Validation - -### Custom Validation Logic - -For more complex validation requirements beyond JSON Schema: - -```python -def validate_dependencies(data): - """Custom validation for dependencies.""" - if 'dependencies' not in data: - return True - - # Check for circular dependencies - deps = data.get('dependencies', {}).get('hatch', []) - dep_names = [d['name'] for d in deps] - - # Add your custom checks here - - return True -``` - -## See Also - -- [Schema Access Guide](access.md) -- [Programmatic Usage Guide](programmatic.md)