Skip to content

Add support for nested data generation#13

Merged
ashish-bagri merged 3 commits intomainfrom
nested-support
Jul 1, 2025
Merged

Add support for nested data generation#13
ashish-bagri merged 3 commits intomainfrom
nested-support

Conversation

@ashish-bagri
Copy link
Member

No description provided.

@vercel
Copy link

vercel bot commented Jun 25, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
glassgen ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 25, 2025 0:56am

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds support for nested data generation by extending the schema parsing and record generation capabilities and by updating the CSV sink to flatten nested dictionaries. Key changes include:

  • Added tests for flat, nested, deeply nested, and mixed schemas.
  • Updated the CSV sink to flatten nested dictionaries when writing CSV data.
  • Enhanced the schema parser to support nested schema fields via a new NestedSchemaField.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
tests/test_sinks.py Updated CSV header assertions to support order-agnostic testing.
tests/test_schema.py Added comprehensive tests for nested and mixed schemas.
glassgen/sinks/csv_sink.py Introduced helper functions for flattening nested dictionaries.
glassgen/schema/schema.py Extended schema parsing and record generation to support nested data.

@github-actions
Copy link
Contributor

Coverage

Test Coverage Report
FileStmtsMissCoverMissing
__init__.py60100% 
config.py490100% 
interface.py43588%32–36
generator
   __init__.py50100% 
   batch_controller.py39392%16, 21–22
   duplication.py51296%25, 33
   generator.py45197%47
   generators.py63493%57, 62, 120, 125
schema
   __init__.py40100% 
   base.py70100% 
   schema.py65296%50, 86
   user_schema.py16193%30
sinks
   __init__.py130100% 
   base.py90100% 
   csv_sink.py47197%41
   kafka_sink.py281353%20–23, 27–28, 33, 36–38, 43–44, 47
   webhook_sink.py260100% 
   yield_sink.py100100% 
TOTAL5263293% 

Copy link
Member

@PabloPardoGarcia PabloPardoGarcia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@ashish-bagri ashish-bagri merged commit 712c9dc into main Jul 1, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants