Added data generator code

Author: vandanamalik6794

data_generator/README.md

@@ -0,0 +1,390 @@
+# Data Generator Project - Summary
+
+## What You Have
+
+A complete, production-ready **Multi-Table Data Generator** with:
+- ✅ Python module (.py file)
+- ✅ Interactive Jupyter notebook (.ipynb file)
+- ✅ Configuration files (.yaml files)
+- ✅ Complete documentation (README.md)
+- ✅ Setup guide (PROJECT_SETUP.md)
+
+---
+
+##  All Files Created
+
+### Core Files (Required)
+1. **`data_generator.py`** - Main Python module
+   - Contains `MultiTableDataGenerator` class
+   - All generation logic
+
+2. **`requirements.txt`** - Dependencies
+   - pyyaml (required)
+   - pandas (recommended)
+
+### Tutorial & Examples
+3. **`DataGenerator_Tutorial.ipynb`** - Jupyter Notebook
+   - Interactive tutorial
+   - Step-by-step examples
+   - Data analysis examples
+   - Ready to run
+
+4. **`config_simple.yaml`** - Simple Example
+   - 2 tables (users + orders)
+   - Foreign key relationship
+   - Easy to understand
+
+5. **`config_ecommerce.yaml`** - E-commerce Example
+   - 4 tables (customers, products, orders, reviews)
+   - Multiple foreign keys
+   - Realistic scenario
+
+6. **`config_all_types.yaml`** - Complete Reference
+   - Shows ALL column types
+   - Reference documentation
+   - Copy-paste templates
+
+### Documentation
+7. **`README.md`** - Complete Documentation
+   - Features overview
+   - Installation guide
+   - API reference
+   - Examples
+   - Troubleshooting
+
+8. **`PROJECT_SETUP.md`** - Setup Guide
+   - Step-by-step setup
+   - Directory structure
+   - Testing instructions
+   - Troubleshooting
+
+9. **`COMPLETE_PROJECT_SUMMARY.md`** - This File
+   - Quick overview
+   - Usage instructions
+   - File descriptions
+
+---
+
+## Quick Start (3 Steps)
+
+### Step 1: Setup
+```bash
+# Create directory
+mkdir data-generator
+cd data-generator
+
+# Save all 9 files in this directory
+
+# Create requirements.txt and add following dependencies in it. Add the dependencies in the cluster
+pyyaml
+pandas
+```
+
+### Step 2: Test
+```python
+# Test basic generation
+from data_generator import MultiTableDataGenerator; \
+
+MultiTableDataGenerator(seed=42).generate_from_config('config_simple.yaml')
+```
+
+### Step 3: Explore
+
+Open DataGenerator_Tutorial.ipynb notebook in AIDP. Run the commands. Kindly change the paths as per your folder.
+
+
+---
+
+## 📋 File Purposes
+
+| File | What It Does | When to Use |
+|------|--------------|-------------|
+| `data_generator.py` | Core generator class | Import in your code |
+| `DataGenerator_Tutorial.ipynb` | Interactive tutorial | Learning & examples |
+| `config_simple.yaml` | Basic 2-table example | Quick testing |
+| `config_ecommerce.yaml` | Real-world scenario | Complex relationships |
+| `config_all_types.yaml` | All features demo | Reference guide |
+| `README.md` | Full documentation | When stuck |
+| `requirements.txt` | Dependencies | Installation |
+
+---
+
+## 💡 Usage Examples
+
+### Example 1: Python Script
+```python
+from data_generator import MultiTableDataGenerator
+
+# Simple usage
+generator = MultiTableDataGenerator(seed=42)
+results = generator.generate_from_config('config_simple.yaml')
+
+# View sample
+generator.print_sample('users', n=5)
+
+# Get as DataFrame
+df = generator.get_dataframe('users')
+```
+
+### Example 3: Custom Configuration
+```python
+config = {
+    'table_name': 'my_data',
+    'rows_count': 100,
+    'output_format': 'both',
+    'columns': [
+        {'name': 'id', 'type': 'integer', 'range': [1, 1000], 'unique': True},
+        {'name': 'name', 'type': 'string', 'length': 8},
+        {'name': 'email', 'type': 'email', 'unique': True}
+    ]
+}
+
+generator = MultiTableDataGenerator(seed=42)
+generator.generate_from_config(config)
+```
+
+### Example 4: From Config File
+```python
+# Use existing config
+generator = MultiTableDataGenerator(seed=42)
+results = generator.generate_from_config('config_ecommerce.yaml')
+
+# Access tables
+df_customers = generator.get_dataframe('customers')
+df_orders = generator.get_dataframe('orders')
+```
+
+---
+
+##  Learning Path
+
+1. **Start**: Read `README.md`
+2. **Learn**: Open `DataGenerator_Tutorial.ipynb`
+3. **Practice**: Modify `config_simple.yaml`
+4. **Build**: Create your own config
+
+---
+
+##  Key Features
+
+### Multi-Table Support
+```yaml
+tables:
+  - table_name: users
+    rows_count: 10
+  - table_name: orders
+    rows_count: 50
+```
+
+### Foreign Keys
+```yaml
+- name: user_id
+  type: reference
+  ref_table: users
+  ref_column: user_id
+```
+
+### 11+ Column Types
+- integer, float, string
+- choice (with weights)
+- boolean
+- date, datetime
+- email, phone, uuid
+- reference (foreign key)
+
+### Automatic Features
+-  Dependency resolution
+-  Unique constraints
+-  Progress indicators
+-  CSV/JSON export
+-  Pandas integration
+
+---
+
+## 📊 Output Structure
+
+After running, you'll get:
+
+```
+Data_generator/
+├── [All your source files]
+│
+└── output/  (or ecommerce_data/, etc.)
+    ├── users.csv
+    ├── users.json
+    ├── orders.csv
+    └── orders.json
+```
+
+---
+
+## 🎯 Common Use Cases
+
+### 1. Testing Databases
+```python
+# Generate test data
+gen = MultiTableDataGenerator(seed=42)
+gen.generate_from_config('config_ecommerce.yaml')
+# Import CSVs into your database
+```
+
+### 2. Prototyping Applications
+```python
+# Quick demo data
+gen = MultiTableDataGenerator()
+gen.generate_from_config('config_simple.yaml')
+# Use in your app prototype
+```
+
+### 3. Data Science Practice
+```python
+# Generate training data
+gen = MultiTableDataGenerator(seed=100)
+results = gen.generate_from_config('my_ml_config.yaml')
+df = gen.get_dataframe('features')
+# Use for ML experiments
+```
+
+### 4. API Testing
+```python
+# Generate test payloads
+gen = MultiTableDataGenerator()
+results = gen.generate_from_config('api_test_config.yaml')
+# Use in API tests
+```
+
+---
+
+##  Configuration Cheat Sheet
+
+### Basic Structure
+```yaml
+table_name: my_table
+rows_count: 100
+output_format: both
+columns: [...]
+```
+
+### Multi-Table Structure
+```yaml
+output_path: ./output
+output_format: both
+tables:
+  - table_name: table1
+    rows_count: 10
+    columns: [...]
+  - table_name: table2
+    rows_count: 50
+    columns: [...]
+```
+
+### Column Template
+```yaml
+- name: column_name
+  type: column_type
+  # type-specific options...
+  unique: false  # optional
+```
+
+---
+
+##  Commands Reference
+
+```python
+
+# Python interactive
+
+>>> from data_generator import MultiTableDataGenerator
+>>> gen = MultiTableDataGenerator(seed=42)
+>>> gen.generate_from_config('config_simple.yaml')
+
+# Check output
+! ls -la output/
+```
+
+---
+
+## 📈 Scaling Tips
+
+| Dataset Size | Recommendation |
+|--------------|----------------|
+| < 1K rows | Any format, instant |
+| 1K - 100K rows | Prefer CSV, seconds |
+| 100K - 1M rows | CSV only, minutes |
+| 1M+ rows | Batch generation |
+
+---
+
+## Quick Troubleshooting
+
+| Problem | Solution |
+|---------|----------|
+| Module not found | `pip install pyyaml pandas` |
+| Config not found | Check file path, use `ls` |
+| Can't generate unique | Increase range |
+| Referenced table error | Parent table must be first |
+
+
+---
+
+##  Success Checklist
+
+- [ ] All 9 files saved
+- [ ] Dependencies installed (`pip install pyyaml pandas`)
+- [ ] Can import: `from data_generator import MultiTableDataGenerator`
+- [ ] `config_simple.yaml` generates successfully
+- [ ] Output files created in `./output/`
+- [ ] Jupyter notebook opens and runs
+- [ ] Can create custom configs
+
+**All checked? You're ready to generate data! **
+
+
+##  Notes
+
+- **Reproducibility**: Use seeds (`seed=42`) for consistent results
+- **Performance**: CSV is faster than JSON for large datasets
+- **Testing**: Start with small `rows_count` (10-20) for testing
+- **Safety**: Generated data is saved automatically
+- **Pandas**: Use `get_dataframe()` for easy data analysis
+
+---
+
+##  What You Can Build
+
+With this generator, you can create:
+- ✅ E-commerce databases
+- ✅ Social media datasets
+- ✅ School management systems
+- ✅ Hospital records
+- ✅ Banking transactions
+- ✅ IoT sensor data
+- ✅ Any relational database!
+
+---
+
+## 🚀 Get Started Now
+
+```bash
+# 1. Install
+pip install pyyaml pandas
+
+# 2. Test
+python -c "from data_generator import MultiTableDataGenerator; \
+           MultiTableDataGenerator(seed=42).generate_from_config('config_simple.yaml')"
+
+```
+
+---
+
+**You have everything you need to generate professional-quality test data! **
+
+**Questions?** Check `README.md` for complete documentation.
+
+**Want examples?** Open `DataGenerator_Tutorial.ipynb` for interactive tutorials.
+
+**Ready to build?** Start with `config_simple.yaml` and customize it!
+
+---
+
+**Happy Data Generating! **