Skip to content

radiantlogicinc/iddm_dataconnector_codegen

Repository files navigation

IDDM Data Connector CodeGen with MCP Server

An AI-powered application for rapid development of IDDM data connectors, enhanced with a comprehensive Model Context Protocol (MCP) server for streamlined development workflows.

πŸš€ Features

  • MCP Server Integration: Full Model Context Protocol server for seamless IDE integration
  • OpenAPI Processing: Automated OpenAPI specification parsing and code generation
  • Java Client Generation: Automatic generation of Java client code from OpenAPI specs
  • Data Connector Templates: Pre-built templates and examples for common connector patterns
  • Few-Shot Learning: AI-powered code generation using example-based learning
  • Maven Integration: Complete Maven project structure with automated builds
  • FastWorkflow Interface: Natural language interface for generating data connectors from OpenAPI specifications

πŸ“ Project Structure

iddm_dataconnector_codegen/
β”œβ”€β”€ mcp_server/                   # MCP Server implementation
β”‚   β”œβ”€β”€ mcp_server.py            # Main MCP server with full tooling
β”‚   └── requirements.txt         # Python dependencies for the server
β”œβ”€β”€ examples/                     # Few-shot learning examples
β”‚   β”œβ”€β”€ HarryPotterDataConnector.java  # Example Java connector class
β”‚   β”œβ”€β”€ harrypotterbooksconnector.json # Example JSON configuration
β”‚   └── HarryPotterDataConnectorTest.java # Example unit tests
β”œβ”€β”€ yamlfiles/                   # OpenAPI specifications
β”‚   β”œβ”€β”€ harry_potter_openapi.yaml
β”‚   └── idp-minimal.yaml
β”œβ”€β”€ sdkfiles/                    # SDK reference documentation
β”‚   β”œβ”€β”€ radiantlogicinc-iddm-sdk.txt
β”‚   β”œβ”€β”€ minimal-radiantlogic-iddm-sdk.txt
β”‚   └── minimal_user_guide.txt
β”œβ”€β”€ java_client/                 # Generated Java client code
β”‚   β”œβ”€β”€ harrypotterapi/
β”‚   └── myaccountmanagement/
β”œβ”€β”€ data_connector_fastworkflow/ # FastWorkflow application for data connector generation
β”‚   β”œβ”€β”€ application/            # Application layer with core business logic
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ data_connector.py   # Main application class using MCP server functions
β”‚   β”‚   β”œβ”€β”€ enhanced_mcp_server.py # Enhanced MCP server functionality
β”‚   β”‚   └── mcp_server.py       # Base MCP server functionality
β”‚   β”œβ”€β”€ _commands/              # Commands layer for natural language interface
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ initialize_data_connector.py # Command to initialize with API spec path
β”‚   β”‚   β”œβ”€β”€ context_inheritance_model.json # Context hierarchy definition
β”‚   β”‚   └── DataConnector/      # DataConnector-specific commands
β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚       β”œβ”€β”€ _DataConnector.py # Context navigation class
β”‚   β”‚       β”œβ”€β”€ list_objects.py # Command to list objects from source JSON
β”‚   β”‚       β”œβ”€β”€ select_object.py # Command to select objects for target JSON
β”‚   β”‚       β”œβ”€β”€ generate_code.py # Command to generate data connector code
β”‚   β”‚       └── zip_code.py     # Command to zip the latest generated code
β”‚   β”œβ”€β”€ fastworkflow.env        # Environment variables with LLM settings
β”‚   β”œβ”€β”€ fastworkflow.passwords.env # API keys for LLMs
β”‚   └── startup_action.json     # Empty startup action (initialization requires parameters)
β”œβ”€β”€ pom.xml                      # Maven project configuration
β”œβ”€β”€ requirements.txt             # Project-level extras (optional)
└── debug_generation.log         # Tooling logs (if present)

πŸš€ Quick Start (Current Workflow)

1. Install Dependencies

# Install Python dependencies
cd mcp_server
pip install -r requirements.txt

# Install Java dependencies (if building)
mvn install

2. Start the MCP Server

# From anywhere: point to your local clone
python /absolute/path/to/iddm_dataconnector_codegen/mcp_server/mcp_server.py
# or
python3 /absolute/path/to/iddm_dataconnector_codegen/mcp_server/mcp_server.py

3. Configure in Cursor (works from any project)

In the project where you want to use these tools, create or edit .cursor/mcp.json and add an entry:

Option A: call Python directly with an absolute path

{
  "mcpServers": {
    "iddm-dataconnector": {
      "command": "python3",
      "args": ["/absolute/path/to/iddm_dataconnector_codegen/mcp_server/mcp_server.py"],
      "disabled": false
    }
  }
}

Option B: ensure the server runs with the correct working directory

{
  "mcpServers": {
    "iddm-dataconnector": {
      "command": "bash",
      "args": ["-lc", "cd '/absolute/path/to/iddm_dataconnector_codegen/mcp_server' && python3 mcp_server.py"],
      "disabled": false
    }
  }
}

Option C: portable configuration (recommended)

{
  "mcpServers": {
    "iddm-dataconnector": {
      "command": "/Library/Frameworks/Python.framework/Versions/3.12/bin/python3",
      "args": [
        "/absolute/path/to/iddm_dataconnector_codegen/mcp_server/mcp_server.py"
      ],
      "env": {
        "PYTHONPATH": "/absolute/path/to/iddm_dataconnector_codegen"
      },
      "cwd": "/absolute/path/to/iddm_dataconnector_codegen"
    }
  }
}

Tip: If your editor supports ${workspaceFolder}, you can use it in place of absolute paths.

4. Use the MCP Tools from Cursor

  • NEW: Use get_project_info to verify the server detected your project root correctly
  • Initialize or load an OpenAPI spec (use one under yamlfiles/, e.g., harry_potter_openapi.yaml)
  • Explore/list objects and methods
  • Select objects and methods to include in the target

5. Generate Code

  • Generate Java client code β†’ output under java_client/
  • Generate Data Connector code β†’ output under src/main/java/com/radiantlogic/custom/dataconnector/

6. Build the Java Project

mvn package -DskipTests
  • Artifacts are written to target/

πŸ”§ Available MCP Tooling (High Level)

  • Project Management: get_project_info - verify project root detection and configuration
  • OpenAPI Operations: initialize specs, list objects/methods, select targets
  • Data Connector Operations: generate Java client code, generate connector code, compile/test with Maven

πŸ“± Data Connector FastWorkflow

The Data Connector FastWorkflow provides a natural language interface for generating data connectors from OpenAPI specifications. It leverages the MCP server functionality for a more user-friendly experience.

Architecture

The FastWorkflow application follows a layered architecture:

  1. Application Layer: Contains the core business logic in the DataConnector class
  2. Commands Layer: Provides natural language interfaces to the application functionality
  3. MCP Integration: Connects to the MCP server to perform operations on OpenAPI specifications

Setting Up and Running the FastWorkflow

  1. Initial Setup

First, clone the repository, create a Python virtual environment, and install all dependencies:

git clone https://github.com/your-org/iddm_dataconnector_codegen.git cd iddm_dataconnector_codegen

Create virtual environment

python3 -m venv venv source venv/bin/activate # On Windows use: venv\Scripts\activate

Install dependencies

pip install -r requirements.txt

Next, add your API key values to fastworkflow.passwords.env before using FastWorkflow:

LLM API Keys

LITELLM_API_KEY_SYNDATA_GEN=your-api-key-here LITELLM_API_KEY_PARAM_EXTRACTION=your-api-key-here LITELLM_API_KEY_RESPONSE_GEN=your-api-key-here LITELLM_API_KEY_AGENT=your-api-key-here

1. Train the FastWorkflow

After cloning the repository and installing all dependencies from the requirements.txt, you must first train the FastWorkflow:

# If you're in the project root directory
fastworkflow train ./data_connector_fastworkflow ./data_connector_fastworkflow/fastworkflow.env ./data_connector_fastworkflow/fastworkflow.passwords.env

# If you're in another directory, use the absolute path
fastworkflow train /path/to/iddm_dataconnector_codegen/data_connector_fastworkflow /path/to/iddm_dataconnector_codegen/data_connector_fastworkflow/fastworkflow.env /path/to/iddm_dataconnector_codegen/data_connector_fastworkflow/fastworkflow.passwords.env

2. Make the Run Script Executable

Before running the FastWorkflow application, make the convenience script executable:

chmod +x run_data_connector.sh

3. Run the FastWorkflow

Now you can run the FastWorkflow application using the provided convenience script:

./run_data_connector.sh

This script runs the FastWorkflow application with the appropriate environment and password files.

Usage

Once the FastWorkflow application is running, you can interact with it using natural language commands:

  1. Initialize the data connector:

    initialize data connector with api spec path /path/to/api-spec.yaml
    
  2. List objects from source JSON:

    list all objects in source json
    
  3. Select objects for target JSON:

    select the users, products and orders object for target
    
  4. Generate data connector code:

    generate data connector code
    
  5. Zip the latest code:

    zip the latest code
    

And add your API keys to fastworkflow.passwords.env:

# LLM API Keys
LITELLM_API_KEY_SYNDATA_GEN=your-api-key-here
LITELLM_API_KEY_PARAM_EXTRACTION=your-api-key-here
LITELLM_API_KEY_RESPONSE_GEN=your-api-key-here
LITELLM_API_KEY_AGENT=your-api-key-here

Output Structure

When generating code, the application creates a structured output directory:

Api_code/
└── {api_name}/
    β”œβ”€β”€ source.json                # Parsed OpenAPI spec
    β”œβ”€β”€ target.json                # Selected objects
    β”œβ”€β”€ src version {version}/     # Generated code
    β”‚   β”œβ”€β”€ main/
    β”‚   β”‚   β”œβ”€β”€ java/
    β”‚   β”‚   β”‚   └── com/
    β”‚   β”‚   β”‚       └── radiantlogic/
    β”‚   β”‚   β”‚           └── custom/
    β”‚   β”‚   β”‚               └── dataconnector/
    β”‚   β”‚   β”‚                   └── {ApiName}DataConnector.java
    β”‚   β”‚   └── resources/
    β”‚   β”‚       └── com/
    β”‚   β”‚           └── radiantlogic/
    β”‚   β”‚               └── custom/
    β”‚   β”‚                   └── dataconnector/
    β”‚   β”‚                       └── {api_name}Connector.json
    β”‚   └── test/
    β”‚       └── java/
    β”‚           └── com/
    β”‚               └── radiantlogic/
    β”‚                   └── custom/
    β”‚                       └── dataconnector/
    β”‚                           └── {ApiName}DataConnectorTest.java
    └── zip/                      # Zipped code archives
        └── src_version_{version}.zip

⚠️ Important Notes

  • File Deletion Limitation: if generated files need to be cleared, remove them manually:
    rm -rf src/main/java/com/radiantlogic/custom/dataconnector/*.java
    rm -rf src/test/java/com/radiantlogic/custom/dataconnector/*.java
  • The requirements.txt at the project root is optional; the server uses mcp_server/requirements.txt.
  • If you work outside the original directory, absolute paths in certain advanced operations (like automatic POM updates) may need adjusting. Core MCP features (OpenAPI, selection, codegen, build) work with the configuration above.

🧩 Tips

  • OpenAPI specs live under yamlfiles/. Point the MCP tools at these when initializing.
  • The src/main/java/com/radiantlogic/custom/dataconnector/ directory will be empty until you generate code.
  • Check debug_generation.log and select_object_debug.log (if present) for troubleshooting information.
  • When using the FastWorkflow interface, you can select multiple objects before generating code.
  • The FastWorkflow application maintains state between commands, so you don't need to re-initialize for each operation.

πŸ“š Examples Folder

The examples/ folder contains few-shot learning examples used by the MCP server to generate high-quality Data Connector code:

  • HarryPotterDataConnector.java - Complete working Java connector class showing proper structure, imports, and patterns
  • harrypotterbooksconnector.json - Example JSON configuration file for the connector
  • HarryPotterDataConnectorTest.java - Example unit tests demonstrating proper testing patterns

These examples are automatically loaded by the server and used as templates when generating new connectors. They ensure consistent code quality and proper adherence to the Data Connector framework requirements.

Customizing Examples

You can modify these example files to:

  • Change import patterns for different frameworks
  • Update annotation styles
  • Modify testing approaches
  • Add new functionality patterns

The server will automatically use your updated examples in future code generation.

πŸ—οΈ Architecture

This project combines:

  • FastMCP: High-performance MCP server framework
  • DSPy: AI-powered code generation and optimization
  • OpenAPI: Standard API specification processing
  • Maven: Java project management and build automation
  • FastWorkflow: Natural language interface for data connector generation

🀝 Contributing

This integration brings together the power of MCP servers with IDDM data connector development, providing a streamlined workflow for rapid connector creation and deployment. We welcome:

  • Feedback: On usability and feature requests
  • Examples: Additional connector patterns and templates
  • Testing: Real-world usage scenarios and edge cases
  • Documentation: Improvements and additional examples

πŸ“„ License

See LICENSE file for details.

About

An AI application for rapid development of IDDM data connectors

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors