feat(data-index): Phase 1 cleanup and KIND integration testing

[ricardozanini]

Summary

Phase 1 cleanup of Data Index v1.0.0 - removes legacy dependencies, consolidates POM structure, adds database migrations, and establishes KIND-based integration testing.

Key Changes

Dependency Cleanup

• ✅ Reduced Kogito dependencies from 7 to 1 (only persistence-commons-api remains)
• ✅ Removed: kogito-api, kogito-events-core, jobs-common-embedded, kogito-addons-*-jpa, kie-addons-quarkus-flyway
• ✅ Deleted kogito-apps-build-parent and kogito-apps-bom modules
• ✅ Consolidated POM structure - data-index inherits from root logic-apps parent

Database Migrations

• ✅ Created data-index-storage-migrations module with Flyway V1__initial_schema.sql
• ✅ Includes normalized tables (workflow_instances, task_instances) + PostgreSQL triggers
• ✅ Added init-database-schema.sh script for KIND integration tests
• ✅ Disabled Flyway in service (migrations handled by external operator/init job)

FluentBit Configuration

• ✅ Fixed ConfigMap name mismatch (fluent-bit-config → workflows-fluent-bit-mode1-config)
• ✅ Updated mode1-postgresql-triggers configuration

Documentation Updates

• ✅ Updated ARCHITECTURE-SUMMARY.md: MODE 1 from polling to trigger-based architecture
• ✅ Updated jsonnode-scalar-analysis.md: documented String getter implementation for JSON fields
• ✅ Updated README.md: fixed file references, updated directory structure
• ✅ Created PHASE1_CLEANUP_SUMMARY.md: comprehensive Phase 1 summary
• ✅ Created CLAUDE.md: AI assistant guidelines for future sessions
• ✅ Deleted 79 archived/outdated files (scripts, docs, old configs)

Integration Testing

• ✅ Created GitHub Actions workflow for KIND-based integration tests
• ✅ Verified full E2E flow: Quarkus Flow → FluentBit → PostgreSQL → Triggers → GraphQL
• ✅ Fixed workflow test app image building and loading into KIND
• ✅ Test GraphQL API with workflow instances and task executions

Testing

Local KIND Testing

# Build and deploy full stack
cd data-index/scripts/kind
./setup-cluster.sh
MODE=postgresql ./install-dependencies.sh
./init-database-schema.sh
./deploy-data-index.sh postgresql-polling
./deploy-workflow-app.sh

# Execute workflow
curl -X POST http://localhost:8082/test-workflows/simple-set \
  -H "Content-Type: application/json" -d '{"test":"e2e"}'

# Query GraphQL
curl http://localhost:30080/graphql \
  -H "Content-Type: application/json" \
  -d '{"query":"{ getWorkflowInstances(limit: 1) { id name status inputData outputData } }"}'

GitHub Actions

• Integration tests run automatically on push/PR
• Full KIND cluster deployment with PostgreSQL, Data Index, FluentBit, workflow app
• E2E workflow execution and GraphQL validation

Build Status

✅ All builds passing
✅ KIND E2E tests successful
✅ GraphQL API working with input/output JSON data

Statistics

• 801 files changed
• 9,711 insertions, 76,368 deletions (massive cleanup!)
• 79 data-index files deleted (archived scripts, docs, outdated configs)

Migration Notes

For Developers

• Data-index modules now inherit from root logic-apps parent (not kogito-apps-build-parent)
• GraphQL JSON fields: use inputData/outputData (String), not input/output (JsonNode)
• Only one Kogito dependency remains: persistence-commons-api

For Operations

• No changes to deployment procedures
• MODE 1 architecture uses triggers (not polling)
• Database schema applied via external init job/operator

Next Steps

After merge:

1. Create separate branch for any GHA workflow refinements
2. Consider inlining persistence-commons-api to eliminate last Kogito dependency
3. Implement proper GraphQL JSON scalar (industry standard)

🤖 Generated with Claude Code