Skip to content

MX-136: Liquibase migration to safely remove unreferenced Pentaho reports#434

Merged
IOhacker merged 3 commits intoopenMF:developfrom
raghavvag:MX-136-remove-pentaho-liquibase
Mar 3, 2026
Merged

MX-136: Liquibase migration to safely remove unreferenced Pentaho reports#434
IOhacker merged 3 commits intoopenMF:developfrom
raghavvag:MX-136-remove-pentaho-liquibase

Conversation

@raghavvag
Copy link
Member

Summary

This PR implements MX-136 by introducing a Liquibase migration in the Mifos Reporting Plugin to remove deprecated Pentaho report definitions from the stretchy_report table.

The migration deletes only unreferenced Pentaho reports and preserves all historical data and FK integrity.

This follows the agreed Option A behavior:

  • Delete report_type = 'Pentaho' entries from stretchy_report
  • Skip deletion if the report is referenced
  • Preserve m_report_mailing_job_run_history
  • Do not delete mailing jobs
  • Do not touch non-Pentaho report types
  • Do not modify Apache Fineract core repository

Implementation Details

Migration File

db/custom-changelog/remove-pentaho-reports.xml

changeSet

  • id: mx136-remove-pentaho-unreferenced-reports
  • author: raghav
  • context: custom_changelog

Deletion Strategy

The migration deletes Pentaho reports only when ALL of the following are true:

  • report_type = 'Pentaho'
  • NOT referenced by m_report_mailing_job
  • NOT referenced by scheduled_email_campaign
  • NOT referenced by sms_campaign

The SQL uses ANSI-compliant NOT EXISTS checks to ensure:

  • FK-safe deletion
  • MariaDB compatibility
  • PostgreSQL compatibility

No DELETE alias syntax is used to avoid MariaDB incompatibility.


Safety Guarantees

  • FK-safe (no constraint violations)
  • Referenced reports are preserved
  • Mailing jobs remain intact
  • Run history remains intact
  • Idempotent (safe on restart)
  • Cross-database compatible
  • No changes to Apache Fineract core
  • Plugin-only scoped migration

Testing Performed

Tested On

Component Version
Apache Fineract 1.15.0-SNAPSHOT (develop branch)
MariaDB 11.4
Java Temurin 21
Tenant default

Phase 1 — Baseline Validation

  • 44 Pentaho reports present
  • 0 references in mailing jobs
  • 0 references in email campaigns
  • 0 references in SMS campaigns

Phase 2 — Liquibase Execution Verification

  • changeSet recorded in DATABASECHANGELOG
  • Execution status = EXECUTED
  • No Liquibase errors in Docker logs
  • Change order incremented correctly

Phase 3 — Post-Migration Validation

Metric Before After
Pentaho reports 44 0
Non-Pentaho reports 84 84
Mailing jobs 0 0
Run history 0 0

All 44 unreferenced Pentaho reports were deleted.
No collateral impact observed.


Phase 4 — Validation (Skip Referenced)

Test Scenario

  1. Inserted a test Pentaho report
  2. Created a referencing mailing job
  3. Executed migration

Results

  • 44 unreferenced Pentaho reports → deleted
  • Referenced Pentaho report → preserved
  • FK constraint correctly blocks manual deletion

Confirmed skip logic works as intended.


Phase 5 — Idempotency

  • Container restarted
  • Migration did not re-run
  • No duplicate entries in DATABASECHANGELOG
  • Pentaho count stable

Phase 6 — Cross-Database Safety

  • No DB-specific functions
  • No DELETE alias syntax
  • ANSI-compliant SQL
  • Successfully executed on MariaDB

The SQL is compatible with PostgreSQL.


Final Result

  • Correct deletion behavior
  • Referenced reports preserved
  • FK integrity maintained
  • No runtime errors
  • API operational post-migration
  • Idempotent and production safe

Impact

This migration safely removes deprecated Pentaho metadata while preserving historical data and system integrity.

Fully aligned with MX-136 requirements.

Ready for review.

…migration

Adds a Liquibase migration under db/custom-changelog to remove
report_type = 'Pentaho' entries from stretchy_report that are not
referenced by mailing jobs or campaigns.

Implementation details:
- Deletes only unreferenced Pentaho reports (Option A behavior)
- Preserves m_report_mailing_job and m_report_mailing_job_run_history
- Uses NOT EXISTS checks for FK-safe deletion
- Cross-database compatible (MariaDB & PostgreSQL)
- Idempotent and safe on restart
- Does not modify Apache Fineract core

Tested against Fineract 1.15.0-SNAPSHOT (develop) with MariaDB 11.4.
Copilot AI review requested due to automatic review settings March 2, 2026 22:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a Liquibase custom changelog to remove deprecated report_type = 'Pentaho' rows from stretchy_report when they are not referenced by jobs/campaigns, keeping historical run data intact.

Changes:

  • Introduces a new Liquibase changeSet to delete unreferenced Pentaho reports from stretchy_report.
  • Adds preconditions to gate execution based on presence of related reference tables.
  • Documents the intended destructive nature of the cleanup (no rollback).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Kindly see my comments

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

@IOhacker IOhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kindly see my comments

@IOhacker
Copy link
Contributor

IOhacker commented Mar 3, 2026

@raghavvag please remove the Copilot suggestion, it was applied by me.

@IOhacker
Copy link
Contributor

IOhacker commented Mar 3, 2026

I have just one comment regarding the author

Copy link
Member Author

@raghavvag raghavvag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed author

@raghavvag raghavvag requested a review from IOhacker March 3, 2026 18:34
Copy link
Contributor

@IOhacker IOhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IOhacker IOhacker merged commit 0b4a990 into openMF:develop Mar 3, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants