Skip to content

Implement GeneratorService data masking & generation execution logic#24

Merged
MaximumTrainer merged 2 commits intomainfrom
copilot/add-data-masking-execution-logic
Apr 1, 2026
Merged

Implement GeneratorService data masking & generation execution logic#24
MaximumTrainer merged 2 commits intomainfrom
copilot/add-data-masking-execution-logic

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 1, 2026

JobService fetched column generators from the DB but never applied them — no GeneratorService existed to transform data. MASK and GENERATE table modes were effectively no-ops.

Changes

  • GeneratorService — new Spring service implementing all 11 GeneratorType variants via datafaker 2.1.0:

    • NAME, EMAIL, PHONE, ADDRESS, SSN, CREDIT_CARD, DATE, UUID — realistic synthetic values from faker
    • CONSTANT — returns params["value"] (JSON-configured)
    • NULL — returns null
    • CUSTOM — returns params["value"] with fallback to original source value
    • applyGenerators(row, generators) — masks an existing row in-place per column config
    • generateRows(generators, count) — produces fully synthetic rows with no source dependency
  • JobService.processTable() — wired GeneratorService into both active modes:

    • MASK: fetch source rows → applyGenerators() per row → write to destination
    • GENERATE: generateRows(generators, rowLimit ?: 100) → write to destination (no source read)
  • GeneratorServiceTest — adds 3 tests for previously uncovered generateRows path: correct row count, zero-count edge case, and CUSTOM originalValue fallback.

// MASK: each source row is transformed
val maskedData = data.map { row -> generatorService.applyGenerators(row, generators) }
destConnector.writeData(tableConfig.tableName, maskedData)

// GENERATE: no source read, pure synthetic output
val generatedData = generatorService.generateRows(generators, rowCount)
destConnector.writeData(tableConfig.tableName, generatedData)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/test/work -Dorg.gradle.native=false -javaagent:/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/.cache/expanded/zip_0bd0851504e6286cb5fc6f37a81e25cc/jacocoagent.jar=destfile=build/jacoco/test.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath9555417616528680429txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1' (dns block)
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dorg.gradle.internal.worker.tmpdir=/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/test/work -Dorg.gradle.native=false -javaagent:/home/REDACTED/work/OpenDataMask/OpenDataMask/backend/build/tmp/.cache/expanded/zip_0bd0851504e6286cb5fc6f37a81e25cc/jacocoagent.jar=destfile=build/jacoco/test.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false @/home/REDACTED/.gradle/.tmp/gradle-worker-classpath11343826022672846332txt -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -ea worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 3' (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@MaximumTrainer MaximumTrainer requested a review from Copilot April 1, 2026 16:41
Copilot AI changed the title [WIP] Add data masking and generation execution logic Implement GeneratorService data masking & generation execution logic Apr 1, 2026
Copilot AI requested a review from MaximumTrainer April 1, 2026 16:42
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds unit tests to validate row generation behavior in GeneratorService, ensuring correct row counts and expected fallback behavior for CUSTOM generators.

Changes:

  • Added tests verifying generateRows returns the requested number of rows and produces non-empty generated values.
  • Added test verifying generateRows returns an empty list when count is zero.
  • Added test verifying GeneratorType.CUSTOM falls back to originalValue when params are not provided.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@MaximumTrainer MaximumTrainer marked this pull request as ready for review April 1, 2026 17:32
@MaximumTrainer MaximumTrainer merged commit e1f3836 into main Apr 1, 2026
9 checks passed
@MaximumTrainer MaximumTrainer deleted the copilot/add-data-masking-execution-logic branch April 1, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data Masking & Generation Execution Logic

3 participants