Skip to content

Comments

Implement modular benchmarking pipeline for patient graph evaluation#14

Merged
gtcha2 merged 11 commits intomainfrom
similarity_benchmark_vicky
May 15, 2025
Merged

Implement modular benchmarking pipeline for patient graph evaluation#14
gtcha2 merged 11 commits intomainfrom
similarity_benchmark_vicky

Conversation

@Vicbi
Copy link
Contributor

@Vicbi Vicbi commented May 10, 2025

Implement modular benchmarking pipeline for patient graph evaluation

♻️ Current Situation & Problem

This PR introduces a new modular benchmarking pipeline for evaluating patient trajectory graphs derived from clinical case reports. It addresses the need to validate and compare different representation techniques (LLM-based reconstructions, BERTScore, structural checks, and trajectory embeddings).

⚙️ Release Notes

  • Added LLMReconstructor using DSPy-compatible APIs for narrative generation from graph data.
  • Integrated BERTScoreEvaluator to compare reconstructed vs. original text.
  • Introduced TopologyValidator for validating DAG structure, timestamp order, and connected components.
  • Added TrajectoryEmbedder based on Bio_ClinicalBERT with pooling strategy for per-patient vectorization.
  • Added CLI scripts (main.py, batch_run.py) to run the pipeline on single or multiple graphs.
  • New visualization utilities support t-SNE, heatmap, and cluster plotting.
  • HTML input support for case report text.

📚 Documentation

  • Each module includes inline docstrings and logging for clarity and debugging.
  • Configurable via config.py to easily switch between LLM backends and embedding models.
  • Visuals and results are saved in output/ and output/plots.
  • Usage scripts added for both single-graph (main.py) and batch mode (batch_run.py).
  • README to be updated in a follow-up PR.

📝 Code of Conduct & Contributing Guidelines

By submitting this pull request, you agree to follow our Coding Guidelines:

@Vicbi Vicbi requested a review from gtcha2 May 10, 2025 00:22
@gtcha2 gtcha2 merged commit 9bb7b55 into main May 15, 2025
2 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants