Skip to content

Autoresearch Pro-1: Fidelity optimization findings (M1/M2/M5/M6) #18

@realityinspector

Description

@realityinspector

Fidelity Cluster Dry-Run Results

Ran 30-iteration dry-run sweeps over the fidelity cluster (M6 tensor compression: compression_method, n_components) on two templates.

board_meeting — Pareto frontier (3 configs)

Run ID Quality Cost CR Config
dry_9fb185ee2d04 0.8111 $0.1082 0.6104 svd, n_components=5
dry_a318e5a5ae8a 0.8122 $0.1100 0.6330 pca, n_components=5
dry_f52c6a0cc910 0.8862 $0.1266 0.7755 nmf, n_components=2
  • Best quality: nmf with n_components=2 (q=0.8862, CR=0.7755)
  • Best efficiency: svd with n_components=5 (eff=7.4979)

mars_mission_portal — Pareto frontier (7 configs)

Run ID Quality Cost CR Config
dry_3831b0561afd 0.6864 $0.1180 0.4240 pca, n_components=5
dry_7b83fe9ef3a7 0.7590 $0.1368 0.5619 pca, n_components=3
dry_c33cce171400 0.8272 $0.1379 0.6608 nmf, n_components=5
dry_e2aef531ddd3 0.8564 $0.1522 0.7465 nmf, n_components=2
dry_ddde2f812ae0 0.8590 $0.1542 0.7301 pca, n_components=9
dry_e61f0290b751 0.8742 $0.1585 0.7625 nmf, n_components=6
dry_f085ecc74357 0.8874 $0.1687 0.7727 nmf, n_components=10
  • Best quality: nmf with n_components=10 (q=0.8874, CR=0.7727)
  • Best efficiency: nmf with n_components=5 (eff=5.9961)

Cross-template observations

  1. NMF consistently dominates both PCA and SVD for quality composite and causal resolution across both templates.
  2. Low n_components (2) with NMF yields highest quality on board_meeting; mars_mission_portal benefits from higher components (6-10), likely due to greater scenario complexity.
  3. SVD at n_components=5 is the most cost-efficient config for board_meeting (7.50 quality/$), while NMF at n_components=5 leads efficiency for mars_mission_portal (5.99 quality/$).
  4. The fidelity cluster search space is only 2 dimensions — these results are high-confidence within the dry-run synthetic metric model.

Recommended next steps

  • Run live (non-dry-run) validation of top NMF configs on both templates
  • Expand fidelity cluster params (add compression ratio, reconstruction error threshold)
  • Cross-cluster sweep: combine fidelity + temporal for joint optimization

Metadata

Metadata

Assignees

No one assigned

    Labels

    autoresearchAutoresearch optimization loopfindingAutoresearch discovery

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions