Feature request: allow users to change retinanet candidate images.

[bw4sz]

Expose RetinaNet detections_per_img and topk_candidates to user config

Problem

The torchvision RetinaNet model has two important parameters that control how many detections are returned per image:

• detections_per_img (default: 300) — the maximum number of detections returned after NMS
• topk_candidates (default: 1000) — the number of top-scoring candidates considered before NMS

Neither of these is currently exposed in the DeepForest config. They are hardcoded at the torchvision defaults. For dense scenes — such as large bird colonies or very dense tree canopies — 300 detections per image may be insufficient, leading to missed objects that the model would otherwise detect.

Example

While running predictions on dense bird imagery (file C2_L2_F52_T20230910_134201_350.jpg from flight JPG_20230910_133900), we appear to be hitting the detection cap in very dense regions. The model scores objects above the score_thresh but they get silently dropped because they exceed detections_per_img.

We also see this in the Bird Fine-Tuning example notebook, which already warns:

UserWarning: Encountered more than 100 detections in a single image. This means that certain detections with the lowest scores will be ignored...

Where the limit is applied

In src/deepforest/models/retinanet.py, RetinaNetHub.__init__ passes **kwargs through to torchvision.models.detection.retinanet.RetinaNet, which accepts detections_per_img and topk_candidates. However, Model.create_model() never passes these parameters, so the torchvision defaults are always used:

# current code in create_model — no detections_per_img or topk_candidates
model = RetinaNetHub(
    backbone_weights="COCO_V1",
    num_classes=self.config.num_classes,
    nms_thresh=self.config.nms_thresh,
    score_thresh=self.config.score_thresh,
    label_dict=label_dict,
)

Proposed solution

1. Add detections_per_img (default 300) and topk_candidates (default 1000) to the DeepForest config schema and config.yaml, so users can override them.
2. Pass them through from Model.create_model() → RetinaNetHub() → RetinaNet.__init__().

Discussion: Can we just increase the default?

Since we already have score_thresh filtering low-confidence detections, one might ask: why not just set detections_per_img very high and rely on score_thresh alone?

Things to consider:

• Performance: Higher detections_per_img means more boxes survive to NMS, which increases compute cost — though this is likely modest for most use cases.
• Memory: More detections means larger output tensors, which could matter on GPU-constrained setups or with very large batch sizes.
• Interaction with topk_candidates: topk_candidates acts as a pre-NMS filter. If detections_per_img is raised but topk_candidates is left at 1000, the improvement may plateau. Both should be tunable together.
• Backwards compatibility: Changing the default could alter existing users' results. Exposing the parameter without changing the default preserves backwards compatibility while giving power users control.

The safest first step is to expose both parameters in the config with their current torchvision defaults.

Questions

1. @jveitchmichaelis does DETR have similar limitations?
2. What are the downsides, this threshold must exist for a reason, is it slower?

[vickysharma-prog]

Hi @bw4sz,
I'd like to work on this!
Will add detections_per_img and topk_candidates to config and pass through create_model(). Will keep defaults unchanged.

[bw4sz]

Sounds good. Let's find some sample tree and bird images that we know are limited by the threshold.

We want to have some understanding of the tradeoff, is it much slower? There must be a reason this threshold exists.

[vickysharma-prog]

Thanks @bw4sz!
Will:

1. Find dense test images (bird colonies/dense canopy)
2. Benchmark current limit behavior
3. Measure performance tradeoff (time/memory at different thresholds)
4. Share findings before implementing
   On it!

[vickysharma-prog]

@bw4sz Here are my benchmark findings:

Test Setup:

• Image: OSBS_029.png (sample data)
• Method: predict_tile with 9 patches
• Hardware: CPU (no GPU)
• Runs: 3 iterations averaged per setting

Results:

detections_per_img	Detections	Avg Time (s)
100	120	28.8
300 (default)	120	33.7
500	120	47.6
1000	120	36.3
2000	120	38.1

Observations:

1. Detection count unchanged - Sample image doesn't hit the 300 limit (NMS reduces 235 raw → 120 final). Would need denser imagery to see actual detection differences.

2. Performance impact is modest - Going from 300 → 1000 adds only ~3-8% overhead on CPU. The spike at 500 appears to be CPU variance.

3. Recommendation - Safe to expose these parameters with current defaults (300/1000). Users who need higher limits can accept the minor performance cost.

Note: Results on GPU would be more reliable. The CPU variance makes it hard to draw precise conclusions, but the overall trend shows acceptable overhead for reasonable increases.

Should I proceed with the implementation, or would you like additional benchmarks on denser images?

[vickysharma-prog]

@bw4sz Following up — I tried a few more images to find one that actually hits the detection cap:

Image	Model	score_thresh	Detections
OSBS_029.png	Tree	0.1	120
OSBS_029.png	Bird (everglades)	0.1	10
AWPE Pigeon Lake DJI_0005.JPG	Bird	0.1	67
2019_YELL_2 (15MB)	Tree	0.1	46
OSBS_029.png	Tree	0.01	56

Couldn't get any sample image to hit the 300 limit — NMS filters them down well before that, even with very low score threshold.

Would you be able to share one of the dense bird colony images you mentioned (like C2_L2_F52_T20230910_134201_350.jpg)? That would help show the actual cap behavior.

Meanwhile, since performance overhead looks fine (~3-8%) and defaults stay unchanged (300/1000), happy to go ahead with the implementation if you'd prefer. Let me know!

[vickysharma-prog]

@bw4sz I've opened draft PR #1314 with an initial implementation.

Changes

• Added detections_per_img and topk_candidates to config.yaml (defaults: 300 / 1000).
• Passed them through create_model() → RetinaNetHub() → torchvision RetinaNet.
• Added a test to verify the parameters are correctly applied to the model.

Benchmarks

From earlier testing:

• Performance overhead was modest (~3–8% on CPU).
• Tested multiple images (tree + bird models, low score_thresh), but none of the repo samples actually hit the 300 detection cap.

Defaults remain unchanged, so existing workflows should not be affected.

Keeping this as a draft for now ,happy to test further with denser imagery if available, or mark it ready for review if this approach looks good.

[bw4sz]

thanks, I know that I see this warning from time to time, I will find an image that is dense enough.

[vickysharma-prog]

Thanks @bw4sz! Sounds good happy to run additional benchmarks once you have a dense example.

[vickysharma-prog]

Hi @bw4sz, just checking in have you had a chance to find a dense image for testing? Happy to run benchmarks as soon as one is available. In the meantime, should I mark PR #1314 as ready for review or keep it as draft?