[Bounding box] Print label in center of generated image.

[aelkhour]

This is quite handy for binary classifiers.

[bsblaw]

@aelkhour When I tried it out, I get the following error:

benl@benl-02:~/ws/aelkhour-inference-sdk/inference-test-tool$ ./send-inference-request.sh -b --host 0.0.0.0 --port 8900 ./study1/
Sending 1 files...
JSON response: {'parts': [], 'bounding_boxes_2d': [{'SOPInstanceUID': '1.2.392.200046.100.13.24550048020077643303031580933179026570', 'top_left': [5, 5], 'label': 'super bbox', 'bottom_right': [10, 10]}], 'protocol_version': '1.0'}
Traceback (most recent call last):
File "run.py", line 143, in
upload_study_me(args.file_path, model_type, args.host, args.port)
File "run.py", line 124, in upload_study_me
test_inference_boxes.generate_images_with_boxes(images, boxes, output_folder)
File "/opt/test_inference_boxes.py", line 34, in generate_images_with_boxes
font = ImageFont.truetype('OpenSans-Regular.ttf', br[1] // 20)
File "/usr/local/lib/python3.8/site-packages/PIL/ImageFont.py", line 642, in truetype
return freetype(font)
File "/usr/local/lib/python3.8/site-packages/PIL/ImageFont.py", line 639, in freetype
return FreeTypeFont(font, size, index, encoding, layout_engine)
File "/usr/local/lib/python3.8/site-packages/PIL/ImageFont.py", line 187, in init
self.font = core.getfont(
OSError: cannot open resource

[bsblaw]

I think you just need to include a font file in the inference-test-tool/fonts directory or something and then you should be able to just do font = ImageFont.truetype('/opt/fonts/FreeSans.ttf', br[1] // 20).

[aelkhour]

I think you just need to include a font file in the inference-test-tool/fonts directory or something and then you should be able to just do font = ImageFont.truetype('/opt/fonts/FreeSans.ttf', br[1] // 20).

Sorry for the late reply @bsblaw , I somehow missed your comments.
Indeed I forget to commit that part. I added an install step for the font in the Dockerfile, let me know if it's ok for you.

[aelkhour]

@bsblaw If you have time and think this PR could be useful, could you retest the updated version please?

[mats-claassen]

In this case br[1] // 20 is the font size right? Now, br[1] is the y coordinate of the bottom right point. That means that boxes that are further down on the image will have a bigger text than boxes at the top. This might work fine for classification but let's make it work for bounding boxes as well.

Maybe dcm.Rows // 20?

[aelkhour]

I agree that this is not very robust.

I made a slightly different proposal relying on the bounding box size instead of the image size, the rationale being that if in the future we want to support multiple bounding boxes in the same image, the labels should be positioned accordingly.

[mats-claassen]

We actually support multiple bounding boxes in the same image

[mats-claassen]

The problem I see with using the text size relative to bbox size is that for small boxes with, say 20px height, the text would be 1px height and unreadable. That is why a fixed font size would be better.
Other implementations I have seen show the text above the bbox, instead of inside. In that case there would be no issue if the text is bigger than the box

[aelkhour]

I agree with you that this PR handles poorly all edge cases, as my intent was to focus on the most common use case (IMHO) of creating a single bbox which has the same size as the image. In that specific case, placing the label above the bbox sadly leads to a text lying outside of the image.

The worst case scenario for this PR would be having a text label being too small to read, somewhat equivalent to running the inference without this PR. What do you think of keeping it this way as I don't see any negative effect?

[mats-claassen]

I see your point regarding placing the label on top of the bounding box. You are right there. However, we are using bboxes for classification as a workaround currently. Ideally we should agree on a format for classification which does not need a bbox. @bsblaw What do you think?

[mats-claassen]

Same happens to this calculation. It does not work for bounding boxes that do not span the whole image.