Issue on creating new DR lesion map

[tien-m-duong]

Dear authors,

I found this work to be very interesting and want to use it for my research. I tried to replicate the results and encountered some problems. When I try to use Retina Template to hand-draw, it gives me this error.

Furthermore, I want to input DR 0-4 and have the model generate a synthetic retina image based on that input. Based on my understanding, you can input any grade and it will generate the image for you. "To generate lesion masks of a specific DR grade, the generator of StyleGAN has been modified to also take DR grades as an input variable." However, I cannot find a way to input DR grade. It would be great if you can assist me on this problem or direct me to sources that can help me.

So far, I was able to randomly generate retina image. I also tried to upload my image and have the model generate a synthetic image, but my uploaded image has to be a DR lesion map image. If I try to upload a real fundus image, it will give error "Mask contain invalid Class Colors". I believe this is meant to be, since the GauGAN can only generate fundus image based on the DR lesion map. I want to put this here to double check whether my understanding for this work is correct. Please let me know if my understanding is not correct. Thank you so much for your work.

[farrell236]

Hi Jaineera,

Thank you for your interest in our work, hope this solves your issues:

1. streamlit error:
   This is likely due to updates in Streamlit's API. In hindsight, I should have included a requirements.txt file with the specific versions, but this project was completed two years ago. I would suggest going onto PyPI and installing a Streamlit version c.2021-2022 (https://pypi.org/project/streamlit/#history).

2. generate lesion masks of a specific DR grade
   I would look at the run_pipeline.py file instead of the streamlit demo to have command line access (https://github.com/farrell236/RetinaGAN/blob/master/run_pipeline.py). Specifically line 15 is where you set the DR grade ([0,1,2,3,4]). conditional_style_gan.call will return generated labels as an array.

3. generating retina image
   Yes that is correct, the pipeline is random noise -> gaugan -> segmentation mask -> conditional stylegan -> retina image

Best,
Benjamin

[tien-m-duong]

Hi Benjamin,

Thank you for the swift response and explaination. I was able to resolved the Streamlit issue by changing:
template = np.uint8(cv2.circle(np.zeros((1024, 1024, 3)), [512, 512], 512, (255, 255, 255), -1)), line 171, web_demo.py ->
template = np.uint8(cv2.circle(np.zeros((1024, 1024, 3)), (512, 512), 512, (255, 255, 255), -1)).

I am a little confused on the pipeline. Based on this

Shouldn't the pipeline be:
random noise/DR grade -> conditional stylegan -> segmentation/DR lesion mask -> gaugan -> retina image
Please let me know if my understanding is incorrect.

Thank you so much for your assistance,
Tien.

[farrell236]

yes, you are right 😅 it has been a while since I had to revisit this project.