Inference speed is only 2FPS

[chaomath]

Hello, I love your code--easy and clean. I have trained and predicted myself using your code, what confused me is that I only got 2FPS at prediction inference speed . My environment is a single GPU(Quadro P5000). Can you give me an advice to speed up my inference?

[Hizhaoyuan]

Hi, are you using predict.py to make predictions? This script saves the predicted results on the hard disk, and you should comment out the code that stores the results to the hard disk. Can you paste the script you used?

[chaomath]

Hi, are you using predict.py to make predictions? This script saves the predicted results on the hard disk, and you should comment out the code that stores the results to the hard disk. Can you paste the script you used?

1--> I use "python train.py " to train the model, and save to pre-trained model to "./weights"
2--> And then "python predict.py --weights_path =./weights/Complex_YOLO_final_train_loss_30.37-16000 --draw_gt_box=True"
3-->For the 1000 prediction samples, my running time is 476 second

[Hizhaoyuan]

You should change the script slightly.
First, set draw_gt_box = False.
Second, comment out the code that save the predicted results.
Start timing at the outer layer of the for loop, this is the right way to calculate the inference time.
You should not calculate the running time of predict.py, because predict.py will continue to run over and over again.

[chaomath]

Thank you very much for your patient answers.
But unfortunately, my speed is still slow. I test again for the predict.py, when I comment out the code that saves the predicted results, the running time is still 425 seconds for 1000 samples.
I guess there may be something wrong with my computating machine, I will continously track this problem, if there is a good result, I will tell you at the first time.

[zbs881314]

Hi, I find you just do the bird-eye-view detection and get the AP, so Have you done the 3D object detection or do you have the code about calculating the 3D AP? Thank you very much.

[Hizhaoyuan]

Yes, I didn't do 3D detection, but the official kitti script has 3D evaluation code.

[zbs881314]

Thank you very much. Your code is very clear and professional. So, if possible, hope you can update the 3D evaluation.

[zbs881314]

A quick question about the Average Precision, when I get the results by running the kitti eval function, how do I get the AP? the format of result is: Car -1 -1 -10 -1 -1 -1 -1 -1 0.55 1.02 1.91 -1000 8.25 1.50 1.00, however the format of target is: Car 0 0 -0.20 712.40 143.00 810.73 307.92 1.89 0.48 1.20 1.84 1.47 8.41 0.01. I have tried to change the target truncated to -1, occluded to -1, alpha to -10, height to -1, y to -1000, but the official evaluation code can not run and report segmentation error. I also try to change the bbox to -1, it still can not run, so could you give some guidance. Thank you very much.

[Hizhaoyuan]

@zbs881314 You should not modify the label. The evaluation script determines which bengchmark to evaluate based on the format of the result you predict. Give me your email and I will send you the evaluation script I used. You can modify the parameters according to the requirements.

[chaomath]

I konw why "Inference speed is only 2FPS"
I add time computing module in the predict.py after line 65, as the following:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
T0 = time.time() #start-compute-time
data = sess.run(y, feed_dict={image: [rgb_map], train_flag: False})
T1 = time.time() #start-compute-time
classes, rois = preprocess_data(data, anchors, important_classes,
grid_w, grid_h, net_scale)
T2 = time.time() #start-compute-time
classes, index = non_max_supression(classes, rois, prob_th, nms_iou_th)
T3 = time.time() #start-compute-time
all_boxes = filter_bbox(classes, rois, index)
T4 = time.time() #start-compute-time
T01 = T1 - T0
T12 = T2 - T1
T23 = T3 - T2
T34 = T4 - T3
print("T01=%f, T12=%f, T23=%f, T34=%f\n"%(T01, T12, T23, T34))
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

One of the output is "T01=0.029227, T12=0.156919, T23=0.097446, T34=0.044286"

As can be seen, the forward propagation is about 30FPS, but other modules which computing in CPU is
much slower than forward propagation. I will try to do some optimization, and if you have any advice, I'd like to listen.

[abhigoku10]

@chaomath can you share your trained model??