model.predict to slow (Low FPS) while running deep_pi_car.py

[fifise77]

Hello,
First of all thanks for the great work and tutorial.
I managed to train my own model following your instruction.
When I run the coco_object_detection.py the FPS are turning around 16 FPS (which seem normale according to your explanation)
but when I execute deep_pi_car.py, the FPS of the model.predict are so low that the pi-car front wheels turn way to late and always get out of the track. (same track working fine in opencv mode).

When I execute python3 end_to_end_lane_follower.py, the diff between the desired and the model never excide 6, so I guess the model is ok, i seem that only its execution is slow.

I am running on an Pi4 with:
Python 3.7.3
tf version 2.3.0
numpy version 1.20.3
keras version 2.4.3

Do you have any idea how to debug/fix this low FPS ?
Thanks in advance for your help

[fifise77]

OK I found some reason/optimisation, after trying multiplae version of TF (2.4, 2.3, 2.0, 1.14 and1.15) an kera (2.4.3, 2.3 and 2.2.4)
I found out that on my Rasp 4 4G as on (05/2021) the faster execution was to use:
tensorflow==1.15.0
keras==2.2.4

You have to train the model with those version install as well in google colab :
!pip install tensorflow==1.15.0
!pip3 install keras==2.2.4

the beginning of my file end_to_end_lane_follower.py look like this now:

import cv2
import numpy as np
import logging
import math
import keras
#from keras.models import load_model #tensorflow 2.x
from tensorflow.keras.models import load_model #tensorflow 1.x
from hand_coded_lane_follower import HandCodedLaneFollower

_SHOW_IMAGE = False

class EndToEndLaneFollower(object):

 def __init__(self, car=None, model_path='/home/pi/DeepPiCar/models/lane_navigation/data/model_result.tf/lane_navigation_final.h5'):
    logging.info('Creating a EndToEndLaneFollower...')
    self.car = car
    self.curr_steering_angle = 90
    self.model = load_model(model_path, compile = False)  ## force keras 
	
def follow_lane(self, frame):
    # Main entry point of the lane follower
    show_image("orig", frame)

    self.curr_steering_angle = self.compute_steering_angle(frame)
    logging.debug("curr_steering_angle = %d" % self.curr_steering_angle)

    if self.car is not None:
        self.car.front_wheels.turn(self.curr_steering_angle)
    final_frame = display_heading_line(frame, self.curr_steering_angle)
    
    return final_frame

def compute_steering_angle(self, frame):
    """ Find the steering angle directly based on video frame
        We assume that camera is calibrated to point to dead center
    """
    keras.backend.set_learning_phase(0)  ## seem to make the model inference with keras run faster
    preprocessed = img_preprocess(frame)
    X = np.asarray([preprocessed])
    steering_angle = self.model.predict(X)[0]


    logging.debug('new steering angle: %s' % steering_angle)
    return int(steering_angle + 0.5) # round the nearest integer

`

[fifise77]

Last detail, I also disactivate all the create_video_recorder in deep_pi_car.py to free CPU ressource to run the model

[dkclssha123]

i got issue to i use raspberry pi 3 B+ if i start deep pi car. py camara FPS 1 and delay 8000ms how can i solve that problem?

[VicoLeChips]

I'm not that far yet (self learning process) @dkclssha123, however I got some really latency on Raspberry pi 4. To fix it, I overclocked it to 2GHz and it works quite well now.