New FPGA Model Streaming

[Daayim]

Related Task

• This PR Closes https://github.com/g14capstone/submissions/issues/103

Changes

• Remove code related to traditional model inference method using ssm and backend
• Added github tokens url and other .env variables for fastapi deployment on ec2. Added to confluence page for .env variables
• Refactored script to pull python3.8 and deploy fastapi server during machine creations via ec2 script
• FastAPI server can now be inferenced and data can be streamed back to user over public Ip
• FastAPI server has multiple options for temperature, max_token, streaming.

Documentation

FastAPI Doc

Model Streaming Testing

Get Inference URL Endpoint

Local Testing

Terminal Log confirming the automated fastapi deployment

Example query prompt

[cmatthews20]

@Daayim merge main to your branch and resolve conflicts - looks like umama merged before you and you didn't have those changes

[K-rolls]

[umama-rahman1]

Overall looks like great work, make sure to create pull request on Config Engine as well so that code is visible for review too. I had a peek on the branch.

Just make sure to add the get_fpga_inference_url endpoint and it should be good.

[K-rolls]

Great work getting those edits in sir!

[umama-rahman1]

Looks good. Thanks for adding in the get_inference_url for FPGA.
Approving now.
The only thing to check is the Frontend @cmatthews20 using the FPGA inference url to do the chat. If there are any issues, we should be able to update the config-engine repo. (Might need to do JSON)