duck-explorer is a full stack project which uses React and Express to fetch duck images and analyze them with the Open AI vision model API .
The Duck Explorer program looks like this in a localhost development environment:
In order to run this project you will need 3 applications installed on your machine
Install Microsoft's Visual Studio Code IDE from this website
Install Node JS from this website onto your machine to be able to run the start commands for the frontend and backend
-Install Git from this website onto your machine to be able to run the start commands for the frontend and backend.
-On this repository site (github.com), create an account that can be used to sign in in later steps.
One you have installed the necessary 3 applications, follow these steps to run the project.
Clone this repository down into your workspace
Run npm run dev in the /frontend directory of a terminal window
Create a .env file in the backend and enter your Open AI API key for usage of the gpt-4-vision model. Have your key stored in a variable called OPENAI_API_KEY.
Run npm start in the /backend directory of a terminal window
After these commands, the backend server should start and the site https://localhost:8081 should be prompted in the terminal. If the server is correctly configured, the terminal should say "Server listening..."
If this is successful, return to the frontend at http://localhost:5173 (or any port number suggested in the terminal response to starting the frontend)
- Instant fetching - when the page loads an initial image url is fetched
- New Duck - button which fetches new image url from random duck API
- Analyze - button which sends image data and prompt to gpt vision model for description
The frontend was developed using React with TypeScript. The project was bootstrapped with Vite. The frontend uses CSS with no pre-processing for custom styling as well as Axios for connection to the backend API.
The backend server was developed using Express JS with Node. The server includes 2 endpoints for data collection and manipulation
- "/random" for fetching a random duck image url from the Random-d.uk API
- Uses GET request
- "/chat" for sending an inputted image url to gpt-4-vision-preview model for analysis with a custom prompt and token amount
- Uses POST request
- Message truncation
- After /chat fetches a response from the gpt model, this function cuts of remaining words that are not a complete sentence
- What was the context for this project?
After having a fascination with different types of rubber ducks, I wanted to find a way to document them or create a project revolving around them. My initial ideas were about a rubber duck encyclopedia but after stumbling across numerous API's which send random duck images I found this idea interesting. Instead of loading a page full of API fetched images, I wanted to take this a step forward by describing the images, however, I didn't want to hard code descriptions for images. This led me to choosing an LLM for this task and I decided to go with the gpt-4-vision-preview model as it was best at giving responses in my testing as compared to the Google image model options.
- What was built?
The final project ended as a full stack web application using NodeJs and Express for the backend and React for the frontend. The backend includes a RESTful API which connects to the frontend to ultimately display duck images fetched from an API and descriptions of those images using the gpt-4-vision model on the frontend.