AllerScan Nearly 1 in 11 people have food allergies in the United States of America. For those in that group, their allergies control the food they buy, avoid, and consume. Allergens also take away precious time, with 3 to 5 minutes per product on average being used to decide on a purchase based on the allergens from the label. Our app AllerScan is a solution that gives users the comfort of safety in their food purchases, along with giving back the time wasted by staring at food labels.
AllerScan is an Android mobile app that uses food product barcodes to check food labels and informs the user whether a product is safe to consume. AllerScan achieves this by using the mobile phone's built-in camera to scan barcodes, with manual barcode insertion available as needed. When creating a profile, the user will be asked to insert their allergens, which, when a barcode is scanned, will be cross-checked against the food label's listed ingredients. All of this happens within seconds, saving the user minutes of reading and eye strain with a quick response prioritizing user safety above all else.
But what makes AllerScan special compared to competitors? AllerScan does not have any form of data collection or "middle-man" interaction. This means that the user's data is truly theirs, with everything being stored and processed locally on device. This gives the user the ability to have full control over their data without the possibility of leaks, data selling, or unwanted sharing of user info. AllerScan offers the same abilities as its competitors without intruding on user privacy.
AllerScan works by utilizing a local database as a persistent library under SQLite, also known as the Room functionality in Android's Kotlin language. This is used to store the required info to cross-check allergens with the info listed on the Open Food Facts website, which we pull data from using their public API.
Below is a visual demonstration of how AllerScan functions between different layers of the scanning process.
The architectural design:
The Frontend: The user is prompted to create their profile, which allows AllerScan to compare allergens when scanning food labels. Once the user has created their profile, they can start scanning barcodes. The user can go to the scan page and enable camera permissions to allow AllerScan to scan and interpret barcodes. The end result will be displayed at the bottom of the screen and will be stored in the history page.
The frontend display seen by the user:
The Backend: In the backend, AllerScan uses the Room functionality to create a local database, storing the user profile (including allergens and the user's name) and the history of the user's scans. When the user scans a barcode, AllerScan sends that barcode to the request handler, where it calls the Open Food Facts API to gather information on that specific food product. Product name and ingredients are both pulled from the website, and the ingredients list will be cross-checked with the user's listed allergens from their previously made profile page. Any overlapping ingredients will cause AllerScan to flag the product as unsafe to eat, adding the product and its scan result to the history in Room for display on the user's history page.
Images demonstrating the allergen properties in the database:
First Second Third
The Welcome Screen is split into three different screens which are traversed through by a "NEXT" button in the bottom right of each slide, or has the option of skipping the tutorial activity by pressing "Skip" in the bottom left.
The first screen introduces the user to the scanning element of the app, stating "Scan Products - Quickly scan barcodes to check for allergens in food products" in a short block of text. There is also an attached icon representing scanning above this text. The first android symbol at the bottom is also illuminated, representing it as the first page in the welcome activity. The user is expected to press the "NEXT" button in order to move to the second page.
The second screen mentions the efficiency/quickness of the app, stating "Instant Results - Get immediate allergen information from our comprehensive database" in a short block of text. There is also an attached icon representing the database above this text. The second android symbol at the bottom is also illuminated, representing it as the second page in the welcome activity. The user is expected to press the "NEXT" button in order to move to the third page.
The third screen introduces the user to the profile element of the app, stating "Manage Your Profile - Set up your allergen profile to get personalized warnings" in a short block of text. There is also an attached icon representing profiles above this text. The third android symbol at the bottom is also illuminated, representing it as the third page in the welcome activity. The user is expected to press the "NEXT" button in order to move to the actual app.
Once the user has gone through the welcome screen one time, the app stores this data and will not prompt the user beyond its first time being opened. This is effective so the user can have a seamless/satisfying experience whenever they open the app in the future.
The backend and camera layers of AllerScan are responsible for turning a raw barcode captured via your device's camera into a safe/unsafe decision that is then stored and shown across the app.
On the camera side, AllerScan uses Android's CameraX API inside a dedicated scan fragment. When the user navigates to the Scan tab, the fragment will request camera permission if needed. Once the camera is working, in parallel, an image analysis is attatched and runs on each frame. As soon as a valid barcode is detected, the analyzer will stop scanning and passes the numeric barcode value to the backend layer.
The backend functions as such:
- Network/API client - A request handler builds a URL to the public Open Food Facts API using the scanned barcode, performing an HTTP GET request and extracting the product name plus a normalized list of ingredients.
- Pre-processing scanned data - Making sure that the barcode from CameraX is validated (trimming, checking format, ignoring bad reads), check to make sure there isn't duplicate reads.
- Recorded entries - After a scan, a data entry is created and saved via Room. Each record includes at least a product name, barcode, and whether it was safe/unsafe, and which allergens were triggered (if any). This will later be utilized in our History page.
The History Screen
The History screen displays previously scanned products from both the camera-based scanning feature and the manual barcode entry option. The History screen utilizes SQL for accessing the Room database to access previously scanned products. A RecyclerView was utilized to display the products as it allows users to scroll through products off the screen. The search feature allows users to find previously scanned products by barcode, as the searched product will appear at the top of the list.
Demonstration of Product Properties in Room Database
The Manual Barcode Insertion Feature
The manual barcode insert feature was an additional protocol taken to ensure the user can check an item in case the user experienced issues with the camera. Similar to the camera, the barcode insertion feature retrieves the barcode, sends API request, analyzes ingredients, makes a safety verdict and saves the product information in the database.
After further evaluation of our primary researched APIs, OpenFoodFacts was found to be the most reliable and completely free to use. Other previously brainstormed APIs included, LookupGoUPC, UPCindex and UPCitembd. All classes and functions for API requests and handling were written in Kotlin. After scanning or entering a barcode, an API request is sent to fetch the product name and ingredients of the product. The ingredients result is then cleaned of brackets, parentheses, slashes, and commas. Certain unnecessary terms such as “and”, “high” and “unbleached” are thrown out as well. This preserves the necessary food ingredient information. This resulting parsed ingredients are then checked for the users allergens. The Allergen Checker class was created to check the parsed list of ingredients meanwhile ensuring entered multi-word custom allergens were checked correctly. The ingredient parsing and allergen checking was tested against multiple items to provide valid verdicts.
Overview Edit Custom Allergen
The first page the user will see is the "Overview" page. It displays the user's name, along with their currently active allergens. It pulls the active allergens from the local database when the page is opened so that it can update in real-time after editing a profile. To edit the profile, the user can press the pen icon in the top right.
The second page is the edit page, where the user can change their listed name, use the checkboxes to easily add major allergens (those seen in the big bold CONTAINS sections of labels), choose to add custom allergens, or delete all user data from the app at once. When entering the edit page, the activated checkboxes will be pre-selected based on their activation status so the user can change their allergens as needed without having to recreate their whole profile. Pressing on a checkbox will change the activation status of that allergen to whatever it currently isn't in the database (so true to false, false to true). Pressing the delete button in the bottom left will set all activation values in the database to false while also clearing out the user's name. The save button on the top right (where the pen icon was) will bring the user back to the previous page, saving all changes.
The last page is where the user can add custom allergens after pressing the "ADD CUSTOM ALLERGEN" button while editing their profile. It works exactly as one would expect. The user types in the allergen in the top bar, and when they press "ADD", the allergen will be added to the database with its activation status set to true. The allergen will also be seen below this bar, showing the user all their current custom allergens, along with giving them the ability to delete custom allergens as needed by pressing the "X" to the right of the allergen. Pressing the "X" will remove the allergen from the screen and set its activation value to false.
Once the profile page is completed and the user has declared their allergens, the database will now contain all the user info needed to cross-check against food labels, enabling the ability to approve or deny the safety of scanned food products.
To ensure compatibility with many devices, and as the only developer using an Android device, I was in charge of ensuring the user interface was both clean and compatible with multiple devices. I was also in charge of testing AllerScan outside of the developer environment due to having a developer-enabled phone.
General UI: Throughout the entire project, I adjusted and changed UI elements as they were created to ensure they were uniform across pages while also adjusting the calculations for object positioning to ensure device compatibility. The calculations are now mostly reliant on the positionings of the screen boundaries, enabling UI elements to shift automatically based on screen size. This way, the app looks the same for all devices, has no UI element being overlapped by another, and shares a uniform theme across all pages, creating a seamless user experience.
Testing: Testing each branch and PR with my own device made it possible to test certain functionality, like camera barcode scanning, user data persistence, UI element positioning, and real-world use in stores when shopping, all things that cannot be done in the emulator. This was done to ensure that all functions of the app were maintained during development, with none being silently broken due to a PR. Using the app allowed me to create feedback that otherwise could not be seen in the emulator due to its constraints, allowing team members and me to take that info and make changes based on it. Testing in my daily life has drastically changed the project for the better, as I ran into scenarios that none of us thought of during development.
There are multiple ways to install the application:
NOTE: AllerScan currently does not support dark mode. Please use light mode when running the app.
ISSUE + WORK AROUND: Some devices don't ask for camera permission when entering the scan page. We are looking into it, but for now, you can manually enable camera permission for AllerScan in the app settings built into your phone.
- Ensure you have a device running Android or an emulator like BlueStacks. If you are using an Android device, we recommend following these steps directly from the device rather than from a computer.
- Go to the releases page of the AllerScan GitHub repository
- Download "AllerScan.apk"
- Import the APK into your emulator and install, or if on a device running Android, open the downloads/files application and run the APK.
- You may be prompted to enable the downloads of unsigned software and a Google Play Security scan of the application. Enable/Accept both of these and allow the scan to run if needed.
- The AllerScan app should now be installed on your device and ready for use!
- The application can also be installed via Android Studio, which can be installed here.
- In Android Studio, you will be prompted to download the Android SDK, which you should accept.
- Once in Android Studio, pull from main and press "Sync with Gradle" after pressing the top left button, a directory warning should pop up letting you know that it will search for the default SDK location, press OK to continue.
- On the top right, you should see a hammer icon. This will build the project. Press the hammer icon.
- Once the project is done building, you can do either of two things.
Option 1. Press the green play button at the top of the screen, which will run the app in the emulator.
Option 2. Plug in an Android device with developer options enabled and USB debugging enabled. If you don't know how to do that, then look to Google's Developer Guides, as they are quite detailed. Then press the green button to install the app locally on your Android phone.
- Enjoy AllerScan!
Rami Nasser (rnass2)
Backend:
- Store user information locally on device using Room database
- Tie frontend actions to backend updates
- Active/deactivate user allergens
Frontend:
- UI for adding allergens to a user profile
- Generalize UI and ensure cross-device uniformity
- Testing UI to backend functionality
Anna Koziol (akozi)
Backend:
- Barcode lookup, fetching ingredients from the API
- Identifying how to separate ingredients to check for allergens
- Confirming if allergen was found or not
- Storing product information with Room Database
Frontend:
- UI for screens (history screen and camera manual insertion feature)
- Ensure cross-device uniformity of UI screens and features
- Redirecting the screens from one another
Aadam Hitchens (ahitc2)
Backend:
- Clean/Optimize data from QR code scan
- Setup baseline database to expand upon
Frontend:
- Get camera functioning
Luke Jacob (ljaco5)
Frontend:
- Make Welcome Page
