[FirebaseAI] Add Live Audio screen

[daymxn]

This adds the LiveAudioScreen to the AI sample app. Since the Live API was released with 12.4.0, this adds a sample app showcasing how you can do audio to audio communication with a Gemini model.

The new screen also supports the following:

• Output audio transcripts
• Function calling (via changing the background color)
• Proper interruption propagation in transcripts
• Minimal support for switching audio devices during a live session
• AEC with speakers
• Automatic disabling of AEC when using bluetooth devices (eg; airpods)
• Documentation for everything added, to assist folks in parsing the source

[gemini-code-assist]

Summary of Changes

Hello @daymxn, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates a new 'Live Audio' feature into the Firebase AI example application. It provides a practical demonstration of how to leverage the Live API for interactive audio experiences with a Gemini model, encompassing real-time transcription, dynamic UI updates via function calls, and sophisticated audio management for various device configurations.

Highlights

• New Live Audio Screen: Introduced a new LiveAudioScreen to the AI sample application, demonstrating real-time audio-to-audio communication with a Gemini model using the Live API (version 12.4.0+).
• Enhanced Audio Interaction: The new screen supports output audio transcripts, function calling (e.g., changing background color), proper interruption propagation in transcripts, and minimal support for switching audio devices during a live session.
• Advanced Audio Processing: Implemented Acoustic Echo Cancellation (AEC) with speakers and automatic disabling of AEC when using Bluetooth devices (like AirPods) to optimize audio quality.
• Comprehensive Documentation: All newly added features and components include documentation to assist developers in understanding and utilizing the source code effectively.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a LiveAudioScreen to the AI sample app, demonstrating audio-to-audio communication with a Gemini model. The implementation is comprehensive, covering audio capture, playback, transcription, and function calling. The code is well-structured, particularly the audio handling logic in AudioController and the typewriter effect for transcripts in TranscriptViewModel. My review focuses on enhancing the robustness and thread safety of the new features. I've provided suggestions to replace fatalError calls with more graceful error handling to prevent application crashes and to convert the AudioController class to an actor to prevent potential race conditions. These changes will improve the stability of the sample app and make it a better reference for production-level code.