Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions evi/evi-react-native/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

This project features a sample implementation of Hume's [Empathic Voice Interface (EVI)](https://dev.hume.ai/docs/empathic-voice-interface-evi/overview) using React Native.

**Targets:** The example supports iOS and web (Android support coming soon!)
**Targets:** The example supports iOS, Android and web

## Setup Instructions

Expand Down Expand Up @@ -36,24 +36,34 @@ This project features a sample implementation of Hume's [Empathic Voice Interfac
```

4. Prebuild, to include the `modules/audio` native module:
- ios:
```shell
npx expo prebuild --platform ios
```
- android:
```shell
npx expo prebuild --platform android
```


## Usage

Run the dev server:

- ios:
```shell
npm run ios
```
- android:
```shell
npm run android
```

## 📝 Notes
* **Echo cancellation**. Echo cancellation is important for a good user experience using EVI. Without echo cancellation, EVI will detect its own speech as user interruptions, and will cut itself off and become incoherent.
* Echo cancellation doesn't seem to work using the iOS simulator when forwarding audio from the host.
* If you need to test using a simulator or emulator, or in an environment where echo cancellation is not provided, use headphones, or enable the mute button while EVI is speaking.

* Because community libraries like `expo-av` module do not support streaming audio recording or echo cancellation, it is necessary to write native code to interface with the microphone and speaker. The example app includes a `modules/audio` with a very simple audio interface written in Swift. It works in simple scenarios, but will not handle scenarios like selecting between multiple possible audio devices, gracefully handling when the user switches audio devices mid-conversation, handling audio interruptions like incoming phone calls, "ducking" audio from other apps that might be playing, etc. You should use the provided module as a starting point and extend it to meet your app's unique requirements.
* Because community libraries like `expo-av` module do not support streaming audio recording or echo cancellation, it is necessary to write native code to interface with the microphone and speaker. The example app includes a `modules/audio` with a very simple audio interface written in Swift for ios and Kotlin for android. It works in simple scenarios, but will not handle scenarios like selecting between multiple possible audio devices, gracefully handling when the user switches audio devices mid-conversation, handling audio interruptions like incoming phone calls, "ducking" audio from other apps that might be playing, etc. You should use the provided module as a starting point and extend it to meet your app's unique requirements.

* This example uses Expo 52, on which ["The New Architecture" is the default](https://docs.expo.dev/guides/new-architecture/). If you use an older version of Expo, you may need to adapt the example to get the native module to work in your app.

52 changes: 52 additions & 0 deletions evi/evi-react-native/modules/audio/android/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
// apply plugin: "com.facebook.react"
apply plugin: 'com.android.library'
apply plugin: 'kotlin-android'

android {
compileSdkVersion 33

defaultConfig {
minSdkVersion 21
targetSdkVersion 33
versionCode 1
versionName "1.0"
}

buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}

lintOptions {
abortOnError false
}

compileOptions {
sourceCompatibility JavaVersion.VERSION_17
targetCompatibility JavaVersion.VERSION_17
}

kotlinOptions {
jvmTarget = '17'
}

buildFeatures {
buildConfig true
}
}

repositories {
mavenCentral()
google()
}

dependencies {
implementation 'org.jetbrains.kotlin:kotlin-stdlib:1.7.10'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.6.4'

// Expo modules
implementation project(':expo-modules-core')
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<manifest package="expo.modules.audio" xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
</manifest>
Loading