AI Companion for Local Machines

A Gradio-based AI companion app leveraging generative AI models (LLMs, Diffusion models, etc.) running in a local environment.

English 한국어 日本語 简体中文 繁體中文

Overview

The AI Companion offers chatbot services along with AI-powered generation of images, videos, and audio content. Utilizing user personas and character personas, the app introduces persona-based chatbot interactions, transforming AI beyond mere tools into genuine friends and partners. Users can engage in conversations, collaborative tasks, and recreational activities with AI characters.

While the user persona mainly targets general users, the image, video, music, and audio generation services also cater to professional users.

Side Project

LLM Backend Image Backend Langchain Integrator

Main Features

Chatbot

Interact with AI using Large Language Models (LLMs).

Supported Models

• API

Provider	Model Name
OpenAI	gpt-3.5-turbo gpt-4 gpt-4-turbo gpt-4o-mini gpt-4o gpt-4.1 gpt-4.1-mini gpt-4.1-nano
Anthropic	claude-3-haiku-20240307 claude-3-sonnet-20240229 claude-3-opus-latest claude-3-5-sonnet-latest claude-3-5-haiku-latest claude-3-7-sonnet-latest
Google GenAI	gemini-1.5-flash gemini-1.5-flash-8b gemini-1.5-pro gemini-2.0-flash

• Local: Transformers, GGUF, MLX (Apple Silicon Macs only)
  Pre-downloaded Transformer models are available via the Download Center.

Provider	Model Name
meta-llama	meta-llama/Llama-3.1-8B meta-llama/Llama-3.1-8B-Instruct meta-llama/Llama-3.2-11B-Vision meta-llama/Llama-3.2-11B-Vision-Instruct
google	google/gemma-2-9b google/gemma-2-9b-it google/gemma-3-12b-pt google/gemma-3-12b-it
Qwen	Qwen/Qwen2.5-7B Qwen/Qwen2.5-7B-Instruct Qwen/Qwen2.5-14B Qwen/Qwen2.5-14B-Instruct
mistralai	mistralai/Mistral-7B-v0.3 mistralai/Mistral-7B-Instruct-v0.3 mistralai/Mistral-Small-24B-Base-2501 mistralai/Mistral-Small-24B-Instruct-2501

Character Persona Settings

• System Prompt(System Message): Instructions that define the chatbot’s role or behavior based on user needs. (Default: You are an useful AI assistant.) Presets or characters automatically apply predefined prompts.
• Character: Choose your AI conversation partner.
• Preset: Applies user-defined system prompts. Presets automatically update when switching characters.

Hyperparameter Settings
If you aren’t familiar with hyperparameters, it’s recommended to stick to the default values.

• Seed: Initial random number seed used in generation (default: 42).
• Temperature: Controls creativity/randomness of responses. Higher values yield more creative answers; lower values result in deterministic responses (default: 0.6).
• Top K: Limits the number of word options, choosing from the most probable words for high-quality outputs (default: 20).
• Top P: Adjusts randomness by setting a probability threshold, sampling from top tokens until the threshold is reached (default: 0.9).
• Repetition Penalty: Controls repetition of words; higher values reduce repetition (default: 1.1).

Image Generation

Create images using models like Stable Diffusion and Flux. Images are generated via ComfyUI backend.

Supported Models

• API Currently, API image generation models have limited support.

Developer	Model repo id
OpenAI	dall-e-3
Google GenAI	imagen-3.0-generate-002

• Local: Diffusers, Checkpoints
  • Diffusers: Currently, Diffusers models can be scanned and selected but cannot yet generate images (feature coming soon).
  • Checkpoints: Image generation via ComfyUI. Model files should be placed in models/checkpoints within the ComfyUI directory.

Supported Local Base Models:

• Stable Diffusion 1.5
• Stable Diffusion 2.x
• Stable Diffusion XL 1.0
• Stable Diffusion 3 Medium
• FLUX.1 Schnell
• FLUX.1 Dev
• Stable Diffusion 3.5 Large
• Stable Diffusion 3.5 Medium
• Illustrious XL 1.0

Detailed options

• LoRA: Supports up to 10 LoRA models per base model. Compatible LoRAs must be applied.
• VAE: User-defined VAE. Defaults to embedded VAE within the Checkpoints.
• Embedding: Use via syntax like embedding:name.
• ControlNet: Interface not yet implemented; coming soon.
• Refiner: For Stable Diffusion XL 1.0, select the start step for refiner sampling.

Generation Options

• Positive Prompt: Generate images based on the input text.
• Negative Prompt: Exclude unwanted features from generated images.
• Width, Height: Adjust image dimensions.
• Recommended Resolutions

Base Model	Recommended Resolution
Stable Diffusion 1.5 Stable Diffusion 2.x	512x512 512x768 768x512
Stable Diffusion XL 1.0 Stable Diffusion 3 Medium FLUX.1 Schnell FLUX.1 Dev Stable Diffusion 3.5 Large Stable Diffusion 3.5 Medium	1024x1024 896x1152 1152x896 832x1216 1216x832 768x1344 1344x768 640x1536 1536x640
Illustrious XL 1.0	1536x1536 1248x1824 1824x1248

• Generation Steps: Higher values result in longer generation times but potentially better quality.
• Denoise Strength: Adjusts the level of noise.

Advanced Settings

• Sampler: Different samplers yield varying outputs.
• Scheduler: Affects output even with the same sampler/prompt.
• CFG Scale: Higher CFG values closely follow prompts; lower values are more creative.
• Seed: Initial random number seed.
• Clip Skip: Skips certain steps during image creation.
• Batch Size: Number of images generated per run.

Image to Image
Modify existing images or use Inpaint for selective alterations.

Storyteller

Utilizes LLMs optimized for text creation, especially for storytelling. (Currently under development.)

Supported Models
Same as Chatbot.

Hyperparameters
Same as Chatbot.
If you are not familiar with how hyperparameters work, we recommend leaving them at the default values.

Video Generation

Coming Soon

Audio Generation

Coming Soon

Translator

Multi-language translation using translation models. Supports text extraction from uploaded images or PDF files for translation.

Installation

Clone a repository

# Clone a repository only
git clone https://github.com/bean980310/ai-companion.git
# Clone a repository with submodules
git clone --recursive https://github.com/bean980310/ai-companion.git
# Init and Update a submodules
git submodule init
git submodule update

• Virtual Environment Setup

# conda (Recommended!)
# Python 3.10
conda create -n ai-companion python=3.10
# Python 3.11
conda create -n ai-companion python=3.11
# Python 3.12
conda create -n ai-companion python=3.12
conda activate ai-companion
# Other virtual environments
cd ai-companion
# venv
python3 -m venv venv
# uv
uv venv --python 3.10 
uv venv --python 3.11
uv venv --python 3.12
# MacOS/Linux/Windows WSL2
source venv/bin/activate # venv
source .venv/bin/activate # uv
# Windows
.\venv\Scripts\activate.bat # venv
.\.venv\Scripts\activate.bat # uv

Install dependencies

• Windows

# on Windows
.\installer_windows_amd64.bat

# on Windows (Powershell)
.\installer_windows_amd64.ps1

# on Windows Subsystem for Linux 2
bash installer_windows_amd64_wsl2.sh
# or
./installer_windows_amd64_wsl2.sh

• macOS(Apple Silicon Mac)

zsh installer_macos_arm64.sh
# or
./installer_macos_arm64.sh

• Linux

bash installer_linux_amd64_cuda.sh
# or
./installer_linux_amd64_cuda.sh

• MeloTTS(Optional)

pip install git+https://github.com/myshell-ai/MeloTTS.git --no-deps

Installing xformers on Mac

brew update
brew install gcc cmake llvm@18 libomp

export PATH="/opt/homebrew/opt/llvm@18/bin:$PATH"

export LDFLAGS="-L/opt/homebrew/opt/libomp/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libomp/include"
export CXXFLAGS="-Xpreprocessor -fopenmp"

export CC=/opt/homebrew/opt/llvm@18/bin/clang
export CXX=/opt/homebrew/opt/llvm@18/bin/clang++
export LDFLAGS="-L/opt/homebrew/opt/llvm@18/lib"
export CPPFLAGS="-I/opt/homebrew/opt/llvm@18/include"

pip install --no-build-isolation --upgrade xformers

Run

python app.py

Support for Intel Macs

AI Companion for Local Machines no longer supports Intel CPU-based Macs.
If you are using an Intel CPU-based Macs, we recommend that you consider migrating to an Apple Silicon Based Macs or a Windows PC or Linux machine with an Nvidia GPU environment. If you have difficulty migrating from an Intel CPU-based Macs, you can use a companion application that supports Intel CPU-based Macs instead.

Guidance for Windows Systems with GPUs That Do Not Support CUDA 12.4 or Higher

AI Companion for Local Machines provides support for dependency installation to ensure compatibility between PyTorch and xformers.
On Windows, versions of CUDA lower than 12.4 cannot install the latest xformers. Therefore, for CUDA 11.8 and 12.1 environments, the PyTorch version is fixed at 2.4.0 to ensure compatibility with the latest supported xformers version, xformers 0.0.27.post2.
Please note that future support for Windows systems with CUDA versions below 12.4 will be discontinued.
If you are using Windows and have a CUDA version below 12.4, we recommend upgrading to CUDA 12.4 or higher and reinstalling PyTorch and xformers accordingly. If your GPU does not support CUDA 12.4 or higher, we recommend upgrading to a compatible GPU or considering migration to a newer PC.

Support for Windows

AI Companion for Local Machines is optimized for UNIX/Linux kernel-based operating systems. While it can be used on Windows, GPU acceleration is unavailable when running directly on Windows. To properly use AI Companion for Local Machines on Windows, we recommend using it within a WSL2 environment.