Skip to content

Add Secret AI as a local app #1690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

secret-ai-dev
Copy link

Hey there! 👋

This PR adds Secret AI as a local app option for running models on Hugging Face. We're excited to contribute to this amazing
project! 🚀

About Secret AI

Secret AI is a cross-platform application that brings the power of Large Language Models to your devices locally. Whether you're on
macOS, iOS, or Android, Secret AI makes it easy to run AI models without relying on cloud services.

Key Features

  • Multi-platform support: Works seamlessly across macOS, iOS, and Android
  • Versatile AI capabilities:
    • Text generation (chat, completion, creative writing)
    • Multimodal support (image + text → text generation)
  • GPU Acceleration:
    • Mobile GPU support for lightning-fast inference
    • Optimized for on-device GPU computing
    • Significantly faster than CPU-only inference
  • Multiple model formats:
    • GGUF models (llama.cpp compatible) - optimized for CPU inference
    • MLX models (Apple Silicon optimized) - blazing fast on M-series chips
    • MNN models (Mobile Neural Network) - lightweight and GPU-accelerated on mobile
  • Easy to get started:
  • Website: https://secretai.io

Performance Highlights

Secret AI leverages mobile GPU capabilities to deliver exceptional performance:

  • GPU-accelerated inference on compatible devices
  • Optimized memory management for mobile constraints

Implementation Details

  1. Added isMnnModel() helper function to intelligently detect MNN models:

    • Checks if model is from taobao-mnn organization
    • Recognizes models with -MNN suffix (following naming conventions)
    • Future-proofed with mnn tag support
  2. Deeplink integration: secret-ai://pages/huggingface/repos/{model_id}?format={format}

    • Smart format detection based on model type
    • Seamless one-click experience from Hugging Face to Secret AI
    • Optional file parameter support for specific model variants
  3. Display logic: Shows for text-generation models supporting GGUF, MLX, or MNN formats

✅ Testing

We've thoroughly tested the implementation:

  • ✅ Deeplink URL construction works correctly for all model types
  • ✅ GGUF models tested (e.g., unsloth/Qwen3-1.7B-GGUF)
  • ✅ MLX models tested with Apple Silicon optimization
  • ✅ MNN models tested with GPU acceleration (e.g., taobao-mnn/deepseek-llm-7b-chat-MNN)
  • ✅ All existing tests pass without issues

🙏 Thank You!

Thanks for considering this PR! We're looking forward to making Hugging Face models more accessible to users across different
platforms, with the added benefit of GPU acceleration for enhanced performance. If you have any questions or need any adjustments,
please don't hesitate to let us know. We're happy to help! 😊

Secret AI is a cross-platform (macOS, Android, iOS) app for running LLMs locally.
It supports GGUF, MLX, and MNN model formats.

- Supports MNN models from taobao-mnn organization
- Available on App Store and Google Play
- Website: https://secretai.io
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant