Skip to content

quickstart.md

Latest commit

 

History

History
90 lines (71 loc) · 2.09 KB

quickstart.md

File metadata and controls

90 lines (71 loc) · 2.09 KB

Quick Start: Shimmy in 30 Seconds

1. Download

# Linux/macOS
curl -L https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy -o shimmy
chmod +x shimmy

# Windows
curl -L https://github.com/Michael-A-Kuykendall/shimmy/releases/latest/download/shimmy.exe -o shimmy.exe

2. Get a Model

Place any .gguf file in one of these locations:

  • ./models/your-model.gguf
  • Set SHIMMY_BASE_GGUF=/path/to/your-model.gguf
  • Or just put it in ~/Downloads/ - Shimmy will find it

Don't have a model? Try microsoft/Phi-3-mini-4k-instruct-gguf

3. Start Shimmy

./shimmy serve

That's it! Shimmy is now running on http://localhost:11435

4. Connect Your Tools

VSCode Copilot:

// settings.json
{
  "github.copilot.advanced": {
    "serverUrl": "http://localhost:11435"
  }
}

Continue.dev:

{
  "models": [{
    "title": "Local Shimmy",
    "provider": "openai",
    "model": "your-model-name", 
    "apiBase": "http://localhost:11435/v1"
  }]
}

Cursor: Set custom endpoint to http://localhost:11435

5. Test It

# List available models
./shimmy list

# Test generation
./shimmy generate --name your-model --prompt "Hello!" --max-tokens 10

# Or use curl
curl -X POST http://localhost:11435/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 10
  }'

Troubleshooting

No models found?

  • Make sure your .gguf file is in ./models/ or set SHIMMY_BASE_GGUF
  • Run ./shimmy discover to see what Shimmy can find

Port already in use?

./shimmy serve --bind 127.0.0.1:11436

Need help?

Next: Check out integrations for more examples!