Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 47 additions & 3 deletions .github/workflows/skyeye.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,26 @@ jobs:
uses: ./.github/actions/setup
- name: Test
run: make test
integration-test:
name: Integration Test (Advisory)
needs: [lint, test]
runs-on: ubuntu-latest
timeout-minutes: 60
continue-on-error: true
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup
uses: ./.github/actions/setup
- name: Cache AI models
uses: actions/cache@v4
with:
path: models
key: ai-models-${{ hashFiles('pkg/recognizer/parakeet/model/version.go', 'pkg/synthesizer/pocket/model/version.go') }}
- name: Download models
run: CGO_ENABLED=0 go run ./cmd/download-models --dir models
- name: Integration Test
run: make integration-test
build-linux-amd64:
name: Build on Linux AMD64
runs-on: ubuntu-latest
Expand All @@ -57,9 +77,17 @@ jobs:
cp LICENSE dist/skyeye-linux-amd64/LICENSE
cp config.yaml dist/skyeye-linux-amd64/config.yaml
cp docs/*.md dist/skyeye-linux-amd64/docs/
- name: Cache AI models
if: startsWith(github.ref, 'refs/tags/')
uses: actions/cache@v4
with:
path: models
key: ai-models-${{ hashFiles('pkg/recognizer/parakeet/model/version.go', 'pkg/synthesizer/pocket/model/version.go') }}
- name: Download models
if: startsWith(github.ref, 'refs/tags/')
run: CGO_ENABLED=0 go run ./cmd/download-models --dir dist/skyeye-linux-amd64/models/parakeet
run: |
CGO_ENABLED=0 go run ./cmd/download-models --dir models
cp -r models dist/skyeye-linux-amd64/models
- name: Create dist archive
shell: bash
run: tar -czf dist/skyeye-linux-amd64.tar.gz -C dist skyeye-linux-amd64
Expand Down Expand Up @@ -96,9 +124,17 @@ jobs:
cp LICENSE dist/skyeye-macos-arm64/LICENSE
cp config.yaml dist/skyeye-macos-arm64/config.yaml
cp docs/*.md dist/skyeye-macos-arm64/docs/
- name: Cache AI models
if: startsWith(github.ref, 'refs/tags/')
uses: actions/cache@v4
with:
path: models
key: ai-models-${{ hashFiles('pkg/recognizer/parakeet/model/version.go', 'pkg/synthesizer/pocket/model/version.go') }}
- name: Download models
if: startsWith(github.ref, 'refs/tags/')
run: CGO_ENABLED=0 go run ./cmd/download-models --dir dist/skyeye-macos-arm64/models/parakeet
run: |
CGO_ENABLED=0 go run ./cmd/download-models --dir models
cp -r models dist/skyeye-macos-arm64/models
- name: Create dist archive
shell: bash
run: tar -czf dist/skyeye-macos-arm64.tar.gz -C dist skyeye-macos-arm64
Expand Down Expand Up @@ -158,10 +194,18 @@ jobs:
cp init/winsw/skyeye-service.yml dist/skyeye-windows-amd64/skyeye-service.yml
cp winsw.exe dist/skyeye-windows-amd64/skyeye-scaler-service.exe
cp init/winsw/skyeye-scaler-service.yml dist/skyeye-windows-amd64/skyeye-scaler-service.yml
- name: Cache AI models
if: startsWith(github.ref, 'refs/tags/')
uses: actions/cache@v4
with:
path: models
key: ai-models-${{ hashFiles('pkg/recognizer/parakeet/model/version.go', 'pkg/synthesizer/pocket/model/version.go') }}
- name: Download models
if: startsWith(github.ref, 'refs/tags/')
shell: msys2 {0}
run: CGO_ENABLED=0 go run ./cmd/download-models --dir dist/skyeye-windows-amd64/models/parakeet
run: |
CGO_ENABLED=0 go run ./cmd/download-models --dir models
cp -r models dist/skyeye-windows-amd64/models
- name: Create dist archive
shell: msys2 {0}
run: |
Expand Down
21 changes: 11 additions & 10 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@

SkyEye is an AI-powered GCI bot for DCS World that uses Parakeet TDT speech recognition (via sherpa-onnx), Tacview telemetry, and TTS to replace in-game AWACS with natural language command processing following real-world aviation brevity codes.

**Stack:** Go 1.26 + CGO, sherpa-onnx (Parakeet TDT), Piper TTS (Windows/Linux), macOS Speech Synthesis, Tacview ACMI, SRS protocol
**Stack:** Go 1.26 + CGO, sherpa-onnx (Parakeet TDT + Pocket TTS), Tacview ACMI, SRS protocol

## Platform Support

| Platform | Arch | Status | TTS | Linking | Runtime Deps |
|----------|------|--------|-----|---------|--------------|
| **Windows** | AMD64 | ✅ | Piper (embedded) | Static | None - fully portable exe |
| **Linux** | AMD64 | ✅ | Piper (embedded) | Dynamic opus/soxr | libopus0, libsoxr0 |
| **macOS** | ARM64 | ✅ | System (Neural Engine) | Dynamic opus/soxr | Homebrew opus, libsoxr |
| **Windows** | AMD64 | ✅ | Pocket TTS (sherpa-onnx) | Static | None - fully portable exe |
| **Linux** | AMD64 | ✅ | Pocket TTS (sherpa-onnx) | Dynamic opus/soxr | libopus0, libsoxr0 |
| **macOS** | ARM64 | ✅ | Pocket TTS (sherpa-onnx) | Dynamic opus/soxr | Homebrew opus, libsoxr |
| macOS Intel | AMD64 | ❌ | - | - | No test hardware |

**Key Differences:**
- **Windows:** MUST build in MSYS2 UCRT64 (not cmd/PowerShell), static linking, portable binary
- **Linux:** Standard Unix build, requires runtime libraries, good for containers
- **macOS:** Uses Apple Clang (system compiler), `--use-system-voice` flag available
- **macOS:** Uses Apple Clang (system compiler)
- **Cross-compilation:** Not supported - must build on target platform

## Critical: Use Make, Not Go Commands
Expand Down Expand Up @@ -52,18 +52,19 @@ pkg/ - Public APIs
recognizer/ - Speech recognition (Parakeet TDT via sherpa-onnx)
recognizer/model/ - Embedded model files (encoder/decoder/joiner ONNX + tokens.txt)
simpleradio/ - SRS protocol client
synthesizer/speakers/ - Platform-specific TTS (macos.go, piper.go)
synthesizer/pocket/ - Pocket TTS speaker (sherpa-onnx)
synthesizer/pocket/model/ - TTS model download/verify (no CGO)
synthesizer/pocket/voice/ - Embedded default reference voice (no CGO)
synthesizer/speakers/ - Speaker interface + resampling helpers
tacview/ - Telemetry parsing
brevity/, parser/, composer/ - GCI command handling
internal/ - Private packages
application/ - Platform detection & glue
application/ - Application glue
controller/, radar/, conf/ - Core logic
```

**Architecture:** Players → SRS → simpleradio.Client → recognizer → parser → controller ← radar ← tacview ← DCS
controller → composer → synthesizer (platform-specific) → simpleradio.Client → SRS

Platform-specific code isolated to `pkg/synthesizer/speakers/{macos,piper}.go` and Makefile. Runtime detection: `runtime.GOOS` ("darwin"/"windows"/"linux").
controller → composer → synthesizer → simpleradio.Client → SRS

## Common Pitfalls

Expand Down
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ run:
test: generate
$(BUILD_VARS) $(GO) tool gotestsum -- $(BUILD_FLAGS) $(TEST_FLAGS) ./...

.PHONY: integration-test
integration-test: generate download-models
SKYEYE_MODELS_PATH=$(CURDIR)/models $(BUILD_VARS) $(GO) tool gotestsum -- -tags 'nolibopusfile integration' -ldflags '$(LDFLAGS)' -timeout 45m $(TEST_FLAGS) ./...

.PHONY: benchmark-parakeet
benchmark-parakeet:
$(BUILD_VARS) $(GO) test -bench=. -run BenchmarkParakeetRecognizer ./pkg/recognizer/parakeet
Expand Down
27 changes: 22 additions & 5 deletions cmd/download-models/main.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// download-models downloads Parakeet TDT model files for bundling into release archives.
// download-models downloads model files for bundling into release archives.
// This tool has no CGO dependencies and can be built with CGO_ENABLED=0.
package main

Expand All @@ -10,17 +10,34 @@ import (
"os/signal"
"path/filepath"

"github.com/dharmab/skyeye/pkg/recognizer/parakeet/model"
parakeetmodel "github.com/dharmab/skyeye/pkg/recognizer/parakeet/model"
pocketmodel "github.com/dharmab/skyeye/pkg/synthesizer/pocket/model"
)

func main() {
dir := flag.String("dir", filepath.Join("models", model.DirName), "directory to download model files into")
dir := flag.String("dir", "models", "base directory to download model files into")
flag.Parse()

ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()

if err := model.Download(ctx, *dir); err != nil {
log.Fatal(err)
parakeetDir := filepath.Join(*dir, parakeetmodel.DirName)
if err := parakeetmodel.Verify(parakeetDir); err != nil {
log.Printf("Parakeet model needs download: %v", err)
if err := parakeetmodel.Download(ctx, parakeetDir); err != nil {
log.Fatal(err)
}
} else {
log.Println("Parakeet model already present and verified")
}

pocketDir := filepath.Join(*dir, pocketmodel.DirName)
if err := pocketmodel.Verify(pocketDir); err != nil {
log.Printf("Pocket TTS model needs download: %v", err)
if err := pocketmodel.Download(ctx, pocketDir); err != nil {
log.Fatal(err)
}
} else {
log.Println("Pocket TTS model already present and verified")
}
}
128 changes: 69 additions & 59 deletions cmd/skyeye/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ import (
"os"
"os/signal"
"path/filepath"
"reflect"
"runtime"
"runtime/pprof"
"strings"
"sync"
Expand All @@ -29,7 +27,7 @@ import (
"github.com/dharmab/skyeye/internal/conf"
"github.com/dharmab/skyeye/pkg/coalitions"
parakeetmodel "github.com/dharmab/skyeye/pkg/recognizer/parakeet/model"
"github.com/dharmab/skyeye/pkg/synthesizer/voices"
pocketmodel "github.com/dharmab/skyeye/pkg/synthesizer/pocket/model"
)

// Used for CLI configuration values.
Expand All @@ -54,12 +52,9 @@ var (
coalitionName string
telemetryUpdateInterval time.Duration
recognizerLockPath string
voiceName string
useSystemVoice bool
voiceFile string
mute bool
voiceSpeed float64
voiceVolume float64
voicePauseLength time.Duration
voiceLockPath string
enableAutomaticPicture bool
automaticPictureInterval time.Duration
Expand Down Expand Up @@ -127,20 +122,10 @@ func init() {
skyeye.Flags().StringVar(&recognizerLockPath, "recognizer-lock-path", "", "Path to lock file for concurrent speech-to-text when using multiple instances")

// Text-to-speech
voiceFlag := cli.NewEnum(&voiceName, "Voice", "", "feminine", "masculine")
skyeye.Flags().Var(voiceFlag, "voice", "Voice to use for SRS transmissions (feminine, masculine). Automatically chosen if not provided.")
skyeye.Flags().Float64Var(&voiceSpeed, "voice-playback-speed", 1.0, "How quickly the GCI speaks (values below 1.0 are faster and above are slower).")
skyeye.Flags().StringVar(&voiceFile, "voice-file", "", "Path to WAV file for custom voice cloning. Uses built-in default if not set.")
skyeye.Flags().Float64Var(&voiceVolume, "voice-volume", voiceVolumeDefault, fmt.Sprintf("Volume level for audio output (%v = silent, %v = normal)", voiceVolumeMin, voiceVolumeDefault))
skyeye.Flags().BoolVar(&mute, "mute", false, "Mute all SRS transmissions. Useful for testing without disrupting play")
skyeye.Flags().StringVar(&voiceLockPath, "voice-lock-path", "", "Path to lock file for concurrent text-to-speech when using multiple instances")
if runtime.GOOS == "darwin" {
skyeye.Flags().BoolVar(&useSystemVoice, "use-system-voice", false, "Use the System Voice chosen in the Spoken Content page in System Settings instead of Samantha.")
if err := skyeye.Flags().MarkDeprecated("voice", "Select a voice in System Settings and use --use-system-voice instead."); err != nil {
log.Fatal().Err(err).Msg("failed to mark flag as deprecated")
}
} else {
skyeye.Flags().DurationVar(&voicePauseLength, "voice-playback-pause", 200*time.Millisecond, "How long the GCI pauses between sentences.")
}

// Controller behavior
skyeye.Flags().BoolVar(&enableAutomaticPicture, "auto-picture", true, "Enable automatic PICTURE broadcasts")
Expand Down Expand Up @@ -258,22 +243,6 @@ func randomizer() (rando *rand.Rand) {
return
}

func loadVoice(rando *rand.Rand) (voice voices.Voice) {
options := map[string]voices.Voice{
"feminine": voices.FeminineVoice,
"masculine": voices.MasculineVoice,
}
if voiceName == "" {
keys := reflect.ValueOf(options).MapKeys()
voice = options[keys[rando.IntN(len(keys))].String()]
log.Info().Type("voice", voice).Msg("randomly selected voice")
} else {
voice = options[voiceName]
log.Info().Type("voice", voice).Msg("selected voice")
}
return
}

func loadCallsign(rando *rand.Rand) (callsign string) {
var options []string
if controllerCallsign != "" {
Expand Down Expand Up @@ -309,28 +278,59 @@ func loadVoiceVolume() float64 {
return clamped
}

func setupParakeetModel(ctx context.Context, parakeetDir string, downloadModels bool) {
log.Info().Msg("verifying Parakeet model files")
if err := parakeetmodel.Verify(parakeetDir); err != nil {
var corruptErr *parakeetmodel.CorruptFileError
if errors.As(err, &corruptErr) {
log.Fatal().Err(err).Msg("Parakeet model files on disk failed verification")
}
var notFoundErr *parakeetmodel.FileNotFoundError
if errors.As(err, &notFoundErr) {
log.Warn().Err(err).Msg("Parakeet model files not found")
if downloadModels {
log.Info().Msg("downloading Parakeet model files")
if err := parakeetmodel.Download(ctx, parakeetDir); err != nil {
log.Fatal().Err(err).Msg("failed to download Parakeet model")
}
} else {
log.Fatal().Err(err).Msg("no Parakeet model files found")
// modelSetup holds the verify and download functions for a model, allowing
// setupModel to work with both Parakeet and Pocket TTS models.
type modelSetup struct {
name string
dir string
verify func(string) error
download func(context.Context, string) error
}

func setupModel(ctx context.Context, m modelSetup, autoDownload bool) {
log.Info().Msgf("verifying %s model files", m.name)
err := m.verify(m.dir)
if err == nil {
log.Info().Msgf("%s model files verified", m.name)
return
}

// Check for corrupt files first — these should not be silently re-downloaded.
if hasCorruptFile(err) {
log.Fatal().Err(err).Msgf("%s model files on disk failed verification", m.name)
}

// Check for missing files.
if hasMissingFile(err) {
log.Warn().Err(err).Msgf("%s model files not found", m.name)
if autoDownload {
log.Info().Msgf("downloading %s model files", m.name)
if dlErr := m.download(ctx, m.dir); dlErr != nil {
log.Fatal().Err(dlErr).Msgf("failed to download %s model", m.name)
}
return
}
} else {
log.Info().Msg("Parakeet model files verified")
log.Fatal().Err(err).Msgf("no %s model files found", m.name)
}

// Unexpected error (e.g. permission denied).
log.Fatal().Err(err).Msgf("failed to verify %s model files", m.name)
}

// hasCorruptFile checks whether err (possibly a joined error) contains a CorruptFileError
// from either the parakeet or pocket model packages.
func hasCorruptFile(err error) bool {
var parakeetCorrupt *parakeetmodel.CorruptFileError
var pocketCorrupt *pocketmodel.CorruptFileError
return errors.As(err, &parakeetCorrupt) || errors.As(err, &pocketCorrupt)
}

// hasMissingFile checks whether err (possibly a joined error) contains a FileNotFoundError
// from either the parakeet or pocket model packages.
func hasMissingFile(err error) bool {
var parakeetNotFound *parakeetmodel.FileNotFoundError
var pocketNotFound *pocketmodel.FileNotFoundError
return errors.As(err, &parakeetNotFound) || errors.As(err, &pocketNotFound)
}

func preRun(cmd *cobra.Command, _ []string) error {
Expand Down Expand Up @@ -371,12 +371,24 @@ func run(_ *cobra.Command, _ []string) {
}()

parakeetDir := filepath.Join(modelsPath, parakeetmodel.DirName)
setupParakeetModel(ctx, parakeetDir, downloadModels)
setupModel(ctx, modelSetup{
name: "Parakeet",
dir: parakeetDir,
verify: parakeetmodel.Verify,
download: parakeetmodel.Download,
}, downloadModels)

pocketDir := filepath.Join(modelsPath, pocketmodel.DirName)
setupModel(ctx, modelSetup{
name: "Pocket TTS",
dir: pocketDir,
verify: pocketmodel.Verify,
download: pocketmodel.Download,
}, downloadModels)

log.Info().Msg("loading configuration")
coalition := loadCoalition()
rando := randomizer()
voice := loadVoice(rando)
callsign := loadCallsign(rando)
parsedSRSFrequencies := cli.LoadFrequencies(srsFrequencies)
voiceLock := loadLock(voiceLockPath)
Expand All @@ -399,13 +411,10 @@ func run(_ *cobra.Command, _ []string) {
Coalition: coalition,
RadarSweepInterval: telemetryUpdateInterval,
RecognizerLock: recognizerLock,
Voice: voice,
UseSystemVoice: useSystemVoice,
VoiceFile: voiceFile,
VoiceLock: voiceLock,
Mute: mute,
VoiceSpeed: voiceSpeed,
Volume: volume,
VoicePauseLength: voicePauseLength,
EnableAutomaticPicture: enableAutomaticPicture,
PictureBroadcastInterval: automaticPictureInterval,
EnableThreatMonitoring: enableThreatMonitoring,
Expand All @@ -428,6 +437,7 @@ func run(_ *cobra.Command, _ []string) {
if err != nil {
log.Fatal().Err(err).Msg("failed to start application")
}
defer app.Close()
err = app.Run(ctx, cancel, &wg)
if err != nil {
log.Fatal().Err(err).Msg("application exited with error")
Expand Down
Loading
Loading