From 58cc177e163de5b54f4976a67e1a6ee0863aa743 Mon Sep 17 00:00:00 2001 From: ChuxiJ Date: Fri, 27 Mar 2026 13:14:53 +0800 Subject: [PATCH] docs: add themed sprint roadmap for research backlog --- .llm/research/automation-parameter-ux.md | 149 ++++++++++ .llm/research/collaboration-project-ux.md | 280 ++++++++++++++++++ .llm/research/daw-synths-samplers.md | 180 +++++++++++ .llm/research/drum-machine-sequencer-ux.md | 142 +++++++++ .llm/research/effects-plugin-ui.md | 144 +++++++++ .llm/research/mixer-channel-strip-ui.md | 101 +++++++ .llm/research/onboarding-workflow-ux.md | 144 +++++++++ .llm/research/piano-roll-midi-ux.md | 115 +++++++ .../research/recording-input-monitoring-ux.md | 152 ++++++++++ .llm/research/sample-browser-library.md | 154 ++++++++++ .llm/research/synth-ui-design.md | 127 ++++++++ .llm/research/timeline-arrangement-ux.md | 120 ++++++++ docs/plans/roadmap-themed-sprints-2026-03.md | 98 ++++++ docs/plans/sprint-01-synth-foundation.md | 86 ++++++ docs/plans/sprint-02-sampling-drum-render.md | 80 +++++ docs/plans/sprint-03-mixer-device-routing.md | 75 +++++ .../plans/sprint-04-midi-rhythm-automation.md | 68 +++++ docs/plans/sprint-05-arrangement-recording.md | 65 ++++ ...int-06-onboarding-library-accessibility.md | 71 +++++ docs/plans/sprint-07-collaboration-cloud.md | 67 +++++ docs/plans/sprint-08-advanced-sound-design.md | 71 +++++ 21 files changed, 2489 insertions(+) create mode 100644 .llm/research/automation-parameter-ux.md create mode 100644 .llm/research/collaboration-project-ux.md create mode 100644 .llm/research/daw-synths-samplers.md create mode 100644 .llm/research/drum-machine-sequencer-ux.md create mode 100644 .llm/research/effects-plugin-ui.md create mode 100644 .llm/research/mixer-channel-strip-ui.md create mode 100644 .llm/research/onboarding-workflow-ux.md create mode 100644 .llm/research/piano-roll-midi-ux.md create mode 100644 .llm/research/recording-input-monitoring-ux.md create mode 100644 .llm/research/sample-browser-library.md create mode 100644 .llm/research/synth-ui-design.md create mode 100644 .llm/research/timeline-arrangement-ux.md create mode 100644 docs/plans/roadmap-themed-sprints-2026-03.md create mode 100644 docs/plans/sprint-01-synth-foundation.md create mode 100644 docs/plans/sprint-02-sampling-drum-render.md create mode 100644 docs/plans/sprint-03-mixer-device-routing.md create mode 100644 docs/plans/sprint-04-midi-rhythm-automation.md create mode 100644 docs/plans/sprint-05-arrangement-recording.md create mode 100644 docs/plans/sprint-06-onboarding-library-accessibility.md create mode 100644 docs/plans/sprint-07-collaboration-cloud.md create mode 100644 docs/plans/sprint-08-advanced-sound-design.md diff --git a/.llm/research/automation-parameter-ux.md b/.llm/research/automation-parameter-ux.md new file mode 100644 index 00000000..2e84d17d --- /dev/null +++ b/.llm/research/automation-parameter-ux.md @@ -0,0 +1,149 @@ +# Automation & Parameter Control UX Research + +> Date: 2026-03-27 | Scope: Automation editing, parameter control, and modulation UI in DAWs vs ACE-Step + +--- + +## 1. Ableton Live Automation + +### Automation Modes +- **No dedicated modes in arrangement**: Draw automation directly on clips or arrangement +- **Session View**: Automation recorded via MIDI mapping or mouse movement during playback +- **Arrangement automation**: Breakpoint envelopes drawn on tracks +- **Clip automation**: Per-clip envelopes that loop with the clip + +### Drawing & Editing +- **Breakpoint editing**: Click to add points, drag to move +- **Line segments**: Straight lines between breakpoints (no curves in standard mode) +- **Draw mode (B key)**: Freehand drawing of automation curves +- **Grid snapping**: Automation points snap to beat grid +- **Select + drag**: Move multiple points together +- **Cmd+A**: Select all automation on track + +### Envelope Display +- **Overlay on clip**: Automation drawn directly on top of audio/MIDI clips +- **Device parameter chooser**: Dropdown to select which parameter to automate +- **Color coding**: Each automated parameter gets distinct color +- **Show/hide**: Toggle automation visibility per track + +## 2. Logic Pro Automation + +### 4 Automation Modes +- **Read**: Plays back existing automation (default) +- **Touch**: Records changes while touching control, returns to previous value on release +- **Latch**: Records changes, holds last value when control released +- **Write**: Overwrites all existing automation with new values + +### Curve Editing +- **Bezier curves**: Drag between points to create curved transitions +- **Step automation**: Square wave style — instant jumps between values +- **Automation Select tool**: Region-based selection for batch editing +- **Relative mode**: Offset existing automation up/down without changing curve shape +- **MIDI Learn**: Move hardware control → auto-assign to parameter + +### Automation Lanes +- **Per-track lanes**: Each parameter gets its own sub-lane below the track +- **Multiple lanes visible**: See volume, pan, filter simultaneously +- **Lane height adjustment**: Resize for precision editing +- **Copy/paste automation**: Between tracks or parameters + +## 3. Studio One — Best Automation Painting + +### Paint Tool +- **Shape painting**: Draw sine, square, triangle, saw, random shapes +- **Grid-synced shapes**: LFO-like automation snapped to beat grid +- **Transform tool**: Scale, stretch, invert automation regions +- **Part automation**: Automation moves with the clip (not fixed to timeline) + +### Automation Modes +- **Touch, Latch, Write**: Standard modes +- **Trim**: Offset automation by percentage (±, multiplicative) +- **Cross-fade automation**: Smooth transitions between automation regions + +## 4. FL Studio Automation + +### Automation Clips +- **Automation as clips**: Automation is placed on playlist as colored clips +- **LFO tool**: Built-in LFO shape generator (sine, saw, triangle + parameters) +- **Curve types**: Single curve, hold, smooth, stairs, half-sine, etc. +- **Event editor**: Per-event automation with fine control +- **Link to controller**: Right-click any knob → Link to controller → assign MIDI CC + +### Unique Features +- **Copy state → Paste as automation**: Copy current parameter values as starting point +- **Scale levels**: Select automation region → scale by percentage +- **LFO shape drawing**: Draw custom LFO shapes that repeat + +## 5. Bitwig Studio + +### Modulation vs Automation +- **Automation**: Traditional timeline-based parameter changes +- **Modulation**: Real-time modulators (LFOs, envelopes, followers) assigned to parameters +- **Per-voice modulation**: Modulate parameters per individual note (MPE) +- **Expressions**: Timbre, pressure, slide — per-note modulation lanes + +### Modulator Devices +- **LFO**: Sync or free-running, multiple shapes +- **Envelope follower**: Track audio amplitude → modulate parameter +- **Step sequencer**: 16-step modulation source +- **Math**: Combine modulation sources (add, multiply, etc.) +- **Keytrack**: Modulate based on played note pitch + +## 6. ACE-Step Current State + +### What Exists +- **TempoLane.tsx**: Tempo change markers on timeline (not general automation) +- **AutomationLane in store**: Data model for per-track automation lanes +- **AutomationPoint type**: `{ beat, value, curveType }` — supports linear/exponential/logarithmic +- **No automation UI**: No drawing, no editing, no visualization of automation curves +- **No MIDI learn**: No controller mapping +- **Knob.tsx**: Supports vertical drag, double-click reset — but no automation recording + +### Key Gaps +| Feature | Competitors | ACE-Step | +|---|---|---| +| Draw automation curves | All | No UI | +| Automation modes (Touch/Latch/Write) | Logic, Studio One, Pro Tools | No | +| Bezier/curved automation | Logic, FL | Data supports, no UI | +| Multiple automation lanes per track | All | Data exists, no UI | +| LFO-shaped automation | FL Studio, Studio One | No | +| MIDI Learn / controller mapping | All | No | +| Relative automation (trim) | Studio One | No | +| Paint/shape tools | Studio One (best) | No | +| Per-clip automation | Ableton, Bitwig | No | +| Modulation routing (non-timeline) | Bitwig, Serum, Vital | No | + +--- + +## 7. Recommendations for ACE-Step + +### Phase 1: Basic Automation Drawing +- **Automation lane UI**: Show/hide lanes below each track +- **Click to add points**: Breakpoint creation on the automation lane +- **Drag points**: Move existing breakpoints +- **Parameter selector**: Dropdown to choose volume, pan, effect params +- **Line segments**: Linear interpolation between points (simplest to implement) + +### Phase 2: Recording & Curves +- **Touch mode**: Record automation by moving knobs during playback +- **Bezier curves**: Drag between points for curved transitions +- **Automation overlay on clips**: Draw automation on top of clip content +- **Copy/paste automation**: Between tracks and parameters +- **Snap to grid**: Automation points snap to beat divisions + +### Phase 3: Creative Automation +- **LFO tool**: Paint repeating shapes (sine, saw, square) synced to grid +- **Trim/relative mode**: Offset existing automation curves +- **Per-clip automation**: Automation that loops with clip +- **MIDI Learn**: Right-click knob → move hardware control → mapped +- **Modulation routing**: LFO/envelope → parameter (Bitwig-style) + +--- + +## Sources + +- [Ableton Live 12: Automation & Envelopes — Ableton Manual](https://www.ableton.com/en/manual/automation-and-editing-envelopes/) +- [Logic Pro: Automation Overview — Apple Support](https://support.apple.com/guide/logicpro/automation-overview-lgce1e9bsf6b/mac) +- [Studio One: Automation — PreSonus Manual](https://www.presonus.com/learn/technical-articles/automation-studio-one) +- [FL Studio: Automation Clips — Image-Line Manual](https://www.image-line.com/fl-studio-learning/fl-studio-online-manual/html/automation_clip.htm) +- [Bitwig Studio: Modulators — Bitwig Documentation](https://www.bitwig.com/stories/modulators/) diff --git a/.llm/research/collaboration-project-ux.md b/.llm/research/collaboration-project-ux.md new file mode 100644 index 00000000..97d1af90 --- /dev/null +++ b/.llm/research/collaboration-project-ux.md @@ -0,0 +1,280 @@ +# Collaboration & Project Management UX Research + +**Date**: 2026-03-26 +**Researcher**: Agent (Competitive Research) +**Topic**: Collaboration, project versioning, sharing, cloud sync, and AI-first workflow + +--- + +## 1. Real-Time Collaboration + +### BandLab (Industry Leader for Browser DAW Collaboration) +- Supports up to **50 simultaneous collaborators** on a single project +- **Permission model**: Owner invites collaborators with either "edit" or "view-only" access. Collaborator accepts invite, then opens project in their own Mix Editor instance +- **Auto-save**: Continuous auto-save captures every change immediately. No manual save needed. Downside: accidental deletions are also saved immediately with no easy rollback +- **Cross-device**: Seamless sync between desktop browser, iOS, and Android apps. Two producers in different countries on different devices see the same project state +- **Forking**: BandLab's version of branching. Users can "Fork" any project that allows it, creating an independent full copy in their account. Changes to the fork do not affect the original. This is analogous to Git's fork model +- **Social integration**: Projects can be shared directly to BandLab's social feed, enabling remixes and community-driven collaboration +- **Architecture**: No public documentation on CRDT vs OT implementation. Given the track-level editing granularity (not note-level), likely uses a simpler last-write-wins or operational transform approach per track rather than full CRDT + +### Soundtrap (Spotify-owned, Education Focus) +- **Real-time multi-user editing**: Multiple cursors visible simultaneously, similar to Google Docs. Each collaborator's active track highlighted in real-time +- **Built-in video calling**: Integrated video chat during collaboration sessions (not just text). This is a significant differentiator for remote music creation +- **Built-in text chat**: In-session messaging for quick communication without leaving the DAW +- **Real-time feedback**: Collaborators can leave comments and suggestions directly on the project +- **Education integration**: Connects with Google Classroom, Schoology, Noteflight, MusicFirst. Teachers can assign projects, monitor student progress, and provide feedback +- **Latency issues**: Some users report synchronization problems during real-time collaboration, particularly for audio recording (MIDI collaboration is smoother) +- **Soundtrap 2.0 (March 2026)**: Major rewrite with lower latency, full effect automation, massive sample library, and improved real-time collaboration + +### LANDR (Professional Collaboration) +- **DAW-to-DAW streaming**: Collaboration plugin streams audio directly from your DAW to collaborators in real-time. Producer in LA works on a mix while singer in NY listens in full fidelity and records ideas +- **HD audio video chat**: Not just basic video calling -- high-definition DAW-audio video sessions +- **Time-synced comments**: Comments tied to specific points in the timeline (similar to SoundCloud model) +- **Mobile app (2025)**: Library, Messages, Sessions (video streaming with collaborators), and Releases tabs +- **Royalty splits**: Automated royalty split management for collaborators at no added cost + +### Udio (AI Platform with Collaboration) +- **Real-time co-creation**: Multiple users work on the same AI-generated track simultaneously +- **Locked ecosystem**: Songs cannot be shared outside the Udio platform as of 2026 +- **Downloads temporarily disabled** during licensing transition (2025-2026) + +### Ableton Link (Local Network Only) +- **Beat/tempo/phase sync** across devices on the same local network +- **No central host**: All participants are equal (unlike MIDI sync). Anyone can change tempo +- **Start/Stop sync**: Transport commands shared across all Link-enabled apps +- **Cross-platform**: Supported by Bitwig, Reason, VCV Rack, Serato DJ, and many iOS apps +- **NOT remote collaboration**: Requires local network. For remote work, Ableton recommends file sharing via WeTransfer or pCloud +- **No note/audio data**: Link syncs timing only, not musical content + +--- + +## 2. Project Versioning & Auto-Save + +### Competitor Approaches + +| Feature | BandLab | Soundtrap | Splice (retired) | FL Studio | Logic Pro | Ableton | +|---|---|---|---|---|---|---| +| Auto-save | Continuous | Continuous | Per-commit | Manual | Auto-save every N minutes | Crash recovery only | +| Version history | Via Fork (copy-based) | Cloud versions | Git-like commits with diffs | Save As with naming | Alternatives (file copies) | Save As with incrementing | +| Rollback | Fork original, abandon current | Restore from cloud version | Restore any commit | Reopen old .flp file | Revert to Saved | Reopen old .als file | +| Named save points | Not supported | Not supported | Supported (commit messages) | Manual "Save new version" | "Save a Copy" | "Save a Copy" | +| Branching | Fork = independent copy | Not supported | Full branching | Not supported | Not supported | Not supported | + +### Best-in-Class: Splice Studio (discontinued but instructive) +- **Git-like model**: Automatic cloud backup, change tracking per commit +- **Dependency tracking**: Tracked which samples and plugins each version used +- **Collaboration**: Shared projects with full version history visible to all collaborators +- **Shutdown reason**: Feature hadn't been maintained since 2017; quality degraded. Splice pivoted to sample marketplace +- **Lesson**: Version control for music is highly desired (community consensus on HN, KVR, Ardour forums) but hard to maintain as a business + +### Community Workarounds +- Some producers use **Git** directly with Reaper (text-based project files). Branch per cue, commit per change +- **Submix/Playback**: Dedicated versioning tool that takes snapshots of DAW projects at key moments +- Desktop DAW convention: Manual "Save As" with incrementing names (song_v1.flp, song_v2.flp) + +--- + +## 3. Stem Sharing & Export + +### FL Studio Project Bones +- **Export components separately**: Automation, channel settings, plugin effects, mixer insert states, piano roll scores saved as individual files +- **Drag-and-drop transfer**: .fsc files (scores) can be dragged onto sounds in Step Sequencer. .fst files (mixer states) onto mixer inserts +- **Limitation**: MIDI notes and channel names exported separately; cannot be recombined easily +- **No audio**: Project Bones do not include audio files -- only metadata and MIDI +- **Better alternative for sharing**: Zipped Loop Package includes everything (audio + project data) + +### Splice Mobile (2025) +- Record ideas on phone, export stems to DAW or share with collaborators +- Stems organized by project, tagged with tempo and key + +### Ableton Live 12.3 (September 2025) +- **Built-in stem separation**: Separates audio into vocals, drums, bass, and other +- **Splice integration**: Browse and audition Splice samples within Live's Browser, synchronized to project tempo and key +- **"Search with Sound"**: Drag audio from a clip to find rhythmically and harmonically compatible samples + +### AIVA +- **MIDI export**: All AI-generated compositions exportable as MIDI for editing in any DAW +- **Full copyright on Pro plan**: Users own 100% of generated content + +--- + +## 4. Comments & Annotations + +### SoundCloud Model (Reference Standard) +- **Timed comments**: Comments appear at the waveform position where the user started typing +- **Visual density indicator**: Cluster of comments at a specific time shows listener engagement hotspots +- **Hyperlinks in comments**: Can include links for credits, references, additional context +- **Hovering text boxes**: Comments appear as floating bubbles over waveform during playback +- **Single-layer limitation**: All comments on one layer. Research (ResearchGate) shows multi-layered annotation timelines would be more useful for collaborative music feedback + +### LANDR +- **Time-synced comments**: Comments tied to timeline position during collaboration sessions +- **Integrated with video chat**: Can discuss while pointing at specific moments + +### Soundtrap +- **In-project feedback**: Collaborators leave comments and suggestions directly on the project +- **Teacher annotations**: In education mode, teachers annotate student work at specific points + +--- + +## 5. Templates & Presets + +### Competitor Approaches +- **Ableton**: Ships with template Sets (e.g., "Default 808", "Lo-Fi Hip-Hop"). User can save any Set as template. Templates include track routing, effects, instruments +- **Logic Pro**: Templates per genre (Electronic, Hip-Hop, Songwriter, Orchestral). Each pre-configures tracks with instruments, effects, and routing +- **FL Studio**: Starter templates with pre-configured channel rack, mixer routing, and pattern structure +- **Udio**: Genre-specific templates as starting points for AI generation. Templates act as style presets +- **BandLab**: SongStarter AI generates initial ideas based on genre/mood selection + +--- + +## 6. Cloud Sync & Storage + +### BandLab +- **Unlimited cloud storage** (free tier): All projects, stems, and audio stored in cloud +- **No local-first option**: Requires internet for full functionality +- **Cross-device sync**: Automatic between all platforms + +### Soundtrap +- **Cloud-native**: All projects in cloud, accessible from any device (Windows, Mac, Chromebook, iOS, Android) +- **No local storage concerns**: Everything server-side + +### LANDR +- **Library tab in app**: Centralized storage for all uploaded tracks +- **Release management**: Schedule and control releases from cloud interface + +--- + +## 7. Export, Share & Distribution + +### LANDR (End-to-End Pipeline) +- **Mastering**: AI-powered, multiple styles, unlimited revisions. Plugin version for in-DAW use +- **Distribution**: Direct to 150+ streaming platforms (Spotify, Apple Music, YouTube Music) +- **Royalty management**: 100% royalties on subscription, 85% if cancelled. Automated splits +- **Pricing**: $9/single, $19/album pay-per-release; or $23.99/year unlimited +- **Reference mastering**: Upload a reference track, AI matches its sonic profile + +### Suno +- **Suno Studio**: Timeline-based editor for arranging sections, adjusting transitions, fine-tuning song structure after AI generation +- **Stem editing**: Edit individual stems of AI-generated songs +- **Commercial rights**: Only while actively subscribed to paid plan + +### Standard DAW Export +- WAV, MP3, FLAC, OGG formats +- Stem export (individual tracks) +- Mixdown with mastering chain applied + +--- + +## 8. AI-First Collaboration Opportunities + +### Generate-Refine-Collaborate Workflow (Novel for ACE-Step) + +Based on competitor analysis, ACE-Step has a unique opportunity to define an **AI-native collaboration model**: + +1. **Generate**: User creates AI stems via prompt (ACE-Step already does this) +2. **Refine**: Edit, arrange, mix the generated material (ACE-Step has this) +3. **Share**: Export stems or full project for collaborator review (partially implemented) +4. **Iterate**: Collaborator can re-generate specific stems, add their own, remix +5. **Publish**: Master and distribute (not implemented) + +### What Competitors Lack +- **Suno/Udio**: Generate music but have minimal DAW editing capabilities +- **BandLab/Soundtrap**: Full DAW editing but AI generation is limited to simple features (SongStarter, AutoPitch) +- **AIVA**: Strong generation but MIDI-only output, no audio collaboration +- **ACE-Step's unique position**: AI generation + full browser DAW + collaboration potential + +--- + +## 9. Project Organization + +### Competitor Approaches +- **Ableton**: File browser with folders, recent files, Places sidebar. Collections (user-defined color-tagged folders) +- **Logic Pro**: "All Projects" browser with search, tags, and last-modified sorting +- **FL Studio**: Browser panel with categorized presets, projects, samples. User data folder for personal content +- **BandLab**: Feed-based with projects, favorites, and collections + +--- + +## 10. ACE-Step Current State Assessment + +### What ACE-Step Has +- **Local project storage**: IndexedDB via idb-keyval. Save, load, delete, list projects +- **Project archive format**: Custom .acedaw binary format with embedded audio blobs +- **Auto-save**: Debounced (1s) auto-save to IndexedDB project library +- **Undo/Redo**: Comprehensive history with scoped undo (arrangement, mixer, track levels) +- **Project templates**: ProjectTemplate type defined with save/load/list/delete in projectStorage.ts +- **Share bundle**: JSON export/import without audio (lightweight sharing) +- **Share links**: URL-based sharing with token, read-only mode, expiration +- **Collaboration store**: Zustand store with viewer mode, share dialog, collaborator list (stubbed for Phase 3) +- **Cloud storage service**: In-memory implementation with version history, shared project records, stem assets +- **Project sharing service**: Renders individual track stems as MP3, creates shared project with audio data URLs +- **Stem export**: ExportMix supports individual track rendering +- **Multiple export formats**: WAV, MP3, FLAC, OGG with metadata + +### What ACE-Step Is Missing + +#### P1 - Critical Gaps +1. **Real cloud backend**: cloudStorageService.ts uses in-memory Map (lost on refresh). No actual cloud persistence +2. **Auto-save version history with rollback**: Current auto-save overwrites single entry. No snapshots, no "go back to 5 minutes ago" +3. **Shareable player/embed**: Share link generates URL but requires same-origin localStorage. No hosted player page + +#### P2 - Important Gaps +4. **Real-time collaboration**: collaborationStore has Collaborator type and add/remove but no WebSocket/CRDT sync. Fully stubbed +5. **Time-stamped comments/annotations**: No comment or annotation system at all +6. **Project organization (folders, tags, search)**: Project list is flat, sorted by updatedAt only. No folders, no tags, no search +7. **Distribution pipeline**: No mastering-for-streaming targets (LUFS), no integration with distribution platforms +8. **Import from other DAWs**: No MIDI file import, no stem import from standard formats + +#### P3 - Nice-to-Have Gaps +9. **Forking/branching**: No way to create an independent copy of a project for experimentation +10. **In-session video/text chat**: No communication during collaboration +11. **Remix/fork from community**: No social layer for discovering and remixing others' projects +12. **Genre templates library**: ProjectTemplate type exists but no pre-built templates shipped +13. **Royalty split management**: No collaborator payment or credit tracking + +--- + +## 11. Recommended Implementation Priority + +### Phase 1: Foundation (P1) +- Implement persistent cloud storage backend (replace in-memory Maps) +- Add version history with named save points and rollback UI +- Build hosted share player page (embeddable, works without full DAW) + +### Phase 2: Collaboration Core (P2) +- MIDI file import/export (standard .mid format) +- Project organization (folders, tags, search, favorites) +- Time-stamped comments on timeline +- Mastering for streaming targets (Spotify -14 LUFS, Apple -16 LUFS) + +### Phase 3: Real-Time & Social (P2-P3) +- WebSocket-based real-time collaboration (start with cursor visibility + track locking) +- Project forking/branching +- Genre template library (ship 5-10 built-in templates) +- In-session text chat + +### Phase 4: Platform (P3) +- Community sharing/remix feed +- Distribution integration (DistroKid/LANDR API) +- Royalty split management +- Video chat integration + +--- + +## Sources + +- [BandLab Collaboration Features: Complete Guide (2026)](https://www.audeobox.com/learn/bandlab/bandlab-collaboration-features/) +- [From DAW to GAW: How BandLab Studio Is Using AI](https://www.makingascene.org/from-daw-to-gaw-how-bandlab-studio-is-using-ai-to-redefine-music-production/) +- [The New Soundtrap: All The Power, None Of The Friction](https://blog.soundtrap.com/new-soundtrap/) +- [Soundtrap Software 2025](https://www.spotsaas.com/blog/soundtrap-software/) +- [Splice Studio is free backup, version control, and collaboration for your DAW](https://cdm.link/splice-studio-is-free-backup-version-control-and-collaboration-for-your-daw/) +- [Ableton Live 12.3: Stem Separation, Splice Integration](https://routenote.com/blog/ableton-live-12-3-update/) +- [Ableton Link features and functions FAQ](https://help.ableton.com/hc/en-us/articles/209776125-Link-features-and-functions-FAQ) +- [Best Practices for Collaborating Remotely - Ableton](https://help.ableton.com/hc/en-us/articles/360012680119-Best-Practices-for-Collaborating-Remotely) +- [How to Use Project Bones in FL Studio](https://itsgratuitous.com/how-to-use-project-bones-in-fl-studio/) +- [Best AI Music Generators in 2026: Suno vs Udio vs AIVA](https://superprompt.com/blog/best-ai-music-generators) +- [LANDR Review (2026)](https://aiquiks.com/ai-tools/landr) +- [LANDR launches new app with AI mastering, collaboration and distribution](https://djmag.com/tech/landr-launches-new-app-ai-mastering-collaboration-and-unlimited-distribution-one) +- [Suno vs AIVA: Best AI Tool for Music Creation](https://www.musicful.ai/vs/suno-vs-aiva/) +- [Best AI Music Generators in 2026: Suno vs Udio vs ElevenLabs](https://jam.com/resources/best-ai-music-generators-2026) diff --git a/.llm/research/daw-synths-samplers.md b/.llm/research/daw-synths-samplers.md new file mode 100644 index 00000000..565f40ed --- /dev/null +++ b/.llm/research/daw-synths-samplers.md @@ -0,0 +1,180 @@ +# DAW Synthesizer & Sampler Competitive Research + +> Date: 2026-03-26 +> Scope: Mainstream DAW built-in instruments vs ACE-Step DAW current state + +--- + +## 1. Ableton Live Built-in Instruments + +### Synthesizers + +| Instrument | Type | Key Features | ACE-Step Status | +|---|---|---|---| +| **Wavetable** | Wavetable synth | 2 oscillators with wavetable morphing, sub-osc, FM/AM modulation, 2 filters with routing matrix, 3 LFOs, 3 envelopes, unison up to 8 voices | **Missing** | +| **Operator** | FM synth | 4 oscillators, 11 FM algorithms, per-osc filter/pitch envelope, additive harmonics editor | **Missing** (FMSynth exists in LoopLibrary but not exposed as track instrument) | +| **Analog** | Virtual analog | 2 oscillators (saw/square/sine/noise), sub-osc, PWM, sync, 2 multimode filters, 2 LFOs, 2 envelopes, unison | **Partial** — basic presets only, no parameter editing UI | +| **Drift** | Modern analog | 2 oscillators with shape/fold, noise, multimode filter, drift modulation, mod matrix | **Missing** | +| **Meld** | MPE synth | 2 engines (oscillator types), MPE per-note expression, mod matrix, advanced modulation | **Missing** | +| **Collision** | Physical modeling | Mallet + noise exciter, membrane/beam/plate/string resonator, LFO, MIDI velocity mapping | **Missing** | +| **Tension** | String physical modeling | Exciter (bow/hammer/pluck), string model with damping/termination, body resonance, filter | **Missing** | +| **Electric** | Electric piano modeling | Mallet/tine fork/tone bar models, pickup simulation, global effects | **Missing** | + +### Samplers + +| Instrument | Type | Key Features | ACE-Step Status | +|---|---|---|---| +| **Simpler** | Quick sampler | Warp modes, filter, LFO, envelope, slice mode, 1-shot mode | **Partial** — SamplerEngine has basic playback modes but no warp/slice | +| **Sampler** | Advanced sampler | Multi-sample zones, velocity layers, round-robin, key/velocity crossfading, modulation matrix, 3 LFOs, 6 envelopes | **Missing** — current sampler is single-sample only | +| **Drum Rack** | Drum instrument | 128 pad slots, per-pad chain (instrument+effects), choke groups, macro knobs, return chains | **Partial** — DrumEngine has 16 synth sounds, no sample loading or per-pad effects | + +### Audio Effects (instrument-relevant) + +| Effect | ACE-Step Status | +|---|---| +| Auto Filter with envelope follower | **Partial** — filter exists but no envelope follower | +| Corpus (resonator) | **Missing** | +| Resonator | **Missing** | +| Spectral Resonator | **Missing** | +| Spectral Time | **Missing** | +| Granulator II (Max for Live) | **Missing** | + +--- + +## 2. Logic Pro Built-in Instruments + +| Instrument | Type | Key Features | ACE-Step Status | +|---|---|---|---| +| **Alchemy** | Hybrid synth | Additive, spectral, granular, wavetable, VA, sample-based — all in one. 4 sources, advanced modulation, morphing | **Missing** — most ambitious synth in any DAW | +| **ES2** | Virtual analog | 3 oscillators, FM, ring mod, sync, 2 filters, mod matrix, vector pad | **Missing** | +| **Retro Synth** | Multi-engine | Analog, sync, wavetable, FM modes in one UI | **Missing** | +| **Drum Machine Designer** | Drum kit | Sample-based pads, per-pad quick sampler, smart controls | **Partial** | +| **Quick Sampler** | Quick sampler | Drag-and-drop, auto-slice, optimized recording, warp modes | **Partial** — SamplerEngine covers basics | +| **Sculpture** | Physical modeling | Component modeling (string+exciter+body), morph pad, record automation | **Missing** | +| **EXS24/Sampler** | Multi-sampler | Key/velocity mapping, round-robin, modulation routing, zone editor | **Missing** | + +--- + +## 3. FL Studio Built-in Instruments + +| Instrument | Type | Key Features | ACE-Step Status | +|---|---|---|---| +| **Sytrus** | FM/subtractive/RM | 6 operators, ring mod matrix, filter per operator, unison, waveshaping | **Missing** | +| **Harmor** | Additive/resynthesis | Additive engine, image resynthesis, prism, blur, pluck, subtractive post-filter | **Missing** | +| **Flex** | Preset-based multi-engine | Simple UI, many sound packs, internal multi-engine | **Similar approach** to ACE-Step presets | +| **3xOsc** | Basic subtractive | 3 oscillators, detuning, phase, basic but effective | **Partial** — similar level to current SynthEngine | +| **FLEX** | Wavetable/multi | Modern presets, macro knobs | **Missing** | +| **DirectWave** | Multi-sampler | SFZ/SF2 import, zone editor, ADSR, filters | **Missing** | +| **Slicex** | Beat slicer | Auto-slice audio, rearrange via piano roll, stretch modes | **Missing** | +| **Fruity Granulizer** | Granular | Grain size, spacing, randomize, attack shape | **Missing** | + +--- + +## 4. Popular 3rd-Party Instruments (Cross-DAW) + +| Plugin | Type | Why It Matters | Web Audio Feasibility | +|---|---|---|---| +| **Serum** (Xfer) | Wavetable | Industry standard for EDM/pop. Visual wavetable editor, drag-and-drop modulation | **High** — wavetable playback is straightforward in Web Audio | +| **Vital** (Matt Tytel) | Wavetable | Free/open-source Serum alternative. Same feature set, spectral warping | **High** — open source, could reference implementation | +| **Surge XT** | Hybrid | Open-source, 3 oscillators (classic/modern/FM/wavetable/string), dual filter, mod matrix | **Medium** — complex but core concepts are portable | +| **Diva** (u-he) | Virtual analog | CPU-heavy analog modeling, zero-delay feedback filters | **Low** — too CPU intensive for Web Audio | +| **Kontakt** (NI) | Multi-sampler | Industry standard sampler, scripting, massive library ecosystem | **Low** — library system not applicable to web | +| **Omnisphere** (Spectrasonics) | Hybrid | Granular, wavetable, sample, most versatile synth | **Low** — too complex | + +--- + +## 5. Web Audio DAW Competitors + +| Platform | Instruments Available | +|---|---| +| **BandLab** | 100+ virtual instruments (MIDI), basic synth presets, drum machine, sample library | +| **Soundtrap** (Spotify) | Built-in synths with presets, pattern beat maker, loops library | +| **Amped Studio** | Subtractive synth, FM synth, drum machine, sampler, SF2 support | +| **Splice** | Sample-based only (drag-and-drop), no built-in synthesis | +| **AudioTool** | Multiple synths (Heisenberg FM, Machiniste subtractive, Pulverisateur granular), most complete web synths | + +**Key takeaway**: AudioTool is the gold standard for web-based synthesis. BandLab and Soundtrap focus on presets over deep editing. ACE-Step can differentiate with AI-integrated synthesis. + +--- + +## 6. Synthesis Types — Priority for ACE-Step DAW + +| Synthesis Type | Description | Web Audio Feasibility | Priority | Justification | +|---|---|---|---|---| +| **Subtractive** (enhanced) | Oscillators → Filter → Amp. Add proper filter ADSR, LFO routing, unison, detune | **Very High** — Tone.js native | **P0** | Current SynthEngine is too basic. Every DAW has this. | +| **FM Synthesis** | Operators modulating each other's frequency | **Very High** — Tone.FMSynth exists | **P1** | Already in Tone.js. Operator-style FM is a staple. | +| **Wavetable** | Cycle through wavetable frames for evolving timbres | **High** — PeriodicWave API + custom buffers | **P1** | Most popular modern synth type (Serum, Vital, Wavetable) | +| **Sample-based (multi)** | Key/velocity-mapped zones, round-robin | **High** — buffer playback with zone logic | **P1** | Required for realistic instruments (piano, strings, drums) | +| **Granular** | Tiny grains from audio buffer, randomized playback | **High** — AudioBufferSourceNode with scheduling | **P2** | Unique textures, ambient/experimental. Feasible in Web Audio | +| **Physical Modeling** | Waveguide/Karplus-Strong string models | **Medium** — needs AudioWorklet for efficiency | **P2** | Unique sounds, CPU moderate in AudioWorklet | +| **Additive** | Sum of sine partials with individual control | **Medium** — many oscillators needed | **P3** | Niche use, CPU heavy with many partials | +| **Spectral** | FFT-based sound manipulation | **Medium** — AnalyserNode + IFFT in AudioWorklet | **P3** | Advanced, unique to few DAWs | + +--- + +## 7. Gap Analysis — ACE-Step DAW vs Mainstream + +### Critical Gaps (P0 — blocks basic music production) + +1. **No synth parameter editing UI** — Users can only pick presets, can't tweak filter cutoff, ADSR, oscillator shape. Every DAW exposes these. +2. **No filter envelope on synth** — Only amplitude ADSR exists. Filter ADSR is fundamental to subtractive synthesis. +3. **No LFO on synth oscillators/filter** — Only global filter effect has LFO. Per-voice LFO is standard. +4. **No unison/detune** — Can't stack voices for thick sounds. Basic feature in all synths. +5. **Effects chain not connected to live audio** — S1-03 blocker: effects are UI-only, not wired to playback. + +### High Priority Gaps (P1 — expected in any serious DAW) + +6. **No FM synthesis on tracks** — Tone.FMSynth exists but not exposed as a track instrument type. +7. **No wavetable synthesis** — Most popular modern synth type, feasible via Web Audio PeriodicWave. +8. **No multi-sample instrument** — Current sampler loads one sample. Can't build realistic piano/strings/drums. +9. **No drum sample loading** — DrumEngine is synthesis-only. Users can't drop their own samples on pads. +10. **No per-pad effects/tuning on drum machine** — Fixed sound parameters per kit. +11. **No velocity layers/crossfading** — No dynamic sample switching based on velocity. +12. **No sample slicing** — Can't auto-slice loops and trigger slices via MIDI. + +### Medium Priority Gaps (P2 — differentiators) + +13. **No granular synthesis engine** — Unique textures for ambient/experimental music. +14. **No physical modeling** — Karplus-Strong / waveguide would add unique pluck/string sounds. +15. **No modulation matrix** — Can't route LFOs/envelopes to arbitrary parameters. +16. **No convolution reverb** — Only algorithmic reverb. Convolver is built into Web Audio API. +17. **No audio warp/time-stretch** — Sampler uses playbackRate which changes pitch with tempo. +18. **No sidechain routing UI** — CompressorParams has `sidechainSourceTrackId` but no UI. + +### Lower Priority Gaps (P3 — advanced/niche) + +19. **No additive synthesis** — Partial-level control for specialized sound design. +20. **No spectral processing** — FFT-based editing/morphing. +21. **No MPE support** — Per-note expression for modern controllers. +22. **No preset browser/manager** — No way to save/load/share synth presets. +23. **No freeze/bounce** — Can't render instrument tracks to audio for CPU savings. + +--- + +## 8. Recommended Implementation Roadmap + +### Phase 1: Enhanced Subtractive Synth (P0) +- Expose synth parameters in UI (oscillator type, filter cutoff/resonance, ADSR for amp+filter) +- Add per-voice filter with ADSR envelope +- Add LFO with routeable destinations (pitch, filter, amp) +- Add unison/detune controls +- Wire effects chain to live audio (S1-03) + +### Phase 2: New Synthesis Engines (P1) +- FM Synth track type (expose Tone.FMSynth with operator controls) +- Wavetable synth (custom wavetable loading + morphing) +- Multi-sample instrument (zone editor, velocity layers) +- Enhanced drum rack (sample loading per pad, per-pad tuning/effects) + +### Phase 3: Advanced Features (P2) +- Granular synthesis engine (AudioWorklet-based) +- Physical modeling (Karplus-Strong in AudioWorklet) +- Modulation matrix (visual routing) +- Convolution reverb (IR loading) +- Audio warp/time-stretch + +### Phase 4: Polish (P3) +- Preset browser and manager +- MPE support +- Freeze/bounce to audio +- Additive/spectral engines diff --git a/.llm/research/drum-machine-sequencer-ux.md b/.llm/research/drum-machine-sequencer-ux.md new file mode 100644 index 00000000..cfefa70b --- /dev/null +++ b/.llm/research/drum-machine-sequencer-ux.md @@ -0,0 +1,142 @@ +# Drum Machine & Step Sequencer UX Research + +> Date: 2026-03-27 | Scope: Drum machine/sequencer UX in hardware + DAWs vs ACE-Step + +--- + +## 1. Elektron — Gold Standard for Step Sequencing + +### Parameter Locks (P-Locks) +- **Per-step parameter changes**: Every step can have unique pitch, amp, filter, decay, sample, etc. +- **Workflow**: Hold step button + turn knob = parameter lock on that step +- **LED feedback**: Locked step LED flashes rapidly +- **Limits**: Up to 72 unique parameter locks per pattern (Analog Rytm) +- **Why it matters**: "Elektron gets so much love because many machines can parameter lock sounds per step" + +### Sound Locks +- **Per-step sample switching**: Assign different sample to each step on same track +- **Use case**: Kick track can also play toms, percussion on specific steps +- **Eliminates track waste**: Don't need 4 tracks for 4 tom hits + +### Conditional Trigs (Probability) +- **Per-step probability**: 1-100% chance of triggering +- **Logical conditions**: "Play on 2nd loop only", "Play if previous trig fired" +- **Fill mode**: Steps that only play when Fill button is held +- **Mutual exclusion**: A/B trigs — one OR the other, never both + +### Digitakt II (2025) — Latest Evolution +- **128 steps** (double previous) +- **Euclidean sequence generator**: Math-based pattern distribution +- **16 levels of velocity**: Per-step velocity grid +- **Retrig mode**: Rapid-fire retriggering (ratchets) +- **Preset pool**: Load multiple sounds per track for sound locks + +### What Users Want in Software Equivalents +- Different sequence length per track (polyrhythm) +- Different time division per track +- Parameter locks on maximum parameters +- Different playback direction per track +- Ratchets, probability, conditional trigs +- Velocity, gate, accent, slide per step + +## 2. Ableton Drum Rack + Step Sequencer + +### Drum Rack +- **128 pad slots**: Far more than the standard 16 +- **Per-pad chain**: Each pad has its own instrument + effect chain +- **Choke groups**: 16 configurable choke groups +- **Macro knobs**: 8 assignable macros +- **Return chains**: Parallel send effects within the rack +- **Drag-and-drop sample assignment**: From browser to pad + +### 16 Velocities Mode +- **Spread velocity across 16 pads**: One sound, 16 velocity levels +- **MPC-style velocity mapping**: Physical finger pressure = velocity + +## 3. Logic Pro Step Sequencer (10.5+) + +### Row-Based Design +- Each row = one sound (kit piece or note) +- **Per-step editing**: Velocity, gate, tie, skip, loop point +- **Pattern regions**: Place step patterns on timeline like clips +- **Visual velocity**: Color gradient on steps (cool→hot) + +## 4. MPC / Maschine + +### Performance-First Design +- **4x4 pad grid**: Industry standard layout +- **Finger drumming**: Real-time velocity from pad pressure +- **Note repeat**: Hold pad + repeat button = rapid-fire at tempo subdivision +- **Pad mode switching**: Pad, Keyboard, 16 Levels, Step Seq +- **Per-pad tuning**: Pitch knob per pad +- **Chop mode**: Slice sample and assign slices to pads + +## 5. Roland TR-808/909 (Hardware Legacy) + +### Step Pattern Entry +- 16 step buttons, each represents one 16th note +- **Toggle on/off**: Press step button to activate +- **Accent**: Separate accent button toggles per step +- **Instrument select**: Choose kick/snare/hat, then edit its pattern +- **LED step indicator**: Current playing step lights up + +## 6. ACE-Step Current State + +### DrumMachineEditor.tsx +- **4x4 pad grid**: 16 pads with unique colors +- **MPC-style velocity**: Mouse Y position determines velocity (innovative!) +- **Keyboard mapping**: ZXCV/ASDF/QWER/1234 for pads +- **Kit selection**: 808, Acoustic, Electronic, Lo-Fi +- **Visual feedback**: 150ms active state highlight +- **Per-pad volume**: Volume fader per pad + +### SequencerEditor.tsx +- Step grid with toggle on/off +- Per-step velocity +- Swing control (0-100%) +- 7 preset patterns (Rock, Pop, Hip-Hop, EDM, Reggae, Jazz, Bossa Nova) + +### Key Gaps +| Feature | Elektron | MPC | Ableton | ACE-Step | +|---|---|---|---|---| +| Per-step parameter locks | Yes (best) | No | No | No | +| Per-step probability | Yes | No | No | No | +| Conditional trigs | Yes | No | No | No | +| Note repeat / ratchet | Yes | Yes | No | No | +| Per-step pitch offset | Yes | No | No | No | +| Polyrhythmic rows | Yes | No | No | No | +| Choke groups | Yes | No | Yes | No | +| Sample loading per pad | Yes | Yes | Yes | No | +| 128 pad slots | No | No | Yes | 16 only | +| Pattern chaining | Yes | Yes | Yes | No | +| Euclidean generator | Yes (Digitakt II) | No | No | No | + +--- + +## 7. Recommendations for ACE-Step + +### Phase 1: Essential +- Per-step velocity painting (drag across velocity bars) +- Sample loading per pad (drag-and-drop audio files) +- Choke groups (hi-hat closed chokes open) + +### Phase 2: Creative +- Per-step probability (right-click step → set %) +- Per-step pitch offset (±12 semitones) +- Note repeat / ratchet (2x/3x/4x subdivisions) +- Pattern chaining (A→B→Fill sequence) + +### Phase 3: Advanced +- Parameter locks (per-step filter, decay, pan) +- Polyrhythmic rows (different step counts per row) +- Euclidean pattern generator +- Micro-timing nudge per step + +--- + +## Sources + +- [Elektron Analog Rytm Manual — Sequencer Features, Parameter Locks](https://www.manualsdir.com/manuals/657159/elektron-analog-rytm.html?page=48) +- [Digitakt II — Creativity-Unlocking 16 Track Drum Computer](https://www.elektron.se/explore/digitakt-ii) +- [Parameter Locking: What Is It Exactly? — Loopy Pro Forum](https://forum.loopypro.com/discussion/26340/parameter-locking-what-is-it-exactly) +- [Software Step Sequencer with Elektron-Like Features? — Gearspace](https://gearspace.com/board/electronic-music-instruments-and-electronic-music-production/1405617-software-step-sequencer-quot-elektron-like-quot-features.html) diff --git a/.llm/research/effects-plugin-ui.md b/.llm/research/effects-plugin-ui.md new file mode 100644 index 00000000..3cc117bc --- /dev/null +++ b/.llm/research/effects-plugin-ui.md @@ -0,0 +1,144 @@ +# Effects & Plugin UI Research + +> Date: 2026-03-27 | Scope: Effects chain UI design in mainstream DAWs vs ACE-Step + +--- + +## 1. Ableton Live Device View + +### Horizontal Device Chain +- **Left-to-right signal flow**: Devices displayed in order of processing +- **Drag-to-reorder**: Grab device title bar, drag left/right to change order +- **Drag-to-add**: From browser directly into chain — insertion point highlighted +- **Collapse/expand**: Click triangle to minimize device to title bar only +- **Hot-swap mode**: Press Q to enter swap mode, browse replacements while hearing audio + +### Audio Effect Rack +- **Parallel processing**: Split signal into chains, each with independent devices +- **Chain zones**: Map velocity, key, or chain selector ranges to activate chains +- **Macro knobs**: 16 assignable macros (expanded from 8 in Live 12) +- **Macro variations**: Save/recall snapshots of macro positions +- **Nested racks**: Racks inside racks for complex routing + +### Device Visualization +- **EQ Eight**: Interactive frequency curve with draggable nodes +- **Compressor**: Real-time gain reduction display with threshold line +- **Spectrum**: Built-in analyzer overlaid on EQ +- **Saturator**: Drive curve visualization +- **Community request**: Mini plugin visualization in mixer (like Studio One's fat channel) + +### Redesign Insights (Nenad Milosevic Case Study) +- Proposed: Contextual preset browsing closer to each device +- Proposed: Enhanced stereo controls per device (width, mid/side) +- Problem identified: Users lose track of which device is selected in long chains + +## 2. Logic Pro Plugin System + +### Channel Strip Insert Slots +- **Vertical stack**: Click empty slot → categorized plugin menu +- **Audio FX / Instrument / MIDI FX**: Three distinct plugin types +- **Bypass per slot**: Option+click to bypass individual plugin +- **Drag to reorder**: Within the insert stack +- **Channel EQ**: Built-in per channel, always available (not an insert) +- **Smart Controls**: 8 macro knobs auto-mapped to most important plugin parameters + +### Plugin Window Management +- **Link mode**: Single floating window updates to show selected plugin +- **Multiple windows**: Open several plugin UIs simultaneously +- **Resize**: Logic plugins are resizable (third-party depends on developer) + +## 3. FL Studio Mixer Effects + +### 10 Slots Per Track +- **Numbered slots (1-10)**: Clear visual ordering +- **Click slot**: Opens plugin picker with categories +- **Route to sidechain**: Right-click → sidechain to any other mixer track +- **Patcher**: Visual node-based effects routing (unique to FL) +- **Fruity Convolver**: Convolution reverb with IR loading and visual display + +## 4. Bitwig Studio Device System + +### Unified Device Architecture +- **Everything is a device**: Instruments, effects, modulators, containers +- **Device nesting**: Any device can contain other devices +- **Grid**: Modular synthesis environment built into the DAW +- **Per-note effects**: Effects that process individual notes independently (MPE-aware) +- **Modulators**: LFOs, envelopes, step sequencers, math operators as modulation sources + +### Visual Signal Flow +- **Color-coded connections**: Audio (orange), modulation (blue), note (green) +- **Hoverable routing**: Hover over connection to see signal path +- **Split/layer containers**: Visual parallel/serial routing + +## 5. Studio One Fat Channel + +### Inline Mixer Integration +- **Mini plugin views**: Compressed plugin UI directly in mixer channel strip +- **Channel strip plugins**: EQ, compressor, gate, limiter as built-in strip modules +- **Drag between channels**: Copy effect chains between tracks +- **Console shaping**: Analog console emulation per channel + +## 6. ACE-Step Current State + +### EffectsEngine.ts (632 lines) +- 10 effect types: reverb, delay, chorus, phaser, distortion, compressor, eq, filter, tremolo, pingPongDelay +- Tone.js wrappers: `Tone.Reverb`, `Tone.FeedbackDelay`, `Tone.Chorus`, etc. +- **Not wired to live audio**: Effects exist in code but aren't connected to the audio graph +- createEffect/updateEffect/removeEffect API exists + +### TrackNode.ts (425 lines) +- Per-track channel strip: 3-band EQ, compressor, reverb send +- These ARE wired to audio (unlike EffectsEngine) +- Fixed processing order, not user-configurable + +### UI: InsertSection in ChannelStrip +- Max 4 insert slots +- Add button opens effect type selector +- Basic parameter display + +### Key Gaps +| Feature | Competitors | ACE-Step | +|---|---|---| +| Drag-to-reorder effects | All | No | +| Drag-from-browser to chain | Ableton, Logic | No | +| Parallel processing (racks) | Ableton, Bitwig | No | +| Effect bypass per slot | All | Toggle exists | +| Visual EQ curve | Logic, Ableton | Knobs only | +| Gain reduction meter | All | No | +| Plugin preset browser | All | No | +| Hot-swap mode | Ableton | No | +| Sidechain routing | FL, Ableton | No | +| 10+ insert slots | Pro Tools, FL | Max 4 | + +--- + +## 7. Recommendations for ACE-Step + +### Phase 1: Usable Effects Chain +- Increase insert slots to 8 +- Drag-to-reorder effects in chain +- Visual EQ curve display (interactive nodes) +- Gain reduction meter on compressor +- Effect preset browser (per-effect presets) + +### Phase 2: Advanced Routing +- Parallel effect chains (rack-like container) +- Sidechain routing between tracks +- Wet/dry mix per effect (parallel compression, etc.) +- Copy/paste effect chains between tracks + +### Phase 3: Creative Tools +- Modulation routing to effect parameters +- Convolution reverb with IR loading +- Multi-band processing container +- Visual signal flow diagram + +--- + +## Sources + +- [Ableton Live 12 Interface — Device View](https://www.ableton.com/en/live/learn-live/interface/) +- [Ableton Live Redesign Case Study](https://nenadmilosevic.co/ableton-live-redesign/) +- [Mixing Music in Ableton Live: Effects Overview](https://www.admiralbumblebee.com/music/2019/04/27/Mixing-music-in-Live.html) +- [Bitwig Studio Device Architecture Overview](https://www.bitwig.com/stories/device-architecture/) +- [Studio One Fat Channel — PreSonus](https://www.presonus.com/learn/technical-articles/fat-channel) diff --git a/.llm/research/mixer-channel-strip-ui.md b/.llm/research/mixer-channel-strip-ui.md new file mode 100644 index 00000000..96044d24 --- /dev/null +++ b/.llm/research/mixer-channel-strip-ui.md @@ -0,0 +1,101 @@ +# Mixer & Channel Strip UI Research + +> Date: 2026-03-27 | Scope: Mixer UI design in mainstream DAWs vs ACE-Step + +--- + +## 1. Ableton Live Mixer + +### Session View Mixer +- **Integrated fader+meter**: Meter is part of the fader track (green=RMS, dark green=peak) +- **Adjustable fader height**: Can resize but limited range +- **Fader scale issue**: Half the fader = -24dB (not great for gain staging) +- **Sends**: Unlabeled in Arrangement view — must count to know which is which +- **Track colors**: Added at bottom of channels in expanded view for navigation +- **Pan**: Actually a stereo balance control (true pan on mono tracks only) +- **Constant power panning**: Sinusoidal gain curves, +3dB at hard L/R + +### Ableton Device View (Effects) +- **Horizontal chain**: Devices displayed left-to-right at bottom of screen +- **No insert slots**: Effects go in Device View, not channel strip +- **Drag-to-add**: From browser to Device View +- **Racks**: Group devices into Audio Effect Rack for parallel processing + +### Redesign Case Study (Nenad Milosevic, 2025) +- Proposed enhanced stereo controls: Pan, Balance, and Stereo Width per channel +- Simplified I/O: 3 elements instead of 4 dropdowns + 3 buttons +- Contextual preset browsing closer to each device +- Community wanted: mini plugin visualization in mixer (like Studio One) + +## 2. Logic Pro Channel Strip +- **Vertical insert list**: Click slot → select plugin from categorized menu +- **Pre/post fader toggle** per send +- **I/O section**: Input source, output bus, clear routing display +- **Channel EQ**: Built-in EQ per channel with visual curve +- **Compressor**: Built-in per channel with gain reduction meter +- **Smart Controls**: 8 macro knobs mapped to most important parameters + +## 3. Pro Tools Mix Window +- **10 insert slots**: Drag to reorder, bypass per slot +- **10 send slots**: Pre/post fader, pan per send +- **Playlists**: Per-track take management +- **VCA faders**: Group control without summing +- **Solo-safe**: Ctrl+click solo button +- **Detailed I/O**: Input, output, bus, and side-chain routing per channel + +## 4. FL Studio Mixer +- **100 mixer tracks**: Numbered, with routing matrix +- **Visual routing**: Lines showing signal flow between tracks +- **10 effect slots per track**: Clear numbered list +- **Separation display**: Audio routing visualized as colored lines +- **Sidechain**: Direct routing between any mixer tracks + +## 5. Web DAW Mixers +- **BandLab**: Simplified mixer with fader + pan per track, no insert slots +- **Soundtrap**: Basic volume/pan only +- **Amped Studio**: More complete — fader, pan, mute/solo, effect inserts + +## 6. ACE-Step Current State + +### What Exists +- MixerPanel with ChannelStrip per track (min 132px width) +- VerticalFader (96px min height) with keyboard support +- Pan knob, 3-band EQ knobs, compressor toggle + threshold/ratio +- Insert section (max 4 effects) +- Send section (max 2 returns) +- LevelMeter with animated height (75ms transition) +- Solo, Mute buttons +- role="group", aria-label, tabindex for keyboard nav + +### Key Gaps vs Competitors +| Feature | Competitors | ACE-Step | +|---|---|---| +| Peak + RMS dual metering | All major DAWs | Peak only | +| dB scale markings | All | None | +| 10+ insert slots | Pro Tools, FL | Max 4 | +| Pre/post fader sends | All | No toggle | +| Solo-safe | All | No | +| Visual routing | FL Studio | No | +| VCA/group faders | Pro Tools, Logic | No | +| Channel EQ curve display | Logic, Ableton | Knobs only | + +--- + +## 7. Recommendations for ACE-Step + +1. **Add RMS metering** alongside peak (light green = RMS, dark = peak, like Ableton) +2. **dB scale markings** on fader track (-inf, -48, -24, -12, -6, -3, 0, +6) +3. **Increase insert slots** to 8 (expandable) +4. **Pre/post fader send toggle** (click label to switch) +5. **Solo-safe mode** (right-click solo → "Solo Safe") +6. **Mini EQ curve** display on channel strip (visualize 3-band settings) +7. **Gain reduction meter** on compressor section + +--- + +## Sources + +- [Ableton Live Redesign Case Study](https://nenadmilosevic.co/ableton-live-redesign/) +- [Mixing Music in Ableton Live: An Overview](https://www.admiralbumblebee.com/music/2019/04/27/Mixing-music-in-Live.html) +- [Mix Tech Series Part 1: Levels & Panning](https://abletunes.com/blog/mix-tech-series-part-1-levels-panning/) +- [Ableton Live 12 Interface](https://www.ableton.com/en/live/learn-live/interface/) diff --git a/.llm/research/onboarding-workflow-ux.md b/.llm/research/onboarding-workflow-ux.md new file mode 100644 index 00000000..4329c297 --- /dev/null +++ b/.llm/research/onboarding-workflow-ux.md @@ -0,0 +1,144 @@ +# Onboarding & Workflow UX Research + +> Date: 2026-03-27 | Scope: First-run experience and workflow design in DAWs vs ACE-Step + +--- + +## 1. GarageBand — Gold Standard for Simplicity + +### Zero-Friction Start +- **Open → pick instrument → play**: Under 30 seconds to first sound +- **No configuration needed**: Auto-detects audio interface, sets buffer/sample rate +- **Live Loops grid**: Tap cells to trigger loops — music within 10 seconds +- **Sound Library**: Curated, categorized, immediate preview +- **Smart instruments**: Auto-chord strumming, drum machine with one-tap patterns + +### Progressive Complexity +- **Drummer track**: AI drummer that follows your song (adjusts complexity, fills, pattern) +- **Track types are clear**: Audio, Instrument, Drummer — no jargon +- **Export to Logic**: Seamless upgrade path when users outgrow GarageBand + +### What Makes It Work +- **Opinionated defaults**: 120 BPM, 4/4, C major — just go +- **No empty state**: Always starts with a template or instrument ready +- **Visual, not technical**: Knobs look like physical knobs, not parameter sliders + +## 2. BandLab — Free Cloud DAW Onboarding + +### Instant Access +- **No download, no payment**: Open browser → sign up → create +- **Mobile-first**: iOS/Android apps with same project format +- **Social features**: Follow artists, remix public tracks, comment on waveforms +- **Templates**: Genre-specific starting points with pre-loaded loops + +### Collaboration as Default +- **Share link → co-edit**: Real-time collaboration built into the core flow +- **Fork/remix**: Public projects can be forked (like GitHub for music) +- **In-app recording**: Record vocals/guitar directly in browser +- **Mastering**: One-click AI mastering (free) + +## 3. Cakewalk Next — Modern Onboarding Redesign + +### Simplified from Cakewalk/SONAR +- **Guided first session**: Step-by-step tutorial overlay on first launch +- **Quick start templates**: Electronic, Rock, Hip-Hop, Singer/Songwriter +- **Contextual help**: ? icons that explain each section on hover +- **Streamlined UI**: Removed power-user features from default view +- **Cloud projects**: Save to cloud by default, local export optional + +## 4. Ableton Live — Session View as Playground + +### Unique Onboarding Advantage +- **Session View**: Non-linear clip launching — experimentation without commitment +- **Record to Arrangement**: Move from exploration to structure naturally +- **Built-in lessons**: Interactive tutorials in the Help menu +- **Pack browser**: Curated sound packs with preview + +### Pain Points (from user research) +- **Steep learning curve**: Dual-view concept confuses beginners +- **No guided tour**: Users must seek out lessons themselves +- **Browser overwhelm**: Too many presets/samples without curation for beginners + +## 5. FL Studio — Pattern-First Workflow + +### Beginner-Friendly Pattern System +- **Channel Rack**: Add instruments, draw patterns — visual and immediate +- **Step sequencer**: Toggle steps on/off for drums — most intuitive beat-making +- **Playlist**: Arrange patterns on timeline — separation of "create" and "arrange" +- **Lifetime free updates**: Buy once, never worry about versions + +### Onboarding Flow +- **Demo songs**: Ships with complete projects to study +- **Suggested channels**: Pre-loaded with kick, clap, hat, snare +- **Right-click help**: Every element has "What's this?" in context menu + +## 6. AI-First Music Tools — New Paradigm + +### Suno / Udio +- **Text prompt → full song**: No musical knowledge required +- **Genre/mood selection**: Dropdown instead of technical parameters +- **Iterate by description**: "Make the chorus more energetic" +- **Relevance to ACE-Step**: ACE-Step already has AI generation — onboarding should lean into this + +### AIVA +- **Genre template → AI composition**: MIDI output that users can edit +- **Emotion-based presets**: "Joyful", "Tense", "Melancholic" +- **Progressive editing**: Start with AI, gradually take manual control + +### Implications for ACE-Step +- **AI generation IS the onboarding**: New users generate a track first, then learn to edit +- **Prompt-to-music eliminates empty canvas fear**: Most powerful onboarding possible +- **Hybrid flow**: Generate → inspect → modify → learn + +## 7. ACE-Step Current State + +### What Exists +- Project loads with empty timeline +- Track type selection: stems, sample, sequencer, pianoroll +- AI generation panel (ACE-Step model integration) +- Keyboard shortcuts dialog +- Template system (project templates exist in store) + +### Key Gaps +| Feature | Competitors | ACE-Step | +|---|---|---| +| Guided first-run tour | GarageBand, Cakewalk Next | No | +| Template gallery on launch | All | Empty project | +| Interactive tutorials | Ableton, GarageBand | No | +| Demo projects | FL Studio, Logic | No | +| Contextual help tooltips | Cakewalk Next, Logic | Keyboard shortcuts only | +| Progressive disclosure | GarageBand, Logic Alchemy | Partial | +| AI-first onboarding flow | Suno/Udio concept | AI exists but not as entry point | +| Quick start wizard | BandLab | No | + +--- + +## 8. Recommendations for ACE-Step + +### Phase 1: First-Run Experience +- **AI-first onboarding**: New users land on "Describe your song" prompt +- **Generate → Edit flow**: AI creates initial track, user learns editing by modifying it +- **Template gallery**: Show genre templates instead of empty project +- **Contextual tooltips**: Hover hints on every major UI element (500ms delay) + +### Phase 2: Learning System +- **Interactive walkthrough**: Step-by-step overlay highlighting timeline, mixer, piano roll +- **Demo projects**: 3-5 complete projects users can explore and modify +- **"What's this?" mode**: Toggle that shows explanations on click +- **Progress badges**: Track user's feature discovery (optional gamification) + +### Phase 3: Workflow Optimization +- **Quick start wizard**: BPM, key, genre, track count → pre-configured project +- **Workflow presets**: "Beat Making", "Songwriting", "Sound Design" — each shows relevant panels +- **Keyboard shortcut trainer**: Interactive drill for common shortcuts + +--- + +## Sources + +- [GarageBand for Mac — Apple](https://www.apple.com/mac/garageband/) +- [BandLab: Make Music Online — Free DAW](https://www.bandlab.com/) +- [Cakewalk Next: A New Beginning — BandLab Blog](https://blog.bandlab.com/cakewalk-next/) +- [Ableton Live 12 — Learn Live Tutorials](https://www.ableton.com/en/live/learn-live/) +- [Suno AI Music Generator](https://suno.com/) +- [AIVA: The AI Music Composition Assistant](https://www.aiva.ai/) diff --git a/.llm/research/piano-roll-midi-ux.md b/.llm/research/piano-roll-midi-ux.md new file mode 100644 index 00000000..0b24db3c --- /dev/null +++ b/.llm/research/piano-roll-midi-ux.md @@ -0,0 +1,115 @@ +# Piano Roll & MIDI Editing UX Research + +> Date: 2026-03-27 | Scope: MIDI editing UX in mainstream DAWs vs ACE-Step + +--- + +## 1. FL Studio Piano Roll — Best in Class + +### Why It's Considered the Best +- "The well-deserved reputation as the best Piano Roll in the business" — industry consensus +- Combination of speed, visual clarity, and creative tools +- Mouse-friendly: Adding, removing, moving, editing — intuitive with minimal clicks +- Any user from any DAW can learn it comfortably in minutes + +### Key Features +- **Ghost Notes**: Semi-transparent notes from other channels visible behind current channel — essential for harmonic reference +- **Chord Stamps**: Place predefined chords (30+ types) with one click via Stamp tool +- **Slide Notes**: Special note type that glides pitch between notes (unique to FL) +- **Scale Highlighting**: Visual highlight of scale notes on piano keyboard +- **Quantize Strength**: 0-100% slider — 75% preserves feel while cleaning timing +- **Tools**: Draw, Paint, Delete, Select, Zoom, Slice modes +- **Multi-Channel Editing**: Switch between MIDI channels within same piano roll + +### What Makes It Mouse-Friendly +- Only 2 basic mechanics: click mouse + hold modifier key +- Discovering functionality happens quickly +- Right-click = delete note (no switching to eraser tool) +- Left-click = draw note (default mode) + +## 2. Ableton MIDI Clip Editor + +### Design Philosophy +- **Fold mode**: Hide unused note rows, show only played notes +- **Note stretch markers**: Drag note edges to adjust duration +- **Velocity lane**: Bottom section with draggable velocity stems +- **Legato button**: Extend all selected notes to touch the next +- **MPE editing**: Per-note pitch bend, slide, and pressure (Live 12+) + +## 3. Logic Pro Piano Roll + +### Unique Strengths +- **Brush tool**: Paint repeated notes at grid divisions +- **Scale Quantize**: Snap notes to selected musical scale +- **MIDI Transform**: Batch operations (transpose, randomize, thin CC data) +- **Articulation management**: Per-note articulation switching for orchestral libraries +- **Step input**: Type notes one at a time using MIDI keyboard + duration keys + +## 4. Bitwig Studio + +### Per-Note Expression +- **Micro-pitch editing**: Detune individual notes in cents +- **Per-note modulation**: Expression data per note (MPE native) +- **Clip modulation**: Modulators that run per-clip, not per-track +- **Operator tool**: Select/draw/split/paint modes with clear visual feedback + +## 5. ACE-Step Current State + +### What Exists (PianoRoll.tsx + PianoRollCanvas.tsx) +- Tool palette: Select, Pencil, Paint, Erase, Slide +- Keyboard shortcuts: V/B/X/1/2/3/4/5 for tools +- Grid sizes: 1/16, 1/8, 1/4, 1/2, whole +- Ghost notes toggle +- Chord shapes selector (Major, Minor, Diminished, etc.) +- Velocity lane at bottom +- Canvas-based rendering +- Zoom X slider +- isSlide property on MidiNote type + +### Key Gaps vs Competitors +| Feature | FL Studio | Ableton | Logic | ACE-Step | +|---|---|---|---|---| +| Ghost notes quality | Best | Good | No | Toggle exists | +| Chord stamp (one-click) | 30+ chords | No | No | Selector exists, UX unclear | +| Scale highlighting | Yes | Fold mode | Yes | No | +| Lock-to-scale | No | No | Yes | No | +| Velocity painting (alt+drag) | Yes | Yes | Yes | Basic lane | +| Quantize strength slider | Yes | Yes | Yes | No strength control | +| Humanize | Yes | No | Yes | No | +| Legato (extend to next) | Yes | Yes | Yes | No | +| Strumming tool | Yes | No | No | No | +| Multi-note time stretch | Yes | Yes | Yes | No | +| Note expression / MPE | No | Yes | No | No | +| Brush/paint tool | Yes | No | Yes | "Paint" tool exists | + +--- + +## 6. Recommendations for ACE-Step + +### Phase 1: Core Editing +- Scale highlighting on piano keyboard (from project key) +- Quantize strength slider (0-100%) +- Velocity painting: Alt+drag in velocity lane +- Legato: Select notes → L key + +### Phase 2: Creative Tools +- Ghost notes rendering quality improvement +- Humanize: Randomize timing (±ticks) and velocity +- Strumming: Apply strum offset to chord +- Chord stamp: One-click chord placement (FL-style) + +### Phase 3: Advanced +- Lock-to-scale mode +- Multi-note time stretch +- MPE/expression editing +- Step input mode + +--- + +## Sources + +- [FL Studio Piano Roll 101: Best Features & Key Functions](https://unison.audio/fl-studio-piano-roll/) +- [How to Use the Piano Roll in FL Studio: Complete Guide (2026)](https://www.audeobox.com/learn/fl-studio/how-to-use-the-piano-roll-in-fl-studio/) +- [DAW with the Best Piano Roll/MIDI Editor? — VI-CONTROL](https://vi-control.net/community/threads/daw-with-the-best-piano-roll-midi-editor.36951/) +- [30 Days with FL Studio 20: Piano Roll Basics](https://www.admiralbumblebee.com/music/2018/06/24/30-days-with-FLStudio-20-Piano-Roll-Basics.html) +- [FL Studio Review: The Producer's Playground in 2025](https://www.thedystopiancollective.com/softwarereviews/fl-studio-review-the-producers-playground-in-2025) diff --git a/.llm/research/recording-input-monitoring-ux.md b/.llm/research/recording-input-monitoring-ux.md new file mode 100644 index 00000000..ec42cf68 --- /dev/null +++ b/.llm/research/recording-input-monitoring-ux.md @@ -0,0 +1,152 @@ +# Recording & Input Monitoring UX Research + +> Date: 2026-03-27 | Scope: Audio recording, comping, and input monitoring in DAWs vs ACE-Step + +--- + +## 1. Pro Tools — Recording Gold Standard + +### Punch Recording +- **Quick Punch (Cmd+Shift+P)**: Instant punch in/out while playing — no pre-roll needed +- **Pre/Post Roll**: Configurable lead-in/out time for context before punch +- **Destructive Punch**: Records directly to file (broadcast/post-production) +- **Non-Destructive Punch**: New region on top, original preserved underneath + +### Loop Recording +- **Define loop region**: Set in/out markers on timeline +- **Each pass = new take**: Stacked automatically in playlist/take lanes +- **Playlist system**: Up to 999 playlists per track — each is a complete alternate take history +- **Clip ratings**: Star rating (1-5) per take for quick evaluation + +### Comping (Take Management) +- **Playlist view**: Expand track to show all takes stacked vertically +- **Swipe comping**: Click-drag across best sections of each take +- **Composite playlist**: Automatically created from selected sections +- **Crossfade at boundaries**: Automatic crossfades between comped sections +- **Flatten comp**: Consolidate final comp into single audio file + +### Input Monitoring +- **3 modes**: Auto (follows record state), Input Only, Playback Only +- **Low-latency monitoring**: Hardware monitoring bypass (when interface supports it) +- **Record-safe**: Prevent accidental recording on armed tracks +- **Gain staging**: Input trim per track, independent of fader + +## 2. Logic Pro Recording + +### Take Folders +- **Loop recording → take folder**: Multiple takes auto-stacked +- **Quick Swipe Comping**: Click-drag to select best parts visually — best comping UX in any DAW +- **Color-coded takes**: Each take has distinct color for visual clarity +- **Flatten and Merge**: Consolidate comp or keep editable +- **Move between takes**: Up/Down arrows to audition different takes + +### Replace vs Merge Recording +- **Replace mode**: New recording replaces existing content +- **Merge mode**: New recording layers on top (useful for building drum patterns) +- **Cycle recording**: Each cycle creates new take in take folder + +### Count-In & Metronome +- **1-bar count-in**: Configurable (1-4 bars) +- **Metronome options**: Tone, click, or external MIDI +- **Only during count-in**: Option to mute click after recording starts + +## 3. Ableton Live Recording + +### Session View Recording +- **Per-clip recording**: Arm slot → record into specific clip +- **Fixed-length recording**: Set clip length before recording +- **Overdub in clip**: Layer recordings in same clip slot +- **MIDI overdub**: Add notes to existing clip by playing + +### Arrangement Recording +- **Punch in/out**: Set loop braces as punch region +- **Session → Arrangement**: Record session clip launches into arrangement +- **Capture MIDI**: Retroactively capture what was just played (even without arming!) +- **Latency compensation**: Automatic delay compensation across all tracks + +### Capture MIDI (Unique Feature) +- **Played something great but wasn't recording?**: Press Capture button +- **Retroactively records**: Last ~30 seconds of MIDI input saved +- **Tempo detection**: Analyzes played timing to set project BPM +- **Game-changer**: Eliminates "I forgot to press record" frustration + +## 4. FL Studio Recording + +### Edison (Built-In Audio Editor) +- **Record directly into Edison**: Per-track audio recording +- **Non-destructive editing**: Cut, trim, normalize, pitch-shift +- **Spectral display**: View frequency content over time +- **Convolution reverb**: Record impulse → apply as reverb + +### Playlist Recording +- **Audio recording tracks**: Record directly to playlist as audio clips +- **Loop recording**: Stacked takes on same track +- **Slip editing**: Adjust clip start point without cutting + +## 5. Modern AI Recording Features + +### Melodyne/Auto-Tune Integration +- **Real-time pitch correction**: During or after recording +- **ARA2 integration**: Direct plugin access to audio in DAW (Logic, Studio One) +- **Note-level editing**: Pitch, timing, vibrato per note + +### AI Transcription +- **Audio → MIDI**: Transcribe recorded audio to MIDI notes +- **Drum separation**: Extract kick/snare/hat from mixed drum recording +- **Chord detection**: Identify chord progression from audio + +## 6. ACE-Step Current State + +### RecordingEngine.ts (522 lines) +- **MediaRecorder API**: Captures from getUserMedia (microphone input) +- **WebM/Opus codec**: Browser-supported format +- **Arm track → record**: Basic record/stop functionality +- **Audio stored as Blob**: Saved to project store +- **Metronome**: Exists in transport (Tone.js MetalSynth click) + +### Key Gaps +| Feature | Competitors | ACE-Step | +|---|---|---| +| Punch in/out | Pro Tools, Logic, Ableton | No | +| Loop recording | All | No | +| Take lanes / comping | Pro Tools (best), Logic | No UI (data model exists) | +| Count-in (bars) | All | No configurable count-in | +| Input monitoring modes | Pro Tools (3 modes) | Basic | +| Capture MIDI (retroactive) | Ableton | No | +| Latency compensation | All | No | +| Clip ratings | Pro Tools | No | +| Non-destructive punch | Pro Tools | No | +| Audio → MIDI transcription | Logic, Ableton 12 | No | +| Overdub / merge recording | Logic, Ableton | No | + +--- + +## 7. Recommendations for ACE-Step + +### Phase 1: Core Recording +- **Count-in**: 1-4 bar count-in before recording starts +- **Punch in/out**: Set markers for record region +- **Loop recording**: Cycle over region, stack takes +- **Input level meter**: Visual input level before/during recording + +### Phase 2: Take Management +- **Take lane UI**: Show stacked takes below track +- **Swipe comping**: Click-drag to select best sections (Logic-style) +- **Crossfade at comp boundaries**: Automatic 4ms crossfades +- **Clip ratings**: Star rating per take + +### Phase 3: Advanced +- **MIDI capture**: Retroactive MIDI recording (Ableton-style) +- **Latency compensation**: Measure and offset recording latency +- **Overdub mode**: Layer MIDI recordings in same clip +- **Audio → MIDI**: Basic pitch detection for monophonic audio + +--- + +## Sources + +- [Pro Tools Recording Modes — Avid Knowledge Base](https://resources.avid.com/SupportFiles/PT/Pro_Tools_Reference_Guide_2024.pdf) +- [Logic Pro: Record Using Cycle Mode — Apple Support](https://support.apple.com/guide/logicpro/record-using-cycle-mode-lgcp31714849/mac) +- [Quick Swipe Comping in Logic Pro — Apple Support](https://support.apple.com/guide/logicpro/quick-swipe-comping-lgce587b14f7/mac) +- [Ableton Live: Capturing MIDI — Ableton Manual](https://www.ableton.com/en/manual/capturing-midi/) +- [Recording Audio in Ableton Live — Ableton Manual](https://www.ableton.com/en/manual/recording-new-clips/) diff --git a/.llm/research/sample-browser-library.md b/.llm/research/sample-browser-library.md new file mode 100644 index 00000000..f8bbd87d --- /dev/null +++ b/.llm/research/sample-browser-library.md @@ -0,0 +1,154 @@ +# Sample Browser & Library UX Research + +> Date: 2026-03-27 | Scope: Sample browsing, library management, and audio preview in DAWs vs ACE-Step + +--- + +## 1. Ableton Live Browser — Industry Reference + +### Architecture +- **Left sidebar**: Always accessible, collapsible with Cmd+Alt+B +- **Category tabs**: Sounds, Drums, Instruments, Audio Effects, MIDI Effects, Max for Live, Samples, Packs +- **Search**: Unified search across all categories with real-time filtering +- **Tags**: Color tags for user organization, auto-tags by content type +- **Collections**: User-created labeled groups (like playlists for sounds) + +### Live 12.3 — Splice Integration +- **In-DAW Splice browsing**: Search Splice catalog directly from Ableton browser +- **One-click download**: Download sample → immediately available in browser +- **Preview in context**: Audition Splice samples at project tempo +- **Seamless workflow**: No app switching to find/download samples + +### Search with Sound (AI Feature) +- **Audio-based search**: Drag audio clip → find similar sounds in library +- **Timbre matching**: Uses ML to match sonic characteristics, not just tags +- **Replacement workflow**: "Find me a similar kick but punchier" + +### Preview System +- **Pre-listen button**: Headphone icon to toggle preview +- **Tempo-synced preview**: Loops play at project BPM (warped automatically) +- **Preview volume**: Independent volume control for preview +- **Hot-swap**: Press Q to swap device — hear replacements live while music plays + +### Drag-and-Drop +- **Drag to track**: Creates new clip with sample +- **Drag to Drum Rack pad**: Assigns sample to pad +- **Drag to Simpler/Sampler**: Loads as instrument +- **Drag to arrangement**: Places audio clip at drop point + +## 2. Logic Pro Sound Library + +### Library Panel +- **Patch browser**: Instruments organized by category (Keyboards, Strings, Synth Leads, etc.) +- **Loop browser**: Apple Loops with key, tempo, genre, instrument tags +- **Sound Effects**: Cinematic, foley, ambient categories +- **Download on demand**: Core library installed, expanded packs downloadable + +### Apple Loops +- **Key/tempo matching**: Loops auto-transpose and time-stretch to project settings +- **Color coding**: Green = audio loop, Blue = MIDI/Software Instrument loop +- **Favorites**: Star to bookmark frequently used loops +- **Column browser**: Filter by genre → instrument → mood + +## 3. FL Studio Browser + +### File Browser +- **Tree structure**: Folders with expandable subfolders +- **Plugin presets**: Browse presets per plugin in tree +- **Favorites**: Drag items to favorites folder +- **Search**: Filter current folder by text +- **Score templates**: MIDI pattern presets (chord progressions, arpeggios) + +### DirectWave Sampler +- **Multi-sample management**: Map samples across keyboard zones +- **Auto-mapping**: Detect pitch and assign to correct notes +- **Batch processing**: Apply processing to all samples at once + +## 4. Splice Desktop Integration + +### Standalone Model +- **Desktop app**: Browse, preview, download independently of DAW +- **Credit system**: Monthly credits for sample downloads +- **Smart search**: Filter by BPM, key, genre, instrument, mood +- **Similar sounds**: "More like this" button on any sample +- **Stems separation**: AI stem extraction from full tracks + +### DAW Integration Patterns +- **Ableton 12.3**: Native browser integration (first DAW to do this) +- **Other DAWs**: Drag from Splice desktop app to DAW (cross-app drag-and-drop) +- **Sync folder**: Downloaded samples appear in configured folder → auto-index + +## 5. Web-Based Sample Platforms + +### Freesound.org +- **Creative Commons samples**: Free, community-contributed +- **API access**: Programmatic search and download +- **Waveform preview**: Visual waveform + play button +- **Tags + text search**: Community-tagged with description search + +### Loopcloud +- **In-DAW plugin**: Browse + audition without leaving DAW +- **Key/BPM detection**: Auto-detected and displayed +- **Effect processing**: Apply effects before downloading +- **AI tagging**: Automatic categorization + +## 6. ACE-Step Current State + +### What Exists +- **LibraryPanel.tsx**: File browser with folder tree +- **Sample preview**: Play button on audio files +- **Drag-and-drop**: From library to timeline (basic) +- **File type support**: WAV, MP3 (via Web Audio API) +- **No tag system**: Files shown by directory structure only +- **No search**: Must navigate folder tree manually + +### Key Gaps +| Feature | Competitors | ACE-Step | +|---|---|---| +| Text search | All | No | +| Tag/category filtering | All | No | +| Tempo-synced preview | Ableton, Logic | No sync | +| Key detection | Ableton, Logic, Splice | No | +| BPM detection | All | No | +| Audio similarity search | Ableton (Search with Sound) | No | +| Favorites/bookmarks | All | No | +| Collections/playlists | Ableton | No | +| Cloud sample integration | Ableton+Splice, Loopcloud | No | +| Hot-swap mode | Ableton | No | +| Drag to instrument/pad | Ableton, FL | Basic drag only | +| Waveform preview display | All | No waveform | + +--- + +## 7. Recommendations for ACE-Step + +### Phase 1: Usable Browser +- **Text search**: Real-time filter as user types +- **Waveform preview**: Show waveform thumbnail + play button +- **Favorites**: Star icon to bookmark samples +- **BPM/key display**: Auto-detect and show metadata +- **Drag-and-drop to drum pads**: Drag sample → assign to pad + +### Phase 2: Smart Organization +- **Auto-tagging**: Categorize by instrument type (kick, snare, hat, bass, vocal, etc.) +- **Tempo-synced preview**: Preview loops at project BPM +- **Collections**: User-created groups for project-specific sounds +- **Recent files**: Quick access to recently used samples +- **Key/BPM filtering**: Filter samples by compatible key and tempo + +### Phase 3: AI-Powered Discovery +- **Audio similarity search**: "Find similar" based on audio analysis +- **Mood/energy tags**: AI-generated mood descriptors +- **Cloud sample marketplace**: Integration with free sample sources (Freesound API) +- **Hot-swap mode**: Audition replacements while playing + +--- + +## Sources + +- [Ableton Live 12.3 — Splice Integration Announcement](https://www.ableton.com/en/live/) +- [Ableton Live 12 — Search with Sound Feature](https://www.ableton.com/en/live/whats-new/) +- [Splice: Sounds, Presets & Creative Tools for Music Makers](https://splice.com/) +- [Logic Pro Sound Library — Apple Developer](https://support.apple.com/guide/logicpro/use-the-sound-library-lgce1e9bsf6b/mac) +- [Loopcloud: Cloud-Based Sample Platform](https://www.loopcloud.com/) +- [Freesound — Collaborative Database of Creative-Commons Licensed Sounds](https://freesound.org/) diff --git a/.llm/research/synth-ui-design.md b/.llm/research/synth-ui-design.md new file mode 100644 index 00000000..0a5e973d --- /dev/null +++ b/.llm/research/synth-ui-design.md @@ -0,0 +1,127 @@ +# Synth UI Design Research + +> Date: 2026-03-27 | Scope: How top synthesizers design their editing UIs vs ACE-Step + +--- + +## 1. Industry Leaders — Layout Patterns + +### Serum 2 (Xfer Records) — Industry Standard +- **Fixed layout with tabbed pages**: Oscillators, Mixer, FX, Matrix, Global +- **Top-down signal flow**: Oscillators at top → Filter center → Envelopes/LFOs left → Mod Matrix + FX on tabs +- **3 oscillators** (new in Serum 2) + sub + noise +- **Drag-and-drop modulation**: Drag LFO/Env source to any knob → creates mod ring (blue arc) +- **Built-in clip sequencer**: Program melodies inside the synth (unique to Serum 2) +- **Wavetable 3D view**: Interactive 3D wavetable display, click to browse frames +- **Preset browser**: Tag-based, mouse wheel scrolling, searchable +- **Full undo/redo**: Every parameter change is undoable + +### Vital (Matt Tytel) — Free, Modern Reference +- **GPU-accelerated visuals**: 60fps animated interface, minimal CPU impact +- **3 oscillators + sampler**: Each with wavetable position, unison, spectral morph +- **Visual modulation arcs**: Colored arcs on destination knobs showing mod depth per source +- **Drag-to-assign modulation**: Drag source → drop on knob → set depth +- **Real-time spectral display**: Live FFT visualization of oscillator output +- **Skinnable UI**: Community skins (retro, minimal, VS Code-inspired, etc.) +- **Layout**: Left column (oscillators) → Center (filter + effects) → Right (envelopes, LFOs, mod matrix) + +### Ableton Wavetable +- **Tab-based sections**: Oscillator 1, Oscillator 2, Sub/Noise, Filter, Mod Matrix, Effects +- **Collapsible panels**: Click header to expand/collapse sections +- **Modulation**: 3 LFOs + 3 Envelopes assignable via matrix tab +- **Wavetable browser**: Built-in wavetable categories, click to load +- **Minimal aesthetic**: Flat design, Ableton's signature clean look + +### Logic Alchemy — Most Ambitious +- **4 source cards**: Each can be additive, spectral, granular, wavetable, VA, or sampled +- **Performance controls**: XY pad, morph pad, mod wheel assignments +- **Transform pad**: Morph between 8 snapshots in real-time +- **Progressive disclosure**: Simple mode vs Advanced mode toggle + +### Surge XT — Open Source Power +- **Dense UI**: All parameters visible simultaneously (no tabs) +- **3 oscillators**: Classic, Modern, FM2, FM3, Wavetable, Window, String, Twist, Alias, S&H +- **Modulation routing**: Right-click any param → assign modulation source +- **Dual filter**: Parallel/serial routing with visual signal flow +- **Open source**: Reference for Web Audio synth UI implementation + +--- + +## 2. Key UI Design Patterns + +### Envelope Visualizers +- **Interactive ADSR curve** with draggable control points (standard in all top synths) +- **Bezier handles**: Serum allows drag between points for curve shape +- **Color coding**: Vital uses green=amplitude, blue=filter, orange=mod +- **Real-time playback cursor**: Moves along envelope during note playback +- **Multiple envelopes visible**: Show all active envelopes stacked or tabbed + +### Knob Design (Industry Standard) +- **Size**: 32-48px diameter, 270° rotation arc +- **Modulation ring**: Colored arc around knob showing modulation depth + - Serum: Blue ring for all modulation + - Vital: Different color per modulation source (multi-colored arcs) +- **Interaction**: Vertical drag (up=increase), with pointer lock to prevent edge hitting +- **Double-click**: Reset to default value +- **Right-click**: Open precision text input +- **Hover tooltip**: Shows parameter name + current value + unit + +### Progressive Disclosure +- **Default view**: Essential params (osc type, cutoff, ADSR, volume) +- **Expanded view**: Unison, detune, LFO, filter routing +- **Expert view**: Full mod matrix, all parameters exposed +- **Logic Alchemy pattern**: Simple/Advanced toggle button + +### Preset Browser Integration +- **Inline browser**: Serum/Vital show preset name in header, click to browse +- **Category tags**: Bass, Lead, Pad, Keys, FX, Pluck, etc. +- **Search**: Text search + tag filter +- **Favorites**: Star to bookmark +- **Previous/Next**: Arrow buttons for quick browsing + +--- + +## 3. ACE-Step Current State + +- **SynthEngine.ts**: 6 hardcoded presets, no parameter editing +- **SynthPreset type**: `'piano' | 'strings' | 'pad' | 'lead' | 'bass' | 'organ' | 'sampler'` +- **UI**: Preset dropdown only — no knobs, no envelopes, no visual feedback +- **Knob.tsx component exists**: 32px, 270° arc, vertical drag, double-click reset, right-click precision — good foundation + +### Gap: Everything Between Preset Dropdown and Full Synth Editor + +--- + +## 4. Recommended ACE-Step Synth Editor Design + +### Layout (Single Panel, ~400px height) + +``` +┌─────────────────────────────────────────────────────────┐ +│ [Preset: Piano ▼] [◀ ▶] [Save] [Browse] [Simple ↔ Advanced] │ +├──────────┬──────────┬──────────┬──────────────────────────┤ +│ OSCILLATOR │ FILTER │ ENVELOPES │ MOD / FX │ +│ │ │ │ │ +│ [Wave ▼] │ [Type ▼] │ ╭─╲___╮ │ LFO Rate [○] │ +│ Shape [○] │ Cutoff [○]│ A D S R │ LFO Depth [○] │ +│ Detune [○] │ Reso [○] │ (drag │ LFO → [Dest ▼] │ +│ Unison [○] │ Env [○] │ points) │ │ +│ Level [○] │ Key [○] │ │ [+ Add Mod Route] │ +│ │ │ Filt Env │ │ +│ │ │ ╭─╲___╮ │ │ +└──────────┴──────────┴──────────┴──────────────────────────┘ +``` + +### Phase 1 (MVP): Oscillator + Filter + Amp ADSR +### Phase 2: Filter ADSR + LFO with routing +### Phase 3: Mod matrix + Wavetable support + +--- + +## Sources + +- [Serum 2 vs Pigments 7 vs Vital: Which Soft Synth Rules in 2026?](https://dawzone.com/serum-2-vs-pigments-6-vs-vital-which-soft-synth-is-the-best) +- [The Ultimate Soft Synth Showdown: Serum 2, Pigments 6, Phase Plant, Vital](https://www.musicradar.com/music-tech/the-ultimate-soft-synth-showdown-serum-2-pigments-6-phase-plant-vital-and-massive-x-but-which-is-best) +- [Serum 2 vs Vital: The Ultimate Wavetable Synth Comparison (2025)](https://theproducerschool.com/blogs/music-production/serum-2-vs-vital-the-ultimate-wavetable-synth-comparison-2025) +- [Serum 2 Tutorial: The Ultimate Guide for Beginners](https://www.edmprod.com/serum-2-guide/) +- [The Best 5 Free Vital VST Skins for 2025](https://vectorpresets.com/blogs/vital-skins/the-best-5-free-vital-vst-skins-for-2025-best-of-the-rest) diff --git a/.llm/research/timeline-arrangement-ux.md b/.llm/research/timeline-arrangement-ux.md new file mode 100644 index 00000000..006d2298 --- /dev/null +++ b/.llm/research/timeline-arrangement-ux.md @@ -0,0 +1,120 @@ +# Timeline & Arrangement UX Research + +> Date: 2026-03-27 | Scope: Arrangement view UX in mainstream DAWs vs ACE-Step + +--- + +## 1. Ableton Live Arrangement View + +### Navigation & Zoom +- **Overview bar**: Shows entire arrangement in miniature, click/drag to navigate +- **Beat-time ruler zoom**: Click+drag vertically in ruler to zoom (drag horizontally to scroll) +- **Pinch gesture**: Trackpad/touchscreen zoom support +- **Cmd+scroll**: Horizontal zoom anchored to cursor position + +### Clip Editing +- **Surface-level editing**: Edit clips without opening detail view +- **Option+Shift+drag**: Scrub through clip's audio position in arrangement +- **Arrow keys**: Nudge selected clips, snaps to grid (Cmd for fine adjust) +- **Cut Time (Cmd+Shift+Delete)**: Removes time selection, moves everything closer +- **Consolidate (Cmd+J)**: Merge selected clips into one +- **Auto-crossfade**: 4ms fade at clip boundaries by default +- **Lock Envelopes**: Automation stays at song position when clips are moved + +### Unique Dual-View System +- **Session View**: Clip launcher for improvisation/looping +- **Arrangement View**: Linear timeline for structured composition +- **Session → Arrangement**: Record session clips into arrangement + +## 2. Logic Pro Tracks Area + +### Region Operations +- **Split (Cmd+T)**: Split at playhead +- **Join (Cmd+J)**: Merge selected regions +- **Flex Time**: Non-destructive time stretching with visible warp markers +- **Take Folders**: Multiple takes stacked, click to select best sections (best comping UX) +- **Quick Swipe Comping**: Click-drag across take lanes to select best parts + +### Automation +- **Show/hide automation lanes** per track +- **Multiple parameters per track**: Each gets own lane +- **4 automation modes**: Read, Touch, Latch, Write + +## 3. FL Studio Playlist + +### Pattern-Based + Audio Clips +- **Patterns**: MIDI patterns placed as clips on timeline +- **Audio clips**: Direct audio on same timeline +- **No track ownership**: Any pattern/audio can go on any track (flexible but unusual) +- **Horizontal zoom**: Mouse wheel on ruler +- **Clip operations**: Split, slice, merge, reverse, time-stretch handles + +## 4. Reaper Arrange + +### Highly Customizable +- **Items**: Clips can contain audio, MIDI, or both +- **Takes**: Multiple takes per item, cycle through with T key +- **Custom actions**: Scriptable macro system +- **Razor editing**: Select time range across multiple tracks visually + +## 5. ACE-Step Current State + +### What Exists +- Timeline.tsx (~500 LOC): Main container with zoom/pan/selection +- TrackLane.tsx: Renders clips per track with drag-and-drop +- ClipBlock.tsx: Individual clip with status overlay, resize handles +- TimeRuler.tsx: Measures/beats with loop braces +- Minimap.tsx: Full project overview sidebar +- Playhead.tsx: Animated with blinking triangle +- GridOverlay.tsx: Bar/beat/eighth/sub divisions +- SelectionFloatingToolbar.tsx: Context actions for selected clips +- TempoLane.tsx: Tempo change markers +- ArrangementMarkers.tsx: Section labels (Intro/Verse/Chorus) +- Drag-and-drop: Move/resize clips +- Multi-select: Shift/Ctrl+click +- Context menus: Right-click for clip/canvas actions + +### Key Gaps +| Feature | Competitors | ACE-Step | +|---|---|---| +| Split at playhead (Cmd+E) | All | No | +| Fade in/out handles | All | Data exists, no drag UI | +| Crossfade on overlap | All | No | +| Duplicate (Cmd+D) | All | No shortcut | +| Loop/repeat clip | All | No | +| Reverse clip | All | No | +| Take lanes / comping | Logic (best) | Data model exists, no UI | +| Clip color override | All | Track color only | +| Time-stretch handles | All | No | +| Consolidate shortcut | Ableton, Logic | Store action only | + +--- + +## 6. Recommendations + +### Phase 1: Essential Clip Operations +- Split at playhead (Cmd+E) +- Duplicate clip (Cmd+D) +- Fade handles (6px drag zones at clip corners) +- Crossfade on overlap + +### Phase 2: Advanced Editing +- Take lane UI with comping +- Clip color override +- Reverse clip +- Loop/repeat + +### Phase 3: Polish +- Time-stretch handles on clips +- Razor editing (Reaper-style) +- Consolidate shortcut + +--- + +## Sources + +- [Arrangement View — Ableton Reference Manual v12](https://www.ableton.com/en/manual/arrangement-view/) +- [Quick Editing in Ableton Live's Arrangement View](https://musictech.com/tutorials/ableton-live/quick-editing-in-ableton-lives-arrangement-view/) +- [Surface Level Clip Editing in Arrangement View](https://www.patches.zone/ableton-tutorials/surface-level-clip-editing) +- [Ableton Live Redesign Case Study](https://nenadmilosevic.co/ableton-live-redesign/) +- [Ableton Live: Session & Arrangement Views](https://www.soundonsound.com/techniques/ableton-live-session-arrangement-views) diff --git a/docs/plans/roadmap-themed-sprints-2026-03.md b/docs/plans/roadmap-themed-sprints-2026-03.md new file mode 100644 index 00000000..aea9b6c3 --- /dev/null +++ b/docs/plans/roadmap-themed-sprints-2026-03.md @@ -0,0 +1,98 @@ +# Research Backlog Themed Sprint Roadmap (#942-#975) + +## Scope + +- Sources: + - `origin/claude/research-daw-synths-KU2At:.llm/research/*` + - GitHub issues `#942-#975` + - current `main` code synced on 2026-03-27 +- Goal: + - convert the research backlog into theme-based sprints + - sequence work around shared engine and store dependencies + - keep implementation reviewable through small, testable PRs + +## Planning Principles + +- Ship data model and engine primitives before surface-level UI. +- Preserve dual-surface parity: every sprint must add or extend `window.__store` actions in addition to visual controls. +- Prefer PR slices that touch one interaction family at a time and land with browser-testable workflows. +- Keep dark-theme quality, keyboard access, and agent automation coverage in every sprint, not only at the end. +- Treat issues `#970`, `#974`, and `#975` as trust multipliers: they improve first-run confidence and should progress in parallel with core audio work. + +## Dependency Summary + +| Sprint | Theme | Issues | Depends On | Planned PRs | +|---|---|---|---|---| +| 01 | Synth Foundation | `#942 #943 #944 #945 #946 #953 #964` | none | 4 | +| 02 | Sampling, Drum Rack, and Render Pipeline | `#948 #949 #950 #951 #952 #958 #961` | 01 | 4 | +| 03 | Mixer, Device Graph, and Routing | `#957 #959 #965 #969` | 01 | 4 | +| 04 | MIDI, Rhythm, and Automation Editors | `#967 #968 #973` | 01, 03 | 3 | +| 05 | Arrangement and Recording Workflows | `#966 #972` | 02, 04 | 3 | +| 06 | Onboarding, Library, and Accessibility | `#970 #971 #975` | none | 3 | +| 07 | Collaboration, Cloud, and Project Lifecycle | `#974` | 06 | 4 | +| 08 | Advanced Sound Design Lab | `#947 #954 #955 #956 #960 #962 #963` | 01, 02, 03 | 4 | + +## Execution Lanes + +### Lane A — Audio Platform + +- Sprint 01 +- Sprint 02 +- Sprint 03 +- Sprint 08 + +### Lane B — Editing Workflows + +- Sprint 04 +- Sprint 05 + +### Lane C — Experience and Trust + +- Sprint 06 +- Sprint 07 + +## Recommended Order + +1. Land Sprint 01 first to replace preset-only instrument assumptions with a real instrument state model. +2. Start Sprint 06 in parallel once Sprint 01 is underway, because onboarding/accessibility mostly avoids audio-engine conflict. +3. Run Sprint 02 and Sprint 03 after Sprint 01 because both need richer instrument and effect schemas. +4. Start Sprint 04 once Sprint 03 has automation-arm and device parameter hooks available. +5. Start Sprint 05 after Sprint 02 and Sprint 04 so recording, comping, and clip operations share the new render and editor primitives. +6. Start Sprint 07 after Sprint 06 so first-run and project-library UX do not diverge from cloud and share-player flows. +7. Keep Sprint 08 last because it depends on the synth, sampler, routing, and modulation foundations from Sprints 01-03. + +## Branch and PR Conventions + +- Sprint branches: + - `feat/v0.0.x-sprint-01-synth-foundation` + - `feat/v0.0.x-sprint-02-sampling-drum-render` + - `feat/v0.0.x-sprint-03-mixer-device-routing` + - `feat/v0.0.x-sprint-04-midi-rhythm-automation` + - `feat/v0.0.x-sprint-05-arrangement-recording` + - `feat/v0.0.x-sprint-06-onboarding-library-accessibility` + - `feat/v0.0.x-sprint-07-collaboration-cloud` + - `feat/v0.0.x-sprint-08-advanced-sound-design` +- PR naming: + - `feat: sprint 01 PR1 instrument state model` + - `feat: sprint 03 PR2 device chain UX` + - `feat: sprint 06 PR1 accessible control primitives` + +## Done Criteria For Every Sprint + +- `npx tsc --noEmit` +- `npm run build` +- targeted Vitest coverage for the new engine/store surface +- at least one browser workflow covering human interaction and one `window.__store` workflow covering agent interaction +- no new console noise outside explicit error handlers +- updated docs/plans for any scope change discovered during implementation + +## Sprint Documents + +- [Sprint 01 — Synth Foundation](./sprint-01-synth-foundation.md) +- [Sprint 02 — Sampling, Drum Rack, and Render Pipeline](./sprint-02-sampling-drum-render.md) +- [Sprint 03 — Mixer, Device Graph, and Routing](./sprint-03-mixer-device-routing.md) +- [Sprint 04 — MIDI, Rhythm, and Automation Editors](./sprint-04-midi-rhythm-automation.md) +- [Sprint 05 — Arrangement and Recording Workflows](./sprint-05-arrangement-recording.md) +- [Sprint 06 — Onboarding, Library, and Accessibility](./sprint-06-onboarding-library-accessibility.md) +- [Sprint 07 — Collaboration, Cloud, and Project Lifecycle](./sprint-07-collaboration-cloud.md) +- [Sprint 08 — Advanced Sound Design Lab](./sprint-08-advanced-sound-design.md) diff --git a/docs/plans/sprint-01-synth-foundation.md b/docs/plans/sprint-01-synth-foundation.md new file mode 100644 index 00000000..c1363cdc --- /dev/null +++ b/docs/plans/sprint-01-synth-foundation.md @@ -0,0 +1,86 @@ +# Sprint 01 — Synth Foundation + +## User Stories + +- As a producer, I want every instrument track to expose editable synth parameters, so that I can shape sounds instead of swapping only preset names. +- As a producer, I want subtractive and FM synth building blocks to behave like real DAW instruments, so that ACE-Step can cover bread-and-butter bass, lead, pad, and keys workflows. +- As an AI agent, I want synth state to be represented in the project store instead of hidden inside Tone instances, so that I can automate instrument design through `window.__store`. + +## Problem + +- Issues `#942 #943 #944 #945 #946 #953 #964` all describe expected baseline synth behavior, but the current app still treats instruments as a preset string plus a fixed Tone setup. +- The backlog mixes UI asks with engine asks, yet the real blocker is missing instrument state and routing primitives. + +## Root Cause + +- `src/types/project.ts:7-13` models instruments as `TrackType` plus a `SynthPreset` string, which cannot represent oscillator, filter, modulation, or engine-specific state. +- `src/engine/SynthEngine.ts:10-52` hardcodes six preset branches on `Tone.PolySynth(Tone.Synth)` and never persists parameter snapshots. +- `src/engine/SynthEngine.ts:66-139` caches one synth per track keyed only by preset, so changing a single parameter would currently require replacing the whole instance. +- `src/components/pianoroll/PianoRoll.tsx:122-155` and `src/components/pianoroll/PianoRoll.tsx:224-260` expose sampler config and editor toolbar state, but there is no synth editor surface or store-backed synth parameter UI. + +## Solution + +### Deliverables + +- Introduce a track-level `instrument` model that separates engine kind from preset label. +- Add a subtractive synth schema with oscillator, amp envelope, filter, filter envelope, LFO, and unison settings. +- Add an FM synth schema with at least carrier/modulator ratio, level, waveform, and modulation index. +- Build a synth editor panel with visual envelope sections and collapsible advanced controls. +- Add preset snapshots backed by factory JSON plus IndexedDB user presets. + +### Issue Map + +- `#942` synth parameter editing baseline +- `#943` filter envelope +- `#944` routeable LFO +- `#945` unison/detune +- `#946` FM instrument +- `#953` preset browser/save/load +- `#964` visual synth editor + +### Proposed PR Slices + +1. `feat: add track instrument state model and migration` + - extend `src/types/project.ts` + - add store migration/helpers in `src/store/projectStore.ts` + - expose agent-safe actions in `src/main.tsx` +2. `feat: add subtractive synth parameter engine and editor essentials` + - update `src/engine/SynthEngine.ts` + - add synth editor component under `src/components/pianoroll/` + - support `#942` and `#943` +3. `feat: add LFO, unison, and FM engine support` + - implement `#944 #945 #946` + - add visual status/routing previews +4. `feat: add preset browser and user preset persistence` + - implement `#953 #964` + - ship factory preset JSON and IndexedDB user presets + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run src/engine/__tests__/SynthEngine.test.ts src/store/__tests__/projectStore.test.ts` +- add E2E coverage: + - create a piano-roll track + - change oscillator, filter, envelope, and preset in UI + - verify the same state through `window.__store.getState().project` +- manual workflow: + - create Bass patch + - duplicate track + - switch duplicate to FM Bell + - save and reload project + - confirm both instruments restore correctly + +## Files To Touch + +- `docs/plans/sprint-01-synth-foundation.md` +- `src/types/project.ts` +- `src/store/projectStore.ts` +- `src/engine/SynthEngine.ts` +- `src/components/pianoroll/PianoRoll.tsx` +- `src/components/pianoroll/` +- `src/main.tsx` +- `src/services/` preset persistence helpers +- `tests/e2e/` +- `src/engine/__tests__/` +- `src/store/__tests__/` diff --git a/docs/plans/sprint-02-sampling-drum-render.md b/docs/plans/sprint-02-sampling-drum-render.md new file mode 100644 index 00000000..071ab5b9 --- /dev/null +++ b/docs/plans/sprint-02-sampling-drum-render.md @@ -0,0 +1,80 @@ +# Sprint 02 — Sampling, Drum Rack, and Render Pipeline + +## User Stories + +- As a producer, I want sampler instruments to use realistic zones, slices, and warp behavior, so that browser-based playback does not feel like a toy pitch shifter. +- As a beat-maker, I want the drum machine to accept my own samples and pad-level shaping, so that I can build real kits instead of only choosing four synthesized kits. +- As a producer and AI agent, I want freeze and bounce workflows to reduce CPU while preserving reversibility, so that large projects stay stable. + +## Problem + +- Issues `#948 #949 #950 #951 #952 #958 #961` all sit on the same missing platform: the app has one-sample playback, synthesized drum pads, and partial bounce primitives, but not a general sample-instrument architecture. + +## Root Cause + +- `src/types/project.ts:57-81` only supports a single `SamplerConfig` buffer with trim and ADSR, and `src/types/project.ts:96-108` only gives each drum pad `sampleKey`, `volume`, and `pan`. +- `src/engine/SamplerEngine.ts:81-220` manages one `AudioBuffer` per track and derives pitch purely from `playbackRate`, which blocks multi-zone, velocity-layer, slice, and pitch-independent stretch workflows. +- `src/engine/DrumEngine.ts:46-220` constructs every drum sound from fixed Tone synth factories instead of per-pad instrument descriptors and effect chains. +- `src/store/projectStore.ts:2292-2385` exposes `freezeTrack`, `unfreezeTrack`, `flattenTrack`, and `bounceInPlace`, but the state model and UX stop short of a complete freeze/unfreeze workflow. +- `src/components/sequencer/DrumMachineEditor.tsx:170-260` only exposes kit selection and per-pad volume details, not sample loading, tuning, or pad processing. + +## Solution + +### Deliverables + +- Replace the single-buffer sampler model with zone-based sampler definitions. +- Add velocity layers, round-robin, and slice maps that remain scriptable through the store. +- Add per-pad sample mode, tune, envelope, filter, drive, and send state to the drum machine. +- Add pitch-independent time stretch and warp markers for sampler clips and slices. +- Complete track freeze/unfreeze/bounce UX on top of the existing offline render hooks. + +### Issue Map + +- `#948` multi-sample zones +- `#949` user sample loading on pads +- `#950` per-pad effects and tuning +- `#951` velocity layers and crossfading +- `#952` audio slice mode +- `#958` warp/time-stretch +- `#961` freeze/bounce track to audio + +### Proposed PR Slices + +1. `feat: add sampler zone model and engine migration` + - zone schema, velocity ranges, round-robin +2. `feat: add drum pad sample mode and per-pad shaping` + - sample loading, choke groups, pad filter/drive/send +3. `feat: add slice mode and warp-ready sample UI` + - transient slicing, manual slices, warp marker model +4. `feat: add freeze-unfreeze workflow and render progress` + - full `#961` UX, store actions, browser tests + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run src/services/__tests__/bounceInPlace.test.ts src/store/__tests__/drumMachine.test.ts` +- add new unit coverage: + - sampler zone selection by pitch and velocity + - slice-to-MIDI mapping + - freeze/unfreeze persistence +- browser workflows: + - drag a sample onto a drum pad, trigger it, reload project + - load a sample into sampler, create slices, trigger them from piano roll + - freeze a synth track, confirm CPU-saving audio playback, unfreeze it + +## Files To Touch + +- `docs/plans/sprint-02-sampling-drum-render.md` +- `src/types/project.ts` +- `src/store/projectStore.ts` +- `src/engine/SamplerEngine.ts` +- `src/engine/DrumEngine.ts` +- `src/components/pianoroll/QuickSamplerEditor.tsx` +- `src/components/sequencer/DrumMachineEditor.tsx` +- `src/components/pianoroll/` +- `src/hooks/useTransport.ts` +- `src/services/bounceInPlace.ts` +- `src/services/audioFileManager.ts` +- `tests/e2e/` +- `src/store/__tests__/` diff --git a/docs/plans/sprint-03-mixer-device-routing.md b/docs/plans/sprint-03-mixer-device-routing.md new file mode 100644 index 00000000..8456ff0d --- /dev/null +++ b/docs/plans/sprint-03-mixer-device-routing.md @@ -0,0 +1,75 @@ +# Sprint 03 — Mixer, Device Graph, and Routing + +## User Stories + +- As a mixer, I want channel strips and device chains to expose real routing, metering, and insert behavior, so that I can mix inside ACE-Step without treating the mixer as a placeholder. +- As a producer, I want routing-heavy effects such as convolution reverb and sidechain ducking, so that modern mix workflows are possible. +- As an AI agent, I want device parameters and routing choices to exist in store state, so that automated mix flows can modify them safely. + +## Problem + +- Issues `#957 #959 #965 #969` look separate, but they are all symptoms of one limitation: there is no unified device graph between the effect model, mixer surface, and audio routing layer. + +## Root Cause + +- `src/components/mixer/MixerPanel.tsx:12-18` caps inserts and sends at four and two, and `src/components/mixer/MixerPanel.tsx:144-247` renders fixed sections without I/O routing, dB markings, or advanced metering. +- `src/components/mixer/EffectChain.tsx:51-141` encodes effect presets inline and `src/components/mixer/EffectChain.tsx:171-279` renders a basic device card row without rack nesting, A/B states, preset library, or searchable add flow. +- `src/engine/EffectsEngine.ts:97-227` builds linear Tone node chains, ships only algorithmic reverb, and has no first-class effect-rack or sidechain input abstraction. +- `src/engine/EffectsEngine.ts:233-247` rebuilds a single linear chain per track, which is insufficient for parallel racks, convolution branches, and visual routing feedback. + +## Solution + +### Deliverables + +- Introduce a track device graph abstraction that supports serial and parallel chains. +- Upgrade channel strips with RMS plus peak metering, dB scale, insert management, pre/post sends, solo-safe, and simple I/O routing. +- Rebuild the device chain UX with searchable add, collapse, reorder, preset persistence, and rack containers. +- Add convolution reverb and sidechain routing as first-class graph nodes. + +### Issue Map + +- `#957` convolution reverb +- `#959` sidechain routing UI +- `#965` pro-grade mixer panel +- `#969` device-view redesign + +### Proposed PR Slices + +1. `feat: add mixer meter upgrade and channel-strip state` + - RMS/peak meters, dB scale, insert slot expansion +2. `feat: add searchable device chain UX and drag reorder` + - searchable add menu, collapse, preset surface +3. `feat: add convolution reverb and visual processor feedback` + - IR loading, EQ curve, gain reduction meter +4. `feat: add sidechain routing and send-state upgrades` + - sidechain source selector, pre/post send toggle, solo-safe + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run src/engine/__tests__/effectsEngineNativeNode.test.ts src/store/__tests__/sendsReturns.test.ts tests/unit/StatusBar.test.tsx` +- add new unit coverage: + - device graph serialization + - sidechain routing selection + - convolution preset persistence +- browser workflows: + - create a reverb rack and reorder devices + - set kick as sidechain source for bass compressor + - verify insert count, send toggles, and meter behavior in mixer + +## Files To Touch + +- `docs/plans/sprint-03-mixer-device-routing.md` +- `src/types/project.ts` +- `src/store/projectStore.ts` +- `src/engine/EffectsEngine.ts` +- `src/components/mixer/MixerPanel.tsx` +- `src/components/mixer/EffectChain.tsx` +- `src/components/mixer/EffectCards.tsx` +- `src/components/mixer/LevelMeter.tsx` +- `src/components/ui/Knob.tsx` +- `src/components/mixer/VerticalFader.tsx` +- `src/engine/__tests__/` +- `src/store/__tests__/` +- `tests/e2e/` diff --git a/docs/plans/sprint-04-midi-rhythm-automation.md b/docs/plans/sprint-04-midi-rhythm-automation.md new file mode 100644 index 00000000..3d55f73e --- /dev/null +++ b/docs/plans/sprint-04-midi-rhythm-automation.md @@ -0,0 +1,68 @@ +# Sprint 04 — MIDI, Rhythm, and Automation Editors + +## User Stories + +- As a composer, I want piano-roll editing tools that feel musical instead of merely technical, so that note editing stays fast. +- As a beat-maker, I want Elektron-style step detail and probability controls, so that the drum workflow moves beyond an on/off grid. +- As a producer, I want automation to support recording, drawing, and modulation tools, so that parameter movement is part of composition instead of an afterthought. + +## Problem + +- Issues `#967 #968 #973` are editor issues, but the shared blocker is that note, step, and automation data are all modeled too simply for advanced interactions. + +## Root Cause + +- `src/components/pianoroll/PianoRoll.tsx:27-120` keeps ghost-note visibility and editor mode state locally and does not derive scale-aware behavior from project harmony. +- `src/components/pianoroll/PianoRollCanvas.tsx:178-238` supports note creation and chord stamping, but the drag model is single-note focused and lacks native operations for humanize, legato, strum, and multi-note stretch. +- `src/components/pianoroll/VelocityLane.tsx:19-99` only draws velocity bars; it does not own an interactive editing model for painted velocity curves. +- `src/engine/DrumEngine.ts:11-27` keeps drum steps to `active` plus `velocity`, which cannot represent probability, micro-timing, ratchets, or per-step parameter locks. +- `src/components/timeline/AutomationLaneView.tsx:37-165` renders a simple polyline editor with draggable points but no recording modes, tool palette, curve handles, or automation-arm state. + +## Solution + +### Deliverables + +- Extend piano-roll actions with scale highlight, lock-to-scale, velocity painting, quantize strength, legato, humanize, strum, and multi-note stretch. +- Extend step data with probability, micro-timing, pitch offset, ratchet, and step-length metadata. +- Add automation editor tool state, recording-arm state, curve handles, LFO tool generation, and copy/paste flows. + +### Issue Map + +- `#967` piano-roll UX +- `#968` drum machine step UX +- `#973` automation recording and toolset + +### Proposed PR Slices + +1. `feat: add piano-roll scale, velocity, and transform tools` +2. `feat: add drum step metadata and Elektron-style editing` +3. `feat: add automation arm, draw tools, and LFO generation` + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run src/store/__tests__/drumMachine.test.ts src/store/__tests__/projectStore.test.ts` +- add new unit coverage: + - quantize-strength math + - legato and humanize transforms + - step probability and micro-timing evaluation + - automation curve and LFO tool generation +- browser workflows: + - create chord notes, apply strum, paint velocity, legato them + - build a drum pattern with step probability and ratchets + - arm a filter parameter and record automation while transport plays + +## Files To Touch + +- `docs/plans/sprint-04-midi-rhythm-automation.md` +- `src/types/project.ts` +- `src/store/projectStore.ts` +- `src/components/pianoroll/PianoRoll.tsx` +- `src/components/pianoroll/PianoRollCanvas.tsx` +- `src/components/pianoroll/VelocityLane.tsx` +- `src/components/sequencer/` +- `src/engine/DrumEngine.ts` +- `src/components/timeline/AutomationLaneView.tsx` +- `src/store/__tests__/` +- `tests/e2e/` diff --git a/docs/plans/sprint-05-arrangement-recording.md b/docs/plans/sprint-05-arrangement-recording.md new file mode 100644 index 00000000..300f3385 --- /dev/null +++ b/docs/plans/sprint-05-arrangement-recording.md @@ -0,0 +1,65 @@ +# Sprint 05 — Arrangement and Recording Workflows + +## User Stories + +- As a producer, I want clip operations and take comping directly on the arrangement, so that editing audio feels like a real DAW workflow. +- As a vocalist or instrumentalist, I want punch ranges, visual count-in, latency feedback, and comp lanes, so that recording sessions are trustworthy. +- As an AI agent, I want arrangement editing and record-state transitions to be explicit store actions, so that end-to-end browser tests can validate them without canvas-only gestures. + +## Problem + +- Issues `#966` and `#972` both depend on a more deliberate arrangement interaction model. The code already contains pieces of take-lane and punch logic, but the user-facing flow is incomplete. + +## Root Cause + +- `src/components/timeline/Timeline.tsx:208-260` focuses on viewport, selection, and drag windows rather than clip-operation commands such as split, duplicate, loop, reverse, and comp actions. +- `src/hooks/useRecording.ts:64-117` and `src/hooks/useRecording.ts:119-201` already accumulate loop and punch takes, but the workflow ends at saving clips and toggling take-lane visibility. +- `src/engine/RecordingEngine.ts:25-169` captures device, monitoring, and level-meter state, but does not expose latency reporting, pre-roll, tuner state, or input clip warnings as a dedicated recording surface. +- `src/store/projectStore.ts:2292-2385` contains freeze/flatten/bounce primitives that can support arrangement clip ops, yet clip-operation shortcuts and comp editing are not organized around those primitives. + +## Solution + +### Deliverables + +- Add arrangement clip commands for split, duplicate, loop, reverse, fade handles, and crossfade behavior. +- Build a recording panel or transport extensions for punch-in/out, pre-roll, latency display, count-in visuals, tuner, and input clip warnings. +- Complete take-lane selection and comp editing on arrangement tracks. + +### Issue Map + +- `#966` timeline clip operations +- `#972` recording UX + +### Proposed PR Slices + +1. `feat: add arrangement clip operations and shortcuts` +2. `feat: add punch, pre-roll, latency, and visual count-in` +3. `feat: add take-lane comp editing and recording polish` + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run src/store/__tests__/toggleClipMuted.test.ts src/store/__tests__/splitClipAtZeroCrossing.test.ts` +- add new unit coverage: + - clip split/duplicate/loop actions + - punch-range clip placement + - take comp selection +- browser workflows: + - split and duplicate clips from keyboard shortcuts + - record with punch-in/out and visual count-in + - create loop takes and comp the best parts on take lanes + +## Files To Touch + +- `docs/plans/sprint-05-arrangement-recording.md` +- `src/store/projectStore.ts` +- `src/hooks/useRecording.ts` +- `src/engine/RecordingEngine.ts` +- `src/components/timeline/Timeline.tsx` +- `src/components/timeline/TrackLane.tsx` +- `src/components/timeline/ClipBlock.tsx` +- `src/components/layout/Toolbar.tsx` +- `src/components/tracks/TrackHeader.tsx` +- `src/store/__tests__/` +- `tests/e2e/` diff --git a/docs/plans/sprint-06-onboarding-library-accessibility.md b/docs/plans/sprint-06-onboarding-library-accessibility.md new file mode 100644 index 00000000..b266f4f3 --- /dev/null +++ b/docs/plans/sprint-06-onboarding-library-accessibility.md @@ -0,0 +1,71 @@ +# Sprint 06 — Onboarding, Library, and Accessibility + +## User Stories + +- As a new user, I want the first-run path to get me to music in under 30 seconds, so that ACE-Step feels AI-native instead of empty. +- As a creator, I want a modern sample browser with preview, search, and drag-to-timeline, so that discovery stays inside the app. +- As a keyboard or assistive-tech user, I want the DAW surface to be navigable and legible, so that accessibility is built into the core experience. + +## Problem + +- Issues `#970 #971 #975` all affect first impression and trust. Today ACE-Step has useful starter pieces, but they are fragmented and do not yet form a guided, accessible creation flow. + +## Root Cause + +- `src/components/dialogs/NewProjectDialog.tsx:89-152` and `src/components/dialogs/NewProjectDialog.tsx:207-245` provide templates and demo starters, but there is no first-run path selector, progress tracking, or contextual help system. +- `src/components/assets/LoopBrowser.tsx:74-191` supports categories, search, preview, and drag payloads, but it is still a loop library rather than a full sample browser with BPM/key metadata, favorites collections, recent items, or AI sample generation. +- `src/components/ui/Knob.tsx:137-210` and `src/components/mixer/VerticalFader.tsx:110-127` expose partial ARIA but omit `aria-valuetext`, value announcements, and shared accessibility primitives. +- `rg` coverage shows almost no reduced-motion, high-contrast, color-blind, skip-link, or focus-trap infrastructure beyond isolated `aria-live` regions. + +## Solution + +### Deliverables + +- Build a first-run shell with three entry paths: generate, template, blank. +- Promote Loop Browser into a sample browser service with metadata, favorites, recents, preview sync, and AI sample generation entry points. +- Standardize accessible control primitives, skip links, reduced-motion handling, and dialog focus management. + +### Issue Map + +- `#970` first-run onboarding +- `#971` sample browser +- `#975` accessibility + +### Proposed PR Slices + +1. `feat: add accessible control primitives and navigation affordances` + - slider ARIA, skip links, reduced motion, focus management +2. `feat: add sample browser metadata, preview sync, and collections` + - extend current loop library into a real browser +3. `feat: add AI-first onboarding and contextual quick help` + - welcome dialog, template path, help panel, shortcut overlay + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run tests/unit/statusBarControls.test.ts tests/unit/StatusBar.test.tsx` +- add new unit coverage: + - accessible slider attributes + - sample browser filtering and favorites + - first-run state gating +- browser workflows: + - first launch from clean storage + - choose "Generate a Song", complete quick-start + - preview and drag a sample to timeline + - complete the same key actions with keyboard-only navigation + +## Files To Touch + +- `docs/plans/sprint-06-onboarding-library-accessibility.md` +- `src/components/dialogs/NewProjectDialog.tsx` +- `src/components/assets/LoopBrowser.tsx` +- `src/components/assets/LoopBrowserItems.tsx` +- `src/components/ui/Knob.tsx` +- `src/components/mixer/VerticalFader.tsx` +- `src/components/ui/DualRangeSlider.tsx` +- `src/store/uiStore.ts` +- `src/index.css` +- `src/App.tsx` +- `tests/e2e/` +- `tests/unit/` diff --git a/docs/plans/sprint-07-collaboration-cloud.md b/docs/plans/sprint-07-collaboration-cloud.md new file mode 100644 index 00000000..7d5a5d17 --- /dev/null +++ b/docs/plans/sprint-07-collaboration-cloud.md @@ -0,0 +1,67 @@ +# Sprint 07 — Collaboration, Cloud, and Project Lifecycle + +## User Stories + +- As a creator, I want projects to persist in the cloud with version history and share links, so that my work survives beyond one browser instance. +- As a collaborator, I want shared playback and later real-time sync to build on the same project model, so that collaboration does not fork into a separate product. +- As an AI agent, I want cloud and share actions to be explicit APIs, so that automated flows can publish, restore, and validate remote project state. + +## Problem + +- Issue `#974` is larger than one feature. It combines storage, sharing, project organization, version history, MIDI IO, and collaboration foundations. Shipping it safely requires a dedicated lifecycle sprint rather than scattered UI additions. + +## Root Cause + +- `src/services/cloudStorageService.ts:61-190` stores cloud projects in memory and shared projects in `localStorage`, which makes sharing local-only and non-persistent across environments. +- `src/store/collaborationStore.ts:12-74` tracks viewer mode, share token, and collaborator metadata, but has no backend session model, sync transport, or conflict state. +- `src/services/projectStorage.ts:14-119` is optimized for local IndexedDB project storage, not remote version history, folder organization, or shared ownership. +- `src/main.tsx:122-133` exposes sharing helpers for the local share API, but not a remote repository client or cloud action surface. + +## Solution + +### Deliverables + +- Introduce a project repository abstraction with local and remote implementations. +- Add persistent remote save/load/list/version APIs and wire them into the collaboration store. +- Build a version timeline, hosted share-player load path, project organization metadata, and MIDI import/export integration. +- Define the real-time sync foundation as phase two on top of the same repository and session model. + +### Issue Map + +- `#974` collaboration and cloud workflow foundation + +### Proposed PR Slices + +1. `feat: add project repository abstraction and cloud API client` +2. `feat: add cloud save-load-version history UI` +3. `feat: add hosted share player and project organization metadata` +4. `feat: add collaboration session foundation for cursors and track locks` + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- `npx vitest run src/services/__tests__/aceStepApi.test.ts src/store/__tests__/projectStore.test.ts` +- add new unit coverage: + - remote repository adapter behavior + - version-history restore logic + - share-token load flows +- browser workflows: + - save project to cloud, refresh, restore it + - publish a share page and open it in a clean session + - restore an earlier version from the timeline + +## Files To Touch + +- `docs/plans/sprint-07-collaboration-cloud.md` +- `src/services/cloudStorageService.ts` +- `src/store/collaborationStore.ts` +- `src/services/projectStorage.ts` +- `src/services/projectSharingService.ts` +- `src/components/sharing/` +- `src/components/dialogs/ShareDialog.tsx` +- `src/components/dialogs/ProjectListDialog.tsx` +- `src/main.tsx` +- `src/store/projectStore.ts` +- `tests/e2e/` +- `src/services/__tests__/` diff --git a/docs/plans/sprint-08-advanced-sound-design.md b/docs/plans/sprint-08-advanced-sound-design.md new file mode 100644 index 00000000..2c626760 --- /dev/null +++ b/docs/plans/sprint-08-advanced-sound-design.md @@ -0,0 +1,71 @@ +# Sprint 08 — Advanced Sound Design Lab + +## User Stories + +- As a sound designer, I want wavetable, granular, physical, additive, and spectral engines, so that ACE-Step can move beyond “browser DAW” expectations and become distinctive. +- As an expressive performer, I want MPE and a modulation matrix, so that per-note motion and controller input can shape advanced engines naturally. +- As an AI agent, I want advanced sound engines to reuse the same instrument and modulation model from earlier sprints, so that automation and preset generation remain coherent. + +## Problem + +- Issues `#947 #954 #955 #956 #960 #962 #963` are the most differentiated features in the backlog, but they should not land until the core synth, sampler, and routing layers are stable. + +## Root Cause + +- `src/engine/SynthEngine.ts:10-139` currently assumes `Tone.Synth` or simple preset playback and has no engine plug-in boundary for wavetable or physical modeling voices. +- `src/types/project.ts:9-18` and `src/types/project.ts:113-124` lack engine-specific instrument schemas and per-note expression metadata. +- `src/engine/EffectsEngine.ts:97-247` handles simple Tone node chains, but there is no AudioWorklet DSP layer for granular or spectral processors. +- `src/main.tsx:22-92` exposes store state to agents, yet there is no advanced modulation bus or per-note expression surface for those agents to target. + +## Solution + +### Deliverables + +- Add a pluggable instrument-engine boundary on top of Sprint 01’s track instrument state. +- Add a modulation matrix and per-note expression model that can be shared by wavetable and MPE-enabled engines. +- Implement advanced DSP engines in an AudioWorklet-backed layer where real-time FFT or grain scheduling is required. + +### Issue Map + +- `#947` wavetable +- `#954` granular +- `#955` physical modeling +- `#956` modulation matrix +- `#960` MPE +- `#962` additive +- `#963` spectral processing + +### Proposed PR Slices + +1. `feat: add modulation matrix and wavetable engine` +2. `feat: add granular and physical-modeling instruments` +3. `feat: add additive and spectral processor surfaces` +4. `feat: add MPE and per-note expression UI` + +## Verification + +- `npx tsc --noEmit` +- `npm run build` +- add new unit coverage: + - modulation routing math + - wavetable frame selection + - MPE channel allocation + - spectral processor serialization +- browser workflows: + - load wavetable patch and modulate position with LFO + - play granular instrument from a loaded sample + - record MPE gestures and verify per-note expression state + +## Files To Touch + +- `docs/plans/sprint-08-advanced-sound-design.md` +- `src/types/project.ts` +- `src/store/projectStore.ts` +- `src/engine/SynthEngine.ts` +- `src/engine/` +- `src/components/pianoroll/` +- `src/components/mixer/` +- `src/main.tsx` +- `src/services/` +- `src/engine/__tests__/` +- `tests/e2e/`