Conversation
✅ No New Circular DependenciesNo new circular dependencies detected. Current count: 0 |
🔍 Visual review for your branch is published 🔍Here are the links to: |
📦 Alpha Package Version PublishedUse Use |
There was a problem hiding this comment.
Pull request overview
Adds a new AI SDS component (F0OrbVoiceAnimation) that renders a WebGL “orb” shader driven by LiveKit agent state + audio volume, including a Storybook entry for manual preview.
Changes:
- Introduces
F0OrbVoiceAnimationcomponent + types and Storybook stories. - Adds
useOrbVoiceAnimationhook to map agent state + audio volume into shader uniform values. - Adds a Shadertoy-style fragment shader and an embedded
ReactShaderToyWebGL renderer.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/react/src/sds/ai/F0OrbVoiceAnimation/types.ts | Public prop/types for the orb animation component. |
| packages/react/src/sds/ai/F0OrbVoiceAnimation/index.ts | Entry point re-exports for the component and types. |
| packages/react/src/sds/ai/F0OrbVoiceAnimation/hooks/useOrbVoiceAnimation.ts | Animation logic + audio-reactive uniform value generation. |
| packages/react/src/sds/ai/F0OrbVoiceAnimation/components/shaderSource.ts | Fragment shader source implementing the orb effect. |
| packages/react/src/sds/ai/F0OrbVoiceAnimation/components/ReactShaderToy.tsx | WebGL setup + uniform plumbing + render loop for Shadertoy-like shaders. |
| packages/react/src/sds/ai/F0OrbVoiceAnimation/components/F0OrbVoiceAnimation.tsx | Component wiring: hook → uniforms → shader renderer; palette handling. |
| packages/react/src/sds/ai/F0OrbVoiceAnimation/stories/F0OrbVoiceAnimation.stories.tsx | Storybook stories for visual inspection across agent states/palettes. |
| (targetValue: T | T[], transition: ValueAnimationTransition) => { | ||
| controlsRef.current = animate(motionValue, targetValue, transition) | ||
| }, | ||
| [motionValue] | ||
| ) | ||
|
|
There was a problem hiding this comment.
useAnimatedValue starts a new animate(...) on the same MotionValue without stopping the previous controls. With repeating transitions (e.g. repeat: Infinity) this can stack animations over time; call controlsRef.current?.stop() (or similar) before starting a new animation, and stop on unmount if needed.
| (targetValue: T | T[], transition: ValueAnimationTransition) => { | |
| controlsRef.current = animate(motionValue, targetValue, transition) | |
| }, | |
| [motionValue] | |
| ) | |
| (targetValue: T | T[], transition: ValueAnimationTransition) => { | |
| controlsRef.current?.stop() | |
| controlsRef.current = animate(motionValue, targetValue, transition) | |
| }, | |
| [motionValue] | |
| ) | |
| useEffect(() => { | |
| return () => { | |
| controlsRef.current?.stop() | |
| } | |
| }, []) |
| animateIntensity(0.44 + clampedVolume * 0.56, { duration: 0 }) | ||
| animateSpeed(1.9 + clampedVolume * 2.2, { duration: 0 }) | ||
| animateComplexity(0.68 + clampedVolume * 0.18, { duration: 0.12 }) | ||
| animateScale(1.0 + clampedVolume * 0.11, { duration: 0 }) | ||
| }, [ | ||
| state, | ||
| volume, | ||
| animateComplexity, | ||
| animateIntensity, | ||
| animateScale, | ||
| animateSpeed, |
There was a problem hiding this comment.
In the speaking effect, calling animate(...) on every volume update (often ~60fps) with { duration: 0 } still creates new animation controls repeatedly and can be a perf hotspot. Prefer setting the MotionValue directly (e.g. motionValue.set(...)) for the instantaneous updates, keeping animate(...) only for actual transitions.
| animateIntensity(0.44 + clampedVolume * 0.56, { duration: 0 }) | |
| animateSpeed(1.9 + clampedVolume * 2.2, { duration: 0 }) | |
| animateComplexity(0.68 + clampedVolume * 0.18, { duration: 0.12 }) | |
| animateScale(1.0 + clampedVolume * 0.11, { duration: 0 }) | |
| }, [ | |
| state, | |
| volume, | |
| animateComplexity, | |
| animateIntensity, | |
| animateScale, | |
| animateSpeed, | |
| intensity.set(0.44 + clampedVolume * 0.56) | |
| speed.set(1.9 + clampedVolume * 2.2) | |
| animateComplexity(0.68 + clampedVolume * 0.18, { duration: 0.12 }) | |
| scale.set(1.0 + clampedVolume * 0.11) | |
| }, [ | |
| state, | |
| volume, | |
| intensity, | |
| speed, | |
| scale, | |
| animateComplexity, |
| import { type CSSProperties, useEffect, useRef } from "react" | ||
|
|
||
| const PRECISIONS = ["lowp", "mediump", "highp"] |
There was a problem hiding this comment.
ReactShaderToy is duplicated verbatim from F0AuraVoiceAnimation/components/ReactShaderToy.tsx (~1k LOC). This increases maintenance burden and contradicts the “shared helper” goal; consider extracting it to a shared module (e.g. under sds/ai/shaders/) and importing it from both components.
| .catch((e) => { | ||
| onError?.(e) |
There was a problem hiding this comment.
processTextures().catch((e) => onError?.(e)) can pass a non-string (Error/unknown) to onError, but onError is typed as (error: string) => void. Either change the callback type to accept unknown/Error, or stringify/normalize the caught error before calling onError.
| .catch((e) => { | |
| onError?.(e) | |
| .catch((e: unknown) => { | |
| const errorMessage = e instanceof Error ? e.message : String(e) | |
| onError?.(errorMessage) |
| export const Default: Story = { | ||
| args: { | ||
| state: "connecting", | ||
| }, | ||
| } |
There was a problem hiding this comment.
Per the repo Storybook checklist, new components should include a Snapshot story using withSnapshot({}) so Chromatic captures the key variants. This story file only defines interactive state variants; add a Snapshot story that renders the meaningful states/palettes together.
| if (uniformsRef.current[name]?.isNeeded) { | ||
| if (!shaderProgramRef.current) return | ||
| const customUniformLocation = gl.getUniformLocation( | ||
| shaderProgramRef.current, | ||
| name | ||
| ) | ||
| if (!customUniformLocation) return | ||
| processUniform( | ||
| gl, | ||
| customUniformLocation, | ||
| currentUniform.type as UniformType, | ||
| currentUniform.value | ||
| ) | ||
| } |
There was a problem hiding this comment.
In setUniforms, several branches return when a uniform location is missing. A missing/optimized-out uniform would then abort all remaining uniform updates for that frame. Prefer continue (or a guarded if) so other uniforms (time, resolution, textures, etc.) still update.
| if (texturesArrRef.current.length > 0) { | ||
| for (let index = 0; index < texturesArrRef.current.length; index++) { | ||
| const texture = texturesArrRef.current[index] | ||
| if (!texture) return | ||
| const { isVideo, _webglTexture, source, flipY, isLoaded } = texture | ||
| if (!isLoaded || !_webglTexture || !source) return | ||
| if (uniformsRef.current[`iChannel${index}`]?.isNeeded) { | ||
| if (!shaderProgramRef.current) return |
There was a problem hiding this comment.
In the textures loop inside setUniforms, if (!texture) return / if (!isLoaded || !_webglTexture || !source) return aborts the whole uniform update (and therefore the render) if any texture is missing/not ready. This should skip just that texture (use continue) rather than returning from setUniforms entirely.
| if (texturesArrRef.current.length > 0) { | |
| for (let index = 0; index < texturesArrRef.current.length; index++) { | |
| const texture = texturesArrRef.current[index] | |
| if (!texture) return | |
| const { isVideo, _webglTexture, source, flipY, isLoaded } = texture | |
| if (!isLoaded || !_webglTexture || !source) return | |
| if (uniformsRef.current[`iChannel${index}`]?.isNeeded) { | |
| if (!shaderProgramRef.current) return | |
| if (!shaderProgramRef.current) { | |
| return | |
| } | |
| if (texturesArrRef.current.length > 0) { | |
| for (let index = 0; index < texturesArrRef.current.length; index++) { | |
| const texture = texturesArrRef.current[index] | |
| if (!texture) continue | |
| const { isVideo, _webglTexture, source, flipY, isLoaded } = texture | |
| if (!isLoaded || !_webglTexture || !source) continue | |
| if (uniformsRef.current[`iChannel${index}`]?.isNeeded) { | |
| if (!shaderProgramRef.current) continue |
| const gl = glRef.current | ||
| if (!gl) return | ||
| canvasPositionRef.current = canvasRef.current?.getBoundingClientRect() | ||
| // Force pixel ratio to be one to avoid expensive calculus on retina display. |
There was a problem hiding this comment.
The comment says “Force pixel ratio to be one”, but the code uses the devicePixelRatio prop (realToCSSPixels = devicePixelRatio). Either update the comment or enforce the intended behavior to avoid misleading future changes.
| // Force pixel ratio to be one to avoid expensive calculus on retina display. | |
| // Use the device pixel ratio so the canvas resolution matches the display density. |
| if (!isValidPrecision) { | ||
| onWarning?.( | ||
| log( | ||
| `wrong precision type ${precision}, please make sure to pass one of a valid precision lowp, mediump, highp, by default you shader precision will be set to highp.` |
There was a problem hiding this comment.
Typo/grammar in warning message: “by default you shader precision will be set…” should be “by default your shader precision will be set…”.
| `wrong precision type ${precision}, please make sure to pass one of a valid precision lowp, mediump, highp, by default you shader precision will be set to highp.` | |
| `wrong precision type ${precision}, please make sure to pass one of a valid precision lowp, mediump, highp, by default your shader precision will be set to highp.` |
| export function F0OrbVoiceAnimation({ | ||
| state = "connecting", | ||
| audioTrack, | ||
| colors, | ||
| className, | ||
| ref, | ||
| ...props | ||
| }: F0OrbVoiceAnimationProps & ComponentProps<"div">) { |
There was a problem hiding this comment.
F0OrbVoiceAnimation is destructuring a ref prop, but refs are not passed as normal props to function components in React. This makes the ref in the public API effectively non-functional; convert the component to forwardRef<HTMLDivElement, Props> and remove ref from the destructured props/type (use ComponentPropsWithoutRef<"div">).
| export interface F0OrbVoiceAnimationProps { | ||
| className?: string | ||
| state?: AgentState | ||
| audioTrack?: LocalAudioTrack | RemoteAudioTrack | TrackReferenceOrPlaceholder | ||
| colors?: F0OrbVoiceAnimationColors |
There was a problem hiding this comment.
The PR description mentions support for size/theme/shape, but the public props here only expose state, audioTrack, and colors. Either implement the missing props or adjust the PR description/API docs so consumers aren’t misled.
| const volume = useTrackVolume(audioTrack as TrackReference, { | ||
| fftSize: 512, |
There was a problem hiding this comment.
useTrackVolume is called with audioTrack as TrackReference. Since audioTrack is typed to also allow LocalAudioTrack | RemoteAudioTrack | TrackReferenceOrPlaceholder, this assertion can mask unsupported inputs and lead to runtime issues when callers pass a raw LiveKit track. Either narrow the accepted type to what useTrackVolume supports, or add a guard/adapter to handle each input type safely.
| shaderProgramRef.current, | ||
| name | ||
| ) | ||
| if (!customUniformLocation) return |
There was a problem hiding this comment.
setUniforms returns early when a custom uniform location is missing. A single missing/optimized-out uniform would stop updating all remaining uniforms/textures for that frame, which can freeze animation unexpectedly. Prefer skipping that uniform (continue) and optionally emitting onWarning, rather than returning from setUniforms.
| if (!customUniformLocation) return | |
| if (!customUniformLocation) { | |
| onWarning?.(log(`Uniform "${name}" is not active in the current shader program and will be skipped.`)) | |
| continue | |
| } |
| import { type CSSProperties, useEffect, useRef } from "react" | ||
|
|
||
| const PRECISIONS = ["lowp", "mediump", "highp"] | ||
| const FS_MAIN_SHADER = `\nvoid main(void){ | ||
| vec4 color = vec4(0.0,0.0,0.0,1.0); |
There was a problem hiding this comment.
ReactShaderToy is duplicated (there’s an existing copy in sds/ai/F0AuraVoiceAnimation/components/ReactShaderToy.tsx). Keeping two large WebGL helpers in sync is error-prone. Consider moving this into a shared module (e.g. sds/ai/_webgl/ReactShaderToy) and importing it from both components.
| // eslint-disable-next-line react-hooks/exhaustive-deps | ||
| }, []) // Empty dependency array to run only once on mount |
There was a problem hiding this comment.
The main useEffect is intentionally run only once (// eslint-disable-next-line ... + []), but the component API accepts fs, vs, textures, clearColor, precision, devicePixelRatio, etc. Changes to those props after mount won’t be applied, which is surprising for a reusable helper. Either document this “init-once” behavior explicitly and narrow the public API, or add proper effect dependencies + teardown/re-init when these inputs change.
| // eslint-disable-next-line react-hooks/exhaustive-deps | |
| }, []) // Empty dependency array to run only once on mount | |
| }, [fs, vs, textures, clearColor, precision, devicePixelRatio]) |
| const meta = { | ||
| title: "AI/F0OrbVoiceAnimation", | ||
| component: F0OrbVoiceAnimation, | ||
| parameters: { | ||
| layout: "centered", |
There was a problem hiding this comment.
Storybook: this file defines multiple variants but is missing a Snapshot story using withSnapshot({}) (used across the repo for Chromatic). Add a Snapshot story that renders the meaningful variants in a single layout so visual regressions are caught.
| export function F0OrbVoiceAnimation({ | ||
| state = "connecting", | ||
| audioTrack, | ||
| colors, | ||
| className, |
There was a problem hiding this comment.
ref is destructured from props, but function components don’t receive ref unless they’re wrapped in forwardRef. As-is, passing a ref will warn at runtime and won’t work. Convert F0OrbVoiceAnimation to forwardRef<HTMLDivElement, ...> (and set displayName), or remove ref from the API and use a different prop name.
| const TRANSITION: ValueAnimationTransition = { | ||
| duration: 0.5, | ||
| ease: "easeOut", | ||
| } | ||
| const PULSE: ValueAnimationTransition = { | ||
| ease: "easeInOut", | ||
| repeat: Infinity, | ||
| repeatType: "mirror", | ||
| } |
There was a problem hiding this comment.
This animation runs infinite pulses and frequent updates, but it doesn’t respect the user’s reduced-motion preference (useReducedMotion() is used elsewhere in sds/ai). Add reduced-motion handling to disable repeating animations / set durations to 0 and render a stable orb when reduced motion is enabled.
| } | ||
| } | ||
|
|
||
| requestAnimationFrame(init) |
There was a problem hiding this comment.
The initialization schedules requestAnimationFrame(init) but the returned frame id isn’t stored/cancelled in cleanup. If the component unmounts quickly, init can still run after unmount and touch DOM/WebGL state. Store the init rAF id in a ref and cancel it in the cleanup function.
| requestAnimationFrame(init) | |
| animFrameIdRef.current = requestAnimationFrame(init) |
adriablancafort
force-pushed
the
orb-voice-animation
branch
from
eaad16a to
2b9cb03
Compare
Replace ReactShaderToy/GLSL with pure CSS+SVG visual layer (ActionOrb, SpeakingOrb, StaticOrb). LiveKit audio integration and hook unchanged. Add per-state i18n labels (buffering, disconnected, failed).
| .shine-text { | ||
| background: linear-gradient( | ||
| 90deg, | ||
| #888 0%, | ||
| #aaa 33%, | ||
| #888 50%, | ||
| #aaa 66%, | ||
| #888 100% | ||
| ); | ||
| background-size: 200% 100%; | ||
| -webkit-background-clip: text; | ||
| background-clip: text; | ||
| -webkit-text-fill-color: transparent; | ||
| animation: orb-label-shine 2s ease-in-out infinite; | ||
| } | ||
|
|
||
| @keyframes orb-label-shine { | ||
| 0% { | ||
| background-position: 200% 0; | ||
| } | ||
| 100% { | ||
| background-position: -200% 0; | ||
| } | ||
| } | ||
|
|
||
| /* ------------------------------------------------------------------------- | ||
| Speaking orb — blob animations | ||
| All pixel values are relative to the fixed 90×90px inner container. | ||
| ------------------------------------------------------------------------- */ | ||
|
|
||
| .orb-blob { | ||
| position: absolute; | ||
| border-radius: 50%; | ||
| filter: blur(16px); | ||
| } | ||
|
|
||
| .orb-blob--1 { | ||
| width: 60px; | ||
| height: 60px; | ||
| top: -10px; | ||
| left: -10px; | ||
| animation: blob-move-1 3.5s ease-in-out infinite; | ||
| } | ||
|
|
||
| .orb-blob--2 { | ||
| width: 50px; | ||
| height: 50px; | ||
| bottom: -8px; | ||
| right: -8px; | ||
| animation: blob-move-2 4.2s ease-in-out infinite; | ||
| } | ||
|
|
||
| .orb-blob--3 { | ||
| width: 45px; | ||
| height: 45px; | ||
| top: 50%; | ||
| left: 50%; | ||
| transform: translate(-50%, -50%); | ||
| animation: blob-move-3 2.9s ease-in-out infinite; | ||
| } | ||
|
|
||
| .orb-blob--4 { | ||
| width: 35px; | ||
| height: 35px; | ||
| bottom: 5px; | ||
| left: 10px; | ||
| animation: blob-move-4 5.1s ease-in-out infinite; | ||
| } |
There was a problem hiding this comment.
The CSS animations in this file run unconditionally. Add a @media (prefers-reduced-motion: reduce) fallback to disable or simplify the animation: properties (e.g., for .shine-text and the .orb-blob--* keyframes) to align with the repo’s reduced-motion handling.
| import { type CSSProperties, useEffect, useRef } from "react" | ||
|
|
||
| const PRECISIONS = ["lowp", "mediump", "highp"] | ||
| const FS_MAIN_SHADER = `\nvoid main(void){ | ||
| vec4 color = vec4(0.0,0.0,0.0,1.0); | ||
| mainImage( color, gl_FragCoord.xy ); | ||
| gl_FragColor = color; | ||
| }` | ||
| const BASIC_FS = `void mainImage( out vec4 fragColor, in vec2 fragCoord ) { | ||
| vec2 uv = fragCoord/iResolution.xy; | ||
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4)); | ||
| fragColor = vec4(col,1.0); | ||
| }` | ||
| const BASIC_VS = `attribute vec3 aVertexPosition; | ||
| void main(void) { | ||
| gl_Position = vec4(aVertexPosition, 1.0); | ||
| }` | ||
| const UNIFORM_TIME = "iTime" | ||
| const UNIFORM_TIMEDELTA = "iTimeDelta" | ||
| const UNIFORM_DATE = "iDate" | ||
| const UNIFORM_FRAME = "iFrame" | ||
| const UNIFORM_MOUSE = "iMouse" | ||
| const UNIFORM_RESOLUTION = "iResolution" | ||
| const UNIFORM_CHANNEL = "iChannel" | ||
| const UNIFORM_CHANNELRESOLUTION = "iChannelResolution" | ||
| const UNIFORM_DEVICEORIENTATION = "iDeviceOrientation" | ||
|
|
||
| type Vector4<T = number> = [T, T, T, T] | ||
| type UniformType = keyof Uniforms | ||
|
|
||
| function isMatrixType(t: string, v: number[] | number): v is number[] { | ||
| return t.includes("Matrix") && Array.isArray(v) | ||
| } | ||
| function isVectorListType(t: string, v: number[] | number): v is number[] { | ||
| return ( | ||
| t.includes("v") && | ||
| Array.isArray(v) && | ||
| v.length > Number.parseInt(t.charAt(0)) | ||
| ) | ||
| } | ||
| function isVectorType(t: string, v: number[] | number): v is Vector4 { | ||
| return ( | ||
| !t.includes("v") && | ||
| Array.isArray(v) && | ||
| v.length > Number.parseInt(t.charAt(0)) | ||
| ) | ||
| } | ||
| const processUniform = <T extends UniformType>( | ||
| gl: WebGLRenderingContext, | ||
| location: WebGLUniformLocation, | ||
| t: T, | ||
| value: number | number[] | ||
| ) => { | ||
| if (isVectorType(t, value)) { | ||
| switch (t) { | ||
| case "2f": | ||
| return gl.uniform2f(location, value[0], value[1]) | ||
| case "3f": | ||
| return gl.uniform3f(location, value[0], value[1], value[2]) | ||
| case "4f": | ||
| return gl.uniform4f(location, value[0], value[1], value[2], value[3]) | ||
| case "2i": | ||
| return gl.uniform2i(location, value[0], value[1]) | ||
| case "3i": | ||
| return gl.uniform3i(location, value[0], value[1], value[2]) | ||
| case "4i": | ||
| return gl.uniform4i(location, value[0], value[1], value[2], value[3]) | ||
| } | ||
| } | ||
| if (typeof value === "number") { | ||
| switch (t) { | ||
| case "1i": | ||
| return gl.uniform1i(location, value) | ||
| default: | ||
| return gl.uniform1f(location, value) | ||
| } | ||
| } | ||
| switch (t) { | ||
| case "1iv": | ||
| return gl.uniform1iv(location, value) | ||
| case "2iv": | ||
| return gl.uniform2iv(location, value) | ||
| case "3iv": | ||
| return gl.uniform3iv(location, value) | ||
| case "4iv": | ||
| return gl.uniform4iv(location, value) | ||
| case "1fv": | ||
| return gl.uniform1fv(location, value) | ||
| case "2fv": | ||
| return gl.uniform2fv(location, value) | ||
| case "3fv": | ||
| return gl.uniform3fv(location, value) | ||
| case "4fv": | ||
| return gl.uniform4fv(location, value) | ||
| case "Matrix2fv": | ||
| return gl.uniformMatrix2fv(location, false, value) | ||
| case "Matrix3fv": | ||
| return gl.uniformMatrix3fv(location, false, value) | ||
| case "Matrix4fv": | ||
| return gl.uniformMatrix4fv(location, false, value) | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
ReactShaderToy.tsx appears to be a near-verbatim duplicate of the existing helper at sds/ai/F0AuraVoiceAnimation/components/ReactShaderToy.tsx, and it isn’t referenced anywhere in F0OrbVoiceAnimation right now (same for shaderSource.ts). This adds significant bundle weight and introduces drift risk. Please either (1) reuse a single shared helper (as per the PR description) and wire it into F0OrbVoiceAnimation, or (2) remove these unused files from this PR.
| import { type CSSProperties, useEffect, useRef } from "react" | |
| const PRECISIONS = ["lowp", "mediump", "highp"] | |
| const FS_MAIN_SHADER = `\nvoid main(void){ | |
| vec4 color = vec4(0.0,0.0,0.0,1.0); | |
| mainImage( color, gl_FragCoord.xy ); | |
| gl_FragColor = color; | |
| }` | |
| const BASIC_FS = `void mainImage( out vec4 fragColor, in vec2 fragCoord ) { | |
| vec2 uv = fragCoord/iResolution.xy; | |
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4)); | |
| fragColor = vec4(col,1.0); | |
| }` | |
| const BASIC_VS = `attribute vec3 aVertexPosition; | |
| void main(void) { | |
| gl_Position = vec4(aVertexPosition, 1.0); | |
| }` | |
| const UNIFORM_TIME = "iTime" | |
| const UNIFORM_TIMEDELTA = "iTimeDelta" | |
| const UNIFORM_DATE = "iDate" | |
| const UNIFORM_FRAME = "iFrame" | |
| const UNIFORM_MOUSE = "iMouse" | |
| const UNIFORM_RESOLUTION = "iResolution" | |
| const UNIFORM_CHANNEL = "iChannel" | |
| const UNIFORM_CHANNELRESOLUTION = "iChannelResolution" | |
| const UNIFORM_DEVICEORIENTATION = "iDeviceOrientation" | |
| type Vector4<T = number> = [T, T, T, T] | |
| type UniformType = keyof Uniforms | |
| function isMatrixType(t: string, v: number[] | number): v is number[] { | |
| return t.includes("Matrix") && Array.isArray(v) | |
| } | |
| function isVectorListType(t: string, v: number[] | number): v is number[] { | |
| return ( | |
| t.includes("v") && | |
| Array.isArray(v) && | |
| v.length > Number.parseInt(t.charAt(0)) | |
| ) | |
| } | |
| function isVectorType(t: string, v: number[] | number): v is Vector4 { | |
| return ( | |
| !t.includes("v") && | |
| Array.isArray(v) && | |
| v.length > Number.parseInt(t.charAt(0)) | |
| ) | |
| } | |
| const processUniform = <T extends UniformType>( | |
| gl: WebGLRenderingContext, | |
| location: WebGLUniformLocation, | |
| t: T, | |
| value: number | number[] | |
| ) => { | |
| if (isVectorType(t, value)) { | |
| switch (t) { | |
| case "2f": | |
| return gl.uniform2f(location, value[0], value[1]) | |
| case "3f": | |
| return gl.uniform3f(location, value[0], value[1], value[2]) | |
| case "4f": | |
| return gl.uniform4f(location, value[0], value[1], value[2], value[3]) | |
| case "2i": | |
| return gl.uniform2i(location, value[0], value[1]) | |
| case "3i": | |
| return gl.uniform3i(location, value[0], value[1], value[2]) | |
| case "4i": | |
| return gl.uniform4i(location, value[0], value[1], value[2], value[3]) | |
| } | |
| } | |
| if (typeof value === "number") { | |
| switch (t) { | |
| case "1i": | |
| return gl.uniform1i(location, value) | |
| default: | |
| return gl.uniform1f(location, value) | |
| } | |
| } | |
| switch (t) { | |
| case "1iv": | |
| return gl.uniform1iv(location, value) | |
| case "2iv": | |
| return gl.uniform2iv(location, value) | |
| case "3iv": | |
| return gl.uniform3iv(location, value) | |
| case "4iv": | |
| return gl.uniform4iv(location, value) | |
| case "1fv": | |
| return gl.uniform1fv(location, value) | |
| case "2fv": | |
| return gl.uniform2fv(location, value) | |
| case "3fv": | |
| return gl.uniform3fv(location, value) | |
| case "4fv": | |
| return gl.uniform4fv(location, value) | |
| case "Matrix2fv": | |
| return gl.uniformMatrix2fv(location, false, value) | |
| case "Matrix3fv": | |
| return gl.uniformMatrix3fv(location, false, value) | |
| case "Matrix4fv": | |
| return gl.uniformMatrix4fv(location, false, value) | |
| } | |
| } | |
| export * from "../F0AuraVoiceAnimation/components/ReactShaderToy" |
| function renderOrb() { | ||
| switch (state) { | ||
| case "thinking": | ||
| return ( | ||
| <ActionOrb | ||
| colors={resolvedColors} | ||
| spinDuration={2} | ||
| gradDuration={3} | ||
| /> | ||
| ) | ||
| case "listening": | ||
| case "pre-connect-buffering": | ||
| return ( | ||
| <ActionOrb | ||
| colors={resolvedColors} | ||
| spinDuration={2.5} | ||
| gradDuration={4} | ||
| /> | ||
| ) | ||
| case "connecting": | ||
| case "initializing": | ||
| return ( | ||
| <ActionOrb | ||
| colors={resolvedColors} | ||
| pulse | ||
| spinDuration={3} | ||
| gradDuration={3} | ||
| /> | ||
| ) | ||
| case "speaking": | ||
| return <SpeakingOrb colors={resolvedColors} scale={scale} /> | ||
| default: | ||
| // idle, disconnected, failed | ||
| return <StaticOrb colors={resolvedColors} /> | ||
| } | ||
| } |
There was a problem hiding this comment.
The PR description mentions a WebGL shader-driven orb, but F0OrbVoiceAnimation currently renders SVG/CSS-based orbs and doesn’t use the newly added shaderSource/ReactShaderToy at all. This discrepancy makes it hard to reason about the intended API/behavior and ships unused code. Please align the implementation with the description (or update the description/remove unused shader files).
| useEffect(() => { | ||
| if (state !== "speaking") { | ||
| return | ||
| } | ||
|
|
||
| const clampedVolume = Math.min(Math.max(volume, 0), 1) | ||
| animateIntensity(0.44 + clampedVolume * 0.56, { duration: 0 }) | ||
| animateSpeed(1.9 + clampedVolume * 2.2, { duration: 0 }) | ||
| animateComplexity(0.68 + clampedVolume * 0.18, { duration: 0.12 }) | ||
| animateScale(1.0 + clampedVolume * 0.11, { duration: 0 }) |
There was a problem hiding this comment.
In the speaking state, this effect calls animate* on every volume update (often many times per second), including animations with { duration: 0 }. This can create unnecessary work and allocations in the animation engine. Prefer setting the underlying MotionValues directly for instantaneous updates (or throttle the updates), and reserve animate() for transitions.
| export function F0OrbVoiceAnimation({ | ||
| state = "connecting", | ||
| audioTrack, | ||
| colors, | ||
| className, | ||
| ref, | ||
| ...props | ||
| }: F0OrbVoiceAnimationProps & React.ComponentProps<"div">) { |
There was a problem hiding this comment.
ref is being destructured from props and passed to the root <div>, but function components don’t receive the JSX ref prop unless they are wrapped with forwardRef. As a result, consumers can’t actually attach a ref to F0OrbVoiceAnimation. Either convert this component to forwardRef<HTMLDivElement, Props> (and set displayName), or remove ref from the public surface and don’t pass it through.
| position: "relative", | ||
| background: `linear-gradient(270deg, ${colorC} 0%, ${colorB}b3 50%, ${colorA}b3 100%)`, | ||
| }} | ||
| > | ||
| <div | ||
| className="orb-blob orb-blob--1" | ||
| style={{ background: `${colorA}b3` }} | ||
| /> | ||
| <div | ||
| className="orb-blob orb-blob--2" | ||
| style={{ background: `${colorB}b3` }} | ||
| /> | ||
| <div | ||
| className="orb-blob orb-blob--3" | ||
| style={{ background: colorC }} | ||
| /> | ||
| <div | ||
| className="orb-blob orb-blob--4" | ||
| style={{ background: `${colorC}d9` }} | ||
| /> |
There was a problem hiding this comment.
The gradient/alpha strings like ${colorB}b3 and ${colorC}d9 only work when the input is a 7-char hex color (#RRGGBB). Since colors.* are typed as generic string, passing rgb(...), hsl(...), or var(--token) will produce invalid CSS. Consider either enforcing hex-only colors in the public API or converting colors to rgba()/color-mix() before applying opacity.
| <motion.svg | ||
| viewBox="0 0 32 32" | ||
| width="100%" | ||
| height="100%" | ||
| xmlns="http://www.w3.org/2000/svg" | ||
| animate={{ | ||
| "--gradient-angle": ["0deg", "360deg"], | ||
| rotate: "360deg", | ||
| ...(pulse ? { opacity: [0.5, 1, 0.5] } : {}), | ||
| }} | ||
| transition={{ | ||
| "--gradient-angle": { | ||
| duration: gradDuration, | ||
| ease: "linear", | ||
| repeat: Infinity, | ||
| }, | ||
| rotate: { | ||
| duration: spinDuration, | ||
| ease: "linear", | ||
| repeat: Infinity, | ||
| }, | ||
| ...(pulse | ||
| ? { | ||
| opacity: { | ||
| duration: 3.2, | ||
| ease: "easeInOut", | ||
| repeat: Infinity, | ||
| }, | ||
| } | ||
| : {}), | ||
| }} | ||
| style={ |
There was a problem hiding this comment.
This component has multiple infinite/continuous animations (Framer Motion + CSS keyframes) but doesn’t appear to respect reduced-motion preferences. Please gate these animations with useReducedMotion() (for Motion transitions) and/or disable keyframes when prefers-reduced-motion: reduce to avoid motion-sickness issues.
| .shine-text { | ||
| background: linear-gradient( | ||
| 90deg, | ||
| #888 0%, | ||
| #aaa 33%, | ||
| #888 50%, | ||
| #aaa 66%, | ||
| #888 100% | ||
| ); | ||
| background-size: 200% 100%; | ||
| -webkit-background-clip: text; | ||
| background-clip: text; | ||
| -webkit-text-fill-color: transparent; | ||
| animation: orb-label-shine 2s ease-in-out infinite; | ||
| } |
There was a problem hiding this comment.
.shine-text is already defined globally in sds/ai/F0ActionItem/styles.css. Defining it again here (with different gradient/animation) will create import-order dependent styling bugs across components. Please rename/scope these classes (e.g., f0-orb-...) to avoid global collisions.
Description
Add Orb Voice Animation: a WebGL shader component for AI Voice Agents
Screenshots
Implementation details