Skip to content

Dev 2.3#20

Open
wlsdml1114 wants to merge 7 commits intomainfrom
dev_2.3
Open

Dev 2.3#20
wlsdml1114 wants to merge 7 commits intomainfrom
dev_2.3

Conversation

@wlsdml1114
Copy link
Owner

track management enhanced

@gemini-code-assist
Copy link

Summary of Changes

Hello @wlsdml1114, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the video editor's track management capabilities and overall user experience. It introduces robust undo/redo functionality for keyframe operations, making the editing process more forgiving and efficient. The timeline has been made more dynamic, automatically adjusting to content and offering improved interaction through a draggable playhead and precise keyframe trimming. Furthermore, audio handling has been upgraded with better track type support and more accurate FFmpeg processing during video export, ensuring superior synchronization and output quality.

Highlights

  • Undo/Redo Functionality: Implemented a comprehensive history management system for keyframe modifications, allowing users to undo and redo changes to their project timeline. This includes tracking additions, updates, and removals of keyframes.
  • Dynamic Timeline Duration & Responsiveness: The video timeline now automatically expands its duration based on the content (keyframes) present, ensuring there's always enough space for new media. The playhead is now draggable, enabling direct seeking, and the timeline ruler's time formatting has been improved for better readability.
  • Enhanced Audio Track Management and Processing: Introduced an explicit 'audio' track type alongside 'music' and 'voiceover', providing more granular control. FFmpeg commands for video export have been refined to precisely trim and synchronize audio files, and the system now handles media duration detection more robustly, especially for blob URLs.
  • Improved User Interface for Tracks: Added a new 'Add Track' dropdown menu in the timeline controls, allowing users to easily create new video or audio tracks. Each track row now includes a delete button for quick removal, and keyframes can be trimmed from both left and right sides directly on the timeline.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • src/app/api/video-export/route.ts
    • Implemented logic to dynamically calculate the composition's duration based on the latest keyframe's end time, ensuring the exported video matches the content length.
    • Modified the audio export process to include keyframe duration, enabling precise audio trimming and synchronization using FFmpeg's atrim and asetpts filters for both single and multiple audio files.
  • src/components/video-editor/TimelineControls.tsx
    • Added a new 'Add Track' dropdown menu, allowing users to create new 'Video Track' or 'Audio Track' instances directly from the timeline controls.
    • Imported new icons (Plus, Video, Music, Mic) and UI components (DropdownMenu, etc.) to support the new track addition feature.
  • src/components/video-editor/TimelineRuler.tsx
    • Refactored the formatTickLabel function to display timeline markers in a more standard and readable MM:SS or MM:SS.ms format.
  • src/components/video-editor/VideoComposition.tsx
    • Adjusted the audio sequence duration calculation to strictly use the keyframe's visual duration, aligning the component's lifecycle with the timeline's visual representation for trimming.
  • src/components/video-editor/VideoEditorView.tsx
    • Integrated undo and redo functionality into the video editor view, allowing users to revert or reapply changes using keyboard shortcuts (Ctrl+Z / Cmd+Z and Ctrl+Shift+Z / Cmd+Shift+Z).
  • src/components/video-editor/VideoTimeline.tsx
    • Implemented auto-expansion of the project's duration based on the latest keyframe, ensuring the timeline always accommodates all content with a buffer.
    • Added a ResizeObserver to dynamically adjust the timeline's display width, improving responsiveness.
    • Made the playhead draggable, allowing users to directly seek to specific points in the video by dragging the indicator.
    • Enhanced media drop handling to map 'tts' media type to 'voiceover' tracks and to broadly find existing audio tracks (audio, music, voiceover) for dropped media.
    • Improved media duration detection for audio and video elements by changing preload to 'auto' and adding more event listeners, particularly beneficial for blob URLs.
    • Updated keyframe drop and move logic to correctly calculate timestamps based on the dynamic display width and visual duration.
  • src/components/video-editor/VideoTrackRow.tsx
    • Added a delete button (TrashIcon) to each track row, visible on hover, allowing users to remove tracks with a confirmation.
    • Introduced a left trim handle for keyframes, enabling trimming from both ends of a media clip on the timeline.
    • Updated track background styling to provide clearer visual distinctions between video and audio track types.
    • Modified keyframe dragging logic to correctly handle moving audio clips between different audio-compatible tracks (audio, music, voiceover).
  • src/lib/context/StudioContext.tsx
    • Expanded the VideoTrack type to include an 'audio' type, providing more flexibility in track categorization.
    • Implemented a history state management system (history, historyIndex, recordHistory, undo, redo) to track changes to keyframes.
    • Integrated history recording into addKeyframe, updateKeyframe, and removeKeyframe operations, ensuring all keyframe modifications are reversible.
  • src/lib/i18n/translations.ts
    • Added new translation keys for track management features, including 'Add Track', 'Add Video Track', and 'Add Audio Track' in both Korean and English.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly enhances the video editor's track management and user experience, introducing features such as track addition/deletion, an undo/redo system for keyframe edits, an improved timeline, and more robust video export functionality. However, it introduces a critical command injection vulnerability in the video export API, where unsanitized user input from the keyframes object is used to construct shell commands executed via execSync. This must be addressed by using secure process execution methods like child_process.spawn and implementing strict input validation. Additionally, a critical issue where project data is loaded twice needs to be resolved.

const filterString = filters.join(',');
const filterComplex = `-filter_complex "[1:a]${filterString}[aout]"`;

const ffmpegCmd = `ffmpeg -y -i "${tempVideoPath}" -i "${ap.path}" ${filterComplex} -c:v copy -c:a aac -map 0:v:0 -map "[aout]" -shortest "${outputPath}"`;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-critical critical

The FFmpeg command is constructed using unsanitized input from the keyframes object (specifically audio URLs, durations, and volumes) and executed via execSync. This allows an attacker to perform command injection by including shell metacharacters in the input.

Remediation: Use child_process.spawn with an array of arguments instead of execSync with a command string to avoid shell execution. Alternatively, use a library like fluent-ffmpeg and strictly validate all user-supplied inputs.

const filterComplex = `${volumeAndDelays};${mixInputs}amix=inputs=${audioPaths.length}:duration=longest[aout]`;
const filterComplex = `${processedInputs};${mixInputs}amix=inputs=${audioPaths.length}:duration=longest[aout]`;

const ffmpegCmd = `ffmpeg -y -i "${tempVideoPath}" ${inputs} -filter_complex "${filterComplex}" -map 0:v:0 -map "[aout]" -c:v copy -c:a aac "${outputPath}"`;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-critical critical

Similar to the previous finding, this FFmpeg command is also constructed using unsanitized user input and executed via execSync, leading to a command injection vulnerability.

Remediation: Use child_process.spawn with an array of arguments and strictly validate all user-supplied inputs.

Comment on lines +54 to +55
loadProjectData();
// eslint-disable-next-line react-hooks/exhaustive-deps

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This duplicate call to loadProjectData() is unnecessary and can lead to performance issues or race conditions due to redundant API requests and state updates. Please remove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant