Skip to content

feat: add freeform drawing gestures#186

Open
aidenybai wants to merge 1 commit intomainfrom
feat/freeform-drawing
Open

feat: add freeform drawing gestures#186
aidenybai wants to merge 1 commit intomainfrom
feat/freeform-drawing

Conversation

@aidenybai
Copy link
Owner

@aidenybai aidenybai commented Feb 20, 2026

Summary

  • Adds freeform drawing gestures (circle to multi-select, arrow to point-select) that open the comment prompt on matched elements
  • Adds a draw mode toolbar button with a dedicated icon so users can draw without holding Alt
  • Fixes the click handler's capture-phase deactivation guard to not kill freeform sessions mid-flight

Test plan

  • Click the draw toolbar button, draw a circle around elements, verify comment prompt opens
  • Draw an arrow pointing at an element, verify comment prompt opens on that element
  • Hold Alt and draw without using the toolbar button, verify same behavior
  • Draw with too few points (quick tap), verify gesture is discarded gracefully
  • Run pnpm test to verify e2e tests pass

Made with Cursor


Summary by cubic

Add freeform drawing to select elements and open the comment prompt. Draw a circle to multi-select or an arrow to select one element, using Alt+drag or the new Draw toolbar button.

  • New Features

    • Freeform draw mode (Alt+drag or toolbar). Circle = multi-select; arrow = single select; opens the comment prompt.
    • Overlay renders pressure-based strokes and fades them after completion; crosshair/selection are hidden while drawing; waits briefly before classifying the gesture.
    • New Draw toolbar button with icon; keeps the overlay active after a gesture.
    • E2E tests cover arrow/circle gestures, crosshair visibility, inactive overlay behavior, insufficient points, and post-gesture overlay state.
  • Bug Fixes

    • Prevent capture-phase deactivation from interrupting freeform sessions.
    • Preserve overlay visibility when entering comment mode after a gesture.

Written for commit 413a2cb. Summary will update on new commits.


Note

Medium Risk
Touches core pointer event handling and overlay state transitions, so regressions could affect drag/select behavior and activation/deactivation flow; changes are contained and covered by new e2e tests.

Overview
Adds a freeform drawing interaction (Alt-drag or new toolbar Draw toggle) that records pointer strokes, classifies them as arrow vs circle, and uses the result to select/freeze either a single element (arrow endpoint) or multiple elements within the stroke bounding box (circle) before entering comment prompt mode.

Updates the overlay renderer to draw and fade pressure-weighted strokes on a new freeform canvas layer, and suppresses crosshair/selection/label rendering while a freeform session is active to avoid UI conflicts.

Introduces StrokePoint typing, new freeform-related constants, a classify-stroke-gesture utility, a new IconDraw, and adds Playwright e2e coverage for gesture recognition, overlay visibility behavior, and insufficient-point discards; also adjusts click/cancel handling so capture-phase deactivation doesn’t interrupt freeform sessions.

Written by Cursor Bugbot for commit 413a2cb. This will update automatically on new commits. Configure here.

Draw on elements using Alt+drag or the toolbar draw button to circle/arrow
select them and open the comment prompt. Adds stroke gesture classification,
overlay canvas rendering, and e2e tests for the full drawing flow.

Co-authored-by: Cursor <cursoragent@cursor.com>
@pullfrog
Copy link
Contributor

pullfrog bot commented Feb 20, 2026

This run croaked 😵

The workflow encountered an error before any progress could be reported. Please check the workflow run logs for details.

Pullfrog  | View workflow run | Triggered by Pullfrogpullfrog.com𝕏

@vercel
Copy link
Contributor

vercel bot commented Feb 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
react-grab-website Ready Ready Preview, Comment Feb 20, 2026 10:02am

@pkg-pr-new
Copy link

pkg-pr-new bot commented Feb 20, 2026

Open in StackBlitz

@react-grab/cli

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/cli@186

grab

npm i https://pkg.pr.new/aidenybai/react-grab/grab@186

@react-grab/ami

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/ami@186

@react-grab/amp

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/amp@186

@react-grab/claude-code

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/claude-code@186

@react-grab/codex

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/codex@186

@react-grab/copilot

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/copilot@186

@react-grab/cursor

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/cursor@186

@react-grab/droid

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/droid@186

@react-grab/gemini

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/gemini@186

@react-grab/opencode

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/opencode@186

react-grab

npm i https://pkg.pr.new/aidenybai/react-grab@186

@react-grab/relay

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/relay@186

@react-grab/utils

npm i https://pkg.pr.new/aidenybai/react-grab/@react-grab/utils@186

commit: 413a2cb

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 issues found across 9 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/react-grab/src/components/overlay-canvas.tsx">

<violation number="1" location="packages/react-grab/src/components/overlay-canvas.tsx:393">
P2: Add a check for `props.freeformStrokeVisible` at the beginning of `renderFreeformLayer` to ensure the stroke is hidden when the prop is false.</violation>

<violation number="2" location="packages/react-grab/src/components/overlay-canvas.tsx:593">
P1: Remove the `else if` block that continuously sets `shouldContinueAnimating = true` when drawing to prevent an infinite `requestAnimationFrame` loop and high CPU usage.</violation>
</file>

<file name="packages/react-grab/e2e/freeform-drawing.spec.ts">

<violation number="1" location="packages/react-grab/e2e/freeform-drawing.spec.ts:6">
P2: Missing test coverage for the draw mode toolbar button. The tests currently only cover the 'Alt' key activation path.</violation>

<violation number="2" location="packages/react-grab/e2e/freeform-drawing.spec.ts:117">
P2: This test completely duplicates the logic from 'should enter comment mode via arrow gesture' (lines 7-29). Consider moving the `isOverlayVisible` assertion into the first test and removing this duplicate to reduce maintenance burden.</violation>
</file>

<file name="packages/react-grab/src/core/index.tsx">

<violation number="1" location="packages/react-grab/src/core/index.tsx:490">
P1: Freeform timers `freeformCleanupTimerId` and `freeformIdleTimerId` are never cleared in `onCleanup`, leaking timers on component disposal. Both should be cleared alongside the other timers in the existing cleanup block.</violation>

<violation number="2" location="packages/react-grab/src/core/index.tsx:1528">
P1: `deactivateRenderer` clears draw mode but does not call `cancelFreeformDraw()`. A mid-stroke deactivation leaves `document.body.style.userSelect = "none"` unreset, freeform signals in a dirty state, and pending timers that will fire on a deactivated renderer.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

freeformFadeOpacity = 1;
shouldContinueAnimating = true;
}
} else if (
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Remove the else if block that continuously sets shouldContinueAnimating = true when drawing to prevent an infinite requestAnimationFrame loop and high CPU usage.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/react-grab/src/components/overlay-canvas.tsx, line 593:

<comment>Remove the `else if` block that continuously sets `shouldContinueAnimating = true` when drawing to prevent an infinite `requestAnimationFrame` loop and high CPU usage.</comment>

<file context>
@@ -517,6 +576,28 @@ export const OverlayCanvas: Component<OverlayCanvasProps> = (props) => {
+        freeformFadeOpacity = 1;
+        shouldContinueAnimating = true;
+      }
+    } else if (
+      props.freeformStrokeVisible &&
+      props.freeformStrokePoints &&
</file context>
Fix with Cubic

);
const [freeformStrokeCompletedAt, setFreeformStrokeCompletedAt] =
createSignal<number | null>(null);
let freeformCleanupTimerId: ReturnType<typeof setTimeout> | null = null;
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Freeform timers freeformCleanupTimerId and freeformIdleTimerId are never cleared in onCleanup, leaking timers on component disposal. Both should be cleared alongside the other timers in the existing cleanup block.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/react-grab/src/core/index.tsx, line 490:

<comment>Freeform timers `freeformCleanupTimerId` and `freeformIdleTimerId` are never cleared in `onCleanup`, leaking timers on component disposal. Both should be cleared alongside the other timers in the existing cleanup block.</comment>

<file context>
@@ -467,6 +477,29 @@ export const init = (rawOptions?: Options): ReactGrabAPI => {
+    );
+    const [freeformStrokeCompletedAt, setFreeformStrokeCompletedAt] =
+      createSignal<number | null>(null);
+    let freeformCleanupTimerId: ReturnType<typeof setTimeout> | null = null;
+    let freeformIdleTimerId: ReturnType<typeof setTimeout> | null = null;
+
</file context>
Fix with Cubic

arrowNavigator.clearHistory();
keyboardSelectedElement = null;
isPendingContextMenuSelect = false;
setIsDrawMode(false);
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: deactivateRenderer clears draw mode but does not call cancelFreeformDraw(). A mid-stroke deactivation leaves document.body.style.userSelect = "none" unreset, freeform signals in a dirty state, and pending timers that will fire on a deactivated renderer.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/react-grab/src/core/index.tsx, line 1528:

<comment>`deactivateRenderer` clears draw mode but does not call `cancelFreeformDraw()`. A mid-stroke deactivation leaves `document.body.style.userSelect = "none"` unreset, freeform signals in a dirty state, and pending timers that will fire on a deactivated renderer.</comment>

<file context>
@@ -1491,6 +1525,7 @@ export const init = (rawOptions?: Options): ReactGrabAPI => {
       arrowNavigator.clearHistory();
       keyboardSelectedElement = null;
       isPendingContextMenuSelect = false;
+      setIsDrawMode(false);
       if (wasDragging) {
         document.body.style.userSelect = "";
</file context>
Fix with Cubic

const context = layer.context;
context.clearRect(0, 0, canvasWidth, canvasHeight);

const strokes = props.freeformStrokePoints;
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Add a check for props.freeformStrokeVisible at the beginning of renderFreeformLayer to ensure the stroke is hidden when the prop is false.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/react-grab/src/components/overlay-canvas.tsx, line 393:

<comment>Add a check for `props.freeformStrokeVisible` at the beginning of `renderFreeformLayer` to ensure the stroke is hidden when the prop is false.</comment>

<file context>
@@ -366,6 +383,46 @@ export const OverlayCanvas: Component<OverlayCanvasProps> = (props) => {
+    const context = layer.context;
+    context.clearRect(0, 0, canvasWidth, canvasHeight);
+
+    const strokes = props.freeformStrokePoints;
+    if (!strokes || strokes.length === 0) return;
+    if (freeformFadeOpacity <= 0) return;
</file context>
Suggested change
const strokes = props.freeformStrokePoints;
if (!props.freeformStrokeVisible) return;
const strokes = props.freeformStrokePoints;
Fix with Cubic

expect(state.isActive).toBe(false);
});

test("should keep overlay active in comment mode after freeform gesture", async ({
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This test completely duplicates the logic from 'should enter comment mode via arrow gesture' (lines 7-29). Consider moving the isOverlayVisible assertion into the first test and removing this duplicate to reduce maintenance burden.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/react-grab/e2e/freeform-drawing.spec.ts, line 117:

<comment>This test completely duplicates the logic from 'should enter comment mode via arrow gesture' (lines 7-29). Consider moving the `isOverlayVisible` assertion into the first test and removing this duplicate to reduce maintenance burden.</comment>

<file context>
@@ -0,0 +1,168 @@
+    expect(state.isActive).toBe(false);
+  });
+
+  test("should keep overlay active in comment mode after freeform gesture", async ({
+    reactGrab,
+  }) => {
</file context>
Fix with Cubic

@@ -0,0 +1,168 @@
import { test, expect } from "./fixtures.js";
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Missing test coverage for the draw mode toolbar button. The tests currently only cover the 'Alt' key activation path.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/react-grab/e2e/freeform-drawing.spec.ts, line 6:

<comment>Missing test coverage for the draw mode toolbar button. The tests currently only cover the 'Alt' key activation path.</comment>

<file context>
@@ -0,0 +1,168 @@
+const FREEFORM_IDLE_TIMEOUT_MS = 600;
+const GESTURE_SETTLE_BUFFER_MS = 300;
+
+test.describe("Freeform Drawing", () => {
+  test("should enter comment mode via arrow gesture", async ({ reactGrab }) => {
+    await reactGrab.activate();
</file context>
Fix with Cubic

Copy link
Contributor

@vercel vercel bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestions:

  1. When deactivateRenderer() is called, active freeform drawing sessions are not cancelled, leaving timers running and potentially causing memory leaks
  1. The pointercancel event listener is missing the capture: true option while pointerdown and pointerup use it, causing inconsistent event handling phases
Fix on Vercel

): StrokePoint => ({
x: clientX,
y: clientY,
pressure: pressure || FREEFORM_STROKE_DEFAULT_PRESSURE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pressure: pressure || FREEFORM_STROKE_DEFAULT_PRESSURE,
pressure: pressure ?? FREEFORM_STROKE_DEFAULT_PRESSURE,

Pressure value of 0 incorrectly replaced with default pressure in createStrokePoint function

Fix on Vercel

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

arrowNavigator.clearHistory();
keyboardSelectedElement = null;
isPendingContextMenuSelect = false;
setIsDrawMode(false);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deactivation doesn't cancel freeform timers or state

High Severity

deactivateRenderer sets isDrawMode(false) but never calls cancelFreeformDraw(). If the renderer deactivates (via keyboard shortcut, toggle button, etc.) while a freeform session is active or during the 600ms idle timeout window, freeformIdleTimerId keeps running and eventually fires finalizeFreeformSession. That calls handleFreeformCircle/handleFreeformArrow, which invoke actions.freeze(), enterCommentModeForElement, and other store mutations on a deactivated renderer — leaving the app in an inconsistent state. Additionally, document.body.style.userSelect may remain "none" since the drag-specific cleanup guard (wasDragging) doesn't cover freeform drawing.

Additional Locations (1)

Fix in Cursor Fix in Web

Copy link

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 1 comment


freeformIdleTimerId = setTimeout(() => {
freeformIdleTimerId = null;
finalizeFreeformSession();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟡 Medium]

The idle timeout finalizes the freeform session unconditionally, even if the renderer was deactivated after pointer-up (for example via toolbar toggle) but before the 600ms idle delay expires. That allows a canceled gesture to still freeze elements and enter prompt mode after the user has explicitly turned the overlay off. ```ts
// packages/react-grab/src/core/index.tsx
freeformIdleTimerId = setTimeout(() => {
freeformIdleTimerId = null;
finalizeFreeformSession();
}, FREEFORM_IDLE_TIMEOUT_MS);


```suggestion
        if (isActivated()) finalizeFreeformSession(); else cancelFreeformDraw();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant