Skip to content

Skip nil input and memoize plconvert#55

Open
tilthouse wants to merge 1 commit intoajrosen:mainfrom
tilthouse:aboster/plconvert-skip-nil-and-memoize
Open

Skip nil input and memoize plconvert#55
tilthouse wants to merge 1 commit intoajrosen:mainfrom
tilthouse:aboster/plconvert-skip-nil-and-memoize

Conversation

@tilthouse
Copy link
Copy Markdown
Contributor

Summary

plconvert (in lib/utils.rb) is called twice per reminder by Reminder#initialize — once for color, once for messaging. Each call forks /usr/bin/plutil. On a typical 4000-reminder store that's ~8000 fork+exec invocations, which dominates cold-start runtime for the tasks command.

Two observations on real data make the work redundant:

  • messaging (zContactHandles) is null for the vast majority of reminders. Every plconvert(nil) call still forks plutil, sends an empty stdin, and comes back with nothing.
  • color is a list-level attribute that the SQL JOIN duplicates onto every reminder row. ~12 unique color blobs across ~40 lists end up getting re-parsed thousands of times.

Empirical impact (3853-reminder store):

  • 100 sequential plutil invocations: ~0.43 s.
  • 7700 sequential invocations (the worst case here): ~33 s.
  • After this fix, ~12 invocations: ~0.05 s.

This is one of two upstream fixes that take this user's icalpal tasks cold start from ~100 s to ~0.6 s. The other is filed separately as an issue (the ZMembershipsOfRemindersInSectionsAsData blob join in Reminder::QUERY — happy to file that as a PR too once we agree on the right SQL approach).

The fix

Two changes in lib/utils.rb:

def plconvert(obj)
  return nil if obj.nil? || (obj.respond_to?(:empty?) && obj.empty?)

  cached = PLCONVERT_CACHE[obj]
  return cached if cached || PLCONVERT_CACHE.key?(obj)

  # ... existing fork+parse logic ...

  PLCONVERT_CACHE[obj] = result
end

Plus a top-level PLCONVERT_CACHE = {} constant for the cache itself.

The cached || PLCONVERT_CACHE.key?(obj) check makes nil results cacheable too — a blob that fails to parse (Plist::UnimplementedElementError returns nil) doesn't get re-tried per row.

Behavior preservation

  • Parsed objects for the same input are byte-identical to the previous behavior (we cache plutil's own output).
  • nil inputs previously went through plutil → the rescue-or-empty-result path → effectively nil. The shim returns nil directly, same observable result.
  • No public API changes. Return shape (Array of plist objects, or nil) is unchanged.
  • Process-scoped cache; not thread-safe (icalPal is single-threaded).

Tests

Adds test/plconvert_test.rb — 5 cases using stdlib minitest:

  • nil input returns nil without spawning a subprocess
  • empty string returns nil without spawning a subprocess
  • repeated calls with the same input only fork plutil once
  • distinct inputs each spawn their own subprocess
  • a blob that produces nil is still cached (no re-fork on subsequent calls)

The Open3 stub is implemented via singleton-method aliasing rather than Minitest::Mock because the latter was removed in minitest 6.x — the test file should run cleanly on either minitest version.

Run with:

ruby test/plconvert_test.rb

I confirmed the suite catches the regression — reverting the patch causes all 5 tests to fail.

Notes

  • No changes to runtime dependencies, gemspec, or executable surface.
  • Rubocop clean against the project's existing style.
  • Happy to adjust naming (PLCONVERT_CACHE could be private inside a module if you'd prefer), wrap the cache in something LRU-bounded, or split the changes if you want them in smaller pieces.

`plconvert` is called twice per reminder by `Reminder#initialize`
(once for `color`, once for `messaging`). Each call forks
`/usr/bin/plutil`. On a 4000-reminder store that's ~8000 fork+exec
invocations, dominating cold-start runtime for `tasks`.

Two observations make the work redundant:

- `messaging` (zContactHandles) is null for the vast majority of
  reminders in real-world data — every `plconvert(nil)` still forks
  plutil only to come back with nothing.
- `color` is a list-level attribute that the SQL JOIN copies onto
  every reminder row. So the same ~12 unique color blobs are
  re-parsed thousands of times.

Fix:

- Early-return `nil` for nil/empty input, skipping the fork.
- Memoize results by input bytes in a process-scoped hash. nil
  results are cached too so failing blobs aren't retried per row.

Behavior is preserved: parsed objects for the same input are
byte-identical (we cache plutil's own output), and nil inputs already
went through the rescue-or-return-nil path implicitly.

## Tests

Adds `test/plconvert_test.rb` — 5 cases using stdlib minitest:

- nil input returns nil without spawning a subprocess
- empty string returns nil without spawning a subprocess
- repeated calls with the same input only fork plutil once
- distinct inputs each spawn their own subprocess
- a blob that produces nil is still cached (no re-fork)

The Open3 stub is implemented via singleton-method aliasing rather
than `Minitest::Mock` because the latter was removed in minitest 6.x.

Run with:

    ruby test/plconvert_test.rb

## Notes

- No changes to runtime dependencies, gemspec, or executable surface.
- No public API changes — `plconvert` signature and return shape are
  unchanged.
- Process-scoped cache; not thread-safe (icalPal is single-threaded).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant