Add a git based sync option #441 by carmiac · Pull Request #722 · GothenburgBitFactory/taskchampion

carmiac · 2026-04-18T05:33:26Z

This add a git backed sync server to TaskChampion.

Repo Layout

Versions are stored as files named v-{parent_uuid}-{child_uuid}, containing encrypted [HistorySegment] bytes.
Snapshots are stored as a single file named snapshot, containing a JSON wrapper around an encrypted full-state blob.
Metadata (meta) holds the latest version UUID and the encryption salt as JSON.

Writes

After each write (add_version, add_snapshot) the server stages the changed files, creates a commit, and pushes to the remote. If the push is rejected , the commit is rolled back and the caller receives an [AddVersionResult::ExpectedParentVersion] or an [Error] so it can retry.

After a snapshot is stored, [GitSyncServer::cleanup] automatically removes all version files whose history is now captured by the snapshot, keeping the repository compact.

Options

There are several configuration options.

branch sets the branch to use for TaskChampion. This would let someone keep tasks alongside a project without dealing with history issues.
remote sets the remote. Same format the git uses.
local_only makes push and pull no-ops. This could be used if the remote isn't ready yet, or is unavailable for some other reason.

General Notes for Reviewers

I haven't done any performance testing, but it seems reasonably quick for manual use. I didn't see a general performance testing module.
Currently is uses the same salt for all files. This isn't great security practice, but does seem to be what the other servers are doing. I could add a per-file salt, at the cost of however long Cryptor::new() takes on each read and write.

and then to bed

djmitche

This is so cool! I would like to take the time to read and experiment with it, but a few initial reactions to the comments and the PR description:

Optional configuration for the path to the git binary will probably be useful for someone, and should be easy to add
It looks like remote accepts the string "None"? Let's make that an Option<String> instead, if that's the case, or make the empty string the special case.
For cleanup, two observations:
- Git keeps all of the data anyway, so "deleting" a version is really just removing the name from the directory. So, I don't think deleting old versions has much impact.
- When a replica syncs, it needs all versions since the last version it saw. Otherwise, it has to restore from the snapshot and lose any local data since it last sync'd.
- So, it's beneficial to keep versions around for as long as is practical. The cloud servers keep a half-year's worth of versions (src/server/cloud/server.rs, MAX_VERSION_AGE_SECS), and I think that's reasonable here, too.

I'd be interested to know what @ryneeverett thinks, too!

carmiac · 2026-04-19T13:45:54Z

Thanks for the review! I'll take a look at those. My reason behind the git rm was to make the globs when adding new versions faster, but I didn't realize how that interacted with restoration.

djmitche · 2026-04-19T18:17:50Z

Hm, good point, but local globs are probably pretty fast even for 100's of dentries.

djmitche

A bunch of comments here, but all toward improving the implementation rather than show-stopping issues. I think most of what I've suggested is relatively straightforward, but if necessary some of it can be done in followup PRs.

One general observation is, despite Server being an async trait, this invokes Git synchronously. I think that's fine for the expected use-cases for this sync model, and it's something that can be improved later if desired.

tl;dr: This looks great, and I look forward to merging it after some minor revisions!

djmitche · 2026-04-19T22:44:40Z

 # Suppport for sync to another SQLite database on the same machine
 server-local = ["dep:rusqlite"]
+# Support for sync via Git
+server-git = ["git-sync"]


Why the two levels of features here? I think one would be sufficient. The cloud syncs have cloud because it enables some common functionality, but there's no such thing for git. So,

server-git = ["dep:serde_with", "dep:glob", "encryption"]

and update the cfg(feature..) in the code.

djmitche · 2026-04-19T22:52:19Z

+//!
+//! - I haven't done any performance testing, but it seems reasonably quick for manual use.
+//! - Currently is uses the same salt for all files. This isn't great security practice,
+//!   but does seem to be what the other servers are doing.


This is a good point! In defense of the idea, the salt is used in the key derivation, and we use the same key for every file, so in that sense it's only used once. If there's further concern, let's open an issue about it -- I'm sure others would like to chime in too!

djmitche · 2026-04-19T22:53:34Z

+//!
+//! Notes for Reviewers
+//!
+//! - I haven't done any performance testing, but it seems reasonably quick for manual use.


This seems fine. We have not focused on performance testing of the sync operations since most of the time is network-related.

djmitche · 2026-04-19T23:01:20Z

+//!   create a 'task' branch and let TaskChampion manage that branch.
+//! - This does support both defining a remote and having `local_only` mode set at the same
+//!   time. The idea is that maybe the remote isn't ready yet, or eithe rtemporarily or
+//!   permanantly down. Either way, you can use this in local mode in the mean time.


What are the risks here? In general, the more user-facing bits of the doc here might be better as docstrings on the configuration enum.

djmitche · 2026-04-19T23:04:54Z

+//!
+//! - Since this shells out to git, it assumes that you havea reasonably functional git
+//!   setup. I.e. 'git init', 'git add', 'git commit', etc shoud just work.
+//! - If you are using a remote, 'git push' and 'git pull' shoud work.


I suspect there's some room for more robustness here, such as disabling prompts. Maybe we can address that as we find issues, but it might be worth thinking about ahead of time.

I see that nothing needs to parse the output of a git command, so that simplifies things!

djmitche · 2026-04-19T23:12:59Z

+}
+
+/// Run a git command in a given directory, returning an error if it exits non-zero.
+fn git_cmd(dir: &Path, args: &[&str]) -> Result<()> {


This ends up putting a lot of git's output in the cargo test output and probably in the taskwarrior output as well. What do you think of amending this function so that it only shows the output on an unexpected error, or logs it all to log::debug or something similar?

djmitche · 2026-04-19T23:22:27Z

+        fs::create_dir_all(local_path)?;
+
+        // Check if path is already a git repo.
+        let is_repo = Command::new("git")


I think this doesn't use git_cmd because a nonzero exit status is expected, and similar for git checkout below. Maybe git_cmd could be extended to support that situation?

djmitche · 2026-04-19T23:26:38Z

+        git_cmd(
+            local_path,
+            &["clean", "-f", "--", "v-*", "snapshot", "meta"],
+        )?;


This happens at least twice and could be a helper function!

djmitche · 2026-04-19T23:42:22Z

+
+    /// Fetch and fast-forward to the remote branch. No-op in local-only mode.
+    /// If the remote branch does not yet exist (e.g. fresh bare repo), this is also a no-op.
+    fn pull(&self) -> Result<()> {


This isn't really a pull, since it does a hard reset. Maybe fn reset_to_remote?

djmitche · 2026-04-19T23:47:15Z

+            parent_version_id,
+            history_segment,
+        };
+        let version_path = self.add_version_by_parent_version_id(&version)?;


For robustness, should this check that the working copy is clean before starting modifications? If a previous invocation of add_version crashed right here, we'd end up with two different version files being committed.

carmiac added 15 commits April 10, 2026 23:04

Basic repo creation implemented, with tests.

06c70bb

add git push/pull helpers

a9b3217

add version

26fbaaf

and then to bed

Add get_version_by_parent_version_id and test

e69b03b

add get_child_version

73989ec

add snapshot system

660881c

fix git cleanup and misc clippy issues

c104e0c

fix/log some git issues from snapshot testing

ec2ae74

fix snapshot encryption

94fbbd4

add some documentation

9b2818a

Add more snapshot tests

d40262f

add cleanup after snapshot

e603030

add documentation

7fc7ecf

cleanup of snapshot code, add documentation

fd09e02

misc cleanup

022948a

carmiac requested a review from djmitche as a code owner April 18, 2026 05:33

carmiac added 3 commits April 17, 2026 22:36

Merge branch 'main' into git-sync

faad1ba

lint

8cb46b2

Merge branch 'git-sync' of github.com:carmiac/taskchampion into git-sync

9a4b371

djmitche reviewed Apr 18, 2026

View reviewed changes

djmitche requested changes Apr 19, 2026

View reviewed changes

Conversation

carmiac commented Apr 18, 2026

Repo Layout

Writes

Options

General Notes for Reviewers

Uh oh!

djmitche left a comment

Choose a reason for hiding this comment

Uh oh!

carmiac commented Apr 19, 2026

Uh oh!

djmitche commented Apr 19, 2026

Uh oh!

djmitche left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants