[CT-9085] Sync main with upstream

.github/workflows/node.js.yml

@@ -16,14 +16,16 @@ jobs:
 
     strategy:
       matrix:
-        node-version: [10.x, 12.x, 14.x]
+        node-version: [16.x, 20.x]
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Use Node.js ${{ matrix.node-version }}
-      uses: actions/setup-node@v1
+      uses: actions/setup-node@v3
       with:
         node-version: ${{ matrix.node-version }}
     - run: npm ci
-    - run: npm run build --if-present
-    - run: npm test
+    - run: npm run lint
+    - run: npm run test
+      env:
+        CI: true

.well-known/funding-manifest-urls

@@ -0,0 +1 @@
+https://yjs.dev

INTERNALS.md

@@ -26,7 +26,7 @@ article](https://blog.kevinjahns.de/are-crdts-suitable-for-shared-editing/).
 
 Each client is assigned a unique *clientID* property on first insert. This is a
 random 53-bit integer (53 bits because that fits in the javascript safe integer
-range).
+range \[JavaScript uses IEEE 754 floats\]).
 
 ## List items
 
@@ -60,7 +60,7 @@ characters have either been deleted or all characters are not deleted. The item
 will be split if the run is interrupted for any reason (eg a character in the
 middle of the run is deleted).
 
-When an item is created, it stores a reference to the IDs of the preceeding and
+When an item is created, it stores a reference to the IDs of the preceding and
 succeeding item. These are stored in the item's `origin` and `originRight`
 fields, respectively. These are used when peers concurrently insert at the same
 location in a document. Though quite rare in practice, Yjs needs to make sure
@@ -88,7 +88,7 @@ When a local insert happens, Yjs needs to map the insert position in the
 document (eg position 1000) to an ID. With just the linked list, this would
 require a slow O(n) linear scan of the list. But when editing a document, most
 inserts are either at the same position as the last insert, or nearby. To
-improve performance, Yjs stores a cache of the 10 most recently looked up
+improve performance, Yjs stores a cache of the 80 most recently looked up
 insert positions in the document. This is consulted and updated when a position
 is looked up to improve performance in the average case. The cache is updated
 using a heuristic that is still changing (currently, it is updated when a new
@@ -149,8 +149,8 @@ concepts that can be used to create a custom network protocol:
 
 * `update`: The Yjs document can be encoded to an *update* object that can be
   parsed to reconstruct the document. Also every change on the document fires
-an incremental document updates that allows clients to sync with each other.
-The update object is an Uint8Array that efficiently encodes `Item` objects and
+an incremental document update that allows clients to sync with each other.
+The update object is a Uint8Array that efficiently encodes `Item` objects and
 the delete set.
 * `state vector`: A state vector defines the known state of each user (a set of
   tuples `(client, clock)`). This object is also efficiently encoded as a

LICENSE

@@ -1,7 +1,7 @@
 The MIT License (MIT)
 
-Copyright (c) 2014
-  - Kevin Jahns <kevin.jahns@rwth-aachen.de>.
+Copyright (c) 2023
+  - Kevin Jahns <kevin.jahns@protonmail.com>.
   - Chair of Computer Science 5 (Databases & Information Systems), RWTH Aachen University, Germany
 
 Permission is hereby granted, free of charge, to any person obtaining a copy

attributing-content.md

@@ -0,0 +1,121 @@
+# IdSets and IdMaps
+
+`IdSet` is a data structure (formerly `DeleteSet`) that allows us to efficiently
+represent ranges of ids in Yjs (all content is identifyable by ids).
+
+`IdMap` is a new data structure that allows us to efficiently map ids to
+attributes. It can be efficiently encoded.
+
+We can perform all usual set operations on `IdMap`s and `IdSet`s: diff, merge,
+intersect.
+
+# Attribution of content
+
+In order to implement a Google Docs-like versioning feature, we want to be able
+to attribute content with additional information (who created the change,
+when was this change created, ..).
+
+When we click on a version in Google Docs, we might get annotated changes like
+this:
+
+```
+# E.g. If Bob appends "world" to the previous version "hello "
+[{ insert: 'hello' }, { insert: 'world', color: 'blue', creator: 'Bob', when: 'yesterday' }]
+# E.g. If Bob deletes "world" from the previous version "hello world"
+[{ insert: 'hello' }, { insert: 'world', backgroundColor: 'red', creator: 'Bob', when: 'yesterday' }]
+```
+
+In Yjs, we can now "attribute" changes with additional information. When we
+render content using methods like `toString()` or `getDelta()`, Yjs will render
+the unattributed content as-is, but it will render the attributed content with
+the additional information. As all changes in Yjs are identifyable by Ids, we
+can use `IdMap`s to map changes to "attributions". For example, we could
+attribute deletions and insertions of a change and render them:
+
+```js
+// We create some initial content "Hello World!". Then we create another
+// document that will have a bunch of changes (make "Hell" italic, replace "World"
+// with "Attribution").
+const ydocVersion0 = new Y.Doc({ gc: false })
+ydocVersion0.getText().insert(0, 'Hello World!')
+const ydoc = new Y.Doc({ gc: false })
+Y.applyUpdate(ydoc, Y.encodeStateAsUpdate(ydocVersion0))
+const ytext = ydoc.getText()
+ytext.applyDelta([{ retain: 4, attributes: { italic: true } }, { retain: 2 }, { delete: 5 }, { insert: 'attributions' }])
+// this represents to all insertions of ydoc
+const insertionSet = Y.createInsertionSetFromStructStore(ydoc.store)
+const deleteSet = Y.createDeleteSetFromStructStore(ydoc.store)
+// exclude the changes from `ydocVersion0`
+const insertionSetDiff = Y.diffIdSet(insertionSet, Y.createInsertionSetFromStructStore(ydocVersion0.store))
+const deleteSetDiff = Y.diffIdSet(deleteSet, Y.createDeleteSetFromStructStore(ydocVersion0.store))
+// assign attributes to the diff
+const attributedInsertions = createIdMapFromIdSet(insertionSetDiff, [new Y.Attribution('insert', 'Bob')])
+const attributedDeletions = createIdMapFromIdSet(deleteSetDiff, [new Y.Attribution('delete', 'Bob')])
+// now we can define an attribution manager that maps these changes to output. One of the
+// implementations is the TwosetAttributionManager
+const attributionManager = new TwosetAttributionManager(attributedInsertions, attributedDeletions)
+// we render the attributed content with the attributionManager
+let attributedContent = ytext.getContent(attributionManager)
+console.log(JSON.stringify(attributedContent.toJSON().ops, null, 2))
+let expectedContent = delta.create().insert('Hell', { italic: true }, { attributes: { italic: ['Bob'] } }).insert('o ').insert('World', {}, { delete: ['Bob'] }).insert('attributions', {}, { insert: ['Bob'] }).insert('!')
+t.assert(attributedContent.equals(expectedContent))
+
+// this is how the output would look like
+const output = [
+  {
+    "insert": "Hell",
+    "attributes": {
+      "italic": true
+    },
+    "attribution": {                // no "insert" attribution: the insertion "Hell" is not attributed to anyone
+      "attributes": {
+        "italic": [                 // the attribute "italic" was added by Bob
+          "Bob"
+        ]
+      }
+    }
+  },
+  {
+    "insert": "o "                  // the insertion "o " has no attributions
+  },
+  {
+    "insert": "World",
+    "attribution": {                // the insertion "World" was deleted by Bob
+      "delete": [
+        "Bob"
+      ]
+    }
+  },
+  {
+    "insert": "attributions",       // the insertion "attributions" was inserted by Bob
+    "attribution": {
+      "insert": [
+        "Bob"
+      ]
+    }
+  },
+  {
+    "insert": "!"                   // the insertion "!" has no attributions
+  }
+]
+```
+
+We get a similar output to Google Docs: Insertions, Deletions, and changes to
+formatting (attributes) are clearly associated to users. It will be the job of
+the editor to render those changes with background-color etc..
+
+Of course, we could associated changes also to multiple users like this:
+
+```js
+const attributedDeletions = createIdMapFromIdSet(deleteSetDiff, [new Y.Attribution('insert', 'Bob'), new Y.Attribution('insert', 'OpenAI o3')])
+```
+
+You could use the same output to calculate a real diff as well (consisting of
+deletions and insertions only, without Attributions). 
+
+`AttributionManager` is an abstract class for mapping attributions. It is
+possible to highlight arbitrary content with this approach. 
+
+The AttributionManager is encodes very efficiently. The ids are encoded using
+run-length encoding and the Attributes are de-duplicated and only encoded once.
+The above example encodes in 20 bytes.