Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog (Sorted by Date in Descending Order)

## 1.3.0.0

* Added overloads of `Upsert` that accept a `string` key, while the `ReadOnlySpan<char>` overloads are amazing in specific cases where its use can prevent a string allocation for the lookup, in other places where the input was originally a `string` that was implicitly converted to a `ReadOnlySpan<char>` for the parameter, this would've caused a copy to be allocated for the key when the key did not exist. The same scenario will now use the `string` overload and use it for the key directly, avoiding the intermediate copy.
* Added a `ValueTask` based `GetOrAddAsync` method, commonly used in caching scenarios.

## 1.2.0.0

* An overload to `Upsert` without `updateCondition` was added and would now act as default path in case `updateCondition` wasn't specified, this should further optimize such cases by removing condition checks and another reference from the stack during runtime.
Expand Down
26 changes: 24 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,23 @@ bool db.TryGetValue<TValue>(ReadOnlySpan<char> key, JsonTypeInfo<TValue> jsonTyp

Notice that all APIs accept keys as `ReadOnlySpan<char>` to avoid unnecessary allocations. This means that if you check for a key by some slice of a string, there is no need to allocate a string just for the lookup.

Upserting (adding or updating) is done via a 2 overloads:
Upserting (adding or updating) is done via 4 overloads:

```csharp
bool db.Upsert<TValue>(string key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo);
bool db.Upsert<TValue>(ReadOnlySpan<char> key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo);
bool db.Upsert<TValue>(ReadOnlySpan<char> key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo, Func<TValue, bool> updateCondition = null); // upserts a value into the ArrowDb instance
bool db.Upsert<TValue>(string, TValue value, JsonTypeInfo<TValue> jsonTypeInfo, Func<TValue, bool> updateCondition = null);
bool db.Upsert<TValue>(ReadOnlySpan<char> key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo, Func<TValue, bool> updateCondition = null);
```

### Upsert Overloads Best Practices

As noted above, There are 2 main upsert methods, but each has 2 options, a `ReadOnlySpan<char>` key, or a `string` key.

The `ReadOnlySpan<char>` methods are best used for scenarios where the key is generated using a slice of a string (whether from regular string, stackallocated buffers, interop and so and forth), and cases where the value for the same key is being frequently updated, in which case this method will replace the value without allocating the key.

The `string` methods are best for addition and cases where the parameter in the caller method is already of type `string`, in these case the direct use of `string` will prevent a `string` allocation by the lookup.

And removal:

```csharp
Expand Down Expand Up @@ -194,6 +204,18 @@ builder.Services.AddSingleton(() => ArrowDb.CreateInMemory().GetAwaiter().GetRes
// Since this isn’t persisted, you may also use it as a Transient or Scoped service (whatever fits your needs).
```

A common code pattern for caching usually consists of some `GetOrAdd` method, that will check if a value exists by the key, and return it, otherwise it will accept a method used to generate the value, which will be used to add the value to the cache, then return it.

`ArrowDb` supports this via the `async ValueTask` method:

```csharp
async ValueTask<TValue> GetOrAddAsync<TValue>(string key, JsonTypeInfo<TValue> jsonTypeInfo, Func<string, ValueTask<TValue>> valueFactory);
```

If the value exists, the asynchronous factory method is not called, and the value is returned synchronously. Otherwise the factory will produce the value, `Upsert` it, then return it.

Since `ArrowDb` was not made specifically to cache, it doesn't store time metadata for values, because of this, there will not be a method that accepts "cache expiration" or similar options in the foreseen future. Such scenarios will need to implemented client-side, best done with a pattern that splits read and write, by called `TryGetValue` which will also check the inner time reference, if false and out of date, will generate the value and use `Upsert`.

## Encryption

As seen earlier, the default recommended serializer is `FileSerializer`, which serializes the db to a file on disk. In addition to it, `ArrowDb` also features a similar serializer that encrypts the db to a file on disk (the `AesFileSerializer`), for that the serializer requires an `Aes` instance to be passed along with the path to the file.
Expand Down
25 changes: 25 additions & 0 deletions src/ArrowDbCore/ArrowDb.GetOrAdd.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
using System.Text.Json;
using System.Text.Json.Serialization.Metadata;

namespace ArrowDbCore;

public partial class ArrowDb {
/// <summary>
/// Tries to retrieve a value stored in the database under <paramref name="key"/>, if doesn't exist, it uses the factory to create and add it, then returns it.
/// </summary>
/// <typeparam name="TValue">The type of the value to get or add</typeparam>
/// <param name="key">The key at which to find or add the value</param>
/// <param name="jsonTypeInfo">The json type info for the value type</param>
/// <param name="valueFactory">The function used to generate a value for the key</param>
/// <returns>The value after finding or adding it</returns>
/// <remarks>
/// </remarks>
public async ValueTask<TValue> GetOrAddAsync<TValue>(string key, JsonTypeInfo<TValue> jsonTypeInfo, Func<string, ValueTask<TValue>> valueFactory) {
if (Lookup.TryGetValue(key, out var source)) {
return JsonSerializer.Deserialize(new ReadOnlySpan<byte>(source), jsonTypeInfo)!;
}
var val = await valueFactory(key);
Upsert(key, val, jsonTypeInfo);
return val;
}
}
37 changes: 37 additions & 0 deletions src/ArrowDbCore/ArrowDb.IDictionaryAccessor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
namespace ArrowDbCore;

public partial class ArrowDb {
/// <summary>
/// Provides an interface that unifies methods of upserting values to ArrowDb
/// </summary>
/// <typeparam name="TKey"></typeparam>
private interface IDictionaryAccessor<TKey> where TKey : allows ref struct {
/// <summary>
/// Assigns the <paramref name="value"/> to the <paramref name="key"/> in <paramref name="instance"/>
/// </summary>
/// <param name="instance">The ArrowDb instance</param>
/// <param name="key">The key to use</param>
/// <param name="value">The value to add/update</param>
void Upsert(ArrowDb instance, TKey key, byte[] value);
}

/// <summary>
/// Implements <see cref="IDictionaryAccessor{TKey}"/> by using the source dictionary directly
/// </summary>
private readonly ref struct StringAccessor : IDictionaryAccessor<string> {
/// <inheritdoc />
public void Upsert(ArrowDb instance, string key, byte[] value) {
instance.Source[key] = value;
}
}

/// <summary>
/// Implements <see cref="IDictionaryAccessor{TKey}"/> by using the lookup
/// </summary>
private readonly ref struct ReadOnlySpanAccessor : IDictionaryAccessor<ReadOnlySpan<char>> {
/// <inheritdoc />
public void Upsert(ArrowDb instance, ReadOnlySpan<char> key, byte[] value) {
instance.Lookup[key] = value;
}
}
}
67 changes: 58 additions & 9 deletions src/ArrowDbCore/ArrowDb.Upsert.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System.Text.Json;
using System.Runtime.CompilerServices;
using System.Text.Json;
using System.Text.Json.Serialization.Metadata;

namespace ArrowDbCore;
Expand All @@ -12,11 +13,33 @@ public partial class ArrowDb {
/// <param name="value">The value to upsert</param>
/// <param name="jsonTypeInfo">The json type info for the value type</param>
/// <returns>True</returns>
public bool Upsert<TValue>(string key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo) {
return UpsertCore<string, TValue, StringAccessor>(key, value, jsonTypeInfo, default);
}

/// <summary>
/// Upsert the specified key with the specified value into the database
/// </summary>
/// <typeparam name="TValue">The type of the value to upsert</typeparam>
/// <param name="key">The key at which to upsert the value</param>
/// <param name="value">The value to upsert</param>
/// <param name="jsonTypeInfo">The json type info for the value type</param>
/// <returns>True</returns>
/// <remarks>
/// This method overload which uses ReadOnlySpan{char} will not allocate a new string for the key if it already exists, instead it will directly replace the value
/// </remarks>
public bool Upsert<TValue>(ReadOnlySpan<char> key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo) {
WaitIfSerializing(); // block if the database is currently serializing
byte[] utf8value = JsonSerializer.SerializeToUtf8Bytes(value, jsonTypeInfo);
Lookup[key] = utf8value;
OnChangeInternal(ArrowDbChangeEventArgs.Upsert); // trigger change event
return UpsertCore<ReadOnlySpan<char>, TValue, ReadOnlySpanAccessor>(key, value, jsonTypeInfo, default);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
private bool UpsertCore<TKey, TValue, TAccessor>(TKey key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo, TAccessor accessor)
where TKey : allows ref struct
where TAccessor : IDictionaryAccessor<TKey>, allows ref struct {
WaitIfSerializing(); // Block if serializing
byte[] utf8Value = JsonSerializer.SerializeToUtf8Bytes(value, jsonTypeInfo);
accessor.Upsert(this, key, utf8Value);
OnChangeInternal(ArrowDbChangeEventArgs.Upsert); // Trigger change event
return true;
}

Expand All @@ -31,11 +54,37 @@ public bool Upsert<TValue>(ReadOnlySpan<char> key, TValue value, JsonTypeInfo<TV
/// <returns>True if the value was upserted, false otherwise</returns>
/// <remarks>
/// <para>
/// <paramref name="updateCondition"/> can be used to resolve write conflicts, the update will be rejected only if all of the following conditions are met:
/// <paramref name="updateCondition"/> can be used to resolve write conflicts, the update will be rejected only if both conditions are met:
/// </para>
/// <para>1. A value for the specified key exists and successfully deserialized to <typeparamref name="TValue"/></para>
/// <para>2. <paramref name="updateCondition"/> on the reference value returns false</para>
/// </remarks>
public bool Upsert<TValue>(string key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo, Func<TValue, bool> updateCondition) {
if (TryGetValue(key, jsonTypeInfo, out TValue existingReference) &&
!updateCondition(existingReference)) {
return false;
}
return Upsert(key, value, jsonTypeInfo);
}

/// <summary>
/// Tries to upsert the specified key with the specified value into the database
/// </summary>
/// <typeparam name="TValue">The type of the value to upsert</typeparam>
/// <param name="key">The key at which to upsert the value</param>
/// <param name="value">The value to upsert</param>
/// <param name="jsonTypeInfo">The json type info for the value type</param>
/// <param name="updateCondition">A conditional check that determines whether this update should be performed</param>
/// <returns>True if the value was upserted, false otherwise</returns>
/// <remarks>
/// <para>
/// <paramref name="updateCondition"/> can be used to resolve write conflicts, the update will be rejected only if both conditions are met:
/// </para>
/// <para>1. A value for the specified key exists and successfully deserialized to <typeparamref name="TValue"/></para>
/// <para>2. <paramref name="updateCondition"/> on the reference value returns false</para>
/// <para>
/// This method overload which uses ReadOnlySpan{char} will not allocate a new string for the key if it already exists, instead it will directly replace the value
/// </para>
/// <para>1. <paramref name="updateCondition"/> is not null</para>
/// <para>2. A value for the specified key exists and successfully deserialized to <typeparamref name="TValue"/></para>
/// <para>3. <paramref name="updateCondition"/> on the reference value returns false</para>
/// </remarks>
public bool Upsert<TValue>(ReadOnlySpan<char> key, TValue value, JsonTypeInfo<TValue> jsonTypeInfo, Func<TValue, bool> updateCondition) {
if (TryGetValue(key, jsonTypeInfo, out TValue existingReference) &&
Expand Down
2 changes: 1 addition & 1 deletion src/ArrowDbCore/ArrowDbCore.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<Version>1.2.0.0</Version>
<Version>1.3.0.0</Version>
<IsAotCompatible>true</IsAotCompatible>
</PropertyGroup>

Expand Down
34 changes: 34 additions & 0 deletions tests/ArrowDbCore.Tests.Unit/GetOrAddAsync.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
namespace ArrowDbCore.Tests.Unit;

public class GetOrAddAsync {
#pragma warning disable xUnit1031 // Do not use blocking task operations in test method
// this is required here for testing purposes
[Fact]
public async Task GetOrAddAsync_ReturnsSynchronously_WhenExists() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
db.Upsert("1", 1, JContext.Default.Int32); // add before
var task = db.GetOrAddAsync("1", JContext.Default.Int32, async _ => {
await Task.Delay(1000);
return 1;
});
Assert.True(task.IsCompletedSuccessfully);

Assert.Equal(1, task.GetAwaiter().GetResult());

}
#pragma warning restore xUnit1031 // Do not use blocking task operations in test method

[Fact]
public async Task GetOrAddAsync_ReturnsAsynchronously_WhenNotExists() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
// doesn't exist
var task = db.GetOrAddAsync("1", JContext.Default.Int32, async _ => {
await Task.Delay(1000);
return 1;
});
Assert.False(task.IsCompletedSuccessfully);
Assert.Equal(1, await task);
}
}
56 changes: 56 additions & 0 deletions tests/ArrowDbCore.Tests.Unit/Upserts.Spans.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
namespace ArrowDbCore.Tests.Unit;

public class Upserts_Spans {
[Fact]
public async Task Upsert_Span_When_Not_Found_Inserts() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
ReadOnlySpan<char> key = "1";
db.Upsert(key, 1, JContext.Default.Int32);
Assert.True(db.ContainsKey(key));
Assert.Equal(1, db.Count);
}

[Fact]
public async Task Upsert_Span_When_Found_Overwrites() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
ReadOnlySpan<char> key = "1";
db.Upsert(key, 1, JContext.Default.Int32);
Assert.True(db.ContainsKey(key));
Assert.Equal(1, db.Count);
db.Upsert(key, 2, JContext.Default.Int32);
Assert.True(db.TryGetValue(key, JContext.Default.Int32, out var value));
Assert.Equal(2, value);
}

[Fact]
public async Task Conditional_Update_When_Not_Found_Inserts() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
ReadOnlySpan<char> key = "1";
Assert.True(db.Upsert(key, 1, JContext.Default.Int32, reference => reference == 3));
}

[Fact]
public async Task Conditional_Update_When_Found_And_Valid_Updates() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
ReadOnlySpan<char> key = "1";
db.Upsert(key, 1, JContext.Default.Int32);
Assert.True(db.Upsert(key, 3, JContext.Default.Int32, reference => reference == 1));
Assert.True(db.TryGetValue(key, JContext.Default.Int32, out var value));
Assert.Equal(3, value);
}

[Fact]
public async Task Conditional_Update_When_Found_And_Invalid_Returns_False() {
var db = await ArrowDb.CreateInMemory();
Assert.Equal(0, db.Count);
ReadOnlySpan<char> key = "1";
db.Upsert(key, 1, JContext.Default.Int32);
Assert.False(db.Upsert(key, 3, JContext.Default.Int32, reference => reference == 3));
Assert.True(db.TryGetValue(key, JContext.Default.Int32, out var value));
Assert.Equal(1, value);
}
}
Loading