Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 134 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ without Tauri:
| -------------------- | --------------- | ---------------- | ------------------- |
| SELECT (multiple) | `fetchAll()` | Read pool | Multiple concurrent |
| SELECT (single) | `fetchOne()` | Read pool | Multiple concurrent |
| SELECT (paginated) | `fetchPage()` | Read pool | Multiple concurrent |
| INSERT/UPDATE/DELETE | `execute()` | Write connection | Serialized |
| DDL (CREATE, etc.) | `execute()` | Write connection | Serialized |

Expand Down Expand Up @@ -268,6 +269,64 @@ if (user) {
}
```

### Pagination

When working with large result sets, loading all rows at once can cause
performance degradation and excessive memory usage on both the Rust and
TypeScript sides. The plugin provides built-in pagination to fetch data in
fixed-size pages, keeping memory usage bounded and queries fast regardless
of total row count.

#### Why Keyset Pagination

The plugin uses keyset (cursor-based) pagination rather than traditional
OFFSET-based pagination. With OFFSET, the database must scan and discard
all skipped rows on every page request, making deeper pages progressively
slower. Keyset pagination uses indexed column values from the last row of
the current page to seek directly to the next page, keeping query time
constant no matter how far you paginate.

```typescript
import type { KeysetColumn } from '@silvermine/tauri-plugin-sqlite';

type Post = { id: number; title: string; category: string; score: number };

const keyset: KeysetColumn[] = [
{ name: 'category', direction: 'asc' },
{ name: 'score', direction: 'desc' },
{ name: 'id', direction: 'asc' },
];

// First page
const page = await db.fetchPage<Post>(
'SELECT id, title, category, score FROM posts',
[],
keyset,
25,
);

// Next page (forward) — pass the cursor from the previous page
if (page.nextCursor) {
const nextPage = await db.fetchPage<Post>(
'SELECT id, title, category, score FROM posts',
[],
keyset,
25,
).after(page.nextCursor);

// Previous page (backward) — rows are returned in original sort order
const prevPage = await db.fetchPage<Post>(
'SELECT id, title, category, score FROM posts',
[],
keyset,
25,
).before(page.nextCursor);
}
```

The base query must not contain `ORDER BY` or `LIMIT` clauses — the builder
appends these automatically based on the keyset definition.

### Transactions

For most cases, use `executeTransaction()` to run multiple statements atomically:
Expand Down Expand Up @@ -338,8 +397,8 @@ Each attached database gets a schema name that acts as a namespace for its
tables.

**Builder Pattern:** All query methods (`execute`, `executeTransaction`,
`fetchAll`, `fetchOne`) return builders that support `.attach()` for
cross-database operations.
`fetchAll`, `fetchOne`, `fetchPage`) return builders that support `.attach()`
for cross-database operations.

```typescript
// Join data from multiple databases
Expand Down Expand Up @@ -509,6 +568,7 @@ await db.remove(); // Close and DELETE database file(s) - irreversible
| `beginInterruptibleTransaction(statements)` | Begin interruptible transaction, returns `InterruptibleTransaction` |
| `fetchAll<T>(query, values?)` | Execute SELECT, return all rows |
| `fetchOne<T>(query, values?)` | Execute SELECT, return single row or `undefined` |
| `fetchPage<T>(query, values, keyset, pageSize)` | Keyset pagination, returns `FetchPageBuilder` |
| `close()` | Close connection, returns `true` if was loaded |
| `remove()` | Close and delete database file(s), returns `true` if was loaded |
| `observe(tables, config?)` | Enable change observation for tables |
Expand All @@ -517,12 +577,15 @@ await db.remove(); // Close and DELETE database file(s) - irreversible

### Builder Methods

All query methods (`execute`, `executeTransaction`, `fetchAll`, `fetchOne`)
return builders that are directly awaitable and support method chaining:
All query methods (`execute`, `executeTransaction`, `fetchAll`, `fetchOne`,
`fetchPage`) return builders that are directly awaitable and support method
chaining:

| Method | Description |
| ------ | ----------- |
| `attach(specs)` | Attach databases for cross-database queries, returns `this` |
| `after(cursor)` | Set cursor for forward pagination (`FetchPageBuilder` only), returns `this` |
| `before(cursor)` | Set cursor for backward pagination (`FetchPageBuilder` only), returns `this` |
| `await builder` | Execute the query (builders implement `PromiseLike`) |

### InterruptibleTransaction Methods
Expand Down Expand Up @@ -569,6 +632,19 @@ interface ObserverConfig {
captureValues?: boolean; // default: true
}

type SortDirection = 'asc' | 'desc';

interface KeysetColumn {
name: string; // Column name in the query result set
direction: SortDirection;
}

interface KeysetPage<T = Record<string, SqlValue>> {
rows: T[];
nextCursor: SqlValue[] | null; // Cursor to continue pagination, null when no more pages
hasMore: boolean;
}

type ChangeOperation = 'insert' | 'update' | 'delete';

type ColumnValue =
Expand Down Expand Up @@ -647,6 +723,47 @@ if let Some(user_data) = user {
}
```

### Pagination (Rust)

See [Pagination](#pagination) above for background on why the plugin uses
keyset pagination. The Rust API works the same way via `fetch_page`:

```rust
use sqlx_sqlite_toolkit::pagination::KeysetColumn;

let keyset = vec![
KeysetColumn::asc("category"),
KeysetColumn::desc("score"),
KeysetColumn::asc("id"),
];

// First page
let page = db.fetch_page(
"SELECT id, title, category, score FROM posts".into(),
vec![],
keyset.clone(),
25,
).await?;

// Next page (forward)
if let Some(cursor) = page.next_cursor {
let next = db.fetch_page(
"SELECT id, title, category, score FROM posts".into(),
vec![],
keyset.clone(),
25,
).after(cursor.clone()).await?;

// Previous page (backward) — rows returned in original sort order
let prev = db.fetch_page(
"SELECT id, title, category, score FROM posts".into(),
vec![],
keyset,
25,
).before(cursor).await?;
}
```

### Simple Transactions

Use `execute_transaction()` for atomic execution of multiple statements:
Expand Down Expand Up @@ -771,6 +888,7 @@ db.remove().await?; // Close and DELETE database file(s)
| `begin_interruptible_transaction()` | Begin interruptible transaction (builder) |
| `fetch_all(query, values)` | Fetch all rows |
| `fetch_one(query, values)` | Fetch single row |
| `fetch_page(query, values, keyset, page_size)` | Keyset pagination (builder, supports `.after()`, `.before()`, `.attach()`) |
| `close()` | Close connection |
| `remove()` | Close and delete database file(s) |

Expand Down Expand Up @@ -818,6 +936,18 @@ fn main() {
}
```

## Examples

Working Tauri demo apps are in the [`examples/`](examples) directory:

* **[`observer-demo`](examples/observer-demo)** — Real-time change
notifications with live streaming of inserts, updates, and deletes
* **[`pagination-demo`](examples/pagination-demo)** — Keyset pagination
with a virtualized list and performance metrics

See the [toolkit crate README](crates/sqlx-sqlite-toolkit/README.md#examples)
for setup instructions.

## Development

This project follows
Expand Down
2 changes: 1 addition & 1 deletion api-iife.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions crates/sqlx-sqlite-toolkit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,24 @@ All errors provide an `error_code()` method returning a machine-readable string:
| `INVALID_COLUMN_NAME` | Keyset column name contains invalid characters |
| `CONFLICTING_CURSORS` | Both `after` and `before` cursors provided |

## Examples

Working Tauri apps demonstrating the toolkit's features are in the
[`examples/`](../../examples) directory:

| App | Description |
| --- | ----------- |
| [`observer-demo`](../../examples/observer-demo) | Real-time change notifications using the observer subsystem — subscribe to table changes and see inserts, updates, and deletes streamed live |
| [`pagination-demo`](../../examples/pagination-demo) | Keyset pagination with a virtualized list — browse large datasets page-by-page with forward/backward navigation and performance metrics |

Both are Vue 3 + Tauri apps. To run one:

```bash
cd examples/observer-demo # or pagination-demo
npm install
cargo tauri dev
```

## Development

```bash
Expand Down
12 changes: 12 additions & 0 deletions examples/pagination-demo/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>SQLite Pagination Demo</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.ts"></script>
</body>
</html>
Loading