making match_documents compatible with n8n structure

[virtuman]

If this makes sense, and purely for compatibility, do you think it would be a good idea to make match_documents in the format as expected by n8n? this would allow to re-use same function for n8n workflows. Minor difference in implementation, but makes it usable with n8n out of the box

Minor differences are arguments in the function:
n8n-compatible:

create function match_documents (
  query_embedding vector(1536),
  match_count int default null,
  filter jsonb DEFAULT '{}'
) returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)

current mind-matrix: https://github.com/khwerhahn/MindMatrix/blob/main/sql/setup.sql#L70-L81

CREATE OR REPLACE FUNCTION public.match_documents(
    query_embedding vector(1536),
    search_vault_id TEXT,
    match_count INT
)
RETURNS TABLE (
    id BIGINT,
    obsidian_id TEXT,
    content TEXT,
    metadata JSONB,
    similarity FLOAT
)

Documentation reference for function structure:
https://supabase.com/docs/guides/ai/langchain?database-method=sql

[khwerhahn]

That would make sense. Feel free to implement that change. Thanks for the suggestion!

[virtuman]

i notice that obsidian Id is already in metadata field, which is the only pre-requirement from what i can tell.

but the confusing part to me - I can't really find where it's using the stored vectors. there's a function semanticSearch in the code, but is it called from anywhere in the obsidian UI or anywhere in the code?

[khwerhahn]

Obsidian_id and vault_id both came from thinking about sync issues, especially across multiple devices. I faced the problem that when you rename an already processed file, Obsidian essentially loses track as it doesn't (to my knowledge) give out uuids for files. I hope that answers your question.

private async handleFileRename(file: TFile, oldPath: string) {
		try {
			if (!this.supabaseService) return;
			// First update the file status record to use the new path.
			await this.supabaseService.updateFilePath(oldPath, file.path);

			// Then, if there are document chunks for the old path, update them as well.
			const chunks = await this.supabaseService.getDocumentChunks(oldPath);
			if (chunks.length > 0) {
				const updatedChunks = chunks.map(chunk => ({
					...chunk,
					metadata: {
						...chunk.metadata,
						obsidianId: file.path,
						path: file.path
					}
				}));
				await this.supabaseService.deleteDocumentChunks(oldPath);
				await this.supabaseService.upsertChunks(updatedChunks);
			}
			if (this.settings.enableNotifications) {
				new Notice(`Updated database entries for renamed file: ${file.name}`);
			}
		} catch (error) {
			this.errorHandler?.handleError(error, { context: 'handleFileRename', metadata: { filePath: file.path, oldPath } });
			if (this.settings.enableNotifications) {
				new Notice(`Failed to update database for renamed file: ${file.name}`);
			}
		}
	}