diff --git a/.opencode/plans/bookfusion-client-tests.md b/.opencode/plans/bookfusion-client-tests.md new file mode 100644 index 0000000..70c3e43 --- /dev/null +++ b/.opencode/plans/bookfusion-client-tests.md @@ -0,0 +1,136 @@ +# BookFusion Client Unit Tests Plan + +## Overview + +Add comprehensive unit tests for `src/api/bookfusion_client.py` — the only file in the BookFusion integration with zero test coverage. The existing `test_bookfusion_routes.py` only tests blueprint routes with mocked clients. + +## Test File: `tests/test_bookfusion_client.py` + +### Test Classes (75+ tests) + +#### 1. `TestBuildMultipart` (12 tests) +Tests the Qt-compatible multipart/form-data builder — the most fragile code in the entire integration. + +- `test_single_text_field` — basic text field with correct boundary +- `test_single_file_field` — file field with filename +- `test_multiple_text_fields` — multiple text fields separated correctly +- `test_mixed_text_and_file_fields` — text + file in same body +- `test_no_content_type_on_text_fields` — Qt format omits Content-Type +- `test_no_content_type_on_file_fields` — file parts also get no Content-Type +- `test_crlf_line_endings` — must use \r\n not \n +- `test_closing_boundary` — proper `--boundary--` terminator +- `test_binary_file_data` — all 256 byte values pass through +- `test_empty_text_value` — empty string field handled +- `test_unicode_text_value` — UTF-8 encoding preserved +- `test_deduplication_of_boundaries` — exactly N boundaries for N fields + +#### 2. `TestCalibreDigest` (7 tests) +Tests SHA-256 digest matching the Calibre plugin's `calculate_digest`. + +- `test_known_digest` — sha256(len + null + content) +- `test_empty_bytes` — edge case for zero-length file +- `test_large_file_chunks` — 100KB file (>64k chunk boundary) +- `test_exactly_64k_boundary` — exactly 65536 bytes +- `test_binary_data` — null bytes and high bytes +- `test_different_data_different_digests` — uniqueness +- `test_same_data_same_digest` — determinism + +#### 3. `TestCalibreAuthHeader` (3 tests) +- `test_format` — `Basic base64(key:)` +- `test_empty_key` — `Basic base64(:)` +- `test_key_with_special_chars` — URL-safe chars in key + +#### 4. `TestCalibreHeaders` (3 tests) +- `test_required_headers` — User-Agent, Authorization, Accept +- `test_extra_headers_merged` — Content-Type passthrough +- `test_extra_does_not_override_auth` — auth header always from key + +#### 5. `TestParseFrontmatterTitle` (8 tests) +- `test_simple_title`, `test_quoted_title`, `test_single_quoted_title` +- `test_multiline_frontmatter`, `test_none_input`, `test_empty_input` +- `test_no_title_field`, `test_title_with_extra_whitespace` + +#### 6. `TestParseFrontmatter` (6 tests) +- `test_all_fields` — title, author, tags, series +- `test_none_input`, `test_empty_input` +- `test_authors_plural` — "authors:" vs "author:" +- `test_missing_fields`, `test_quoted_values` + +#### 7. `TestParseHighlightDate` (6 tests) +- `test_valid_date` — standard format +- `test_date_with_extra_content` — embedded in highlight markdown +- `test_no_date`, `test_invalid_date_format`, `test_empty_string` +- `test_extra_whitespace` + +#### 8. `TestParseHighlightQuote` (6 tests) +- `test_single_quote`, `test_multiple_quote_lines` +- `test_mixed_content` — quote among other markdown +- `test_no_quote`, `test_empty_quote_line`, `test_empty_string` + +#### 9. `TestBookFusionClientConfig` (6 tests) +- `test_is_configured_false_when_disabled` — BOOKFUSION_ENABLED=false +- `test_is_configured_true_with_highlights_key` +- `test_is_configured_true_with_upload_key` +- `test_is_configured_true_with_both_keys` +- `test_is_configured_ignores_enabled_false_with_keys` +- `test_is_configured_no_env_vars` + +#### 10. `TestBookFusionClientConnection` (6 tests) +- `test_check_connection_success` — HTTP 200 +- `test_check_connection_http_error` — HTTP 401 +- `test_check_connection_no_key` — missing key +- `test_check_connection_network_error` — ConnectionError +- `test_check_upload_connection_success` +- `test_check_upload_connection_no_key` + +#### 11. `TestBookFusionClientUpload` (7 tests) +Full 3-step upload flow coverage: + +- `test_upload_book_no_api_key` — early return None +- `test_upload_book_already_exists` — check_exists returns data, skips upload +- `test_upload_book_init_fails` — POST /uploads/init returns 500 +- `test_upload_book_init_success_no_s3_url` — init returns null URL +- `test_upload_book_full_flow` — init → S3 → finalize, all 200s +- `test_upload_book_s3_upload_fails` — S3 returns 500 +- `test_upload_book_finalize_fails` — finalize returns 400 + +#### 12. `TestBookFusionClientLibrary` (5 tests) +- `test_fetch_library_no_api_key` — returns [] +- `test_fetch_library_single_page` — <100 books, single request +- `test_fetch_library_pagination` — 101 books, 2 requests +- `test_fetch_library_http_error` — returns [] +- `test_fetch_library_network_error` — returns [] + +#### 13. `TestBookFusionClientHighlights` (8 tests) +- `test_fetch_highlights_no_key_raises` — ValueError +- `test_fetch_highlights_success` — proper response structure +- `test_fetch_highlights_http_error` — raises HTTPError +- `test_sync_all_highlights_basic` — single page, saves highlights + books +- `test_sync_all_highlights_pagination` — multi-page sync +- `test_sync_all_highlights_pagination_loop_detection` — stops on same cursor +- `test_sync_all_highlights_skips_non_book_pages` — filters by type +- `test_sync_all_highlights_library_fetch_failure` — sync succeeds even if library fetch fails + +#### 14. `TestBookFusionClientCheckExists` (4 tests) +- `test_no_api_key_returns_none` +- `test_existing_book` — HTTP 200 with data +- `test_nonexistent_book` — HTTP 404 +- `test_network_error_returns_none` + +## Testing Approach + +- **Real client instances**: Uses `BookFusionClient()` directly, not Mock, so actual logic is exercised +- **Mocked network**: `requests.Session` patched at class level +- **Env var isolation**: `patch.dict(os.environ, ...)` for each test +- **Follows project patterns**: Matches `test_grimmory_client.py` and `test_storyteller_api_client.py` conventions + +## What This Catches + +1. Multipart encoding regressions that would break all book uploads +2. Digest calculation errors causing false "already exists" matches +3. Auth header format changes +4. Frontmatter parsing edge cases (quoted titles, missing fields) +5. Highlight date/quote extraction failures +6. Pagination loops in sync +7. All error paths in the 3-step upload flow +8. Network failure resilience diff --git a/alembic/versions/r5s6t7u8v9w0_remove_bookfusion_matched_abs_id.py b/alembic/versions/r5s6t7u8v9w0_remove_bookfusion_matched_abs_id.py new file mode 100644 index 0000000..715df72 --- /dev/null +++ b/alembic/versions/r5s6t7u8v9w0_remove_bookfusion_matched_abs_id.py @@ -0,0 +1,28 @@ +"""Remove matched_abs_id columns from BookFusion tables. + +Revision ID: r5s6t7u8v9w0 +Revises: p6q7r8s9t0u1 +Create Date: 2026-04-05 +""" + +import sqlalchemy as sa + +from alembic import op + +revision = "r5s6t7u8v9w0" +down_revision = "p6q7r8s9t0u1" +branch_labels = None +depends_on = None + + +def upgrade(): + with op.batch_alter_table("bookfusion_books") as batch_op: + batch_op.drop_column("matched_abs_id") + + with op.batch_alter_table("bookfusion_highlights") as batch_op: + batch_op.drop_column("matched_abs_id") + + +def downgrade(): + op.add_column("bookfusion_books", sa.Column("matched_abs_id", sa.String(255), nullable=True)) + op.add_column("bookfusion_highlights", sa.Column("matched_abs_id", sa.String(255), nullable=True)) diff --git a/alembic/versions/t9u0v1w2x3y4_merge_detected_books_head.py b/alembic/versions/t9u0v1w2x3y4_merge_detected_books_head.py new file mode 100644 index 0000000..e462ff3 --- /dev/null +++ b/alembic/versions/t9u0v1w2x3y4_merge_detected_books_head.py @@ -0,0 +1,21 @@ +"""merge grimmory rename and bookfusion cleanup heads + +Revision ID: t9u0v1w2x3y4 +Revises: 5308a8e2c930, r5s6t7u8v9w0 +Create Date: 2026-04-25 +""" + +from collections.abc import Sequence + +revision: str = "t9u0v1w2x3y4" +down_revision: str | Sequence[str] | None = ("5308a8e2c930", "r5s6t7u8v9w0") +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def upgrade() -> None: + """Merge parallel heads into a single linear history.""" + + +def downgrade() -> None: + """Unmerge the parallel heads.""" diff --git a/pyproject.toml b/pyproject.toml index 0ba916f..e3a7970 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,31 @@ [project] name = "pagekeeper" requires-python = ">=3.11" +dependencies = [ + "requests==2.32.5", + "schedule==1.2.2", + "faster-whisper==1.2.1", + "deepgram-sdk==6.0.1", + "ebooklib==0.20", + "epubcfi==0.0.6", + "beautifulsoup4==4.14.3", + "rapidfuzz==3.14.3", + "fuzzysearch==0.8.1", + "tqdm==4.67.3", + "gTTS==2.5.4", + "flask==3.1.3", + "nh3==0.3.1", + "lxml==5.4.0", + "defusedxml==0.7.1", + "dependency-injector==4.48.3", + "sqlalchemy==2.0.48", + "alembic==1.18.4", + "python-socketio[client]==5.16.1", + "h11==0.16.0", + "mkdocs-material==9.7.4", + "mistune==3.2.0", + "pytest-asyncio==1.3.0", +] [tool.pytest.ini_options] markers = ["docker: tests requiring Docker (epubcfi, ffmpeg)"] diff --git a/pyrightconfig.json b/pyrightconfig.json new file mode 100644 index 0000000..1421e35 --- /dev/null +++ b/pyrightconfig.json @@ -0,0 +1,25 @@ +{ + "include": [ + "src", + "tests" + ], + "typeCheckingMode": "basic", + "reportMissingImports": "none", + "reportMissingModuleSource": "none", + "reportAttributeAccessIssue": "none", + "reportArgumentType": "none", + "reportOptionalMemberAccess": "none", + "reportOptionalOperand": "none", + "reportOptionalSubscript": "none", + "reportAssignmentType": "none", + "reportCallIssue": "none", + "reportIncompatibleMethodOverride": "none", + "executionEnvironments": [ + { + "root": ".", + "extraPaths": [ + "." + ] + } + ] +} diff --git a/requirements.txt b/requirements.txt index bc8865a..7d0dec5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -19,3 +19,5 @@ alembic==1.18.4 python-socketio[client]==5.16.1 h11==0.16.0 mkdocs-material==9.7.4 +mistune==3.2.0 +pytest-asyncio==1.3.0 diff --git a/src/api/hardcover_client.py b/src/api/hardcover_client.py index a707504..d4d09ab 100644 --- a/src/api/hardcover_client.py +++ b/src/api/hardcover_client.py @@ -19,12 +19,13 @@ import requests +from src.api.http_client_base import JsonHttpClientBase from src.utils.string_utils import calculate_similarity, clean_book_title logger = logging.getLogger(__name__) -class HardcoverClient: +class HardcoverClient(JsonHttpClientBase): def __init__(self): self.api_url = "https://api.hardcover.app/v1/graphql" self.user_id = None @@ -88,33 +89,26 @@ def query(self, query: str, variables: dict | None = None) -> dict | None: self._rate_limit() try: - r = requests.post( + payload = {"query": query, "variables": variables or {}} + + def _mark_retry(_attempt, _response): + with self._rate_lock: + self._last_request_time = time.monotonic() + + r = self.post_json_with_retries( self.api_url, - json={"query": query, "variables": variables or {}}, + json_body=payload, headers=self.headers, timeout=20, + max_retries=3, + retry_statuses={429}, + backoff_seconds=5, + retry_label="Hardcover request", + on_retry=_mark_retry, ) - # Handle rate limiting (429) with exponential backoff - max_retries = 3 - backoff = 5 - attempt = 0 - while r.status_code == 429 and attempt < max_retries: - attempt += 1 - logger.warning(f"Hardcover rate limit hit (429), retry {attempt}/{max_retries} after {backoff}s") - time.sleep(backoff) - with self._rate_lock: - self._last_request_time = time.monotonic() - r = requests.post( - self.api_url, - json={"query": query, "variables": variables or {}}, - headers=self.headers, - timeout=20, - ) - backoff *= 2 - if r.status_code == 429: - logger.error(f"Hardcover rate limit persisted after {max_retries} retries, giving up") + logger.error("Hardcover rate limit persisted after 3 retries, giving up") return None if r.status_code == 200: diff --git a/src/api/hardcover_routes.py b/src/api/hardcover_routes.py index cd290bc..1126f72 100644 --- a/src/api/hardcover_routes.py +++ b/src/api/hardcover_routes.py @@ -1,30 +1,25 @@ +# pyright: reportMissingImports=false # Hardcover Routes - Flask Blueprint for Hardcover API endpoints import ipaddress import logging import socket +from typing import Any, cast from urllib.parse import urlparse -from flask import Blueprint, flash, jsonify, redirect, request, url_for +from flask import Blueprint, current_app, flash, jsonify, redirect, request, url_for + +from src.utils.http import json_error logger = logging.getLogger(__name__) # Create Blueprint for Hardcover endpoints hardcover_bp = Blueprint("hardcover", __name__) -# Module-level references - set via init_hardcover_routes() -_database_service = None -_container = None - - -def init_hardcover_routes(database_service, container): - """Initialize Hardcover routes with required dependencies.""" - global _database_service, _container - _database_service = database_service - _container = container - def _get_dependencies(): - if _database_service is None or _container is None: + database_service = current_app.config.get("database_service") + container = current_app.config.get("container") + if database_service is None or container is None: logger.error("Hardcover routes not initialized") return ( None, @@ -34,7 +29,7 @@ def _get_dependencies(): 500, ), ) - return _database_service, _container, None + return cast(Any, database_service), cast(Any, container), None def _validate_custom_cover_url(raw_url): @@ -85,16 +80,18 @@ def api_hardcover_resolve(): database_service, container, error_response = _get_dependencies() if error_response: return error_response + database_service = cast(Any, database_service) + container = cast(Any, container) abs_id = request.args.get("abs_id", "").strip() manual_input = request.args.get("input", "").strip() if not abs_id: - return jsonify({"found": False, "message": "Missing abs_id parameter"}), 400 + return json_error("Missing abs_id parameter", 400, found=False, user_message="Missing abs_id parameter") hardcover_client = container.hardcover_client() if not hardcover_client.is_configured(): - return jsonify({"found": False, "message": "Hardcover not configured"}), 400 + return json_error("Hardcover not configured", 400, found=False, user_message="Hardcover not configured") book_data = None author = None @@ -113,7 +110,7 @@ def api_hardcover_resolve(): if not book_data: # No existing link (or fetch failed) - fall back to auto-match from ABS metadata if not book: - return jsonify({"found": False, "message": "Book not found"}), 404 + return json_error("Book not found", 404, found=False, user_message="Book not found") # Get metadata from ABS (if available) item = container.abs_client().get_item_details(abs_id) @@ -188,6 +185,8 @@ def link_hardcover(abs_id): database_service, container, error_response = _get_dependencies() if error_response: return error_response + database_service = cast(Any, database_service) + container = cast(Any, container) # Check if JSON request (new flow) or form data (legacy flow) if request.is_json: @@ -210,18 +209,20 @@ def link_hardcover(abs_id): # Determine cover URL: use provided cached_image, or preserve existing cover_url = cached_image book = database_service.get_book_by_ref(abs_id) + if not book: + return jsonify({"error": "Book not found"}), 404 if not cover_url and book: existing = database_service.get_hardcover_details(book.id) if existing and existing.hardcover_cover_url: cover_url = existing.hardcover_cover_url hardcover_details = HardcoverDetails( abs_id=abs_id, - book_id=book.id if book else None, + book_id=book.id, hardcover_book_id=str(book_id), - hardcover_slug=slug, - hardcover_edition_id=str(edition_id) if edition_id else None, - hardcover_pages=hardcover_pages, - hardcover_audio_seconds=audio_seconds if audio_seconds else None, + hardcover_slug=slug or "", + hardcover_edition_id=str(edition_id) if edition_id else "", + hardcover_pages=int(hardcover_pages) if hardcover_pages is not None else 0, + hardcover_audio_seconds=int(audio_seconds) if audio_seconds is not None else 0, hardcover_cover_url=cover_url, matched_by="manual", ) @@ -261,13 +262,16 @@ def link_hardcover(abs_id): try: book = database_service.get_book_by_ref(abs_id) + if not book: + flash("Book not found", "error") + return redirect(url_for("dashboard.index")) hardcover_details = HardcoverDetails( abs_id=abs_id, - book_id=book.id if book else None, - hardcover_book_id=book_data["book_id"], - hardcover_slug=book_data.get("slug"), - hardcover_edition_id=book_data.get("edition_id"), - hardcover_pages=book_data.get("pages"), + book_id=book.id, + hardcover_book_id=str(book_data["book_id"]), + hardcover_slug=book_data.get("slug") or "", + hardcover_edition_id=str(book_data.get("edition_id") or ""), + hardcover_pages=int(book_data.get("pages") or 0), matched_by="manual", ) @@ -303,6 +307,7 @@ def api_cover_search(): database_service, container, error_response = _get_dependencies() if error_response: return error_response + container = cast(Any, container) query = request.args.get("query", "").strip() if not query: @@ -324,6 +329,7 @@ def set_book_cover(abs_id): database_service, container, error_response = _get_dependencies() if error_response: return error_response + database_service = cast(Any, database_service) data = request.get_json() if not data: @@ -359,8 +365,8 @@ def set_book_cover(abs_id): details = HardcoverDetails( abs_id=abs_id, book_id=book.id, - hardcover_book_id=str(book_id) if book_id else None, - hardcover_slug=slug, + hardcover_book_id=str(book_id) if book_id else "", + hardcover_slug=slug or "", hardcover_cover_url=cover_url, matched_by="cover_picker", ) @@ -399,6 +405,7 @@ def delete_book_cover(abs_id): database_service, container, error_response = _get_dependencies() if error_response: return error_response + database_service = cast(Any, database_service) book = database_service.get_book_by_ref(abs_id) if not book: diff --git a/src/api/http_client_base.py b/src/api/http_client_base.py new file mode 100644 index 0000000..cbafdb3 --- /dev/null +++ b/src/api/http_client_base.py @@ -0,0 +1,50 @@ +# pyright: reportMissingImports=false, reportMissingModuleSource=false + +import logging +import time + +import requests + +logger = logging.getLogger(__name__) + + +class JsonHttpClientBase: + """Small helper for JSON POST calls with retry/backoff handling.""" + + def post_json_with_retries( + self, + url, + *, + json_body, + headers, + timeout=20, + max_retries=0, + retry_statuses=None, + backoff_seconds=1, + request_func=None, + retry_label="request", + on_retry=None, + ): + retry_statuses = set(retry_statuses or []) + request_func = request_func or requests.post + response = request_func(url, json=json_body, headers=headers, timeout=timeout) + attempt = 0 + backoff = backoff_seconds + + while response.status_code in retry_statuses and attempt < max_retries: + attempt += 1 + logger.warning( + "%s returned %s, retry %s/%s after %ss", + retry_label, + response.status_code, + attempt, + max_retries, + backoff, + ) + time.sleep(backoff) + if on_retry: + on_retry(attempt, response) + response = request_func(url, json=json_body, headers=headers, timeout=timeout) + backoff *= 2 + + return response diff --git a/src/app_runtime.py b/src/app_runtime.py new file mode 100644 index 0000000..a107489 --- /dev/null +++ b/src/app_runtime.py @@ -0,0 +1,329 @@ +# pyright: reportMissingImports=false, reportMissingModuleSource=false + +import logging +import os +import secrets +import sys +import threading +import time +from pathlib import Path + +import schedule +from flask import Flask + +from src.api.kosync_server import kosync_sync_bp +from src.utils.runtime_config import get_bool, get_float, get_str +from src.version import get_update_status + +logger = logging.getLogger(__name__) + + +def reconfigure_logging(): + """Update root logger level from LOG_LEVEL.""" + try: + new_level_str = get_str("LOG_LEVEL", "INFO").upper() + new_level = getattr(logging, new_level_str, logging.INFO) + logging.getLogger().setLevel(new_level) + logger.info("Logging level updated to %s", new_level_str) + except Exception as exc: + logger.warning("Failed to reconfigure logging: %s", exc) + + +def reconcile_socket_listener(app): + """Start, stop, or restart the ABS socket listener to match current config.""" + from src.services.abs_socket_listener import ABSSocketListener + + instant_sync = get_bool("INSTANT_SYNC_ENABLED", True) + socket_enabled = get_bool("ABS_SOCKET_ENABLED", True) + abs_server = get_str("ABS_SERVER", "") + abs_key = get_str("ABS_KEY", "") + should_run = instant_sync and socket_enabled and abs_server and abs_key + + current: ABSSocketListener | None = app.config.get("abs_listener") + current_server = app.config.get("_abs_listener_server", "") + current_key = app.config.get("_abs_listener_key", "") + + if should_run and current is None: + listener = ABSSocketListener( + abs_server_url=abs_server, + abs_api_token=abs_key, + database_service=app.config["database_service"], + sync_manager=app.config["sync_manager"], + ) + threading.Thread(target=listener.start, daemon=True).start() + app.config["abs_listener"] = listener + app.config["_abs_listener_server"] = abs_server + app.config["_abs_listener_key"] = abs_key + logger.info("ABS Socket.IO listener started via hot-reload") + return + + if not should_run and current is not None: + current.stop() + app.config["abs_listener"] = None + app.config["_abs_listener_server"] = "" + app.config["_abs_listener_key"] = "" + logger.info("ABS Socket.IO listener stopped via hot-reload") + return + + if should_run and current is not None and (abs_server != current_server or abs_key != current_key): + current.stop() + listener = ABSSocketListener( + abs_server_url=abs_server, + abs_api_token=abs_key, + database_service=app.config["database_service"], + sync_manager=app.config["sync_manager"], + ) + threading.Thread(target=listener.start, daemon=True).start() + app.config["abs_listener"] = listener + app.config["_abs_listener_server"] = abs_server + app.config["_abs_listener_key"] = abs_key + logger.info("ABS Socket.IO listener restarted via hot-reload (credentials changed)") + + +def apply_settings(app): + """Hot-reload settings that do not propagate automatically via os.environ.""" + errors = [] + reconfigure_logging() + + try: + sync_mgr = app.config.get("sync_manager") + raw_period = get_str("SYNC_PERIOD_MINS", "5") + new_period = int(raw_period) + if new_period <= 0: + raise ValueError("SYNC_PERIOD_MINS must be an integer greater than 0") + + schedule.clear("sync_cycle") + if sync_mgr: + schedule.every(new_period).minutes.do(sync_mgr.sync_cycle).tag("sync_cycle") + logger.info("Sync schedule updated to every %s minutes", new_period) + except Exception as exc: + errors.append(f"sync reschedule failed: {exc}") + + try: + reconcile_socket_listener(app) + except Exception as exc: + errors.append(f"socket listener reconciliation failed: {exc}") + + app.config["ABS_COLLECTION_NAME"] = get_str("ABS_COLLECTION_NAME", "Synced with KOReader") + app.config["SUGGESTIONS_ENABLED"] = get_bool("SUGGESTIONS_ENABLED", False) + + try: + from src.utils.logging_utils import reconcile_telegram_logging + + reconcile_telegram_logging() + except Exception as exc: + errors.append(f"telegram logging reconciliation failed: {exc}") + + if errors: + error_message = "; ".join(errors) + logger.error("Failed to apply one or more settings: %s", error_message) + raise RuntimeError(error_message) + + return True + + +def wait_for_split_port_healthcheck(timeout=30): + """Wait for the split-port KoSync server to become available before the first sync.""" + kosync_port = get_str("KOSYNC_PORT", "") + if not kosync_port or kosync_port == "4477": + return + + import urllib.request + + url = f"http://127.0.0.1:{kosync_port}/healthcheck" + deadline = time.time() + timeout + while time.time() < deadline: + try: + urllib.request.urlopen(url, timeout=2) + logger.info("Split-port KoSync server ready on port %s", kosync_port) + return + except Exception: + time.sleep(1) + logger.warning("Split-port KoSync server not ready after %ss — proceeding anyway", timeout) + + +def run_sync_daemon(sync_manager, sync_period_mins): + """Run the schedule-based background sync loop.""" + try: + schedule.every(int(sync_period_mins)).minutes.do(sync_manager.sync_cycle).tag("sync_cycle") + schedule.every(1).minutes.do(sync_manager.check_pending_jobs).tag("check_jobs") + logger.info("Sync daemon started (period: %s minutes)", sync_period_mins) + + wait_for_split_port_healthcheck() + + try: + sync_manager.sync_cycle() + except Exception as exc: + logger.error("Initial sync cycle failed: %s", exc) + + while True: + try: + schedule.run_pending() + time.sleep(30) + except Exception as exc: + logger.error("Sync daemon error: %s", exc) + time.sleep(60) + except Exception as exc: + logger.error("Sync daemon crashed: %s", exc) + + +def start_sync_daemon_thread(sync_manager, sync_period_mins): + thread = threading.Thread(target=run_sync_daemon, args=(sync_manager, sync_period_mins), daemon=True) + thread.start() + logger.info("Sync daemon thread started") + return thread + + +def get_or_create_secret_key() -> str: + """Return a persistent random secret key, falling back to ephemeral.""" + data_dir = Path(get_str("DATA_DIR", "/data")) + key_file = data_dir / ".flask_secret_key" + try: + if key_file.exists(): + key = key_file.read_text().strip() + if key: + return key + key = secrets.token_hex(32) + data_dir.mkdir(parents=True, exist_ok=True) + key_file.write_text(key) + key_file.chmod(0o600) + return key + except Exception: + logger.warning("Could not persist Flask secret key — using ephemeral key") + return secrets.token_hex(32) + + +def log_security_warnings(): + """Log warnings for common security misconfigurations at startup.""" + kosync_user = get_str("KOSYNC_USER", "") + kosync_key = get_str("KOSYNC_KEY", "") + kosync_port = get_str("KOSYNC_PORT", "") + public_url = get_str("KOSYNC_PUBLIC_URL", "") + + if not kosync_user or not kosync_key: + logger.warning("SECURITY: KOSYNC_USER/KOSYNC_KEY not configured — sync endpoints will reject all requests") + elif len(kosync_key) < 8: + logger.warning("SECURITY: KOSYNC_KEY is shorter than 8 characters — consider using a stronger password") + + if not kosync_port or kosync_port == "4477": + logger.warning( + "SECURITY: Split-port mode not active — dashboard and sync API share port 4477. " + "Set KOSYNC_PORT to a different port before exposing sync to the internet." + ) + + if public_url: + from urllib.parse import urlsplit, urlunsplit + + parts = urlsplit(public_url) + safe_netloc = parts.hostname or "" + if parts.port: + safe_netloc = f"{safe_netloc}:{parts.port}" + safe_url = urlunsplit((parts.scheme, safe_netloc, parts.path or "", "", "")) + logger.info("KOSync public URL: %s", safe_url) + elif kosync_port and kosync_port != "4477": + logger.info("Tip: Set KOSYNC_PUBLIC_URL in settings if you expose KOSync through a reverse proxy") + + +def initialize_abs_listener(app, container, database_service, sync_manager): + """Start or disable the ABS listener based on runtime config.""" + instant_sync_enabled = get_bool("INSTANT_SYNC_ENABLED", True) + abs_socket_enabled = get_bool("ABS_SOCKET_ENABLED", True) + + if instant_sync_enabled and abs_socket_enabled and container.abs_client().is_configured(): + from src.services.abs_socket_listener import ABSSocketListener + + abs_listener = ABSSocketListener( + abs_server_url=get_str("ABS_SERVER", ""), + abs_api_token=get_str("ABS_KEY", ""), + database_service=database_service, + sync_manager=sync_manager, + ) + threading.Thread(target=abs_listener.start, daemon=True).start() + app.config["abs_listener"] = abs_listener + app.config["_abs_listener_server"] = get_str("ABS_SERVER", "") + app.config["_abs_listener_key"] = get_str("ABS_KEY", "") + logger.info("ABS Socket.IO listener started (instant sync enabled)") + return abs_listener + + app.config["abs_listener"] = None + app.config["_abs_listener_server"] = "" + app.config["_abs_listener_key"] = "" + if not instant_sync_enabled: + logger.info("ABS Socket.IO listener disabled (INSTANT_SYNC_ENABLED=false)") + elif not abs_socket_enabled: + logger.info("ABS Socket.IO listener disabled (ABS_SOCKET_ENABLED=false)") + return None + + +def start_client_poller(database_service, sync_manager, sync_clients_dict): + from src.services.client_poller import ClientPoller + + client_poller = ClientPoller( + database_service=database_service, + sync_manager=sync_manager, + sync_clients_dict=sync_clients_dict, + ) + thread = threading.Thread(target=client_poller.start, daemon=True) + thread.start() + return client_poller, thread + + +def log_ebook_source_configuration(container): + grimmory_configured = container.grimmory_client().is_configured() + books_volume_exists = container.books_dir().exists() + + if grimmory_configured: + logger.info("Grimmory integration enabled - ebooks sourced from API") + elif books_volume_exists: + logger.info("Ebooks directory mounted at %s", container.books_dir()) + else: + logger.info( + "NO EBOOK SOURCE CONFIGURED: Neither Grimmory integration nor /books volume is available. " + "New book matches will fail. Enable Grimmory (GRIMMORY_SERVER, GRIMMORY_USER, GRIMMORY_PASSWORD) " + "or mount the ebooks directory to /books." + ) + + +def start_split_port_server(app, port): + def run_sync_only_server(server_port): + sync_app = Flask(__name__) + sync_app.config["kosync_service"] = app.config["kosync_service"] + sync_app.config["debounce_manager"] = app.config["debounce_manager"] + sync_app.config["rate_limiter"] = app.config["rate_limiter"] + sync_app.register_blueprint(kosync_sync_bp) + + @sync_app.route("/") + def sync_health(): + return "Sync Server OK", 200 + + sync_app.run(host="0.0.0.0", port=server_port, debug=False, use_reloader=False) + + thread = threading.Thread(target=run_sync_only_server, args=(int(port),), daemon=True) + thread.start() + logger.info("Split-Port Mode Active: Sync-only server on port %s", port) + return thread + + +def handle_exit_signal(signum, frame): + logger.warning("Received signal %s - Shutting down...", signum) + for handler in logger.handlers: + handler.flush() + if hasattr(logging.getLogger(), "handlers"): + for handler in logging.getLogger().handlers: + handler.flush() + sys.exit(0) + + +def start_runtime_services(app, container, database_service, sync_manager): + logger.info("=== Unified ABS Manager Started (Integrated Mode) ===") + log_security_warnings() + sync_period_mins = int(get_float("SYNC_PERIOD_MINS", 5)) + start_sync_daemon_thread(sync_manager, sync_period_mins) + threading.Thread(target=get_update_status, daemon=True).start() + initialize_abs_listener(app, container, database_service, sync_manager) + start_client_poller(database_service, sync_manager, container.sync_clients()) + log_ebook_source_configuration(container) + + sync_port = get_str("KOSYNC_PORT", "") + if sync_port and int(sync_port) != 4477: + start_split_port_server(app, sync_port) diff --git a/src/app_setup.py b/src/app_setup.py new file mode 100644 index 0000000..cb691a9 --- /dev/null +++ b/src/app_setup.py @@ -0,0 +1,110 @@ +# pyright: reportMissingImports=false, reportMissingModuleSource=false + +import logging +import os +from pathlib import Path + +from dependency_injector import providers + +from src.api.hardcover_routes import hardcover_bp +from src.api.kosync_admin import kosync_admin_bp +from src.api.kosync_server import kosync_sync_bp +from src.utils.config_loader import ConfigLoader +from src.utils.runtime_config import get_bool, get_str + +logger = logging.getLogger(__name__) + +container = None +manager = None +database_service = None +SYNC_PERIOD_MINS = 5.0 + + +def _get_float_env(key, default): + try: + return float(os.environ.get(key, str(default))) + except (ValueError, TypeError): + logger.warning("Invalid '%s' value, defaulting to %s", key, default) + return float(default) + + +def _migrate_abs_library_ids(database_service): + old_lib_id = get_str("ABS_LIBRARY_ID", "") + new_lib_ids = get_str("ABS_LIBRARY_IDS", "") + if old_lib_id and not new_lib_ids: + old_only_search = get_str("ABS_ONLY_SEARCH_IN_ABS_LIBRARY_ID", "false") + if old_only_search.lower() == "true": + database_service.set_setting("ABS_LIBRARY_IDS", old_lib_id) + os.environ["ABS_LIBRARY_IDS"] = old_lib_id + logger.info("Migrated ABS_LIBRARY_ID '%s' to ABS_LIBRARY_IDS", old_lib_id) + + +def setup_dependencies(app, test_container=None, logging_reconfigure=None): + """Initialize database, DI container, shared services, and app.config state.""" + global container, manager, database_service, SYNC_PERIOD_MINS + + from src.db.migration_utils import initialize_database + from src.services.kosync_service import KosyncService + from src.utils.debounce_manager import DebounceManager + from src.utils.rate_limiter import TokenBucketRateLimiter + + database_service = initialize_database(get_str("DATA_DIR", "/data")) + + if database_service: + ConfigLoader.bootstrap_config(database_service) + ConfigLoader.load_settings(database_service) + logger.info("Settings loaded into environment variables") + _migrate_abs_library_ids(database_service) + if logging_reconfigure: + logging_reconfigure() + + SYNC_PERIOD_MINS = _get_float_env("SYNC_PERIOD_MINS", 5) + logger.info("Globals reloaded from settings (ABS_SERVER=%s)", get_str("ABS_SERVER", "")) + + if test_container is not None: + container = test_container + else: + from src.utils.di_container import create_container + + container = create_container() + container.database_service.override(providers.Object(database_service)) + + manager = container.sync_manager() + data_dir = container.data_dir() + ebook_dir = container.books_dir() + covers_dir = data_dir / "covers" + covers_dir.mkdir(parents=True, exist_ok=True) + + app.config["container"] = container + app.config["sync_manager"] = manager + app.config["database_service"] = database_service + if hasattr(container, "abs_service"): + app.config["abs_service"] = container.abs_service() + else: + from src.services.abs_service import ABSService + + app.config["abs_service"] = ABSService(container.abs_client()) + app.config["DATA_DIR"] = data_dir + app.config["EBOOK_DIR"] = ebook_dir + app.config["COVERS_DIR"] = covers_dir + app.config["ABS_COLLECTION_NAME"] = get_str("ABS_COLLECTION_NAME", "Synced with KOReader") + app.config["SUGGESTIONS_ENABLED"] = get_bool("SUGGESTIONS_ENABLED", False) + + rate_limiter = TokenBucketRateLimiter() + kosync_service = KosyncService(database_service, container, manager, ebook_dir) + debounce_manager = DebounceManager(database_service, manager, rate_limiter=rate_limiter) + app.config["kosync_service"] = kosync_service + app.config["debounce_manager"] = debounce_manager + app.config["rate_limiter"] = rate_limiter + + app.register_blueprint(kosync_sync_bp) + app.register_blueprint(kosync_admin_bp) + + app.register_blueprint(hardcover_bp) + + logger.info("Web server dependencies initialized (DATA_DIR=%s)", data_dir) + return container, manager, database_service + + +def get_runtime_state(): + return container, manager, database_service, SYNC_PERIOD_MINS diff --git a/src/app_template_context.py b/src/app_template_context.py new file mode 100644 index 0000000..fe73aec --- /dev/null +++ b/src/app_template_context.py @@ -0,0 +1,96 @@ +# pyright: reportMissingImports=false + +import os + +from flask import current_app, request + +TEMPLATE_DEFAULTS = { + "TZ": "America/New_York", + "LOG_LEVEL": "INFO", + "DATA_DIR": "/data", + "BOOKS_DIR": "/books", + "ABS_COLLECTION_NAME": "Synced with KOReader", + "GRIMMORY_SHELF_NAME": "Kobo", + "SYNC_PERIOD_MINS": "5", + "SYNC_DELTA_ABS_SECONDS": "60", + "SYNC_DELTA_KOSYNC_PERCENT": "0.5", + "SYNC_DELTA_BETWEEN_CLIENTS_PERCENT": "0.5", + "SYNC_DELTA_KOSYNC_WORDS": "400", + "FUZZY_MATCH_THRESHOLD": "80", + "WHISPER_MODEL": "tiny", + "JOB_MAX_RETRIES": "5", + "JOB_RETRY_DELAY_MINS": "15", + "MONITOR_INTERVAL": "3600", + "AUDIOBOOKS_DIR": "/audiobooks", + "ABS_PROGRESS_OFFSET_SECONDS": "0", + "EBOOK_CACHE_SIZE": "3", + "KOSYNC_HASH_METHOD": "content", + "TELEGRAM_LOG_LEVEL": "ERROR", + "ABS_ENABLED": "true", + "KOSYNC_ENABLED": "false", + "STORYTELLER_ENABLED": "false", + "GRIMMORY_ENABLED": "false", + "HARDCOVER_ENABLED": "false", + "TELEGRAM_ENABLED": "false", + "SUGGESTIONS_ENABLED": "false", + "BOOKFUSION_ENABLED": "false", + "REPROCESS_ON_CLEAR_IF_NO_ALIGNMENT": "true", +} + + +def _get_val(key, default_val=None): + if key in os.environ: + return os.environ[key] + if key in TEMPLATE_DEFAULTS: + return TEMPLATE_DEFAULTS[key] + return default_val if default_val is not None else "" + + +def _get_bool(key): + return _get_val(key, "false").lower() in ("true", "1", "yes", "on") + + +def _get_header_service_url(service_name): + from src.utils.service_url_helper import get_service_web_url + + prefix = service_name.upper() + if not _get_bool(f"{prefix}_ENABLED"): + return "" + return get_service_web_url(prefix) + + +def _is_active_path(path): + req_path = request.path.rstrip("/") or "/" + target_path = path.rstrip("/") or "/" + if target_path == "/": + return req_path == "/" + return req_path == target_path or req_path.startswith(f"{target_path}/") + + +def inject_global_vars(): + """Provide common template variables and helpers for Jinja templates.""" + pagekeeper_env = os.environ.get("PAGEKEEPER_ENV", "").strip().lower() + is_dev_container = pagekeeper_env == "dev" + title_prefix = "[DEV] " if is_dev_container else "" + + suggestion_count = 0 + if _get_bool("SUGGESTIONS_ENABLED"): + try: + db_svc = current_app.config.get("database_service") + if db_svc: + suggestion_count = db_svc.get_pending_suggestion_count() + except Exception: + pass + + return dict( + abs_server=os.environ.get("ABS_SERVER", ""), + grimmory_server=os.environ.get("GRIMMORY_SERVER", ""), + pagekeeper_env=pagekeeper_env, + is_dev_container=is_dev_container, + title_prefix=title_prefix, + get_val=_get_val, + get_bool=_get_bool, + get_header_service_url=_get_header_service_url, + is_active_path=_is_active_path, + suggestion_count=suggestion_count, + ) diff --git a/src/blueprints/api.py b/src/blueprints/api.py index 5fedb9e..36728bc 100644 --- a/src/blueprints/api.py +++ b/src/blueprints/api.py @@ -7,6 +7,8 @@ from flask import Blueprint, current_app, jsonify, request +from src.utils.http import json_detail_error, json_error + from src.blueprints.helpers import ( find_in_grimmory, get_book_or_404, @@ -142,10 +144,10 @@ def hide_suggestion(source_id): database_service = get_database_service() source = request.args.get("source", "abs") if source not in _VALID_SUGGESTION_SOURCES: - return jsonify({"success": False, "error": "Invalid source"}), 400 + return json_error("Invalid source", 400) if database_service.hide_suggestion(source_id, source=source): return jsonify({"success": True}) - return jsonify({"success": False, "error": "Not found"}), 404 + return json_error("Not found", 404) @api_bp.route("/api/suggestions//unhide", methods=["POST"]) @@ -153,10 +155,10 @@ def unhide_suggestion(source_id): database_service = get_database_service() source = request.args.get("source", "abs") if source not in _VALID_SUGGESTION_SOURCES: - return jsonify({"success": False, "error": "Invalid source"}), 400 + return json_error("Invalid source", 400) if database_service.unhide_suggestion(source_id, source=source): return jsonify({"success": True}) - return jsonify({"success": False, "error": "Not found"}), 404 + return json_error("Not found", 404) @api_bp.route("/api/suggestions//ignore", methods=["POST"]) @@ -164,10 +166,10 @@ def ignore_suggestion(source_id): database_service = get_database_service() source = request.args.get("source", "abs") if source not in _VALID_SUGGESTION_SOURCES: - return jsonify({"success": False, "error": "Invalid source"}), 400 + return json_error("Invalid source", 400) if database_service.ignore_suggestion(source_id, source=source): return jsonify({"success": True}) - return jsonify({"success": False, "error": "Not found"}), 404 + return json_error("Not found", 404) @api_bp.route("/api/suggestions/clear_stale", methods=["POST"]) @@ -185,21 +187,21 @@ def link_suggestion_bookfusion(source_id): data = request.get_json(silent=True) or {} source = data.get("source", "abs") if source not in _VALID_SUGGESTION_SOURCES: - return jsonify({"success": False, "error": "Invalid source"}), 400 + return json_error("Invalid source", 400) suggestion = database_service.get_pending_suggestion(source_id, source=source) if not suggestion: - return jsonify({"success": False, "error": "Suggestion not found"}), 404 + return json_error("Suggestion not found", 404) match_index = data.get("match_index") matches = suggestion.matches or [] if match_index is None or not isinstance(match_index, int) or match_index < 0 or match_index >= len(matches): - return jsonify({"success": False, "error": "Valid match_index required"}), 400 + return json_error("Valid match_index required", 400) match = matches[match_index] bookfusion_ids = match.get("bookfusion_ids") or [] if match.get("source_family") != "bookfusion" or not bookfusion_ids: - return jsonify({"success": False, "error": "Selected match is not a BookFusion candidate"}), 400 + return json_error("Selected match is not a BookFusion candidate", 400) # Find or create the book to link BookFusion to if source == "abs": @@ -249,7 +251,7 @@ def link_suggestion_bookfusion(source_id): book = database_service.get_book_by_id(book.id) if not book: - return jsonify({"success": False, "error": "Could not find or create book"}), 500 + return json_error("Could not find or create book", 500) for bid in bookfusion_ids: database_service.set_bookfusion_book_match_by_book_id(bid, book.id) @@ -276,7 +278,7 @@ def api_storyteller_search(): container = get_container() query = request.args.get("q", "") if not query: - return jsonify({"success": False, "error": "Query parameter 'q' is required"}), 400 + return json_error("Query parameter 'q' is required", 400) results = container.storyteller_client().search_books(query) return jsonify(results) @@ -287,7 +289,7 @@ def api_storyteller_link(book_ref): data = request.get_json() if not data or "uuid" not in data: - return jsonify({"success": False, "error": "Missing 'uuid' in JSON payload"}), 400 + return json_error("Missing 'uuid' in JSON payload", 400) storyteller_uuid = data["uuid"] book = get_book_or_404(book_ref) @@ -315,7 +317,7 @@ def _get_grimmory_libraries(client_getter, name): container = get_container() client = client_getter(container) if not client.is_configured(): - return jsonify({"success": False, "error": f"{name} not configured"}), 400 + return json_error(f"{name} not configured", 400) return jsonify(client.get_libraries()) @@ -370,15 +372,15 @@ def api_grimmory_link(book_ref): data = request.get_json(silent=True) if not isinstance(data, dict): - return jsonify({"success": False, "error": "No data provided"}), 400 + return json_error("No data provided", 400) if "filename" not in data: - return jsonify({"success": False, "error": "Missing 'filename' in JSON payload"}), 400 + return json_error("Missing 'filename' in JSON payload", 400) filename_raw = data.get("filename") if filename_raw is None: filename = "" elif not isinstance(filename_raw, str): - return jsonify({"success": False, "error": "'filename' must be a string or null"}), 400 + return json_error("'filename' must be a string or null", 400) else: filename = filename_raw.strip() diff --git a/src/blueprints/bookfusion_bp.py b/src/blueprints/bookfusion_bp.py index 01ad9b1..21b8c5d 100644 --- a/src/blueprints/bookfusion_bp.py +++ b/src/blueprints/bookfusion_bp.py @@ -1,76 +1,55 @@ """BookFusion blueprint — upload books and sync highlights.""" -import difflib import logging -from collections import defaultdict +import re from datetime import datetime -from flask import Blueprint, current_app, jsonify, render_template, request +from flask import Blueprint, jsonify, request -from src.blueprints.helpers import get_container, get_database_service, get_grimmory_client -from src.db.models import Book, HardcoverDetails -from src.utils.title_utils import clean_book_title, normalize_title +from src.blueprints.helpers import get_container, get_database_service +from src.db.models import BookfusionBook logger = logging.getLogger(__name__) bookfusion_bp = Blueprint("bookfusion", __name__) +BOOKFUSION_ENTRY_SPLIT = "\n— " -SUPPORTED_FORMATS = {".epub", ".mobi", ".azw3", ".pdf", ".azw", ".fb2", ".cbz", ".cbr"} +def _normalize_bookfusion_chapter(chapter): + chapter = re.sub(r"^#{1,6}\s*", "", (chapter or "").strip()) + return re.sub(r"^[*_]+|[*_]+$", "", chapter).strip() -def _is_supported(filename: str) -> bool: - return any(filename.lower().endswith(ext) for ext in SUPPORTED_FORMATS) +def _bookfusion_entry_key(entry): + if not entry: + return ("", "") -@bookfusion_bp.route("/bookfusion") -def bookfusion_page(): - return render_template("bookfusion.html") + text = entry.strip() + if text.startswith("\U0001f4d6"): + text = text[1:].lstrip() + if text.startswith("\U0001f4d6"): + text = text[1:].lstrip() + quote, chapter = text, "" + if BOOKFUSION_ENTRY_SPLIT in text: + quote, chapter = text.split(BOOKFUSION_ENTRY_SPLIT, 1) + return (quote.strip(), _normalize_bookfusion_chapter(chapter)) -@bookfusion_bp.route("/api/bookfusion/grimmory-books") -def grimmory_books(): - """List Grimmory books for upload selection, filtered by supported formats.""" - q = request.args.get("q", "").strip() - results = [] - client = get_grimmory_client() - if client.is_configured(): - try: - label = current_app.config.get("GRIMMORY_LABEL", "Grimmory") - books = client.search_books(q) if q else client.get_all_books() - for b in books or []: - fname = b.get("fileName", "") - if not _is_supported(fname): - continue - results.append( - { - "id": b.get("id"), - "title": b.get("title", ""), - "authors": b.get("authors", ""), - "fileName": fname, - "source": label, - } - ) - except Exception as e: - logger.warning(f"Grimmory search failed: {e}") - - return jsonify(results) +def _bookfusion_highlight_key(quote, chapter): + return ((quote or "").strip(), _normalize_bookfusion_chapter(chapter)) @bookfusion_bp.route("/api/bookfusion/upload", methods=["POST"]) def upload_book(): - """Upload a book from Grimmory to BookFusion.""" + """Upload a book from PageKeeper to BookFusion.""" data = request.get_json() if not data: return jsonify({"error": "No data provided"}), 400 - book_id = data.get("book_id") - title = data.get("title") or "" - authors = data.get("authors") or "" - filename = data.get("fileName", "") - - if not book_id: - return jsonify({"error": "book_id required"}), 400 + abs_id = data.get("abs_id") + if not abs_id: + return jsonify({"error": "abs_id required"}), 400 container = get_container() bf_client = container.bookfusion_client() @@ -78,26 +57,71 @@ def upload_book(): if not bf_client.upload_api_key: return jsonify({"error": "BookFusion upload API key not configured"}), 400 - bl_client = get_grimmory_client() - if not bl_client.is_configured(): - return jsonify({"error": "Grimmory not configured"}), 400 + db_service = get_database_service() + book = db_service.get_book_by_ref(abs_id) + if not book: + return jsonify({"error": "Book not found"}), 404 + + if not book.ebook_filename and not book.original_ebook_filename: + return jsonify({"error": "No ebook file associated with this book"}), 400 + + ebook_filename = book.original_ebook_filename or book.ebook_filename + + from src.utils.epub_resolver import get_local_epub + + books_dir = container.config.get("BOOKS_DIR") or "/books" + epub_cache_dir = container.config.get("EPUB_CACHE_DIR") or "/tmp/epub_cache" + grimmory_client = container.grimmory_client() if hasattr(container, "grimmory_client") else None + + file_path = get_local_epub(ebook_filename, books_dir, epub_cache_dir, grimmory_client) + if not file_path: + return jsonify({"error": "Could not locate ebook file"}), 500 + + try: + with open(file_path, "rb") as f: + file_bytes = f.read() + except Exception as e: + logger.error("Failed to read ebook file: %s", e) + return jsonify({"error": "Failed to read ebook file"}), 500 + + title = book.title or "" + authors = book.author or "" - # Download from Grimmory - file_bytes = bl_client.download_book(book_id) - if not file_bytes: - return jsonify({"error": "Failed to download book from Grimmory"}), 500 + logger.info( + "BookFusion upload request: title='%s', authors='%s', filename='%s'", + title, + authors, + ebook_filename, + ) + result = bf_client.upload_book(ebook_filename, file_bytes, title, authors) + if not result: + return jsonify({"error": "Upload to BookFusion failed"}), 500 + + bf_book_id = result.get("id") + already_linked = False + if bf_book_id: + existing = db_service.get_bookfusion_book_by_book_id(book.id) + if existing: + already_linked = True + logger.info(f"BookFusion book already linked: bf_id={bf_book_id}, book_id={book.id}") + else: + db_service.save_bookfusion_book( + BookfusionBook( + bookfusion_id=bf_book_id, + title=title, + authors=authors, + filename=ebook_filename, + matched_book_id=book.id, + ) + ) + logger.info(f"BookFusion book linked: bf_id={bf_book_id}, book_id={book.id}") - # Upload to BookFusion - logger.info(f"BookFusion upload request: title='{title}', authors='{authors}', filename='{filename}'") - result = bf_client.upload_book(filename, file_bytes, title, authors) - if result: - return jsonify({"success": True, "result": result}) - return jsonify({"error": "Upload to BookFusion failed"}), 500 + return jsonify({"success": True, "already_linked": already_linked, "result": result}) @bookfusion_bp.route("/api/bookfusion/sync-highlights", methods=["POST"]) def sync_highlights(): - """Trigger highlight sync from BookFusion.""" + """Trigger highlight sync from BookFusion for a specific book or all books.""" container = get_container() bf_client = container.bookfusion_client() db_service = get_database_service() @@ -106,18 +130,17 @@ def sync_highlights(): return jsonify({"error": "BookFusion highlights API key not configured"}), 400 data = request.get_json(silent=True) or {} + if data.get("full_resync"): db_service.set_bookfusion_sync_cursor(None) try: result = bf_client.sync_all_highlights(db_service) - matched = _auto_match_highlights(db_service) return jsonify( { "success": True, "new_highlights": result["new_highlights"], "books_saved": result["books_saved"], - "auto_matched": matched, "new_ids": result.get("new_ids", []), } ) @@ -126,231 +149,53 @@ def sync_highlights(): return jsonify({"error": "BookFusion highlight sync failed"}), 500 -def _auto_match_highlights(db_service) -> int: - """Auto-match unlinked BookFusion highlights to PageKeeper books by title similarity.""" - unmatched = db_service.get_unmatched_bookfusion_highlights() - if not unmatched: - return 0 - - books = db_service.get_all_books() - if not books: - return 0 - - # Build normalized title → book_id list map (detect ambiguous duplicates) - book_map: dict[str, list[int]] = defaultdict(list) - for b in books: - if b.title: - norm = normalize_title(b.title) - book_map[norm].append(b.id) - - # Group unmatched by book_title - title_groups: dict[str, list] = {} - for hl in unmatched: - title = clean_book_title(hl.book_title or "") - title_groups.setdefault(title, []).append(hl) - - matched_count = 0 - norm_keys = list(book_map.keys()) - - for bf_title, highlights in title_groups.items(): - norm_bf = normalize_title(bf_title) - matched_book_id = None +@bookfusion_bp.route("/api/bookfusion/sync-book", methods=["POST"]) +def sync_book_highlights(): + """Sync highlights for a specific book from BookFusion.""" + data = request.get_json() + if not data: + return jsonify({"error": "No data provided"}), 400 - # Exact match (only if unambiguous) - if norm_bf in book_map and len(book_map[norm_bf]) == 1: - matched_book_id = book_map[norm_bf][0] - else: - # Fuzzy match (only if unambiguous) - candidates = [] - for norm_pk in norm_keys: - if len(book_map[norm_pk]) != 1: - continue - ratio = difflib.SequenceMatcher(None, norm_bf, norm_pk).ratio() - if ratio > 0.85: - candidates.append((ratio, book_map[norm_pk][0])) - if candidates: - candidates.sort(key=lambda c: c[0], reverse=True) - if len(candidates) == 1 or (candidates[0][0] - candidates[1][0]) >= 0.05: - matched_book_id = candidates[0][1] + abs_id = data.get("abs_id") + if not abs_id: + return jsonify({"error": "abs_id required"}), 400 - if matched_book_id: - bf_ids = {hl.bookfusion_book_id for hl in highlights if hl.bookfusion_book_id} - for bf_id in bf_ids: - db_service.link_bookfusion_highlights_by_book_id(bf_id, matched_book_id) - matched_count += len(highlights) + db_service = get_database_service() + book = db_service.get_book_by_ref(abs_id) + if not book: + return jsonify({"error": "Book not found"}), 404 - return matched_count + container = get_container() + bf_client = container.bookfusion_client() + if not bf_client.highlights_api_key: + return jsonify({"error": "BookFusion highlights API key not configured"}), 400 -def _estimate_reading_dates(db_service, abs_id: str, bookfusion_ids: list[str], title: str) -> dict: - """Attempt to set reading dates on a newly-linked book. Returns date info for the response.""" - book = db_service.get_book_by_ref(abs_id) - if not book or book.started_at or book.finished_at: - return {} + bf_books = db_service.get_bookfusion_books_by_book_id(book.id) + if not bf_books: + return jsonify({"error": "BookFusion link not found for this book"}), 404 - container = get_container() - started_at = None - finished_at = None - source = None - estimated = False + bf_book_ids = [b.bookfusion_id for b in bf_books] - # Priority 1: Hardcover (actual dates) try: - hc_client = container.hardcover_client() - if hc_client.is_configured(): - hc_details = db_service.get_hardcover_details(book.id) - user_book = None - if hc_details and hc_details.hardcover_book_id: - try: - user_book = hc_client.find_user_book(int(hc_details.hardcover_book_id)) - except (ValueError, TypeError): - logger.debug("Invalid hardcover_book_id: %s", hc_details.hardcover_book_id) - user_book = None - elif title: - search_result = hc_client.search_by_title_author(title) - if search_result: - # Persist HardcoverDetails so the cover URL and link survive - if not hc_details: - new_details = HardcoverDetails( - abs_id=abs_id, - book_id=book.id, - hardcover_book_id=str(search_result["book_id"]), - hardcover_slug=search_result.get("slug"), - hardcover_cover_url=search_result.get("cached_image"), - matched_by="title", - ) - db_service.save_hardcover_details(new_details) - user_book = hc_client.find_user_book(search_result["book_id"]) - if user_book: - reads = user_book.get("user_book_reads", []) - if reads: - read = reads[0] - if read.get("started_at"): - started_at = read["started_at"] - if read.get("finished_at"): - finished_at = read["finished_at"] - if started_at or finished_at: - source = "hardcover" - except Exception as e: - logger.debug(f"Hardcover date lookup failed for '{abs_id}': {e}") - - # Priority 2: Highlight date range (estimated) - if not source and bookfusion_ids: - date_range = db_service.get_bookfusion_highlight_date_range(bookfusion_ids) - if date_range: - earliest, latest, count = date_range - started_at = earliest.strftime("%Y-%m-%d") if earliest else None - if count > 1 and latest: - finished_at = latest.strftime("%Y-%m-%d") - source = "highlights" - estimated = True - - if not source: - return {} - - # Apply dates and status - updates = {} - if started_at: - updates["started_at"] = started_at - if finished_at: - updates["finished_at"] = finished_at - if updates: - updated_book = db_service.update_book_reading_fields(book.id, **updates) - if updated_book: - book = updated_book - - # Update status via ReadingService for proper journal entries + HC sync - if finished_at or started_at: - from src.services.reading_service import ReadingService - - reading_svc = ReadingService(db_service) - target_status = "completed" if finished_at else "active" - if book.status != target_status: - reading_svc.update_status(abs_id, target_status, container) - - return { - "dates_set": True, - "dates_source": source, - "dates_estimated": estimated, - "started_at": started_at, - "finished_at": finished_at, - } - - -@bookfusion_bp.route("/api/bookfusion/highlights") -def get_highlights(): - """Return cached highlights from DB, grouped by book.""" - db_service = get_database_service() - highlights = db_service.get_bookfusion_highlights() + result = bf_client.sync_all_highlights(db_service) - grouped = {} - for hl in highlights: - key = (hl.bookfusion_book_id, hl.matched_abs_id, clean_book_title(hl.book_title or "Unknown Book")) - if key not in grouped: - grouped[key] = { - "highlights": [], - "matched_abs_id": hl.matched_abs_id, - "bookfusion_book_id": hl.bookfusion_book_id, - "display_title": clean_book_title(hl.book_title or "Unknown Book"), - } - date_str = hl.highlighted_at.strftime("%Y-%m-%d %H:%M:%S") if hl.highlighted_at else None - grouped[key]["highlights"].append( + linked_count = 0 + for bf_id in bf_book_ids: + db_service.link_bookfusion_highlights_by_book_id(bf_id, book.id) + linked_count += 1 + + return jsonify( { - "id": hl.id, - "highlight_id": hl.highlight_id, - "quote": hl.quote_text or hl.content, - "date": date_str, - "chapter_heading": hl.chapter_heading, - "matched_abs_id": hl.matched_abs_id, + "success": True, + "new_highlights": result["new_highlights"], + "books_saved": result["books_saved"], + "linked_books": linked_count, } ) - - # Sort highlights within each book by date - for key in grouped: - grouped[key]["highlights"].sort(key=lambda h: h["date"] or "", reverse=True) - - # Re-key by display title for the frontend (API contract uses title as key) - display = {} - for _key, group in grouped.items(): - title = group.pop("display_title") - # Disambiguate if two different books share the same cleaned title - display_key = title - if display_key in display: - display_key = f"{title} ({group['bookfusion_book_id']})" - display[display_key] = group - - cursor = db_service.get_bookfusion_sync_cursor() - - # Include list of PageKeeper books for journal matching - books = db_service.get_all_books() - book_list = [{"abs_id": b.abs_id, "title": b.title} for b in books if b.title] - - return jsonify({"highlights": display, "has_synced": cursor is not None, "books": book_list}) - - -@bookfusion_bp.route("/api/bookfusion/link-highlight", methods=["POST"]) -def link_highlight(): - """Manually link or unlink a BookFusion book's highlights to a PageKeeper book.""" - data = request.get_json() - if not data: - return jsonify({"error": "No data provided"}), 400 - - bookfusion_book_id = data.get("bookfusion_book_id") - abs_id = data.get("abs_id") # None or empty to unlink - - if not bookfusion_book_id: - return jsonify({"error": "bookfusion_book_id required"}), 400 - - db_service = get_database_service() - if abs_id: - book = db_service.get_book_by_ref(abs_id) - if not book: - return jsonify({"error": "Book not found"}), 404 - book_id = book.id - else: - book_id = None - db_service.link_bookfusion_highlights_by_book_id(bookfusion_book_id, book_id) - return jsonify({"success": True}) + except Exception: + logger.exception("BookFusion highlight sync failed for book") + return jsonify({"error": "BookFusion highlight sync failed"}), 500 @bookfusion_bp.route("/api/bookfusion/save-journal", methods=["POST"]) @@ -366,11 +211,13 @@ def save_highlight_to_journal(): if not abs_id: return jsonify({"error": "abs_id required"}), 400 - # When no highlights provided in the request, fetch them server-side + db_service = get_database_service() + book = db_service.get_book_by_ref(abs_id) + if not book: + return jsonify({"error": "Book not found"}), 404 + if not highlights: - db_service = get_database_service() - book = db_service.get_book_by_ref(abs_id) - bf_highlights = db_service.get_bookfusion_highlights_for_book_by_book_id(book.id) if book else [] + bf_highlights = db_service.get_bookfusion_highlights_for_book_by_book_id(book.id) if not bf_highlights: return jsonify({"error": "No highlights found for this book"}), 400 highlights = [] @@ -383,22 +230,33 @@ def save_highlight_to_journal(): } ) - db_service = get_database_service() - book = db_service.get_book_by_ref(abs_id) - if not book: - return jsonify({"error": "Book not found"}), 404 - - cleanup_stats = db_service.cleanup_bookfusion_import_notes(abs_id) + db_service.cleanup_bookfusion_import_notes(book.id) + existing_entries = db_service.get_reading_journal_entries_for_book(book.id, "highlight") + existing_keys = set() + for entry in existing_entries: + if entry.entry: + existing_keys.add(_bookfusion_entry_key(entry.entry)) saved = 0 + skipped = 0 + for hl in highlights: quote = hl.get("quote", "").strip() - chapter = hl.get("chapter", "") + chapter = hl.get("chapter", "").strip() highlighted_at_raw = (hl.get("highlighted_at") or "").strip() + if not quote: continue - entry = quote + + highlight_key = _bookfusion_highlight_key(quote, chapter) + if highlight_key in existing_keys: + skipped += 1 + continue + + entry_text = quote if chapter: - entry += f"\n— {chapter}" + chapter_clean = chapter.lstrip("#").strip() + entry_text += f"\n— *{chapter_clean}*" + created_at = None if highlighted_at_raw: for fmt in ("%Y-%m-%d %H:%M:%S", "%b %d, %Y"): @@ -409,221 +267,14 @@ def save_highlight_to_journal(): continue if not created_at: logger.debug("Could not parse BookFusion highlight timestamp '%s'", highlighted_at_raw) + try: - db_service.add_reading_journal(book.id, "highlight", entry=entry, created_at=created_at, abs_id=book.abs_id) + db_service.add_reading_journal( + book.id, "highlight", entry=entry_text, created_at=created_at, abs_id=book.abs_id + ) + existing_keys.add(highlight_key) saved += 1 except Exception as e: logger.warning(f"Failed to save journal entry: {e}") - return jsonify({"success": True, "saved": saved, "cleanup": cleanup_stats}) - - -@bookfusion_bp.route("/api/bookfusion/library") -def get_library(): - """Return BookFusion library catalog for the Library tab, merging duplicate titles.""" - db_service = get_database_service() - bf_books = db_service.get_bookfusion_books() - - # Check which books are already on the dashboard (by bf- prefix or highlight match) - all_books = db_service.get_all_books() - dashboard_ids = {b.abs_id for b in all_books} - book_list = [{"abs_id": b.abs_id, "title": b.title} for b in all_books if b.title] - - # Group by normalized title to merge format duplicates - groups = defaultdict(list) - for b in bf_books: - norm = normalize_title(b.title or b.filename or "") - groups[norm].append(b) - - result = [] - for _norm_title, group in groups.items(): - # Pick the entry with the most highlights as the "primary" for metadata - group.sort(key=lambda b: b.highlight_count or 0, reverse=True) - primary = group[0] - - title = clean_book_title(primary.title or primary.filename or "") - authors = "" - series = "" - tags = "" - for b in group: - if not authors and b.authors: - authors = b.authors - if not series and b.series: - series = b.series - if not tags and b.tags: - tags = b.tags - - filenames = list(dict.fromkeys(b.filename for b in group if b.filename)) - bookfusion_ids = [b.bookfusion_id for b in group] - highlight_count = sum(b.highlight_count or 0 for b in group) - - # Check dashboard match across all entries in the group - matched_abs_id = None - for b in group: - bf_abs_id = f"bf-{b.bookfusion_id}" - if b.matched_abs_id and b.matched_abs_id in dashboard_ids: - matched_abs_id = b.matched_abs_id - break - elif bf_abs_id in dashboard_ids: - matched_abs_id = bf_abs_id - break - - # A group is hidden if any entry in the group is hidden - is_hidden = any(b.hidden for b in group) - - result.append( - { - "bookfusion_id": bookfusion_ids[0], - "bookfusion_ids": bookfusion_ids, - "title": title, - "authors": authors, - "filenames": filenames, - "filename": primary.filename or "", - "series": series, - "tags": tags, - "highlight_count": highlight_count, - "on_dashboard": matched_abs_id is not None, - "abs_id": matched_abs_id, - "hidden": is_hidden, - } - ) - - return jsonify({"books": result, "dashboard_books": book_list}) - - -@bookfusion_bp.route("/api/bookfusion/add-to-dashboard", methods=["POST"]) -def add_to_dashboard(): - """Add a BookFusion book to the reading dashboard.""" - data = request.get_json() - if not data: - return jsonify({"error": "No data provided"}), 400 - - bookfusion_ids = data.get("bookfusion_ids") or [] - if not bookfusion_ids: - single = data.get("bookfusion_id") - if single: - bookfusion_ids = [single] - if not bookfusion_ids: - return jsonify({"error": "bookfusion_id required"}), 400 - - primary_id = bookfusion_ids[0] - db_service = get_database_service() - bf_book = db_service.get_bookfusion_book(primary_id) - if not bf_book: - return jsonify({"error": "BookFusion book not found in catalog"}), 404 - - abs_id = f"bf-{primary_id}" - - # Check if already on dashboard - existing = db_service.get_book_by_ref(abs_id) - if existing: - return jsonify({"success": True, "abs_id": abs_id, "already_existed": True}) - - # Create dashboard book entry - title = clean_book_title(bf_book.title or bf_book.filename or "Unknown") - initial_status = data.get("status", "not_started") - if initial_status not in ("not_started", "active"): - initial_status = "not_started" - book = Book( - abs_id=abs_id, - title=title, - status=initial_status, - sync_mode="ebook_only", - ) - db_service.save_book(book, is_new=True) - - # Re-fetch book to get the auto-assigned ID - saved_book = db_service.get_book_by_ref(abs_id) - saved_book_id = saved_book.id if saved_book else None - - # Auto-link ALL catalog books + highlights in the group - for bid in bookfusion_ids: - db_service.set_bookfusion_book_match_by_book_id(bid, saved_book_id) - db_service.link_bookfusion_highlights_by_book_id(bid, saved_book_id) - - # Auto-populate reading dates - date_info = _estimate_reading_dates(db_service, abs_id, bookfusion_ids, title) - - resp = {"success": True, "abs_id": abs_id} - resp.update(date_info) - return jsonify(resp) - - -@bookfusion_bp.route("/api/bookfusion/match-to-book", methods=["POST"]) -def match_to_book(): - """Match a BookFusion catalog book to an existing dashboard book (link highlights).""" - data = request.get_json() - if not data: - return jsonify({"error": "No data provided"}), 400 - - bookfusion_ids = data.get("bookfusion_ids") or [] - if not bookfusion_ids: - single = data.get("bookfusion_id") - if single: - bookfusion_ids = [single] - abs_id = data.get("abs_id") # None/empty to unlink - - if not bookfusion_ids: - return jsonify({"error": "bookfusion_id required"}), 400 - - db_service = get_database_service() - - book = db_service.get_book_by_ref(abs_id) if abs_id else None - if abs_id and not book: - return jsonify({"error": "Book not found"}), 404 - - book_id = book.id if book else None - - # Link ALL catalog books + highlights in the group - for bid in bookfusion_ids: - db_service.set_bookfusion_book_match_by_book_id(bid, book_id) - db_service.link_bookfusion_highlights_by_book_id(bid, book_id) - - resp = {"success": True, "abs_id": abs_id} - - # Auto-populate reading dates if linking (not unlinking) - if abs_id: - title = book.title if book else "" - date_info = _estimate_reading_dates(db_service, abs_id, bookfusion_ids, title) - resp.update(date_info) - - return jsonify(resp) - - -@bookfusion_bp.route("/api/bookfusion/hide", methods=["POST"]) -def hide_book(): - """Hide or unhide a BookFusion library book.""" - data = request.get_json() - if not data: - return jsonify({"error": "No data provided"}), 400 - - bookfusion_ids = data.get("bookfusion_ids") or [] - if not bookfusion_ids: - single = data.get("bookfusion_id") - if single: - bookfusion_ids = [single] - if not bookfusion_ids: - return jsonify({"error": "bookfusion_id required"}), 400 - - hidden = data.get("hidden", True) - db_service = get_database_service() - db_service.set_bookfusion_books_hidden(bookfusion_ids, hidden) - return jsonify({"success": True}) - - -@bookfusion_bp.route("/api/bookfusion/unlink", methods=["POST"]) -def unlink_book(): - """Unlink a BookFusion book from a dashboard book.""" - data = request.get_json() - if not data: - return jsonify({"error": "No data provided"}), 400 - - abs_id = data.get("abs_id") - if not abs_id: - return jsonify({"error": "abs_id required"}), 400 - - db_service = get_database_service() - book = db_service.get_book_by_ref(abs_id) - if book: - db_service.unlink_bookfusion_by_book_id(book.id) - return jsonify({"success": True}) + return jsonify({"success": True, "saved": saved, "skipped": skipped}) diff --git a/src/blueprints/reading_bp.py b/src/blueprints/reading_bp.py index 7bd9518..9f0cdb5 100644 --- a/src/blueprints/reading_bp.py +++ b/src/blueprints/reading_bp.py @@ -8,6 +8,8 @@ from flask import Blueprint, abort, jsonify, render_template, request +from src.utils.http import json_error + from src.blueprints.helpers import ( find_grimmory_metadata, get_abs_service, @@ -21,6 +23,7 @@ from src.services.reading_service import ReadingService from src.services.reading_stats_service import ReadingStatsService from src.utils.cover_resolver import resolve_book_covers +from src.utils.markdown import render_markdown_html logger = logging.getLogger(__name__) @@ -344,6 +347,9 @@ def reading_detail(book_ref): or database_service.is_bookfusion_linked_by_book_id(book.id) ) + # Check if book is eligible for BookFusion upload (has ebook file) + bookfusion_upload_eligible = bool(book.ebook_filename or book.original_ebook_filename) + container = get_container() metadata = build_book_metadata(book, container, database_service, abs_service) hardcover = metadata.get("_hardcover") @@ -401,6 +407,7 @@ def reading_detail(book_ref): journals=journals, bf_highlights=bf_highlights, has_bookfusion_link=has_bookfusion_link, + bookfusion_upload_eligible=bookfusion_upload_eligible, has_linked_tbr=has_linked_tbr, metadata=metadata, services_enabled=services_enabled, @@ -465,15 +472,15 @@ def update_rating(book_ref): try: rating = float(rating) except (TypeError, ValueError): - return jsonify({"success": False, "error": "Invalid rating value"}), 400 + return json_error("Invalid rating value", 400) if not math.isfinite(rating) or rating < 0 or rating > 5: - return jsonify({"success": False, "error": "Rating must be between 0 and 5"}), 400 + return json_error("Rating must be between 0 and 5", 400) if abs((rating * 2) - round(rating * 2)) > 1e-9: - return jsonify({"success": False, "error": "Rating must be in 0.5 increments"}), 400 + return json_error("Rating must be in 0.5 increments", 400) book = database_service.update_book_reading_fields(book.id, rating=rating) if not book: - return jsonify({"success": False, "error": "Book not found"}), 404 + return json_error("Book not found", 404) hardcover_synced = False hardcover_error = None @@ -485,7 +492,8 @@ def update_rating(book_ref): hardcover_synced = bool(sync_result.get("hardcover_synced")) hardcover_error = sync_result.get("hardcover_error") except Exception as e: - hardcover_error = str(e) + logger.warning("Hardcover rating sync failed for book %s: %s", book.id, e) + hardcover_error = "Hardcover sync failed" return jsonify( { @@ -505,15 +513,15 @@ def update_progress(book_ref): percentage = data.get("percentage") if percentage is None: - return jsonify({"success": False, "error": "percentage is required"}), 400 + return json_error("percentage is required", 400) try: percentage = float(percentage) except (TypeError, ValueError): - return jsonify({"success": False, "error": "Invalid percentage value"}), 400 + return json_error("Invalid percentage value", 400) if not math.isfinite(percentage) or percentage < 0 or percentage > 1: - return jsonify({"success": False, "error": "percentage must be between 0 and 1"}), 400 + return json_error("percentage must be between 0 and 1", 400) container = get_container() result = _get_reading_service().set_progress(book.id, percentage, container) @@ -538,20 +546,20 @@ def update_dates(book_ref): try: datetime.strptime(val, "%Y-%m-%d") except ValueError: - return jsonify({"success": False, "error": f"Invalid date format for {field}"}), 400 + return json_error(f"Invalid date format for {field}", 400) updates[field] = val or None if not updates: - return jsonify({"success": False, "error": "No date fields provided"}), 400 + return json_error("No date fields provided", 400) effective_started = updates.get("started_at") or (book.started_at if "started_at" not in updates else None) effective_finished = updates.get("finished_at") or (book.finished_at if "finished_at" not in updates else None) if effective_started and effective_finished and effective_started > effective_finished: - return jsonify({"success": False, "error": "started_at cannot be after finished_at"}), 400 + return json_error("started_at cannot be after finished_at", 400) book = database_service.update_book_reading_fields(book.id, **updates) if not book: - return jsonify({"success": False, "error": "Book not found"}), 404 + return json_error("Book not found", 404) # Sync corresponding journal entry timestamps to match the edited dates event_map = {"started_at": "started", "finished_at": "finished"} @@ -581,7 +589,7 @@ def sync_dates_to_hardcover(book_ref): synced, message = container.reading_date_service().push_dates_to_hardcover(book.id, force=True) if synced: return jsonify({"success": True, "message": message}) - return jsonify({"success": False, "error": message}), 400 + return json_error(message, 400) @reading_bp.route("/api/reading/book//dates/pull-hardcover", methods=["POST"]) @@ -592,7 +600,7 @@ def pull_dates_from_hardcover(book_ref): success, message, dates = container.reading_date_service().pull_dates_from_hardcover(book.id) if success: return jsonify({"success": True, "message": message, "dates": dates}) - return jsonify({"success": False, "error": message}), 400 + return json_error(message, 400) @reading_bp.route("/api/reading/book//journal", methods=["POST"]) @@ -604,7 +612,7 @@ def add_journal(book_ref): entry = (data.get("entry") or "").strip() if not entry: - return jsonify({"success": False, "error": "Entry text is required"}), 400 + return json_error("Entry text is required", 400) # Get current progress for the journal entry book_states = database_service.get_states_for_book(book.id) @@ -625,6 +633,7 @@ def add_journal(book_ref): "id": journal.id, "event": journal.event, "entry": journal.entry, + "entry_html": render_markdown_html(journal.entry), "percentage": journal.percentage, "created_at": journal.created_at.isoformat() if journal.created_at else None, }, @@ -640,14 +649,14 @@ def delete_journal(journal_id): # Look up the journal before deleting so we can cascade for started/finished journal = database_service.get_reading_journal(journal_id) if not journal: - return jsonify({"success": False, "error": "Journal entry not found"}), 404 + return json_error("Journal entry not found", 404) book_id = journal.book_id event = journal.event deleted = database_service.delete_reading_journal(journal_id) if not deleted: - return jsonify({"success": False, "error": "Journal entry not found"}), 404 + return json_error("Journal entry not found", 404) # If this was the last started/finished journal, clear the corresponding book field cleared_field = None @@ -669,13 +678,13 @@ def update_journal(journal_id): existing = database_service.get_reading_journal(journal_id) if not existing: - return jsonify({"success": False, "error": "Journal entry not found"}), 404 + return json_error("Journal entry not found", 404) # Started/finished entries: only allow editing the date (created_at), not text if existing.event in ("started", "finished"): date_str = (data.get("created_at") or "").strip() if not date_str: - return jsonify({"success": False, "error": "created_at date is required for started/finished entries"}), 400 + return json_error("created_at date is required for started/finished entries", 400) try: new_dt = datetime.strptime(date_str, "%Y-%m-%d") except ValueError: @@ -691,6 +700,7 @@ def update_journal(journal_id): "id": journal.id, "event": journal.event, "entry": journal.entry, + "entry_html": render_markdown_html(journal.entry), "percentage": journal.percentage, "created_at": journal.created_at.isoformat() if journal.created_at else None, }, @@ -698,9 +708,9 @@ def update_journal(journal_id): ) if existing.event != "note": - return jsonify({"success": False, "error": "Only notes can be edited"}), 400 + return json_error("Only notes can be edited", 400) if not entry: - return jsonify({"success": False, "error": "entry is required"}), 400 + return json_error("entry is required", 400) journal = database_service.update_reading_journal(journal_id, entry=entry) return jsonify( @@ -710,6 +720,7 @@ def update_journal(journal_id): "id": journal.id, "event": journal.event, "entry": journal.entry, + "entry_html": render_markdown_html(journal.entry), "percentage": journal.percentage, "created_at": journal.created_at.isoformat() if journal.created_at else None, }, @@ -741,15 +752,15 @@ def set_goal(year): target = data.get("target_books") if target is None: - return jsonify({"success": False, "error": "target_books is required"}), 400 + return json_error("target_books is required", 400) try: target = int(target) except (TypeError, ValueError): - return jsonify({"success": False, "error": "target_books must be an integer"}), 400 + return json_error("target_books must be an integer", 400) if target < 1: - return jsonify({"success": False, "error": "target_books must be at least 1"}), 400 + return json_error("target_books must be at least 1", 400) goal = database_service.save_reading_goal(year, target) return jsonify({"success": True, "year": goal.year, "target_books": goal.target_books}) diff --git a/src/blueprints/tbr_bp.py b/src/blueprints/tbr_bp.py index 0abd22e..6bd7f21 100644 --- a/src/blueprints/tbr_bp.py +++ b/src/blueprints/tbr_bp.py @@ -5,6 +5,8 @@ from flask import Blueprint, jsonify, request +from src.utils.http import json_error + from src.blueprints.helpers import get_container, get_database_service from src.services.hardcover_service import HC_IGNORED, HC_WANT_TO_READ @@ -120,11 +122,11 @@ def add_tbr_from_library(): data = request.json or {} book_ref = (data.get("abs_id") or data.get("book_ref") or "").strip() if not book_ref: - return jsonify({"success": False, "error": "Book reference is required"}), 400 + return json_error("Book reference is required", 400) book = database_service.get_book_by_ref(book_ref) if not book: - return jsonify({"success": False, "error": "Book not found"}), 404 + return json_error("Book not found", 404) # Dedup: if TBR item already linked to this book, return it existing = database_service.find_tbr_by_book_id(book.id) @@ -179,7 +181,7 @@ def add_tbr_item(): title = (data.get("title") or "").strip() if not title: - return jsonify({"success": False, "error": "Title is required"}), 400 + return json_error("Title is required", 400) # Determine source from which fields are present source = "manual" @@ -262,7 +264,7 @@ def delete_tbr_item(item_id): database_service = get_database_service() item = database_service.get_tbr_item(item_id) if not item: - return jsonify({"success": False, "error": "Item not found"}), 404 + return json_error("Item not found", 404) # Optionally push "Ignored" status to Hardcover before deleting locally hc_removed = False @@ -290,11 +292,11 @@ def update_tbr_item(item_id): database_service = get_database_service() item = database_service.get_tbr_item(item_id) if not item: - return jsonify({"success": False, "error": "Item not found"}), 404 + return json_error("Item not found", 404) data = request.json or {} if not data: - return jsonify({"success": False, "error": "No fields to update"}), 400 + return json_error("No fields to update", 400) allowed = { "notes", @@ -316,18 +318,18 @@ def update_tbr_item(item_id): updates[key] = value if not updates: - return jsonify({"success": False, "error": "No valid fields to update"}), 400 + return json_error("No valid fields to update", 400) # Dedupe check: prevent reassigning hardcover_book_id to a duplicate new_hc_id = updates.get("hardcover_book_id") if new_hc_id and str(new_hc_id) != str(item.hardcover_book_id or ""): existing = database_service.find_tbr_by_hardcover_id(new_hc_id) if existing and existing.id != item_id: - return jsonify({"success": False, "error": "Another TBR item already has this Hardcover ID"}), 409 + return json_error("Another TBR item already has this Hardcover ID", 409) updated = database_service.update_tbr_item(item_id, **updates) if not updated: - return jsonify({"success": False, "error": "Update failed"}), 500 + return json_error("Update failed", 500) return jsonify({"success": True, "item": _serialize_tbr_item(updated)}) @@ -338,9 +340,9 @@ def start_tbr_item(item_id): database_service = get_database_service() item = database_service.get_tbr_item(item_id) if not item: - return jsonify({"success": False, "error": "TBR item not found"}), 404 + return json_error("TBR item not found", 404) if not item.book_id and not item.book_abs_id: - return jsonify({"success": False, "error": "Book not in library — cannot start reading"}), 400 + return json_error("Book not in library — cannot start reading", 400) book = ( database_service.get_book_by_id(item.book_id) @@ -348,7 +350,7 @@ def start_tbr_item(item_id): else database_service.get_book_by_abs_id(item.book_abs_id) ) if not book: - return jsonify({"success": False, "error": "Linked book not found"}), 404 + return json_error("Linked book not found", 404) # Transition book to active book.status = "active" @@ -455,9 +457,9 @@ def import_hardcover_wtr(): try: hc_client = get_container().hardcover_client() if not hc_client.is_configured(): - return jsonify({"success": False, "error": "Hardcover not configured"}), 400 + return json_error("Hardcover not configured", 400) except Exception: - return jsonify({"success": False, "error": "Hardcover not available"}), 400 + return json_error("Hardcover not available", 400) database_service = get_database_service() wtr_books = hc_client.get_want_to_read_books() @@ -541,24 +543,24 @@ def import_hardcover_list(): data = request.json or {} list_id = data.get("list_id") if not list_id: - return jsonify({"success": False, "error": "list_id is required"}), 400 + return json_error("list_id is required", 400) try: list_id = int(list_id) except (TypeError, ValueError): - return jsonify({"success": False, "error": "Invalid list_id"}), 400 + return json_error("Invalid list_id", 400) try: hc_client = get_container().hardcover_client() if not hc_client.is_configured(): - return jsonify({"success": False, "error": "Hardcover not configured"}), 400 + return json_error("Hardcover not configured", 400) except Exception: - return jsonify({"success": False, "error": "Hardcover not available"}), 400 + return json_error("Hardcover not available", 400) database_service = get_database_service() list_data = hc_client.get_list_books(list_id) if not list_data: - return jsonify({"success": False, "error": "List not found"}), 404 + return json_error("List not found", 404) list_name = list_data.get("name", "") @@ -651,15 +653,15 @@ def link_tbr_to_library(item_id): data = request.json or {} book_ref = (data.get("abs_id") or data.get("book_ref") or "").strip() if not book_ref: - return jsonify({"success": False, "error": "Book reference is required"}), 400 + return json_error("Book reference is required", 400) book = database_service.get_book_by_ref(book_ref) if not book: - return jsonify({"success": False, "error": "Book not found in library"}), 404 + return json_error("Book not found in library", 404) updated = database_service.link_tbr_to_book(item_id, book.id) if not updated: - return jsonify({"success": False, "error": "TBR item not found"}), 404 + return json_error("TBR item not found", 404) return jsonify({"success": True}) @@ -670,6 +672,6 @@ def unlink_tbr_from_library(item_id): database_service = get_database_service() updated = database_service.link_tbr_to_book(item_id, None) if not updated: - return jsonify({"success": False, "error": "TBR item not found"}), 404 + return json_error("TBR item not found", 404) return jsonify({"success": True}) diff --git a/src/db/base_repository.py b/src/db/base_repository.py index 35e32ea..38a249a 100644 --- a/src/db/base_repository.py +++ b/src/db/base_repository.py @@ -1,3 +1,5 @@ +# pyright: reportMissingImports=false + """Base repository with shared query helpers to reduce boilerplate.""" import logging @@ -30,13 +32,32 @@ def get_session(self): finally: session.close() - def _get_one(self, model, *filters): - """Query a single row, expunge and return it (or None).""" - with self.get_session() as session: - obj = session.query(model).filter(*filters).first() + def _expunge_items(self, session, items): + for item in items: + session.expunge(item) + return items + + def _query_and_expunge(self, session, query, first=False): + if first: + obj = query.first() if obj: session.expunge(obj) return obj + items = query.all() + return self._expunge_items(session, items) + + def _paginate(self, query, page=1, per_page=50): + safe_page = max(1, int(page)) + safe_per_page = max(1, int(per_page)) + total = query.count() + items = query.offset((safe_page - 1) * safe_per_page).limit(safe_per_page).all() + return items, total + + def _get_one(self, model, *filters): + """Query a single row, expunge and return it (or None).""" + with self.get_session() as session: + query = session.query(model).filter(*filters) + return self._query_and_expunge(session, query, first=True) def _get_all(self, model, *filters, order_by=None): """Query multiple rows, expunge and return them.""" @@ -46,10 +67,7 @@ def _get_all(self, model, *filters, order_by=None): query = query.filter(*filters) if order_by is not None: query = query.order_by(order_by) - items = query.all() - for item in items: - session.expunge(item) - return items + return self._query_and_expunge(session, query) def _delete_one(self, model, *filters): """Find and delete a single row. Returns True if deleted.""" diff --git a/src/db/bookfusion_repository.py b/src/db/bookfusion_repository.py index fc8d3ba..8ce9da6 100644 --- a/src/db/bookfusion_repository.py +++ b/src/db/bookfusion_repository.py @@ -97,9 +97,14 @@ def get_unmatched_bookfusion_highlights(self): def link_bookfusion_highlights_by_book_id(self, bookfusion_book_id, book_id): """Link all highlights for a BookFusion book to a library book by book_id.""" + normalized_id = bookfusion_book_id + if normalized_id.startswith("book-"): + normalized_id = normalized_id[5:] with self.get_session() as session: session.query(BookfusionHighlight).filter( - BookfusionHighlight.bookfusion_book_id == bookfusion_book_id + (BookfusionHighlight.bookfusion_book_id == bookfusion_book_id) + | (BookfusionHighlight.bookfusion_book_id == normalized_id) + | (BookfusionHighlight.bookfusion_book_id == f"book-{normalized_id}") ).update({BookfusionHighlight.matched_book_id: book_id}, synchronize_session=False) def get_bookfusion_highlights_for_book_by_book_id(self, book_id): @@ -180,10 +185,22 @@ def get_bookfusion_book(self, bookfusion_id): def get_bookfusion_book_by_book_id(self, book_id): return self._get_one(BookfusionBook, BookfusionBook.matched_book_id == book_id) + def get_bookfusion_books_by_book_id(self, book_id): + """Get all BookFusion catalog entries linked to a specific PageKeeper book.""" + with self.get_session() as session: + books = session.query(BookfusionBook).filter(BookfusionBook.matched_book_id == book_id).all() + for b in books: + session.expunge(b) + return books + + def save_bookfusion_book(self, bookfusion_book): + """Save a single BookFusion catalog entry.""" + return self._save_new(bookfusion_book) + def unlink_bookfusion_by_book_id(self, book_id): with self.get_session() as session: session.query(BookfusionBook).filter(BookfusionBook.matched_book_id == book_id).update( - {BookfusionBook.matched_book_id: None, BookfusionBook.matched_abs_id: None}, synchronize_session=False + {BookfusionBook.matched_book_id: None}, synchronize_session=False ) session.query(BookfusionHighlight).filter(BookfusionHighlight.matched_book_id == book_id).update( {BookfusionHighlight.matched_book_id: None}, synchronize_session=False diff --git a/src/db/database_service.py b/src/db/database_service.py index 8c4ccb4..f5b2961 100644 --- a/src/db/database_service.py +++ b/src/db/database_service.py @@ -1,3 +1,5 @@ +# pyright: reportMissingImports=false + """ Unified SQLAlchemy database service for PageKeeper. Facade that delegates to domain-specific repositories. @@ -7,6 +9,7 @@ import logging from contextlib import contextmanager from pathlib import Path +from typing import Any, cast from .book_repository import BookRepository from .bookfusion_repository import BookFusionRepository @@ -84,8 +87,7 @@ def _run_alembic_migrations(self): try: from alembic.config import Config from alembic.runtime.migration import MigrationContext - - from alembic import command + import alembic.command as alembic_command alembic_dir = Path(__file__).parent.parent.parent / "alembic" alembic_ini = alembic_dir.parent / "alembic.ini" @@ -119,8 +121,8 @@ def _run_alembic_migrations(self): "before upgrading to head", LEGACY_BASELINE_REVISION, ) - command.stamp(alembic_cfg, LEGACY_BASELINE_REVISION) - command.upgrade(alembic_cfg, "head") + alembic_command.stamp(alembic_cfg, LEGACY_BASELINE_REVISION) + alembic_command.upgrade(alembic_cfg, "head") else: # Unknown populated schema with no revision history. We keep the # previous safety behavior here because we cannot infer a correct @@ -129,7 +131,7 @@ def _run_alembic_migrations(self): "Populated database has no alembic_version table but does not " "match the known pre-Alembic schema; stamping at head" ) - command.stamp(alembic_cfg, "head") + alembic_command.stamp(alembic_cfg, "head") else: # Normal migration path with engine.connect() as conn: @@ -139,9 +141,9 @@ def _run_alembic_migrations(self): if current_rev is None and not tables: # Fresh database — will be created by create_all, then stamp Base.metadata.create_all(engine) - command.stamp(alembic_cfg, "head") + alembic_command.stamp(alembic_cfg, "head") else: - command.upgrade(alembic_cfg, "head") + alembic_command.upgrade(alembic_cfg, "head") logger.debug("Alembic migrations completed successfully") finally: @@ -256,7 +258,7 @@ def _cleanup_bookfusion_md_titles(self): from .models import BookfusionBook with self.get_session() as session: - dirty = session.query(BookfusionBook).filter(BookfusionBook.title.like("%.md")).all() + dirty = session.query(BookfusionBook).filter(cast(Any, BookfusionBook.title).like("%.md")).all() for b in dirty: stripped = b.title[:-3].strip() b.title = stripped if stripped else b.title @@ -297,6 +299,15 @@ def get_session(self): "_tbr", ) + def _delegated_method_names(self): + names = set() + for repo_name in DatabaseService._REPOS: + repo = object.__getattribute__(self, repo_name) + for attr in dir(repo): + if not attr.startswith("_"): + names.add(attr) + return names + def __getattr__(self, name): if name.startswith("_"): raise AttributeError(name) @@ -307,6 +318,9 @@ def __getattr__(self, name): return method raise AttributeError(f"'{type(self).__name__}' has no attribute '{name}'") + def __dir__(self): + return sorted(set(super().__dir__()) | self._delegated_method_names()) + # ── Bulk data helpers (avoid N+1 in views) ── def get_states_by_book(self): diff --git a/src/db/hardcover_repository.py b/src/db/hardcover_repository.py index 635bcd9..834f12f 100644 --- a/src/db/hardcover_repository.py +++ b/src/db/hardcover_repository.py @@ -51,8 +51,6 @@ def add_hardcover_sync_log(self, entry): return self._save_new(entry) def get_hardcover_sync_logs(self, page=1, per_page=50, direction=None, action=None, search=None): - safe_page = max(1, int(page)) - safe_per_page = max(1, int(per_page)) with self.get_session() as session: query = session.query(HardcoverSyncLog) if direction: @@ -66,16 +64,8 @@ def get_hardcover_sync_logs(self, page=1, per_page=50, direction=None, action=No | (HardcoverSyncLog.detail.ilike(like)) | (HardcoverSyncLog.error_message.ilike(like)) ) - total = query.count() - items = ( - query.order_by(HardcoverSyncLog.created_at.desc()) - .offset((safe_page - 1) * safe_per_page) - .limit(safe_per_page) - .all() - ) - for item in items: - session.expunge(item) - return items, total + items, total = self._paginate(query.order_by(HardcoverSyncLog.created_at.desc()), page=page, per_page=per_page) + return self._expunge_items(session, items), total def prune_hardcover_sync_logs(self, before_date): with self.get_session() as session: diff --git a/src/db/models.py b/src/db/models.py index f2ae99d..e5997b8 100644 --- a/src/db/models.py +++ b/src/db/models.py @@ -2,7 +2,7 @@ SQLAlchemy ORM models for PageKeeper database. """ -from datetime import datetime +from datetime import UTC, datetime import sqlalchemy as sa from sqlalchemy import ( @@ -18,12 +18,15 @@ UniqueConstraint, create_engine, ) -from sqlalchemy.ext.declarative import declarative_base -from sqlalchemy.orm import relationship, sessionmaker +from sqlalchemy.orm import declarative_base, relationship, sessionmaker Base = declarative_base() +def utc_now(): + return datetime.now(UTC) + + class KosyncDocument(Base): """ Model for raw KOSync documents (mirroring the official server's schema). @@ -42,8 +45,8 @@ class KosyncDocument(Base): # Bridge specific fields — linked_book_id is the canonical FK linked_abs_id = Column(String(255), nullable=True, index=True) linked_book_id = Column(Integer, ForeignKey("books.id"), nullable=True, index=True) - first_seen = Column(DateTime, default=datetime.utcnow) - last_updated = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) + first_seen = Column(DateTime, default=utc_now) + last_updated = Column(DateTime, default=utc_now, onupdate=utc_now) # Hash cache replacement fields filename = Column(String(500), nullable=True) @@ -81,8 +84,8 @@ def __init__( self.source = source self.grimmory_id = grimmory_id self.mtime = mtime - self.first_seen = datetime.utcnow() - self.last_updated = datetime.utcnow() + self.first_seen = utc_now() + self.last_updated = utc_now() def __repr__(self): return f"" @@ -197,7 +200,7 @@ class ReadingJournal(Base): event = Column(String(20), nullable=False) # started|progress|finished|note|paused|resumed|dnf entry = Column(Text, nullable=True) # freeform text note percentage = Column(Float, nullable=True) # progress at time of entry - created_at = Column(DateTime, default=datetime.utcnow) + created_at = Column(DateTime, default=utc_now) book = relationship("Book", back_populates="reading_journals", foreign_keys=[book_id]) @@ -215,7 +218,7 @@ def __init__( self.event = event self.entry = entry self.percentage = percentage - self.created_at = created_at or datetime.utcnow() + self.created_at = created_at or utc_now() def __repr__(self): return f"" @@ -319,7 +322,7 @@ class HardcoverSyncLog(Base): detail = Column(Text, nullable=True) # JSON blob success = Column(Boolean, default=True) error_message = Column(Text, nullable=True) - created_at = Column(DateTime, default=datetime.utcnow, index=True) + created_at = Column(DateTime, default=utc_now, index=True) book = relationship("Book", backref="hardcover_sync_logs", foreign_keys=[book_id]) @@ -343,7 +346,7 @@ def __init__( self.detail = detail self.success = success self.error_message = error_message - self.created_at = created_at or datetime.utcnow() + self.created_at = created_at or utc_now() def __repr__(self): return f"" @@ -361,7 +364,7 @@ class StorytellerSubmission(Base): submission_dir = Column(String(500), nullable=True) storyteller_uuid = Column(String(36), nullable=True) error = Column(Text, nullable=True) - submitted_at = Column(DateTime, default=datetime.utcnow) + submitted_at = Column(DateTime, default=utc_now) last_checked_at = Column(DateTime, nullable=True) book = relationship("Book", backref="storyteller_submissions", foreign_keys=[book_id]) @@ -381,7 +384,7 @@ def __init__( self.submission_dir = submission_dir self.storyteller_uuid = storyteller_uuid self.error = error - self.submitted_at = datetime.utcnow() + self.submitted_at = utc_now() def __repr__(self): return f"" @@ -484,7 +487,7 @@ class PendingSuggestion(Base): cover_url = Column(String(500)) matches_json = Column(Text) status = Column(String(20), default="pending") - created_at = Column(DateTime, default=datetime.utcnow) + created_at = Column(DateTime, default=utc_now) def __init__( self, @@ -503,7 +506,7 @@ def __init__( self.cover_url = cover_url self.matches_json = matches_json self.status = status - self.created_at = datetime.utcnow() + self.created_at = utc_now() @property def matches(self): @@ -549,7 +552,7 @@ class BookAlignment(Base): abs_id = Column(String(255), nullable=True) alignment_map_json = Column(Text, nullable=False) # JSON-encoded list of dicts or optimized structure source = Column(String(20), nullable=True) # storyteller, smil, whisper - last_updated = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) + last_updated = Column(DateTime, default=utc_now, onupdate=utc_now) # Relationship book = relationship("Book", back_populates="alignment", foreign_keys=[book_id]) @@ -572,10 +575,9 @@ class BookfusionHighlight(Base): book_title = Column(String(500)) content = Column(Text, nullable=False) chapter_heading = Column(String(500)) - fetched_at = Column(DateTime, default=datetime.utcnow) + fetched_at = Column(DateTime, default=utc_now) highlighted_at = Column(DateTime, nullable=True) quote_text = Column(Text, nullable=True) - matched_abs_id = Column(String(255), nullable=True) matched_book_id = Column(Integer, ForeignKey("books.id", ondelete="SET NULL"), nullable=True, index=True) matched_book = relationship("Book", foreign_keys=[matched_book_id]) @@ -589,7 +591,6 @@ def __init__( chapter_heading: str = None, highlighted_at=None, quote_text: str = None, - matched_abs_id: str = None, matched_book_id: int = None, ): self.bookfusion_book_id = bookfusion_book_id @@ -597,10 +598,9 @@ def __init__( self.content = content self.book_title = book_title self.chapter_heading = chapter_heading - self.fetched_at = datetime.utcnow() + self.fetched_at = utc_now() self.highlighted_at = highlighted_at self.quote_text = quote_text - self.matched_abs_id = matched_abs_id self.matched_book_id = matched_book_id def __repr__(self): @@ -621,11 +621,10 @@ class BookfusionBook(Base): tags = Column(String(500)) series = Column(String(500)) highlight_count = Column(Integer, default=0, nullable=False, server_default="0") - matched_abs_id = Column(String(255), nullable=True) matched_book_id = Column(Integer, ForeignKey("books.id", ondelete="SET NULL"), nullable=True, index=True) hidden = Column(Boolean, default=False, nullable=False, server_default="0") - fetched_at = Column(DateTime, default=datetime.utcnow) - last_updated = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) + fetched_at = Column(DateTime, default=utc_now) + last_updated = Column(DateTime, default=utc_now, onupdate=utc_now) matched_book = relationship("Book", foreign_keys=[matched_book_id]) @@ -639,7 +638,6 @@ def __init__( tags: str = None, series: str = None, highlight_count: int = 0, - matched_abs_id: str = None, matched_book_id: int = None, hidden: bool = False, ): @@ -651,11 +649,10 @@ def __init__( self.tags = tags self.series = series self.highlight_count = highlight_count - self.matched_abs_id = matched_abs_id self.matched_book_id = matched_book_id self.hidden = hidden - self.fetched_at = datetime.utcnow() - self.last_updated = datetime.utcnow() + self.fetched_at = utc_now() + self.last_updated = utc_now() def __repr__(self): return f"" @@ -673,7 +670,7 @@ class TbrItem(Base): cover_url = Column(String(500), nullable=True) notes = Column(Text, nullable=True) priority = Column(Integer, default=0) - added_at = Column(DateTime, default=datetime.utcnow) + added_at = Column(DateTime, default=utc_now) # Hardcover link (if sourced from or synced to HC) hardcover_book_id = Column(Integer, nullable=True, index=True) @@ -749,7 +746,7 @@ def __init__( self.release_year = release_year self.genres = genres self.subtitle = subtitle - self.added_at = datetime.utcnow() + self.added_at = utc_now() def __repr__(self): return f"" @@ -768,7 +765,7 @@ class GrimmoryBook(Base): title = Column(String(500)) authors = Column(String(500)) raw_metadata = Column(Text) - last_updated = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow) + last_updated = Column(DateTime, default=utc_now, onupdate=utc_now) __table_args__ = (UniqueConstraint("server_id", "filename", name="uq_grimmory_server_filename"),) diff --git a/src/db/reading_repository.py b/src/db/reading_repository.py index aacae23..cc88893 100644 --- a/src/db/reading_repository.py +++ b/src/db/reading_repository.py @@ -40,6 +40,17 @@ def get_reading_journals(self, book_id): order_by=ReadingJournal.created_at.desc(), ) + def get_reading_journal_entries_for_book(self, book_id, event=None): + """Get all journal entries for a book, optionally filtered by event type.""" + with self.get_session() as session: + query = session.query(ReadingJournal).filter(ReadingJournal.book_id == book_id) + if event: + query = query.filter(ReadingJournal.event == event) + journals = query.order_by(ReadingJournal.created_at.desc()).all() + for j in journals: + session.expunge(j) + return journals + def get_reading_journal(self, journal_id): return self._get_one(ReadingJournal, ReadingJournal.id == journal_id) diff --git a/src/services/alignment_service.py b/src/services/alignment_service.py index 275d5a1..4342614 100644 --- a/src/services/alignment_service.py +++ b/src/services/alignment_service.py @@ -7,7 +7,7 @@ import json import logging import re -from datetime import datetime +from datetime import UTC, datetime from src.db.models import Book, BookAlignment, Job from src.utils.logging_utils import sanitize_exception, time_execution @@ -634,7 +634,7 @@ def _save_alignment(self, book_id: int, alignment_map: list[dict], source: str = existing = session.query(BookAlignment).filter_by(book_id=book_id).first() if existing: existing.alignment_map_json = json_blob - existing.last_updated = datetime.utcnow() + existing.last_updated = datetime.now(UTC) if source: existing.source = source else: diff --git a/src/services/cache_cleanup_service.py b/src/services/cache_cleanup_service.py new file mode 100644 index 0000000..83fc853 --- /dev/null +++ b/src/services/cache_cleanup_service.py @@ -0,0 +1,49 @@ +import logging + +logger = logging.getLogger(__name__) + + +class CacheCleanupService: + """Delete orphaned EPUB cache files not referenced by books or suggestions.""" + + def __init__(self, database_service, epub_cache_dir): + self.database_service = database_service + self.epub_cache_dir = epub_cache_dir + + def cleanup(self): + if not self.epub_cache_dir.exists(): + return + + logger.info("Starting ebook cache cleanup...") + try: + valid_filenames = set() + + for book in self.database_service.get_all_books(): + if book.ebook_filename: + valid_filenames.add(book.ebook_filename) + + for suggestion in self.database_service.get_all_actionable_suggestions(): + for match in suggestion.matches: + if match.get("filename"): + valid_filenames.add(match["filename"]) + + deleted_count = 0 + reclaimed_bytes = 0 + for file_path in self.epub_cache_dir.iterdir(): + if file_path.is_file() and file_path.name not in valid_filenames: + try: + size = file_path.stat().st_size + file_path.unlink() + deleted_count += 1 + reclaimed_bytes += size + logger.debug(" Deleted orphaned cache file: %s", file_path.name) + except Exception as exc: + logger.warning(" Failed to delete %s: %s", file_path.name, exc) + + if deleted_count > 0: + mb = reclaimed_bytes / (1024 * 1024) + logger.info("Cache cleanup complete: Removed %s files (%.2f MB)", deleted_count, mb) + else: + logger.info("Cache is clean (no orphaned files found)") + except Exception as exc: + logger.error("Error during cache cleanup: %s", exc) diff --git a/src/services/kosync_progress_service.py b/src/services/kosync_progress_service.py new file mode 100644 index 0000000..b24795e --- /dev/null +++ b/src/services/kosync_progress_service.py @@ -0,0 +1,227 @@ +import calendar +import logging +import os +import threading +import time +from datetime import UTC, datetime + +from src.db.models import KosyncDocument +from src.utils.constants import INTERNAL_DEVICE_NAMES + +logger = logging.getLogger(__name__) + + +class KosyncProgressService: + """KoSync GET/PUT progress flow extracted from KosyncService.""" + + def __init__(self, service): + self.service = service + self.db = service._db + self.container = service._container + self.manager = service._manager + + def handle_put_progress(self, data, remote_addr, debounce_manager=None): + if not data: + logger.warning("KOSync: PUT progress with no JSON data from %s", remote_addr) + return {"error": "No data"}, 400 + + doc_hash = data.get("document") + if not doc_hash or not isinstance(doc_hash, str): + logger.warning("KOSync: PUT progress with no document ID from %s", remote_addr) + return {"error": "Missing document ID"}, 400 + if len(doc_hash) > 64: + return {"error": "Document hash too long"}, 400 + + percentage = data.get("percentage", 0) + try: + percentage = float(percentage) + except (TypeError, ValueError): + return {"error": "Invalid percentage value"}, 400 + if percentage < 0.0 or percentage > 1.0: + return {"error": "Percentage must be between 0.0 and 1.0"}, 400 + + logger.info( + "KOSync: PUT progress request for doc %s... from %s (device: %s)", + doc_hash[:8], + remote_addr, + data.get("device", "unknown"), + ) + + progress = str(data.get("progress", ""))[:512] + device = str(data.get("device", ""))[:128] + device_id = str(data.get("device_id", ""))[:64] + now = datetime.now(UTC) + + kosync_doc = self.db.get_kosync_document(doc_hash) + furthest_wins = os.environ.get("KOSYNC_FURTHEST_WINS", "true").lower() == "true" + force_update = data.get("force", False) + same_device = kosync_doc and kosync_doc.device_id == device_id + + if furthest_wins and kosync_doc and kosync_doc.percentage and not force_update and not same_device: + existing_pct = float(kosync_doc.percentage) + new_pct = float(percentage) + if new_pct < existing_pct - 0.0001: + logger.info( + "KOSync: Ignored progress from '%s' for doc %s... (server has higher: %.2f%% vs new %.2f%%)", + device, + doc_hash[:8], + existing_pct, + new_pct, + ) + return { + "document": doc_hash, + "timestamp": int(kosync_doc.timestamp.timestamp()) if kosync_doc.timestamp else int(now.timestamp()), + }, 200 + + if kosync_doc is None: + kosync_doc = KosyncDocument( + document_hash=doc_hash, + progress=progress, + percentage=percentage, + device=device, + device_id=device_id, + timestamp=now, + ) + logger.info("KOSync: New document tracked: %s... from device '%s'", doc_hash[:8], device) + else: + logger.info( + "KOSync: Received progress from '%s' for doc %s... -> %.2f%% (Updated from %.2f%%)", + device, + doc_hash[:8], + float(percentage), + float(kosync_doc.percentage) if kosync_doc.percentage else 0, + ) + kosync_doc.progress = progress + kosync_doc.percentage = percentage + kosync_doc.device = device + kosync_doc.device_id = device_id + kosync_doc.timestamp = now + + self.db.save_kosync_document(kosync_doc) + linked_book = self._resolve_linked_book(doc_hash, kosync_doc) + + if not linked_book: + self._handle_unlinked_document(doc_hash, kosync_doc, device) + + if linked_book: + if linked_book.status in ("paused", "dnf", "not_started") and not linked_book.activity_flag: + linked_book.activity_flag = True + self.db.save_book(linked_book) + logger.info("KOSync PUT: Activity detected on %s book '%s'", linked_book.status, linked_book.title) + + logger.debug("KOSync: Updated linked book '%s' to %.2f%%", linked_book.title, percentage) + is_internal = device and device.lower() in INTERNAL_DEVICE_NAMES + instant_sync_enabled = os.environ.get("INSTANT_SYNC_ENABLED", "true").lower() != "false" + if linked_book.status == "active" and self.manager and not is_internal and instant_sync_enabled and debounce_manager: + logger.debug("KOSync PUT: Progress event recorded for '%s'", linked_book.title) + debounce_manager.record_event(linked_book.id, linked_book.title) + + response_timestamp = now.isoformat() + if device and device.lower() == "booknexus": + response_timestamp = int(calendar.timegm(now.timetuple())) + + return {"document": doc_hash, "timestamp": response_timestamp}, 200 + + def handle_get_progress(self, doc_id, remote_addr): + if len(doc_id) > 64: + return {"error": "Document ID too long"}, 400 + + logger.info("KOSync: GET progress for doc %s... from %s", doc_id[:8], remote_addr) + + kosync_doc = self.db.get_kosync_document(doc_id) + if kosync_doc: + if kosync_doc.linked_book_id: + book = self.db.get_book_by_id(kosync_doc.linked_book_id) + if book: + return self.resolve_best_progress(doc_id, book) + elif kosync_doc.linked_abs_id: + book = self.db.get_book_by_abs_id(kosync_doc.linked_abs_id) + if book: + return self.resolve_best_progress(doc_id, book) + + has_progress = kosync_doc.percentage and float(kosync_doc.percentage) > 0 + if has_progress: + return self.service.serialize_progress(kosync_doc, device_default=""), 200 + + book = self.db.get_book_by_kosync_id(doc_id) + if book: + return self.resolve_best_progress(doc_id, book) + + resolved_book = self.service.resolve_book_by_sibling_hash(doc_id, existing_doc=kosync_doc) + if resolved_book: + self.service.register_hash_for_book(doc_id, resolved_book) + return self.resolve_best_progress(doc_id, resolved_book) + + auto_create = os.environ.get("AUTO_CREATE_EBOOK_MAPPING", "true").lower() == "true" + if auto_create and self.service.start_discovery_if_available(doc_id): + stub = KosyncDocument(document_hash=doc_id) + self.db.save_kosync_document(stub) + logger.info("KOSync: Created stub for unknown hash %s..., starting background discovery", doc_id[:8]) + threading.Thread(target=self.service.run_get_auto_discovery, args=(doc_id,), daemon=True).start() + + logger.warning("KOSync: Document not found: %s... (GET from %s)", doc_id[:8], remote_addr) + return {"message": "Document not found on server"}, 502 + + def resolve_best_progress(self, doc_id, book): + states = self.db.get_states_for_book(book.id) + sibling_docs = self.db.get_kosync_documents_for_book_by_book_id(book.id) + now_ts = time.time() + docs_with_progress = [ + d + for d in sibling_docs + if d.percentage and float(d.percentage) > 0 and d.timestamp and (now_ts - d.timestamp.timestamp()) < 30 * 86400 + ] + if not docs_with_progress: + docs_with_progress = [d for d in sibling_docs if d.percentage and float(d.percentage) > 0 and d.timestamp] + if docs_with_progress: + best_doc = max(docs_with_progress, key=lambda d: float(d.percentage)) + logger.info( + "KOSync: Resolved %s... to '%s' via sibling hash %s... (%.2f%%)", + doc_id[:8], + book.title, + best_doc.document_hash[:8], + float(best_doc.percentage), + ) + return self.service.serialize_progress(best_doc, doc_id), 200 + + if not states: + return {"message": "Document not found on server"}, 502 + + kosync_state = next((s for s in states if s.client_name.lower() == "kosync"), None) + latest_state = kosync_state or max(states, key=lambda s: s.last_updated or datetime.min) + return { + "device": "pagekeeper", + "device_id": "pagekeeper", + "document": doc_id, + "percentage": float(latest_state.percentage) if latest_state.percentage else 0, + "progress": (latest_state.xpath or latest_state.cfi) if hasattr(latest_state, "xpath") else "", + "timestamp": int(latest_state.last_updated) if latest_state.last_updated else 0, + }, 200 + + def _resolve_linked_book(self, doc_hash, kosync_doc): + if kosync_doc.linked_book_id: + return self.db.get_book_by_id(kosync_doc.linked_book_id) + if kosync_doc.linked_abs_id: + return self.db.get_book_by_abs_id(kosync_doc.linked_abs_id) + + linked_book = self.db.get_book_by_kosync_id(doc_hash) + if linked_book: + self.db.link_kosync_document(doc_hash, linked_book.id, linked_book.abs_id) + return linked_book + + def _handle_unlinked_document(self, doc_hash, kosync_doc, device): + auto_create = os.environ.get("AUTO_CREATE_EBOOK_MAPPING", "true").lower() == "true" + discovery_started = auto_create and self.service.start_discovery_if_available(doc_hash) + if discovery_started: + threading.Thread(target=self.service.run_put_auto_discovery, args=(doc_hash,), daemon=True).start() + return + + try: + suggestion_svc = self.container.suggestion_service() + suggestion_svc.queue_kosync_suggestion( + doc_hash, + filename=kosync_doc.filename, + device=device, + ) + except Exception as exc: + logger.debug("KoSync suggestion attempt failed for %s...: %s", doc_hash[:8], exc) diff --git a/src/services/kosync_service.py b/src/services/kosync_service.py index 7f99199..0416d08 100644 --- a/src/services/kosync_service.py +++ b/src/services/kosync_service.py @@ -4,18 +4,16 @@ document management. Route handlers in kosync_server.py delegate here. """ -import calendar import json import logging import os import re import threading import time -from datetime import datetime from pathlib import Path from src.db.models import Book, KosyncDocument -from src.utils.constants import INTERNAL_DEVICE_NAMES +from src.services.kosync_progress_service import KosyncProgressService from src.utils.logging_utils import sanitize_log_data from src.utils.path_utils import is_safe_path_within @@ -73,6 +71,7 @@ def __init__(self, database_service, container, manager=None, ebook_dir=None): self._ebook_dir = ebook_dir self._active_scans = set() self._active_scans_lock = threading.Lock() + self._progress = KosyncProgressService(self) # ------------------------------------------------------------------ # # Progress serialization (was duplicated 3x in kosync_server.py) @@ -543,220 +542,14 @@ def clear_orphaned_hash(self, book_id): return book # ------------------------------------------------------------------ # - # HTTP handler logic (moved from kosync_server.py route handlers) + # HTTP handler logic (delegated to extracted progress service) # ------------------------------------------------------------------ # def handle_put_progress(self, data, remote_addr, debounce_manager=None): - """Process a KoSync PUT progress request. Returns (response_dict, status_code).""" - - if not data: - logger.warning(f"KOSync: PUT progress with no JSON data from {remote_addr}") - return {"error": "No data"}, 400 - - doc_hash = data.get("document") - if not doc_hash or not isinstance(doc_hash, str): - logger.warning(f"KOSync: PUT progress with no document ID from {remote_addr}") - return {"error": "Missing document ID"}, 400 - if len(doc_hash) > 64: - return {"error": "Document hash too long"}, 400 - - percentage = data.get("percentage", 0) - try: - percentage = float(percentage) - except (TypeError, ValueError): - return {"error": "Invalid percentage value"}, 400 - if percentage < 0.0 or percentage > 1.0: - return {"error": "Percentage must be between 0.0 and 1.0"}, 400 - - logger.info( - f"KOSync: PUT progress request for doc {doc_hash[:8]}... from {remote_addr} (device: {data.get('device', 'unknown')})" - ) - - progress = str(data.get("progress", ""))[:512] - device = str(data.get("device", ""))[:128] - device_id = str(data.get("device_id", ""))[:64] - - now = datetime.utcnow() - - kosync_doc = self._db.get_kosync_document(doc_hash) - - # Optional "furthest wins" protection - furthest_wins = os.environ.get("KOSYNC_FURTHEST_WINS", "true").lower() == "true" - force_update = data.get("force", False) - same_device = kosync_doc and kosync_doc.device_id == device_id - - if furthest_wins and kosync_doc and kosync_doc.percentage and not force_update and not same_device: - existing_pct = float(kosync_doc.percentage) - new_pct = float(percentage) - if new_pct < existing_pct - 0.0001: - logger.info( - f"KOSync: Ignored progress from '{device}' for doc {doc_hash[:8]}... (server has higher: {existing_pct:.2f}% vs new {new_pct:.2f}%)" - ) - return { - "document": doc_hash, - "timestamp": int(kosync_doc.timestamp.timestamp()) - if kosync_doc.timestamp - else int(now.timestamp()), - }, 200 - - if kosync_doc is None: - kosync_doc = KosyncDocument( - document_hash=doc_hash, - progress=progress, - percentage=percentage, - device=device, - device_id=device_id, - timestamp=now, - ) - logger.info(f"KOSync: New document tracked: {doc_hash[:8]}... from device '{device}'") - else: - logger.info( - f"KOSync: Received progress from '{device}' for doc {doc_hash[:8]}... -> {float(percentage):.2f}% (Updated from {float(kosync_doc.percentage) if kosync_doc.percentage else 0:.2f}%)" - ) - kosync_doc.progress = progress - kosync_doc.percentage = percentage - kosync_doc.device = device - kosync_doc.device_id = device_id - kosync_doc.timestamp = now - - self._db.save_kosync_document(kosync_doc) - - # Update linked book if exists - linked_book = None - if kosync_doc.linked_book_id: - linked_book = self._db.get_book_by_id(kosync_doc.linked_book_id) - elif kosync_doc.linked_abs_id: - linked_book = self._db.get_book_by_abs_id(kosync_doc.linked_abs_id) - else: - linked_book = self._db.get_book_by_kosync_id(doc_hash) - if linked_book: - self._db.link_kosync_document(doc_hash, linked_book.id, linked_book.abs_id) - - # AUTO-DISCOVERY + SUGGESTION - if not linked_book: - auto_create = os.environ.get("AUTO_CREATE_EBOOK_MAPPING", "true").lower() == "true" - discovery_started = auto_create and self.start_discovery_if_available(doc_hash) - if discovery_started: - threading.Thread(target=self.run_put_auto_discovery, args=(doc_hash,), daemon=True).start() - else: - # Auto-discovery disabled or slots full — try suggestion via title matching - try: - suggestion_svc = self._container.suggestion_service() - suggestion_svc.queue_kosync_suggestion( - doc_hash, - filename=kosync_doc.filename, - device=device, - ) - except Exception as e: - logger.debug(f"KoSync suggestion attempt failed for {doc_hash[:8]}...: {e}") - - if linked_book: - # Flag activity on paused/DNF books - if linked_book.status in ("paused", "dnf", "not_started") and not linked_book.activity_flag: - linked_book.activity_flag = True - self._db.save_book(linked_book) - logger.info(f"KOSync PUT: Activity detected on {linked_book.status} book '{linked_book.title}'") - - logger.debug(f"KOSync: Updated linked book '{linked_book.title}' to {percentage:.2%}") - - # Debounce sync trigger - is_internal = device and device.lower() in INTERNAL_DEVICE_NAMES - instant_sync_enabled = os.environ.get("INSTANT_SYNC_ENABLED", "true").lower() != "false" - if linked_book.status == "active" and self._manager and not is_internal and instant_sync_enabled: - if debounce_manager: - logger.debug(f"KOSync PUT: Progress event recorded for '{linked_book.title}'") - debounce_manager.record_event(linked_book.id, linked_book.title) - - response_timestamp = now.isoformat() + "Z" - if device and device.lower() == "booknexus": - response_timestamp = int(calendar.timegm(now.timetuple())) - - return {"document": doc_hash, "timestamp": response_timestamp}, 200 + return self._progress.handle_put_progress(data, remote_addr, debounce_manager) def handle_get_progress(self, doc_id, remote_addr): - """Process a KoSync GET progress request. Returns (response_dict, status_code).""" - - if len(doc_id) > 64: - return {"error": "Document ID too long"}, 400 - - logger.info(f"KOSync: GET progress for doc {doc_id[:8]}... from {remote_addr}") - - # Step 1: Direct hash lookup - kosync_doc = self._db.get_kosync_document(doc_id) - if kosync_doc: - if kosync_doc.linked_book_id: - book = self._db.get_book_by_id(kosync_doc.linked_book_id) - if book: - return self.resolve_best_progress(doc_id, book) - elif kosync_doc.linked_abs_id: - book = self._db.get_book_by_abs_id(kosync_doc.linked_abs_id) - if book: - return self.resolve_best_progress(doc_id, book) - - has_progress = kosync_doc.percentage and float(kosync_doc.percentage) > 0 - if has_progress: - return self.serialize_progress(kosync_doc, device_default=""), 200 - - # Step 2: Book lookup by kosync_doc_id - book = self._db.get_book_by_kosync_id(doc_id) - if book: - return self.resolve_best_progress(doc_id, book) - - # Step 3: Sibling hash resolution - resolved_book = self.resolve_book_by_sibling_hash(doc_id, existing_doc=kosync_doc) - if resolved_book: - self.register_hash_for_book(doc_id, resolved_book) - return self.resolve_best_progress(doc_id, resolved_book) - - # Step 4: Unknown hash — register stub and start background discovery - auto_create = os.environ.get("AUTO_CREATE_EBOOK_MAPPING", "true").lower() == "true" - if auto_create and self.start_discovery_if_available(doc_id): - stub = KosyncDocument(document_hash=doc_id) - self._db.save_kosync_document(stub) - logger.info(f"KOSync: Created stub for unknown hash {doc_id[:8]}..., starting background discovery") - threading.Thread(target=self.run_get_auto_discovery, args=(doc_id,), daemon=True).start() - - logger.warning(f"KOSync: Document not found: {doc_id[:8]}... (GET from {remote_addr})") - return {"message": "Document not found on server"}, 502 + return self._progress.handle_get_progress(doc_id, remote_addr) def resolve_best_progress(self, doc_id, book): - """Find the best progress data for a book across sibling docs and states. - - Returns (response_dict, status_code). - """ - - states = self._db.get_states_for_book(book.id) - - sibling_docs = self._db.get_kosync_documents_for_book_by_book_id(book.id) - now_ts = time.time() - docs_with_progress = [ - d - for d in sibling_docs - if d.percentage - and float(d.percentage) > 0 - and d.timestamp - and (now_ts - d.timestamp.timestamp()) < 30 * 86400 - ] - if not docs_with_progress: - docs_with_progress = [d for d in sibling_docs if d.percentage and float(d.percentage) > 0 and d.timestamp] - if docs_with_progress: - best_doc = max(docs_with_progress, key=lambda d: float(d.percentage)) - logger.info( - f"KOSync: Resolved {doc_id[:8]}... to '{book.title}' via sibling hash {best_doc.document_hash[:8]}... ({float(best_doc.percentage):.2%})" - ) - return self.serialize_progress(best_doc, doc_id), 200 - - if not states: - return {"message": "Document not found on server"}, 502 - - kosync_state = next((s for s in states if s.client_name.lower() == "kosync"), None) - latest_state = kosync_state or max(states, key=lambda s: s.last_updated or datetime.min) - - return { - "device": "pagekeeper", - "device_id": "pagekeeper", - "document": doc_id, - "percentage": float(latest_state.percentage) if latest_state.percentage else 0, - "progress": (latest_state.xpath or latest_state.cfi) if hasattr(latest_state, "xpath") else "", - "timestamp": int(latest_state.last_updated) if latest_state.last_updated else 0, - }, 200 + return self._progress.resolve_best_progress(doc_id, book) diff --git a/src/services/storyteller_submission_service.py b/src/services/storyteller_submission_service.py index 92c7cb2..e19c50c 100644 --- a/src/services/storyteller_submission_service.py +++ b/src/services/storyteller_submission_service.py @@ -179,8 +179,10 @@ def check_status(self, abs_id: str) -> str: # Timeout: if submission has been non-terminal for too long, mark failed max_wait = int(os.environ.get("STORYTELLER_MAX_WAIT_HOURS", "12")) if submission.submitted_at: - # submitted_at is naive UTC (from datetime.utcnow() in the model) - elapsed = (datetime.utcnow() - submission.submitted_at).total_seconds() / 3600 + submitted_at = submission.submitted_at + if submitted_at.tzinfo is None: + submitted_at = submitted_at.replace(tzinfo=UTC) + elapsed = (datetime.now(UTC) - submitted_at).total_seconds() / 3600 if elapsed > max_wait: logger.warning( f"Storyteller submission timed out after {elapsed:.1f}h for abs_id={abs_id} " diff --git a/src/services/sync_manager_startup.py b/src/services/sync_manager_startup.py new file mode 100644 index 0000000..de1945b --- /dev/null +++ b/src/services/sync_manager_startup.py @@ -0,0 +1,72 @@ +import logging +import time + +from src.utils.logging_utils import sanitize_exception + +logger = logging.getLogger(__name__) + + +class SyncManagerStartup: + """Startup checks and one-time initialization for SyncManager.""" + + def __init__(self, sync_clients, library_service, abs_client, migration_service): + self.sync_clients = sync_clients + self.library_service = library_service + self.abs_client = abs_client + self.migration_service = migration_service + + def run(self): + for client_name, client in (self.sync_clients or {}).items(): + first_err = RuntimeError("check_connection() returned False") + try: + if client.check_connection(): + logger.info("'%s' connection verified", client_name) + continue + except Exception as err: + first_err = err + + time.sleep(2) + try: + if client.check_connection(): + logger.info("'%s' connection verified (retry)", client_name) + else: + raise RuntimeError("check_connection() returned False") + except Exception as exc: + logger.warning( + "'%s' connection failed after retry: %s (first attempt: %s)", + client_name, + sanitize_exception(exc), + sanitize_exception(first_err), + ) + + if self.library_service and self.library_service.cwa_client: + cwa = self.library_service.cwa_client + if cwa.is_configured(): + if cwa.check_connection(): + template = cwa._get_search_template() + if template: + logger.info(" CWA search template: %s", template) + else: + logger.debug("CWA not configured (disabled or missing server URL)") + else: + logger.debug("CWA not available (library_service or cwa_client missing)") + + if self.abs_client and self.abs_client.is_configured(): + try: + if hasattr(self.abs_client, "get_ebook_files") and hasattr(self.abs_client, "search_ebooks"): + logger.info("ABS ebook methods available (get_ebook_files, search_ebooks)") + else: + logger.warning("ABS ebook methods missing - ebook search may not work") + except Exception as exc: + logger.warning("ABS ebook check failed: %s", sanitize_exception(exc)) + + if self.migration_service: + logger.info("Checking for legacy data to migrate...") + self.migration_service.migrate_legacy_data() + + hc_client = self.sync_clients.get("Hardcover") if self.sync_clients else None + if hc_client and getattr(hc_client, "hardcover_service", None): + try: + hc_client.hardcover_service.backfill_hardcover_states() + except Exception as exc: + logger.warning("Hardcover state backfill failed (non-fatal): %s", sanitize_exception(exc)) diff --git a/src/sync_manager.py b/src/sync_manager.py index 7cb72cf..09f5261 100644 --- a/src/sync_manager.py +++ b/src/sync_manager.py @@ -6,16 +6,16 @@ from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path -import schedule - from src.api.storyteller_api import StorytellerAPIClient from src.db.models import State from src.services.alignment_service import AlignmentService, normalize_for_cross_format_comparison from src.services.background_job_service import BackgroundJobService +from src.services.cache_cleanup_service import CacheCleanupService from src.services.library_service import LibraryService from src.services.migration_service import MigrationService from src.services.progress_reset_service import ProgressResetService from src.services.suggestion_service import SuggestionService +from src.services.sync_manager_startup import SyncManagerStartup from src.sync_clients.sync_client_interface import ( LocatorResult, SyncClient, @@ -127,7 +127,7 @@ def __init__( self._pending_clears_lock = threading.Lock() self._last_library_sync = 0 - self._setup_sync_clients(sync_clients) + self._setup_sync_clients(sync_clients or {}) # ProgressResetService created internally (shares lock references with sync cycle) self.progress_reset_service = ProgressResetService( @@ -139,6 +139,14 @@ def __init__( pending_clears_lock=self._pending_clears_lock, ) + self._startup = SyncManagerStartup( + sync_clients=self.sync_clients, + library_service=self.library_service, + abs_client=self.abs_client, + migration_service=self.migration_service, + ) + self._cache_cleanup = CacheCleanupService(self.database_service, self.epub_cache_dir) + self.startup_checks() self.background_job_service.cleanup_stale_jobs() self.background_job_service.prune_hardcover_sync_logs() @@ -153,69 +161,7 @@ def _setup_sync_clients(self, clients: dict[str, SyncClient]): logger.debug(f"Sync client disabled/unconfigured: '{name}'") def startup_checks(self): - # Check configured sync clients - for client_name, client in (self.sync_clients or {}).items(): - try: - if client.check_connection(): - logger.info(f"'{client_name}' connection verified") - continue - first_err = RuntimeError("check_connection() returned False") - except Exception as first_err: - pass - - time.sleep(2) - try: - if client.check_connection(): - logger.info(f"'{client_name}' connection verified (retry)") - else: - raise RuntimeError("check_connection() returned False") - except Exception as e: - logger.warning( - f"'{client_name}' connection failed after retry: {sanitize_exception(e)} (first attempt: {sanitize_exception(first_err)})" - ) - - # Check CWA Integration Status - if self.library_service and self.library_service.cwa_client: - cwa = self.library_service.cwa_client - if cwa.is_configured(): - # check_connection() logs its own Success/Fail messages and verifies Authentication - if cwa.check_connection(): - # If connected, ensure search template is cached - template = cwa._get_search_template() - if template: - logger.info(f" CWA search template: {template}") - else: - logger.debug("CWA not configured (disabled or missing server URL)") - else: - logger.debug("CWA not available (library_service or cwa_client missing)") - - # Check ABS ebook search capability - if self.abs_client and self.abs_client.is_configured(): - try: - # Just verify methods exist (don't actually search during startup) - if hasattr(self.abs_client, "get_ebook_files") and hasattr(self.abs_client, "search_ebooks"): - logger.info("ABS ebook methods available (get_ebook_files, search_ebooks)") - else: - logger.warning("ABS ebook methods missing - ebook search may not work") - except Exception as e: - logger.warning(f"ABS ebook check failed: {sanitize_exception(e)}") - - # Run one-time migration - if self.migration_service: - logger.info("Checking for legacy data to migrate...") - self.migration_service.migrate_legacy_data() - - # Backfill Hardcover state records for linked books missing them - hc_client = self.sync_clients.get("Hardcover") - if hc_client and getattr(hc_client, "hardcover_service", None): - try: - hc_client.hardcover_service.backfill_hardcover_states() - except Exception as e: - logger.warning(f"Hardcover state backfill failed (non-fatal): {sanitize_exception(e)}") - - # Cleanup orphaned cache files - # DISABLED: Current logic is too aggressive (deletes original_ebook_filename for linked books). - # We rely on delete_mapping in web_server.py to handle explicit deletions. + self._startup.run() def cleanup_stale_jobs(self): """Delegate to BackgroundJobService.""" @@ -223,53 +169,7 @@ def cleanup_stale_jobs(self): def cleanup_cache(self): """Delete files from ebook cache that are not referenced in the DB.""" - if not self.epub_cache_dir.exists(): - return - - logger.info("Starting ebook cache cleanup...") - - try: - # 1. Collect all valid filenames from DB - valid_filenames = set() - - # From Active Books - books = self.database_service.get_all_books() - for book in books: - if book.ebook_filename: - valid_filenames.add(book.ebook_filename) - - # From Pending Suggestions (covers auto-discovery matches) - suggestions = self.database_service.get_all_actionable_suggestions() - for suggestion in suggestions: - # matches property automatically parses the JSON - for match in suggestion.matches: - if match.get("filename"): - valid_filenames.add(match["filename"]) - - # 2. Iterate cache and delete orphans - deleted_count = 0 - reclaimed_bytes = 0 - - for file_path in self.epub_cache_dir.iterdir(): - # Only check files, and ensure we don't delete if it's in our valid list - if file_path.is_file() and file_path.name not in valid_filenames: - try: - size = file_path.stat().st_size - file_path.unlink() - deleted_count += 1 - reclaimed_bytes += size - logger.debug(f" Deleted orphaned cache file: {file_path.name}") - except Exception as e: - logger.warning(f" Failed to delete {file_path.name}: {e}") - - if deleted_count > 0: - mb = reclaimed_bytes / (1024 * 1024) - logger.info(f"Cache cleanup complete: Removed {deleted_count} files ({mb:.2f} MB)") - else: - logger.info("Cache is clean (no orphaned files found)") - - except Exception as e: - logger.error(f"Error during cache cleanup: {e}") + self._cache_cleanup.cleanup() def get_audiobook_title(self, ab): media = ab.get("media", {}) @@ -860,27 +760,3 @@ def clear_progress(self, abs_id): """Clear progress data for a specific book and reset all sync clients to 0%.""" return self.progress_reset_service.clear_progress(abs_id) - def run_daemon(self): - """Legacy method - daemon is now run from web_server.py""" - logger.warning("run_daemon() called — daemon should be started from web_server.py instead") - schedule.every(int(os.getenv("SYNC_PERIOD_MINS", 5))).minutes.do(self.sync_cycle) - schedule.every(1).minutes.do(self.check_pending_jobs) - logger.info("Daemon started") - self.sync_cycle() - while True: - schedule.run_pending() - time.sleep(30) - - -if __name__ == "__main__": - # This is only used for standalone testing - production uses web_server.py - logger.info("Running sync manager in standalone mode (for testing)") - - from src.utils.di_container import create_container - - di_container = create_container() - # Try to use dependency injection, fall back to legacy if there are issues - sync_manager = di_container.sync_manager() - logger.info("Using dependency injection") - - sync_manager.run_daemon() diff --git a/src/utils/http.py b/src/utils/http.py new file mode 100644 index 0000000..a691a5d --- /dev/null +++ b/src/utils/http.py @@ -0,0 +1,20 @@ +# pyright: reportMissingImports=false +from flask import jsonify + + +def json_error(error_message, status=400, **payload): + body = {"success": False, "error": error_message} + body.update(payload) + return jsonify(body), status + + +def json_detail_error(detail, status=400, **payload): + body = {"success": False, "detail": detail} + body.update(payload) + return jsonify(body), status + + +def json_success(status=200, **payload): + body = {"success": True} + body.update(payload) + return jsonify(body), status diff --git a/src/utils/markdown.py b/src/utils/markdown.py new file mode 100644 index 0000000..edc5464 --- /dev/null +++ b/src/utils/markdown.py @@ -0,0 +1,41 @@ +import mistune +import nh3 +from markupsafe import Markup + +ALLOWED_HTML_TAGS = { + "p", + "br", + "b", + "i", + "em", + "strong", + "ul", + "ol", + "li", + "blockquote", + "span", + "h1", + "h2", + "h3", + "h4", + "h5", + "h6", +} + + +def sanitize_html(value): + if not value: + return Markup("") + cleaned = nh3.clean(str(value), tags=ALLOWED_HTML_TAGS, attributes={}) + return Markup(cleaned) + + +def render_markdown_html(value): + if not value: + return "" + html = mistune.html(value) + return nh3.clean(html, tags=ALLOWED_HTML_TAGS, attributes={}) + + +def render_markdown_markup(value): + return Markup(render_markdown_html(value)) diff --git a/src/utils/runtime_config.py b/src/utils/runtime_config.py new file mode 100644 index 0000000..c56484c --- /dev/null +++ b/src/utils/runtime_config.py @@ -0,0 +1,26 @@ +"""Lightweight environment-backed config helpers for runtime code.""" + +import os + + +def get_str(key, default=""): + return os.environ.get(key, default) + + +def get_bool(key, default=False): + raw_default = "true" if default else "false" + return os.environ.get(key, raw_default).strip().lower() in ("true", "1", "yes", "on") + + +def get_int(key, default): + try: + return int(os.environ.get(key, str(default))) + except (TypeError, ValueError): + return int(default) + + +def get_float(key, default): + try: + return float(os.environ.get(key, str(default))) + except (TypeError, ValueError): + return float(default) diff --git a/src/web_server.py b/src/web_server.py index 1dc80af..890ccdf 100644 --- a/src/web_server.py +++ b/src/web_server.py @@ -1,529 +1,66 @@ +# pyright: reportMissingImports=false + import logging import os -import secrets -import sys -import threading -import time -from pathlib import Path - -import nh3 -import schedule -from dependency_injector import providers -from flask import Flask, current_app, request -from markupsafe import Markup - -from src.api.hardcover_routes import hardcover_bp, init_hardcover_routes -from src.api.kosync_admin import kosync_admin_bp -from src.api.kosync_server import kosync_sync_bp +import signal + +from flask import Flask + +from src.app_runtime import ( + apply_settings, + get_or_create_secret_key, + handle_exit_signal, + reconfigure_logging, + reconcile_socket_listener, + start_runtime_services, +) +from src.app_setup import get_runtime_state, setup_dependencies as _setup_dependencies +from src.app_template_context import inject_global_vars from src.blueprints import register_blueprints from src.blueprints.helpers import safe_folder_name -from src.utils.config_loader import ConfigLoader -from src.version import get_update_status - - -def _reconfigure_logging(): - """ - Update the root logger's level from the LOG_LEVEL environment variable. - - Reads LOG_LEVEL (default "INFO"), resolves it to a logging level constant, sets the root logger to that level, and logs the outcome. On failure, emits a warning describing the error. - """ - try: - new_level_str = os.environ.get("LOG_LEVEL", "INFO").upper() - new_level = getattr(logging, new_level_str, logging.INFO) - - root = logging.getLogger() - root.setLevel(new_level) - - logger.info(f"Logging level updated to {new_level_str}") - except Exception as e: - logger.warning(f"Failed to reconfigure logging: {e}") - - -def apply_settings(app): - """Hot-reload settings that don't propagate automatically via os.environ. - - Handles the five edge cases that previously required a full server restart: - 1. LOG_LEVEL — reconfigure the root logger - 2. SYNC_PERIOD_MINS — clear and re-register the schedule job - 3. ABS Socket.IO listener — start/stop/restart to match current config - 4. Refresh config values that blueprints read from app.config - 5. Telegram logging handler — add/remove/update handler to match current config - """ - errors = [] - - # 1. Reconfigure logging level - _reconfigure_logging() - - # 2. Reschedule sync_cycle job with new period - try: - sync_mgr = app.config.get("sync_manager") - raw_period = os.environ.get("SYNC_PERIOD_MINS", "5") - new_period = int(raw_period) - if new_period <= 0: - raise ValueError("SYNC_PERIOD_MINS must be an integer greater than 0") - - schedule.clear("sync_cycle") - if sync_mgr: - schedule.every(new_period).minutes.do(sync_mgr.sync_cycle).tag("sync_cycle") - logger.info(f"Sync schedule updated to every {new_period} minutes") - except Exception as e: - errors.append(f"sync reschedule failed: {e}") - - # 3. Reconcile ABS Socket.IO listener state - try: - _reconcile_socket_listener(app) - except Exception as e: - errors.append(f"socket listener reconciliation failed: {e}") - - # 4. Refresh config values that blueprints read from app.config - app.config["ABS_COLLECTION_NAME"] = os.environ.get("ABS_COLLECTION_NAME", "Synced with KOReader") - app.config["SUGGESTIONS_ENABLED"] = os.environ.get("SUGGESTIONS_ENABLED", "false").lower() == "true" - - # 5. Reconcile Telegram logging handler state - try: - from src.utils.logging_utils import reconcile_telegram_logging - - reconcile_telegram_logging() - except Exception as e: - errors.append(f"telegram logging reconciliation failed: {e}") - - if errors: - error_message = "; ".join(errors) - logger.error(f"Failed to apply one or more settings: {error_message}") - raise RuntimeError(error_message) - - return True - - -def _reconcile_socket_listener(app): - """ - Ensure the ABS Socket.IO listener is started, stopped, or restarted to reflect current environment settings. - - Reads INSTANT_SYNC_ENABLED, ABS_SOCKET_ENABLED, ABS_SERVER, and ABS_KEY from the environment and: - - starts the listener when instant sync and socket are enabled and server + key are present, - - stops the listener when it is disabled or credentials are missing, - - restarts the listener when credentials change. - - Updates app.config entries 'abs_listener', '_abs_listener_server', and '_abs_listener_key' and uses app.config['database_service'] and app.config['sync_manager'] when creating the listener. - - Parameters: - app: The Flask application whose config holds listener state and required services. - """ - from src.services.abs_socket_listener import ABSSocketListener - - instant_sync = os.environ.get("INSTANT_SYNC_ENABLED", "true").lower() != "false" - socket_enabled = os.environ.get("ABS_SOCKET_ENABLED", "true").lower() != "false" - abs_server = os.environ.get("ABS_SERVER", "") - abs_key = os.environ.get("ABS_KEY", "") - should_run = instant_sync and socket_enabled and abs_server and abs_key - - current: ABSSocketListener | None = app.config.get("abs_listener") - current_server = app.config.get("_abs_listener_server", "") - current_key = app.config.get("_abs_listener_key", "") - - if should_run and current is None: - # Start new listener - listener = ABSSocketListener( - abs_server_url=abs_server, - abs_api_token=abs_key, - database_service=app.config["database_service"], - sync_manager=app.config["sync_manager"], - ) - threading.Thread(target=listener.start, daemon=True).start() - app.config["abs_listener"] = listener - app.config["_abs_listener_server"] = abs_server - app.config["_abs_listener_key"] = abs_key - logger.info("ABS Socket.IO listener started via hot-reload") - - elif not should_run and current is not None: - # Stop running listener - current.stop() - app.config["abs_listener"] = None - app.config["_abs_listener_server"] = "" - app.config["_abs_listener_key"] = "" - logger.info("ABS Socket.IO listener stopped via hot-reload") - - elif should_run and current is not None and (abs_server != current_server or abs_key != current_key): - # Credentials changed — restart listener - current.stop() - listener = ABSSocketListener( - abs_server_url=abs_server, - abs_api_token=abs_key, - database_service=app.config["database_service"], - sync_manager=app.config["sync_manager"], - ) - threading.Thread(target=listener.start, daemon=True).start() - app.config["abs_listener"] = listener - app.config["_abs_listener_server"] = abs_server - app.config["_abs_listener_key"] = abs_key - logger.info("ABS Socket.IO listener restarted via hot-reload (credentials changed)") +from src.utils.markdown import render_markdown_markup, sanitize_html +logger = logging.getLogger(__name__) -# ---------------- APP SETUP ---------------- container = None manager = None database_service = None +SYNC_PERIOD_MINS = 5.0 -def setup_dependencies(app, test_container=None): - """ - Initialize dependencies for the web server. - - Args: - test_container: Optional test container for dependency injection during testing. - If None, creates production container from environment. - """ - global container, manager, database_service, DATA_DIR, EBOOK_DIR, COVERS_DIR - - # Initialize Database Service - from src.db.migration_utils import initialize_database - - database_service = initialize_database(os.environ.get("DATA_DIR", "/data")) - - # Load settings from DB - if database_service: - ConfigLoader.bootstrap_config(database_service) - ConfigLoader.load_settings(database_service) - logger.info("Settings loaded into environment variables") - - # Migrate ABS_LIBRARY_ID -> ABS_LIBRARY_IDS - old_lib_id = os.environ.get("ABS_LIBRARY_ID", "") - new_lib_ids = os.environ.get("ABS_LIBRARY_IDS", "") - if old_lib_id and not new_lib_ids: - old_only_search = os.environ.get("ABS_ONLY_SEARCH_IN_ABS_LIBRARY_ID", "false") - if old_only_search.lower() == "true": - database_service.set_setting("ABS_LIBRARY_IDS", old_lib_id) - os.environ["ABS_LIBRARY_IDS"] = old_lib_id - logger.info(f"Migrated ABS_LIBRARY_ID '{old_lib_id}' to ABS_LIBRARY_IDS") - - # Force reconfigure logging level based on new settings - _reconfigure_logging() - - # RELOAD GLOBALS from updated os.environ - global SYNC_PERIOD_MINS - - def _get_float_env(key, default): - try: - return float(os.environ.get(key, str(default))) - except (ValueError, TypeError): - logger.warning(f"Invalid '{key}' value, defaulting to {default}") - return float(default) - - SYNC_PERIOD_MINS = _get_float_env("SYNC_PERIOD_MINS", 5) - - logger.info(f"Globals reloaded from settings (ABS_SERVER={os.environ.get('ABS_SERVER')})") - - if test_container is not None: - # Use injected test container - container = test_container - else: - # Create production container AFTER loading settings - from src.utils.di_container import create_container - - container = create_container() - - # Override the container's database_service with our already-initialized instance - if test_container is None: - container.database_service.override(providers.Object(database_service)) - - # Initialize manager and services - manager = container.sync_manager() - - # Get data directories - DATA_DIR = container.data_dir() - EBOOK_DIR = container.books_dir() - - # Initialize covers directory - COVERS_DIR = DATA_DIR / "covers" - if not COVERS_DIR.exists(): - COVERS_DIR.mkdir(parents=True, exist_ok=True) - - # Store shared state on app.config for blueprint access - app.config["container"] = container - app.config["sync_manager"] = manager - app.config["database_service"] = database_service - if hasattr(container, "abs_service"): - app.config["abs_service"] = container.abs_service() - else: - from src.services.abs_service import ABSService - - app.config["abs_service"] = ABSService(container.abs_client()) - app.config["DATA_DIR"] = DATA_DIR - app.config["EBOOK_DIR"] = EBOOK_DIR - app.config["COVERS_DIR"] = COVERS_DIR - app.config["ABS_COLLECTION_NAME"] = os.environ.get("ABS_COLLECTION_NAME", "Synced with KOReader") - app.config["SUGGESTIONS_ENABLED"] = os.environ.get("SUGGESTIONS_ENABLED", "false").lower() == "true" - - # Register KoSync blueprints and initialize services - from src.services.kosync_service import KosyncService - from src.utils.debounce_manager import DebounceManager - from src.utils.rate_limiter import TokenBucketRateLimiter - - rate_limiter = TokenBucketRateLimiter() - kosync_service = KosyncService(database_service, container, manager, EBOOK_DIR) - debounce_manager = DebounceManager(database_service, manager, rate_limiter=rate_limiter) - - app.config["kosync_service"] = kosync_service - app.config["debounce_manager"] = debounce_manager - app.config["rate_limiter"] = rate_limiter - - app.register_blueprint(kosync_sync_bp) - app.register_blueprint(kosync_admin_bp) - - # Register Hardcover Blueprint and initialize with dependencies - init_hardcover_routes(database_service, container) - app.register_blueprint(hardcover_bp) +# Backward-compatible exports for tests and existing callers +_reconfigure_logging = reconfigure_logging +_reconcile_socket_listener = reconcile_socket_listener - logger.info(f"Web server dependencies initialized (DATA_DIR={DATA_DIR})") - -# ---------------- CONTEXT PROCESSORS ---------------- -def inject_global_vars(): - """ - Provide global template variables and helper functions for Jinja templates. - - Returns: - dict: Mapping made available to templates containing: - - abs_server (str): Value of ENV['ABS_SERVER'] or empty string. - - grimmory_server (str): Value of ENV['GRIMMORY_SERVER'] or empty string. - - get_val (callable): get_val(key, default_val=None) returns, in order: - 1) the environment variable value for `key` if present, - 2) a built-in default for well-known keys, - 3) `default_val` if provided, - 4) an empty string otherwise. - - get_bool (callable): get_bool(key) returns `True` if get_val(key, 'false') - yields a case-insensitive value in ('true', '1', 'yes', 'on'), `False` otherwise. - """ - pagekeeper_env = os.environ.get("PAGEKEEPER_ENV", "").strip().lower() - is_dev_container = pagekeeper_env == "dev" - title_prefix = "[DEV] " if is_dev_container else "" - - def get_val(key, default_val=None): - if key in os.environ: - return os.environ[key] - DEFAULTS = { - "TZ": "America/New_York", - "LOG_LEVEL": "INFO", - "DATA_DIR": "/data", - "BOOKS_DIR": "/books", - "ABS_COLLECTION_NAME": "Synced with KOReader", - "GRIMMORY_SHELF_NAME": "Kobo", - "SYNC_PERIOD_MINS": "5", - "SYNC_DELTA_ABS_SECONDS": "60", - "SYNC_DELTA_KOSYNC_PERCENT": "0.5", - "SYNC_DELTA_BETWEEN_CLIENTS_PERCENT": "0.5", - "SYNC_DELTA_KOSYNC_WORDS": "400", - "FUZZY_MATCH_THRESHOLD": "80", - "WHISPER_MODEL": "tiny", - "JOB_MAX_RETRIES": "5", - "JOB_RETRY_DELAY_MINS": "15", - "MONITOR_INTERVAL": "3600", - "AUDIOBOOKS_DIR": "/audiobooks", - "ABS_PROGRESS_OFFSET_SECONDS": "0", - "EBOOK_CACHE_SIZE": "3", - "KOSYNC_HASH_METHOD": "content", - "TELEGRAM_LOG_LEVEL": "ERROR", - "ABS_ENABLED": "true", - "KOSYNC_ENABLED": "false", - "STORYTELLER_ENABLED": "false", - "GRIMMORY_ENABLED": "false", - "HARDCOVER_ENABLED": "false", - "TELEGRAM_ENABLED": "false", - "SUGGESTIONS_ENABLED": "false", - "BOOKFUSION_ENABLED": "false", - "REPROCESS_ON_CLEAR_IF_NO_ALIGNMENT": "true", - } - if key in DEFAULTS: - return DEFAULTS[key] - return default_val if default_val is not None else "" - - def get_bool(key): - """ - Interpret the value of an environment variable as a boolean. - - Parameters: - key (str): Environment variable name to read via get_val. - - Returns: - bool: `True` if the variable's value (case-insensitive) is one of `'true'`, `'1'`, `'yes'`, or `'on'`; `False` otherwise. - """ - val = get_val(key, "false") - return val.lower() in ("true", "1", "yes", "on") - - def get_header_service_url(service_name): - from src.utils.service_url_helper import get_service_web_url - - prefix = service_name.upper() - if not get_bool(f"{prefix}_ENABLED"): - return "" - return get_service_web_url(prefix) - - def is_active_path(path): - req_path = request.path.rstrip("/") or "/" - target_path = path.rstrip("/") or "/" - if target_path == "/": - return req_path == "/" - return req_path == target_path or req_path.startswith(f"{target_path}/") - - suggestion_count = 0 - if get_bool("SUGGESTIONS_ENABLED"): - try: - db_svc = current_app.config.get("database_service") - if db_svc: - suggestion_count = db_svc.get_pending_suggestion_count() - except Exception: - pass - - return dict( - abs_server=os.environ.get("ABS_SERVER", ""), - grimmory_server=os.environ.get("GRIMMORY_SERVER", ""), - pagekeeper_env=pagekeeper_env, - is_dev_container=is_dev_container, - title_prefix=title_prefix, - get_val=get_val, - get_bool=get_bool, - get_header_service_url=get_header_service_url, - is_active_path=is_active_path, - suggestion_count=suggestion_count, +def setup_dependencies(app, test_container=None): + """Initialize app dependencies and expose runtime globals for legacy callers/tests.""" + global container, manager, database_service, SYNC_PERIOD_MINS + container, manager, database_service = _setup_dependencies( + app, + test_container=test_container, + logging_reconfigure=reconfigure_logging, ) - - -# ---------------- SYNC DAEMON ---------------- -def sync_daemon(): - """ - Run the background synchronization daemon that schedules and executes periodic sync tasks. - - Schedules the main sync cycle to run every SYNC_PERIOD_MINS minutes and a pending-job checker every minute, performs an initial sync once at startup, then enters a loop that runs scheduled jobs and sleeps between checks. Errors during the initial sync or in the main loop are logged; the daemon continues retrying after failures. - """ - try: - schedule.every(int(SYNC_PERIOD_MINS)).minutes.do(manager.sync_cycle).tag("sync_cycle") - schedule.every(1).minutes.do(manager.check_pending_jobs).tag("check_jobs") - - logger.info(f"Sync daemon started (period: {SYNC_PERIOD_MINS} minutes)") - - # Wait for the built-in KoSync split-port server to be ready - def _wait_for_local_services(timeout=30): - kosync_port = os.environ.get("KOSYNC_PORT") - if not kosync_port or kosync_port == "4477": - return # No split-port server to wait for - - import urllib.request - - url = f"http://127.0.0.1:{kosync_port}/healthcheck" - deadline = time.time() + timeout - while time.time() < deadline: - try: - urllib.request.urlopen(url, timeout=2) - logger.info(f"Split-port KoSync server ready on port {kosync_port}") - return - except Exception: - time.sleep(1) - logger.warning(f"Split-port KoSync server not ready after {timeout}s — proceeding anyway") - - _wait_for_local_services() - - # Run initial sync cycle - try: - manager.sync_cycle() - except Exception as e: - logger.error(f"Initial sync cycle failed: {e}") - - # Main daemon loop - while True: - try: - schedule.run_pending() - time.sleep(30) - except Exception as e: - logger.error(f"Sync daemon error: {e}") - time.sleep(60) - - except Exception as e: - logger.error(f"Sync daemon crashed: {e}") - - -# --- Logger setup --- -logger = logging.getLogger(__name__) - - -def _get_or_create_secret_key() -> str: - """Return a persistent random secret key, falling back to ephemeral.""" - data_dir = Path(os.environ.get("DATA_DIR", "/data")) - key_file = data_dir / ".flask_secret_key" - try: - if key_file.exists(): - key = key_file.read_text().strip() - if key: - return key - key = secrets.token_hex(32) - data_dir.mkdir(parents=True, exist_ok=True) - key_file.write_text(key) - key_file.chmod(0o600) - return key - except Exception: - logger.warning("Could not persist Flask secret key — using ephemeral key") - return secrets.token_hex(32) - - -def _log_security_warnings(): - """Log warnings for common security misconfigurations at startup.""" - kosync_user = os.environ.get("KOSYNC_USER", "") - kosync_key = os.environ.get("KOSYNC_KEY", "") - kosync_port = os.environ.get("KOSYNC_PORT", "") - public_url = os.environ.get("KOSYNC_PUBLIC_URL", "") - - if not kosync_user or not kosync_key: - logger.warning("SECURITY: KOSYNC_USER/KOSYNC_KEY not configured — sync endpoints will reject all requests") - elif len(kosync_key) < 8: - logger.warning("SECURITY: KOSYNC_KEY is shorter than 8 characters — consider using a stronger password") - - if not kosync_port or kosync_port == "4477": - logger.warning( - "SECURITY: Split-port mode not active — dashboard and sync API share port 4477. " - "Set KOSYNC_PORT to a different port before exposing sync to the internet." - ) - - if public_url: - from urllib.parse import urlsplit, urlunsplit - - parts = urlsplit(public_url) - safe_netloc = parts.hostname or "" - if parts.port: - safe_netloc = f"{safe_netloc}:{parts.port}" - safe_url = urlunsplit((parts.scheme, safe_netloc, parts.path or "", "", "")) - logger.info(f"KOSync public URL: {safe_url}") - elif kosync_port and kosync_port != "4477": - logger.info("Tip: Set KOSYNC_PUBLIC_URL in settings if you expose KOSync through a reverse proxy") - - -_ALLOWED_HTML_TAGS = {"p", "br", "b", "i", "em", "strong", "ul", "ol", "li"} - - -def _sanitize_html(value): - """Allow only safe formatting tags and strip all attributes/protocols.""" - if not value: - return "" - cleaned = nh3.clean(str(value), tags=_ALLOWED_HTML_TAGS, attributes={}) - return Markup(cleaned) + _, _, _, SYNC_PERIOD_MINS = get_runtime_state() + return container, manager, database_service # --- Application Factory --- def create_app(test_container=None): - STATIC_DIR = os.environ.get("STATIC_DIR", "/app/static") - TEMPLATE_DIR = os.environ.get("TEMPLATE_DIR", "/app/templates") - app = Flask(__name__, static_folder=STATIC_DIR, static_url_path="/static", template_folder=TEMPLATE_DIR) - app.secret_key = _get_or_create_secret_key() + static_dir = os.environ.get("STATIC_DIR", "/app/static") + template_dir = os.environ.get("TEMPLATE_DIR", "/app/templates") + app = Flask(__name__, static_folder=static_dir, static_url_path="/static", template_folder=template_dir) + app.secret_key = get_or_create_secret_key() app.config["SESSION_COOKIE_SAMESITE"] = "Lax" app.config["SESSION_COOKIE_HTTPONLY"] = True - # Setup dependencies and inject into app context setup_dependencies(app, test_container=test_container) - # Register context processors, jinja globals app.context_processor(inject_global_vars) app.jinja_env.globals["safe_folder_name"] = safe_folder_name - app.jinja_env.filters["sanitize_html"] = _sanitize_html + app.jinja_env.filters["sanitize_html"] = sanitize_html + app.jinja_env.filters["markdown"] = render_markdown_markup - # Register all application blueprints register_blueprints(app) @app.after_request @@ -532,110 +69,16 @@ def set_security_headers(response): response.headers["X-Content-Type-Options"] = "nosniff" return response - # Return both app and container for external reference return app, container -# ---------------- MAIN ---------------- if __name__ == "__main__": - # Setup signal handlers to catch unexpected kills - import signal - - def handle_exit_signal(signum, frame): - logger.warning(f"Received signal {signum} - Shutting down...") - for handler in logger.handlers: - handler.flush() - if hasattr(logging.getLogger(), "handlers"): - for handler in logging.getLogger().handlers: - handler.flush() - sys.exit(0) - signal.signal(signal.SIGTERM, handle_exit_signal) signal.signal(signal.SIGINT, handle_exit_signal) app, container = create_app() - logger.info("=== Unified ABS Manager Started (Integrated Mode) ===") - _log_security_warnings() - - # Start sync daemon in background thread - sync_daemon_thread = threading.Thread(target=sync_daemon, daemon=True) - sync_daemon_thread.start() - threading.Thread(target=get_update_status, daemon=True).start() - logger.info("Sync daemon thread started") - - # Start ABS Socket.IO listener for real-time / instant sync - instant_sync_enabled = os.environ.get("INSTANT_SYNC_ENABLED", "true").lower() != "false" - abs_socket_enabled = os.environ.get("ABS_SOCKET_ENABLED", "true").lower() != "false" - if instant_sync_enabled and abs_socket_enabled and container.abs_client().is_configured(): - from src.services.abs_socket_listener import ABSSocketListener - - abs_listener = ABSSocketListener( - abs_server_url=os.environ.get("ABS_SERVER", ""), - abs_api_token=os.environ.get("ABS_KEY", ""), - database_service=database_service, - sync_manager=manager, - ) - threading.Thread(target=abs_listener.start, daemon=True).start() - app.config["abs_listener"] = abs_listener - app.config["_abs_listener_server"] = os.environ.get("ABS_SERVER", "") - app.config["_abs_listener_key"] = os.environ.get("ABS_KEY", "") - logger.info("ABS Socket.IO listener started (instant sync enabled)") - else: - app.config["abs_listener"] = None - app.config["_abs_listener_server"] = "" - app.config["_abs_listener_key"] = "" - if not instant_sync_enabled: - logger.info("ABS Socket.IO listener disabled (INSTANT_SYNC_ENABLED=false)") - elif not abs_socket_enabled: - logger.info("ABS Socket.IO listener disabled (ABS_SOCKET_ENABLED=false)") - - # Start per-client poller - from src.services.client_poller import ClientPoller - - client_poller = ClientPoller( - database_service=database_service, - sync_manager=manager, - sync_clients_dict=container.sync_clients(), - ) - poller_thread = threading.Thread(target=client_poller.start, daemon=True) - poller_thread.start() - - # Check ebook source configuration - grimmory_configured = container.grimmory_client().is_configured() - books_volume_exists = container.books_dir().exists() - - if grimmory_configured: - logger.info("Grimmory integration enabled - ebooks sourced from API") - elif books_volume_exists: - logger.info(f"Ebooks directory mounted at {container.books_dir()}") - else: - logger.info( - "NO EBOOK SOURCE CONFIGURED: Neither Grimmory integration nor /books volume is available. " - "New book matches will fail. Enable Grimmory (GRIMMORY_SERVER, GRIMMORY_USER, GRIMMORY_PASSWORD) " - "or mount the ebooks directory to /books." - ) + start_runtime_services(app, container, database_service, manager) logger.info("Web interface starting on port 4477") - - # --- Split-Port Mode --- - sync_port = os.environ.get("KOSYNC_PORT") - if sync_port and int(sync_port) != 4477: - - def run_sync_only_server(port): - sync_app = Flask(__name__) - sync_app.config["kosync_service"] = app.config["kosync_service"] - sync_app.config["debounce_manager"] = app.config["debounce_manager"] - sync_app.config["rate_limiter"] = app.config["rate_limiter"] - sync_app.register_blueprint(kosync_sync_bp) - - @sync_app.route("/") - def sync_health(): - return "Sync Server OK", 200 - - sync_app.run(host="0.0.0.0", port=port, debug=False, use_reloader=False) - - threading.Thread(target=run_sync_only_server, args=(int(sync_port),), daemon=True).start() - logger.info(f"Split-Port Mode Active: Sync-only server on port {sync_port}") - app.run(host="0.0.0.0", port=4477, debug=False) diff --git a/static/css/bookfusion.css b/static/css/bookfusion.css deleted file mode 100644 index d58e074..0000000 --- a/static/css/bookfusion.css +++ /dev/null @@ -1,716 +0,0 @@ -/* PageKeeper — BookFusion Page Styles */ - -.bf-container { - max-width: 1200px; - margin: 0 auto; - padding: 24px; -} -.bf-header { - display: flex; - justify-content: space-between; - align-items: center; - margin-bottom: 28px; - flex-wrap: wrap; - gap: 12px; -} -.bf-page-title { - font-family: var(--font-heading); - font-size: 28px; - font-weight: 700; - color: var(--color-text); - margin: 0; -} - -/* Pill Tabs */ -.bf-tabs { - display: inline-flex; - gap: 4px; - background: var(--color-surface); - border: 1px solid var(--color-border-light); - border-radius: var(--radius-lg); - padding: 4px; - margin-bottom: 24px; - max-width: 100%; -} -.bf-tab { - padding: 8px 18px; - border-radius: 20px; - border: 1px solid transparent; - background: none; - color: var(--color-text-muted); - cursor: pointer; - font-family: var(--font-body); - font-size: 0.9rem; - transition: var(--transition); - white-space: nowrap; -} -.bf-tab:hover { - color: var(--color-text); - background: var(--color-surface-hover); -} -.bf-tab:focus-visible { - color: var(--color-text); - outline: 2px solid var(--color-bookfusion); - outline-offset: -2px; - border-radius: 20px; -} -.bf-tab.active { - background: rgba(59, 130, 246, 0.15); - border-color: rgba(59, 130, 246, 0.3); - color: var(--color-bookfusion); - font-weight: 600; -} -.bf-panel { display: none; } -.bf-panel.active { display: block; } - -/* Search */ -.bf-search { - display: flex; - gap: 12px; - margin-bottom: 20px; -} -.bf-search input { - flex: 1; - border-color: var(--color-border-light); - background: var(--color-bg-input); - width: 100%; -} -.bf-search input:focus { - border-color: var(--color-primary-hover); - box-shadow: 0 0 0 3px rgba(124, 58, 237, 0.1); - outline: none; -} - -/* Book list */ -.bf-book-list { - display: flex; - flex-direction: column; - gap: 8px; -} -.bf-book-item { - display: flex; - align-items: center; - gap: 16px; - padding: 16px 20px; - background: var(--color-bg-raised); - border-radius: var(--radius-lg); - border: 1px solid var(--color-border-light); - transition: border-color var(--transition-bounce), box-shadow var(--transition-bounce); -} -.bf-book-item:hover { - border-color: var(--color-border); - box-shadow: 0 4px 16px rgba(0, 0, 0, 0.2); -} -.bf-book-info { flex: 1; min-width: 0; } -.bf-book-title { - font-family: var(--font-heading); - font-weight: 600; - font-size: 0.95rem; - white-space: nowrap; - overflow: hidden; - text-overflow: ellipsis; -} -.bf-book-meta { - color: var(--color-text-muted); - font-size: 0.85rem; - margin-top: 2px; - line-height: 1.5; - overflow-wrap: anywhere; -} -.bf-source-tag { - display: inline-block; - padding: 1px 6px; - border-radius: 3px; - background: var(--color-surface-hover); - font-size: 0.75rem; - color: var(--color-text-muted); - margin-left: 4px; -} - -/* Action Buttons */ -.bf-upload-btn { - flex-shrink: 0; - padding: 6px 14px; - border: none; - border-radius: var(--radius-sm); - background: var(--color-primary-hover); - color: #fff; - cursor: pointer; - font-size: 0.85rem; - font-weight: 600; - transition: all var(--transition-bounce); - display: inline-flex; - align-items: center; - justify-content: center; - min-height: 40px; -} -.bf-upload-btn:hover { - transform: translateY(-1px); - box-shadow: 0 2px 8px var(--color-primary-glow); -} -.bf-upload-btn:focus-visible { - outline: 2px solid var(--color-primary-hover); - outline-offset: 2px; -} -.bf-upload-btn:disabled { opacity: 0.5; cursor: not-allowed; transform: none; box-shadow: none; } -.bf-upload-btn:disabled:focus-visible { outline: none; } -.bf-upload-btn.done { background: var(--color-success); } -.bf-upload-btn.error { background: var(--color-error); } -.bf-upload-btn.bf-start-btn { background: var(--color-success); } - -/* Highlights */ -.bf-sync-bar { - display: flex; - align-items: center; - gap: 12px; - margin-bottom: 20px; - background: var(--color-surface); - border: 1px solid var(--color-border-light); - border-radius: var(--radius-lg); - padding: 12px 16px; -} -.bf-resync-btn { - background: var(--color-surface-hover) !important; - color: var(--color-text-muted) !important; - border: 1px solid var(--color-border-light) !important; -} -.bf-resync-btn:hover { - background: var(--color-surface) !important; - color: var(--color-text) !important; - border-color: var(--color-border) !important; -} -.bf-sync-info { - color: var(--color-text-muted); - font-size: 0.85rem; - min-width: 0; - flex: 1 1 220px; -} -.bf-highlight-group { - margin-bottom: 20px; -} -.bf-group-header { - display: flex; - align-items: flex-start; - gap: 8px; - padding: 12px 16px; - background: var(--color-surface); - border: 1px solid var(--color-border-light); - border-radius: var(--radius-lg); - cursor: pointer; - font-weight: 600; - font-family: var(--font-heading); - user-select: none; - transition: all var(--transition); -} - -.bf-group-title { - min-width: 0; - overflow-wrap: anywhere; -} -.bf-group-header:hover { - border-color: var(--color-border); - background: var(--color-surface-hover); -} -.bf-group-header:focus-visible { - outline: 2px solid var(--color-bookfusion); - outline-offset: 2px; -} -.bf-group-header .chevron { - transition: transform 0.2s; - font-size: 0.8rem; -} -.bf-group-header.collapsed .chevron { - transform: rotate(-90deg); -} -.bf-group-body { - display: flex; - flex-direction: column; - gap: 10px; - padding: 10px 0 0 20px; -} -.bf-group-body.hidden { display: none; } -.bf-highlight { - padding: 14px 18px; - background: var(--color-bg-raised); - border: 1px solid var(--color-border-light); - border-radius: var(--radius-lg); - border-left: 3px solid var(--color-bookfusion); - transition: border-color var(--transition); -} -.bf-highlight:hover { - border-color: var(--color-border); - border-left-color: var(--color-bookfusion); -} -.bf-highlight-content { - font-size: 0.95rem; - font-style: italic; - line-height: 1.6; -} -.bf-highlight-chapter { - color: var(--color-text-muted); - font-size: 0.8rem; - margin-top: 6px; -} -.bf-highlight-chapter::before { - content: "\2014\00a0"; -} -.bf-journal-controls { - display: inline-flex; - gap: 8px; - align-items: center; - margin-left: auto; - padding-left: 12px; - flex-wrap: wrap; - justify-content: flex-end; -} -.bf-book-select { - background: var(--color-surface); - max-width: 200px; -} -.bf-book-select:focus-visible { - outline: 2px solid var(--color-bookfusion); - outline-offset: 1px; -} - -/* Custom Combobox */ -.bf-combobox { - position: relative; - display: inline-block; - width: 200px; - text-align: left; - min-width: 0; -} -.bf-combobox-input { - width: 100%; - background: var(--color-bg-input); - box-sizing: border-box; - min-height: 40px; -} -.bf-combobox-input:focus { - border-color: var(--color-primary-hover); - box-shadow: 0 0 0 3px rgba(124, 58, 237, 0.1); - outline: none; -} -.bf-combobox-dropdown { - position: absolute; - top: calc(100% + 2px); - left: 0; - right: 0; - max-height: 200px; - overflow-y: auto; - background: var(--color-bg-raised); - border: 1px solid var(--color-border-light); - border-radius: var(--radius); - z-index: 100; - display: none; - box-shadow: 0 4px 12px rgba(0,0,0,0.3); - padding: 2px; -} -.bf-combobox.open .bf-combobox-dropdown { - display: block; -} -.bf-combobox-option { - padding: 8px 10px; - font-size: 0.8rem; - cursor: pointer; - color: var(--color-text); - white-space: nowrap; - overflow: hidden; - text-overflow: ellipsis; - border-radius: var(--radius-sm); - transition: background var(--transition); -} -.bf-combobox-option:hover, .bf-combobox-option.active { - background: var(--color-surface-hover); -} -.bf-combobox-option.hidden { - display: none; -} - -/* Empty States */ -.bf-empty { - text-align: center; - padding: 60px 40px; - color: var(--color-text-muted); -} -.bf-empty-icon { - font-size: 3rem; - margin-bottom: 12px; - opacity: 0.3; -} -.bf-empty-heading { - font-family: var(--font-heading); - font-size: 1.1rem; - font-weight: 600; - color: var(--color-text-muted); - margin-bottom: 6px; -} -.bf-empty-desc { - font-size: 0.9rem; - color: var(--color-text-faint); -} - -/* Badges */ -.bf-count { - color: var(--color-text-muted); - font-size: 0.85rem; - font-weight: 400; -} -.bf-new-badge { - display: inline-block; - padding: 1px 7px; - border-radius: 3px; - background: var(--color-primary-light); - color: var(--color-primary-hover); - font-size: 0.7rem; - font-weight: 600; - font-style: normal; - margin-right: 6px; - vertical-align: middle; - letter-spacing: 0.02em; -} -.bf-match-badge { - display: inline-block; - padding: 1px 8px; - border-radius: 3px; - background: var(--color-success-light); - color: var(--color-success); - border: 1px solid var(--color-success-border); - font-size: 0.75rem; - font-weight: 600; - margin-left: 4px; -} - -/* Library tab */ -.bf-dashboard-badge { - display: inline-block; - padding: 2px 8px; - border-radius: 3px; - background: var(--color-success-light); - color: var(--color-success); - border: 1px solid var(--color-success-border); - font-size: 0.75rem; - font-weight: 600; - text-decoration: none; - transition: all var(--transition); -} -.bf-dashboard-badge:hover { - background: var(--color-success-border); -} -.bf-hl-count { - display: inline-block; - padding: 1px 6px; - border-radius: 3px; - background: var(--color-primary-light); - color: var(--color-primary-hover); - font-size: 0.75rem; - margin-left: 4px; -} -.bf-format-tag { - display: inline-block; - padding: 1px 5px; - border-radius: 3px; - background: var(--color-surface-hover); - color: var(--color-text-muted); - font-size: 0.65rem; - font-weight: 600; - letter-spacing: 0.03em; - margin-left: 3px; -} -.bf-library-actions { - display: flex; - gap: 8px; - align-items: center; - flex-shrink: 0; -} -.bf-link-btn { - background: var(--color-success); -} -.bf-unlink-btn { - background: var(--color-surface-hover); - color: var(--color-text-muted); - font-size: 0.7rem; -} -.bf-hide-btn { - background: var(--color-surface-hover); - color: var(--color-text-muted); - font-size: 0.7rem; -} -.bf-unhide-btn { - background: var(--color-surface-hover); - color: var(--color-text-muted); - font-size: 0.7rem; -} - -/* Hidden section */ -.bf-hidden-section { - margin-top: 24px; - border-top: 1px solid var(--color-border-light); - padding-top: 16px; -} -.bf-hidden-header { - display: flex; - align-items: center; - gap: 8px; - padding: 10px 16px; - background: var(--color-surface); - border: 1px solid var(--color-border-light); - border-radius: var(--radius-lg); - cursor: pointer; - font-weight: 600; - font-family: var(--font-heading); - font-size: 0.9rem; - color: var(--color-text-muted); - user-select: none; - transition: all var(--transition); -} -.bf-hidden-header:hover { - border-color: var(--color-border); - background: var(--color-surface-hover); -} -.bf-hidden-header:focus-visible { - outline: 2px solid var(--color-primary-hover); - outline-offset: 2px; -} -.bf-hidden-header .chevron { - transition: transform 0.2s; - font-size: 0.8rem; -} -.bf-hidden-header.collapsed .chevron { - transform: rotate(-90deg); -} -.bf-hidden-body { - display: flex; - flex-direction: column; - gap: 8px; - margin-top: 10px; -} -.bf-hidden-body.hidden { display: none; } -.bf-hidden-body .bf-book-item { - opacity: 0.6; -} -.bf-hidden-body .bf-book-item:hover { - opacity: 1; -} - -@media (max-width: 768px) { - .bf-container { - padding: 16px; - padding-bottom: max(24px, env(safe-area-inset-bottom)); - } - - .bf-header { - margin-bottom: 18px; - } - - .bf-page-title { - font-size: 24px; - } - - .bf-tabs { - display: flex; - width: 100%; - overflow-x: auto; - flex-wrap: nowrap; - margin-bottom: 18px; - scrollbar-width: none; - -webkit-overflow-scrolling: touch; - } - - .bf-tabs::-webkit-scrollbar { - display: none; - } - - /* Stack book items vertically */ - .bf-book-item { - flex-direction: column; - align-items: stretch; - gap: 12px; - padding: 14px; - border-radius: 18px; - } - - .bf-book-info { min-width: 0; } - .bf-book-title { - white-space: normal; - line-height: 1.3; - } - - .bf-book-meta { - margin-top: 6px; - font-size: 0.9rem; - } - - .bf-search { - margin-bottom: 16px; - } - - .bf-search input { - min-height: 48px; - font-size: 16px; - } - - .bf-book-list { - gap: 12px; - } - - .bf-book-item > .bf-upload-btn { - width: 100%; - min-height: 44px; - } - - /* Actions fill width and wrap */ - .bf-library-actions { - width: 100%; - flex-wrap: wrap; - gap: 8px; - } - - .bf-library-actions .bf-book-select, - .bf-library-actions .bf-combobox, - .bf-library-actions .bf-dashboard-badge { - flex: 1; - min-width: 0; - max-width: none; - width: 100%; - } - - .bf-library-actions .bf-upload-btn { - flex: 1 1 calc(50% - 4px); - width: auto; - min-height: 42px; - } - - /* Highlights header: stack controls below title */ - .bf-group-header { - flex-wrap: wrap; - gap: 8px 10px; - padding: 12px; - } - - .bf-group-title { - flex: 1 1 180px; - } - - .bf-journal-controls { - width: 100%; - margin-left: 0; - padding-left: 0; - margin-top: 8px; - justify-content: flex-start; - } - - .bf-journal-controls .bf-combobox { - width: 100%; - } - - .bf-journal-controls .bf-upload-btn { - width: 100%; - min-height: 42px; - } - - .bf-group-body { - padding: 10px 0 0; - } - - .bf-highlight { - padding: 12px 14px; - border-radius: 16px; - } - - .bf-highlight-content { - font-size: 0.9rem; - line-height: 1.55; - overflow-wrap: anywhere; - } - - /* Sync bar wrap */ - .bf-sync-bar { - flex-direction: column; - align-items: stretch; - gap: 10px; - padding: 10px 12px; - } - - .bf-sync-bar .btn { - width: 100%; - justify-content: center; - min-height: 44px; - } - - .bf-sync-info { - flex: auto; - font-size: 0.8rem; - } - - .bf-hidden-header { - padding: 10px 12px; - } - - /* Tabs: smaller padding */ - .bf-tab { - flex: 0 0 auto; - padding: 8px 14px; - font-size: 0.85rem; - } -} - -@media (max-width: 480px) { - .bf-container { - padding: 14px; - } - - .bf-page-title { - font-size: 22px; - } - - .bf-tabs { - margin-bottom: 16px; - } - - .bf-tab { - padding: 8px 12px; - font-size: 0.8rem; - } - - .bf-book-item { - padding: 12px; - } - - .bf-book-title { - font-size: 0.92rem; - } - - .bf-book-meta { - font-size: 0.85rem; - } - - .bf-library-actions .bf-upload-btn { - flex-basis: 100%; - } - - .bf-highlight { - padding: 11px 12px; - } - - .bf-hidden-header, - .bf-group-header { - border-radius: 16px; - } - - .bf-empty { - padding: 42px 18px; - } - - .bf-empty-icon { - font-size: 2.4rem; - } - - .bf-empty-heading { - font-size: 1rem; - } - - .bf-empty-desc { - font-size: 0.85rem; - } -} diff --git a/static/css/reading.css b/static/css/reading.css index 3a6c9d6..1034d79 100644 --- a/static/css/reading.css +++ b/static/css/reading.css @@ -2037,6 +2037,14 @@ a.r-service-row-name:hover { text-decoration: underline; } white-space: pre-wrap; } +.r-tl-text > :first-child { + margin-top: 0; +} + +.r-tl-text > :last-child { + margin-bottom: 0; +} + .r-tl-menu { position: absolute; top: 12px; diff --git a/static/js/bookfusion.js b/static/js/bookfusion.js deleted file mode 100644 index 777698b..0000000 --- a/static/js/bookfusion.js +++ /dev/null @@ -1,888 +0,0 @@ -/* ═══════════════════════════════════════════ - PageKeeper — BookFusion page - ═══════════════════════════════════════════ - Depends on: utils.js - No Jinja2 vars — clean extraction. - ═══════════════════════════════════════════ */ - -/* ── Helpers ── */ - -function getSpinnerHtml() { - return ''; -} - -function isMobileViewport() { - return window.matchMedia('(max-width: 768px)').matches; -} - -function keepElementVisible(el, block) { - if (!el || !isMobileViewport()) return; - window.setTimeout(function () { - try { - el.scrollIntoView({ behavior: 'smooth', block: block || 'center', inline: 'nearest' }); - } catch (e) { - el.scrollIntoView(); - } - }, 180); -} - -function revealFirstMobileResult(listId) { - if (!isMobileViewport()) return; - var active = document.activeElement; - if (!active || (active.id !== 'bf-search-input' && active.id !== 'bf-library-search')) return; - var first = document.querySelector('#' + listId + ' .bf-book-item, #' + listId + ' .bf-highlight-group, #' + listId + ' .bf-empty'); - if (first) keepElementVisible(first, 'nearest'); -} - -function scrollActiveTabIntoView(tab) { - var activeTab = document.querySelector('.bf-tab[data-tab="' + tab + '"]'); - if (!activeTab || !isMobileViewport()) return; - activeTab.scrollIntoView({ behavior: 'smooth', inline: 'center', block: 'nearest' }); -} - -/* - * All user-facing text in dynamically generated HTML is passed through - * escapeHtml() (from utils.js) before insertion. - */ - -function createComboboxHtml(options, placeholder, onChangeFnName, extraAttrs) { - extraAttrs = extraAttrs || ''; - var optionsHtml = options.map(function (opt) { - return '
' + escapeHtml(opt.label) + '
'; - }).join(''); - - var selectedOpt = options.find(function (o) { return o.selected; }); - var initialValue = selectedOpt ? escapeHtml(selectedOpt.label) : ''; - var initialDataValue = selectedOpt ? escapeHtml(selectedOpt.value) : ''; - - return '
' + - '' + - '
' + optionsHtml + '
' + - '
'; -} - -function handleComboboxFilter(input) { - var filter = input.value.toLowerCase(); - var options = input.nextElementSibling.querySelectorAll('.bf-combobox-option'); - options.forEach(function (opt) { - if (opt.textContent.toLowerCase().indexOf(filter) !== -1) { - opt.classList.remove('hidden'); - } else { - opt.classList.add('hidden'); - } - }); - input.dataset.selectedValue = ''; -} - -function handleComboboxSelect(optionEl) { - var combobox = optionEl.closest('.bf-combobox'); - var input = combobox.querySelector('.bf-combobox-input'); - input.value = optionEl.textContent; - input.dataset.selectedValue = optionEl.dataset.value; - combobox.classList.remove('open'); - - if (combobox.dataset.onChange) { - window[combobox.dataset.onChange](combobox); - } -} - -var _newHighlightIds = []; - -/* ── Tab switching ── */ -function switchBFTab(tab) { - _newHighlightIds = []; - document.querySelectorAll('.bf-tab').forEach(function (t) { t.classList.remove('active'); }); - document.querySelectorAll('.bf-panel').forEach(function (p) { p.classList.remove('active'); }); - document.getElementById('bf-panel-' + tab).classList.add('active'); - document.querySelectorAll('.bf-tab').forEach(function (t) { - if (t.dataset.tab === tab) t.classList.add('active'); - }); - scrollActiveTabIntoView(tab); - - if (tab === 'highlights') loadHighlights(); - if (tab === 'library') loadLibrary(); -} - -/* ── Upload Tab ── */ -var searchTimer; - -function debounceSearch() { - clearTimeout(searchTimer); - searchTimer = setTimeout(searchBooks, 300); -} - -function searchBooks() { - var q = document.getElementById('bf-search-input').value.trim(); - var list = document.getElementById('bf-book-list'); - if (!q) { - list.textContent = ''; - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = 'Search your Grimmory library'; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; d.textContent = 'Type above to find books to upload'; - emptyEl.appendChild(h); emptyEl.appendChild(d); - list.appendChild(emptyEl); - return; - } - fetch('/api/bookfusion/grimmory-books?q=' + encodeURIComponent(q)) - .then(function (r) { - if (!r.ok) throw new Error('Search failed'); - return r.json(); - }) - .then(function (books) { - list.textContent = ''; - if (!books.length) { - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\uD83D\uDCDA'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = 'No books found'; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; d.textContent = 'Try a different search term'; - emptyEl.appendChild(icon); emptyEl.appendChild(h); emptyEl.appendChild(d); - list.appendChild(emptyEl); - return; - } - books.forEach(function (b) { - var item = document.createElement('div'); - item.className = 'bf-book-item'; - - var info = document.createElement('div'); - info.className = 'bf-book-info'; - - var titleEl = document.createElement('div'); - titleEl.className = 'bf-book-title'; - titleEl.textContent = b.title || b.fileName; - - var metaEl = document.createElement('div'); - metaEl.className = 'bf-book-meta'; - var metaText = ''; - if (b.authors) metaText += b.authors + ' \u00B7 '; - metaText += b.fileName; - metaEl.textContent = metaText; - var sourceTag = document.createElement('span'); - sourceTag.className = 'bf-source-tag'; - sourceTag.textContent = b.source; - metaEl.appendChild(document.createTextNode(' ')); - metaEl.appendChild(sourceTag); - - info.appendChild(titleEl); - info.appendChild(metaEl); - - var btn = document.createElement('button'); - btn.className = 'bf-upload-btn'; - btn.textContent = 'Upload'; - btn.dataset.bookId = b.id; - btn.dataset.title = b.title || ''; - btn.dataset.authors = b.authors || ''; - btn.dataset.fileName = b.fileName || ''; - btn.addEventListener('click', function () { handleUploadClick(btn); }); - - item.appendChild(info); - item.appendChild(btn); - list.appendChild(item); - }); - revealFirstMobileResult('bf-book-list'); - }) - .catch(function (err) { - list.textContent = ''; - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\u26A0\uFE0F'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = 'Search failed'; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; d.textContent = 'Please try again'; - emptyEl.appendChild(icon); emptyEl.appendChild(h); emptyEl.appendChild(d); - list.appendChild(emptyEl); - }); -} - -function handleUploadClick(btn) { - var book = { - id: btn.dataset.bookId, - title: btn.dataset.title, - authors: btn.dataset.authors, - fileName: btn.dataset.fileName - }; - uploadBook(btn, book); -} - -function uploadBook(btn, book) { - btn.disabled = true; - btn.innerHTML = getSpinnerHtml() + 'Uploading\u2026'; - - fetch('/api/bookfusion/upload', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - book_id: book.id, - title: book.title, - authors: book.authors, - fileName: book.fileName - }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Upload failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - btn.textContent = 'Done'; - btn.classList.add('done'); - } else { - btn.textContent = data.error || 'Error'; - btn.classList.add('error'); - btn.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = err.message || 'Upload failed'; - btn.classList.add('error'); - btn.disabled = false; - }); -} - -/* ── Highlights Tab ── */ -function syncHighlights(fullResync) { - var btn = document.getElementById('bf-sync-btn'); - var resyncBtn = document.getElementById('bf-resync-btn'); - var info = document.getElementById('bf-sync-info'); - - var activeBtn = fullResync ? resyncBtn : btn; - var originalText = activeBtn.textContent; - - btn.disabled = true; - resyncBtn.disabled = true; - activeBtn.innerHTML = getSpinnerHtml() + (fullResync ? 'Re-syncing\u2026' : 'Syncing\u2026'); - info.textContent = ''; - - fetch('/api/bookfusion/sync-highlights', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ full_resync: !!fullResync }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Sync failed'); - return r.json(); - }) - .then(function (data) { - btn.disabled = false; - resyncBtn.disabled = false; - activeBtn.textContent = originalText; - if (data.success) { - var parts = []; - if (data.new_highlights) parts.push(data.new_highlights + ' new highlight' + (data.new_highlights !== 1 ? 's' : '')); - if (data.books_saved) parts.push(data.books_saved + ' book' + (data.books_saved !== 1 ? 's' : '') + ' cataloged'); - info.textContent = parts.length ? 'Synced: ' + parts.join(', ') : 'Up to date'; - _newHighlightIds = data.new_ids || []; - loadHighlights(); - } else { - info.textContent = data.error || 'Sync failed'; - } - }) - .catch(function (err) { - btn.disabled = false; - resyncBtn.disabled = false; - activeBtn.textContent = originalText; - info.textContent = err.message || 'Sync failed'; - }); -} - -var _pkBooks = []; -var _currentHighlightGroups = {}; - -function loadHighlights() { - fetch('/api/bookfusion/highlights') - .then(function (r) { - if (!r.ok) throw new Error('Failed to load highlights'); - return r.json(); - }) - .then(function (data) { - var container = document.getElementById('bf-highlights-container'); - - _pkBooks = data.books || []; - var groups = data.highlights; - _currentHighlightGroups = groups; - var bookNames = Object.keys(groups); - - if (!bookNames.length) { - container.textContent = ''; - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\uD83D\uDCDD'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; - h.textContent = data.has_synced ? 'No highlights found' : 'Sync your highlights'; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; - d.textContent = data.has_synced ? 'Your synced books have no highlights yet' : 'Click "Sync Highlights" to fetch highlights from BookFusion'; - emptyEl.appendChild(icon); emptyEl.appendChild(h); emptyEl.appendChild(d); - container.appendChild(emptyEl); - return; - } - - /* Highlight data is escaped via escapeHtml before insertion */ - container.innerHTML = bookNames.map(function (book) { // eslint-disable-line no-unsanitized/property - var groupData = groups[book]; - var hls = groupData.highlights; - var matchedAbsId = groupData.matched_abs_id; - - var options = _pkBooks.map(function (b) { - return { - value: b.abs_id, - label: b.title, - selected: (matchedAbsId && b.abs_id === matchedAbsId) - }; - }); - - var comboboxHtml = createComboboxHtml(options, 'Match to book\u2026', 'handleHighlightLinkChange', 'data-book-title="' + escapeHtml(book) + '" data-bf-id="' + escapeHtml(groupData.bookfusion_book_id) + '"'); - - var hlsHtml = hls.map(function (h) { - var metaParts = []; - if (h.chapter_heading) metaParts.push(h.chapter_heading.replace(/^#{1,6}\s*/, '')); - if (h.date) metaParts.push(h.date); - var metaText = metaParts.map(escapeHtml).join(' · '); - var isNew = _newHighlightIds.length && h.highlight_id && _newHighlightIds.indexOf(h.highlight_id) !== -1; - var newBadge = isNew ? 'New' : ''; - var newAttr = isNew ? ' data-new-highlight="1"' : ''; - - return '
' + - '
' + newBadge + escapeHtml(h.quote || '') + '
' + - '
' + metaText + '
' + - '
'; - }).join(''); - - return '
' + - '
' + - '\u25BC' + - '' + escapeHtml(book) + '' + - '(' + hls.length + ')' + - (matchedAbsId ? '\u2714 Linked' : '') + - '' + - comboboxHtml + - '' + - '' + - '
' + - '
' + hlsHtml + '
' + - '
'; - }).join(''); - revealFirstMobileResult('bf-highlights-container'); - - if (_newHighlightIds.length) { - var firstNew = container.querySelector('[data-new-highlight]'); - if (firstNew) { - var group = firstNew.closest('.bf-group-body'); - if (group && group.classList.contains('hidden')) { - group.classList.remove('hidden'); - var header = group.previousElementSibling; - if (header) header.classList.remove('collapsed'); - } - setTimeout(function () { - firstNew.scrollIntoView({ behavior: 'smooth', block: 'center' }); - }, 100); - } - } - }) - .catch(function (err) { - var container = document.getElementById('bf-highlights-container'); - container.textContent = ''; - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\u26A0\uFE0F'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = 'Failed to load highlights'; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; d.textContent = 'Please try again'; - emptyEl.appendChild(icon); emptyEl.appendChild(h); emptyEl.appendChild(d); - container.appendChild(emptyEl); - }); -} - -function toggleGroup(e, headerEl) { - if (e.target.closest('.bf-journal-controls')) return; - headerEl.classList.toggle('collapsed'); - headerEl.nextElementSibling.classList.toggle('hidden'); -} - -function handleHighlightLinkChange(comboboxEl) { - var input = comboboxEl.querySelector('.bf-combobox-input'); - var absId = input.dataset.selectedValue; - var bookfusionBookId = comboboxEl.dataset.bfId; - linkHighlight(bookfusionBookId, absId); -} - -function handleSaveJournalClick(btn) { - var bookTitle = btn.dataset.bookTitle; - var groupData = _currentHighlightGroups[bookTitle]; - if (!groupData) return; - var comboboxEl = btn.previousElementSibling; - var input = comboboxEl.querySelector('.bf-combobox-input'); - saveToJournal(btn, input.dataset.selectedValue, groupData.highlights); -} - -function saveToJournal(btn, absId, highlights) { - if (!absId) { - btn.textContent = 'Select a book first'; - btn.classList.add('error'); - setTimeout(function () { - btn.textContent = 'Save to Journal'; - btn.classList.remove('error'); - }, 2000); - return; - } - btn.disabled = true; - btn.innerHTML = getSpinnerHtml() + 'Saving\u2026'; - - var payload = highlights.map(function (h) { - return { - quote: h.quote || '', - chapter: (h.chapter_heading || '').replace(/^#{1,6}\s*/, ''), - highlighted_at: h.date || '' - }; - }); - - fetch('/api/bookfusion/save-journal', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ abs_id: absId, highlights: payload }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Save failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - btn.textContent = '\u2714 Saved ' + data.saved; - btn.classList.add('done'); - } else { - btn.textContent = data.error || 'Error'; - btn.classList.add('error'); - btn.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = err.message || 'Save failed'; - btn.classList.add('error'); - btn.disabled = false; - }); -} - -function linkHighlight(bookfusionBookId, absId) { - var inputs = document.querySelectorAll('.bf-combobox[data-book-title] .bf-combobox-input'); - inputs.forEach(function (s) { s.disabled = true; }); - fetch('/api/bookfusion/link-highlight', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ bookfusion_book_id: bookfusionBookId, abs_id: absId || null }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Link failed'); - return r.json(); - }) - .then(function (data) { - inputs.forEach(function (s) { s.disabled = false; }); - if (data.success) { - loadHighlights(); - } else { - console.error('Link failed:', data.error || 'unknown error'); - loadHighlights(); - } - }) - .catch(function (err) { - inputs.forEach(function (s) { s.disabled = false; }); - console.error('Link request failed:', err); - loadHighlights(); - }); -} - -/* ── Library Tab ── */ -var _libraryData = []; -var _dashboardBooks = []; -var _currentRenderedBooks = []; - -function loadLibrary() { - var list = document.getElementById('bf-library-list'); - list.textContent = ''; - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\u23F3'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = 'Loading\u2026'; - emptyEl.appendChild(icon); emptyEl.appendChild(h); - list.appendChild(emptyEl); - - fetch('/api/bookfusion/library') - .then(function (r) { - if (!r.ok) throw new Error('Failed to load library'); - return r.json(); - }) - .then(function (data) { - _libraryData = data.books || []; - _dashboardBooks = data.dashboard_books || []; - var urlQ = new URLSearchParams(window.location.search).get('q'); - if (urlQ) { - document.getElementById('bf-library-search').value = urlQ; - filterLibrary(); - } else { - renderLibrary(_libraryData); - } - }) - .catch(function (err) { - list.textContent = ''; - var errEl = document.createElement('div'); - errEl.className = 'bf-empty'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\u26A0\uFE0F'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = 'Failed to load library'; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; d.textContent = 'Please try again'; - errEl.appendChild(icon); errEl.appendChild(h); errEl.appendChild(d); - list.appendChild(errEl); - }); -} - -function filterLibrary() { - var q = document.getElementById('bf-library-search').value.trim().toLowerCase(); - if (!q) { - renderLibrary(_libraryData); - return; - } - var filtered = _libraryData.filter(function (b) { - var searchable = (b.title || '') + ' ' + (b.authors || '') + ' ' + (b.series || '') + ' ' + (b.filenames || []).join(' '); - return searchable.toLowerCase().indexOf(q) !== -1; - }); - renderLibrary(filtered); -} - -function _extractExt(filename) { - var dot = filename.lastIndexOf('.'); - if (dot > 0) return filename.substring(dot + 1).toUpperCase(); - return ''; -} - -function _formatDateNote(data) { - if (!data.dates_set) return null; - var parts = []; - if (data.started_at) parts.push(data.started_at); - if (data.finished_at) parts.push(data.finished_at); - if (!parts.length) return null; - var range = parts.join(' to '); - if (data.dates_source === 'hardcover') { - return 'Reading dates set from Hardcover \u2014 ' + range; - } - return 'Dates estimated from highlights \u2014 ' + range; -} - -/* Library item rendering — all user-facing text is escapeHtml-sanitized */ -function _renderBookItem(b, i) { - var metaHtml = ''; - if (b.authors) metaHtml += escapeHtml(b.authors); - if (b.series) { - if (b.authors) metaHtml += ' · '; - metaHtml += escapeHtml(b.series); - } - if (b.highlight_count > 0) { - metaHtml += ' ' + b.highlight_count + ' highlight' + (b.highlight_count !== 1 ? 's' : '') + ''; - } - - var filenames = b.filenames || (b.filename ? [b.filename] : []); - if (filenames.length) { - metaHtml += ' '; - var exts = []; - filenames.forEach(function (fn) { - var ext = _extractExt(fn); - if (ext && ext !== 'MD' && exts.indexOf(ext) === -1) exts.push(ext); - }); - exts.forEach(function (ext) { - metaHtml += '' + escapeHtml(ext) + ''; - }); - } - - var actionsHtml = ''; - if (b.on_dashboard) { - actionsHtml = '\u2714 Matched' + - ''; - } else { - var options = _dashboardBooks.map(function (db) { - return { value: db.abs_id, label: db.title, selected: false }; - }); - var comboboxHtml = createComboboxHtml(options, 'Match to book\u2026', ''); - actionsHtml = comboboxHtml + - '' + - '' + - ''; - } - - var hideBtn = b.hidden - ? '' - : ''; - - return '
' + - '
' + - '
' + escapeHtml(b.title || b.filename) + '
' + - '
' + metaHtml + '
' + - '
' + - '
' + - actionsHtml + - hideBtn + - '
' + - '
'; -} - -function renderLibrary(books) { - var list = document.getElementById('bf-library-list'); - _currentRenderedBooks = books; - - var visible = books.filter(function (b) { return !b.hidden; }); - var hidden = books.filter(function (b) { return b.hidden; }); - - if (!books.length) { - list.textContent = ''; - var emptyEl = document.createElement('div'); - emptyEl.className = 'bf-empty'; - var heading = _libraryData.length ? 'No matches' : 'No books in catalog'; - var desc = _libraryData.length ? 'Try a different filter' : 'Run a Full Re-sync to populate your library'; - var icon = document.createElement('div'); icon.className = 'bf-empty-icon'; icon.textContent = '\uD83D\uDCDA'; - var h = document.createElement('div'); h.className = 'bf-empty-heading'; h.textContent = heading; - var d = document.createElement('div'); d.className = 'bf-empty-desc'; d.textContent = desc; - emptyEl.appendChild(icon); emptyEl.appendChild(h); emptyEl.appendChild(d); - list.appendChild(emptyEl); - return; - } - - /* Library book data is escaped via escapeHtml before HTML insertion */ - var html = ''; - - if (visible.length) { - html += visible.map(function (b) { return _renderBookItem(b, books.indexOf(b)); }).join(''); - } else if (hidden.length) { - html += '
\uD83D\uDCDA
All books are hidden
Expand the hidden section below to manage them
'; - } - - if (hidden.length) { - html += '
' + - '
' + - '\u25BC' + - 'Hidden' + - '(' + hidden.length + ')' + - '
' + - '' + - '
'; - } - - list.innerHTML = html; // eslint-disable-line no-unsanitized/property - revealFirstMobileResult('bf-library-list'); -} - -/* toggleHiddenSection — provided by utils.js */ - -function handleHideClick(btn, index) { - var book = _currentRenderedBooks[index]; - btn.disabled = true; - btn.innerHTML = getSpinnerHtml() + 'Hiding\u2026'; - fetch('/api/bookfusion/hide', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ bookfusion_ids: book.bookfusion_ids || [book.bookfusion_id], hidden: true }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Hide failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - book.hidden = true; - renderLibrary(_currentRenderedBooks); - } else { - btn.textContent = 'Hide'; - btn.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = 'Hide'; - btn.disabled = false; - }); -} - -function handleUnhideClick(btn, index) { - var book = _currentRenderedBooks[index]; - btn.disabled = true; - btn.innerHTML = getSpinnerHtml() + 'Unhiding\u2026'; - fetch('/api/bookfusion/hide', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ bookfusion_ids: book.bookfusion_ids || [book.bookfusion_id], hidden: false }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Unhide failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - book.hidden = false; - renderLibrary(_currentRenderedBooks); - } else { - btn.textContent = 'Unhide'; - btn.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = 'Unhide'; - btn.disabled = false; - }); -} - -function handleUnlinkClick(btn, index) { - var book = _currentRenderedBooks[index]; - btn.disabled = true; - btn.innerHTML = getSpinnerHtml() + 'Unlinking\u2026'; - fetch('/api/bookfusion/unlink', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ abs_id: book.abs_id }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Unlink failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - book.on_dashboard = false; - book.abs_id = null; - renderLibrary(_currentRenderedBooks); - } else { - btn.textContent = 'Unlink'; - btn.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = 'Unlink'; - btn.disabled = false; - }); -} - -function handleLinkClick(btn, index) { - var book = _currentRenderedBooks[index]; - var comboboxEl = btn.previousElementSibling; - var input = comboboxEl.querySelector('.bf-combobox-input'); - if (input.dataset.selectedValue) { - matchToBook(btn, input, book, index); - } else { - btn.classList.add('error'); - btn.textContent = 'Select a book first'; - setTimeout(function () { - btn.classList.remove('error'); - btn.textContent = 'Link'; - }, 2000); - } -} - -function handleAddClick(btn, index, status) { - var book = _currentRenderedBooks[index]; - addToDashboard(btn, book, index, status); -} - -function _showDateNote(actionsEl, data) { - var note = _formatDateNote(data); - if (!note) return; - var noteEl = document.createElement('div'); - noteEl.className = 'bf-date-note'; - noteEl.textContent = note; - noteEl.style.cssText = 'font-size: 0.75rem; color: var(--color-text-muted); margin-top: 4px; opacity: 0; transition: opacity 0.4s; width: 100%;'; - actionsEl.appendChild(noteEl); - requestAnimationFrame(function () { noteEl.style.opacity = '1'; }); -} - -function matchToBook(btn, input, book, index) { - var absId = input.dataset.selectedValue; - btn.disabled = true; - input.disabled = true; - btn.innerHTML = getSpinnerHtml() + 'Linking\u2026'; - - fetch('/api/bookfusion/match-to-book', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ bookfusion_ids: book.bookfusion_ids || [book.bookfusion_id], abs_id: absId }) - }) - .then(function (r) { - if (!r.ok) throw new Error('Match failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - book.on_dashboard = true; - book.abs_id = absId; - renderLibrary(_currentRenderedBooks); - setTimeout(function () { - var actionsEls = document.querySelectorAll('.bf-library-actions'); - var actionsEl = Array.from(actionsEls).find(function (el) { return el.dataset.index == index; }); - if (actionsEl) _showDateNote(actionsEl, data); - }, 50); - } else { - btn.textContent = 'Link'; - btn.disabled = false; - input.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = 'Link'; - btn.disabled = false; - input.disabled = false; - }); -} - -function addToDashboard(btn, book, index, status) { - btn.disabled = true; - btn.textContent = 'Adding\u2026'; - - var payload = { bookfusion_ids: book.bookfusion_ids || [book.bookfusion_id] }; - if (status) payload.status = status; - - fetch('/api/bookfusion/add-to-dashboard', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify(payload) - }) - .then(function (r) { - if (!r.ok) throw new Error('Add failed'); - return r.json(); - }) - .then(function (data) { - if (data.success) { - book.on_dashboard = true; - book.abs_id = data.abs_id; - renderLibrary(_currentRenderedBooks); - setTimeout(function () { - var actionsEls = document.querySelectorAll('.bf-library-actions'); - var actionsEl = Array.from(actionsEls).find(function (el) { return el.dataset.index == index; }); - if (actionsEl) _showDateNote(actionsEl, data); - }, 50); - } else { - btn.textContent = data.error || 'Error'; - btn.classList.add('error'); - btn.disabled = false; - } - }) - .catch(function (err) { - btn.textContent = err.message || 'Error'; - btn.classList.add('error'); - btn.disabled = false; - }); -} - -/* ── Init ── */ -document.addEventListener('focusin', function (e) { - if (e.target.matches('#bf-search-input, #bf-library-search, .bf-combobox-input')) { - keepElementVisible(e.target, 'center'); - } -}); - -window.addEventListener('resize', function () { - var active = document.activeElement; - if (active && active.matches && active.matches('#bf-search-input, #bf-library-search, .bf-combobox-input')) { - keepElementVisible(active, 'center'); - } -}); - -var urlTab = new URLSearchParams(window.location.search).get('tab'); -if (urlTab && document.getElementById('bf-panel-' + urlTab)) { - switchBFTab(urlTab); -} else { - loadLibrary(); -} diff --git a/static/js/reading.js b/static/js/reading.js index 21df948..2d78606 100644 --- a/static/js/reading.js +++ b/static/js/reading.js @@ -726,7 +726,7 @@ function initReadingDetail() { if (editJournalId) { const item = timeline.querySelector(`.r-tl-item[data-journal-id="${editJournalId}"]`); const text = item?.querySelector('.r-tl-text'); - if (text) text.textContent = data.journal.entry; + if (text) renderJournalEntry(text, data.journal); if (item) item.dataset.entry = data.journal.entry; } else { const empty = timeline.querySelector('.r-journal-empty'); @@ -786,7 +786,7 @@ function initReadingDetail() { return; } if (eventType !== 'note') return; - const text = item?.querySelector('.r-tl-text')?.textContent || ''; + const text = item?.dataset.entry || ''; form.dataset.editJournalId = journalId; textarea.value = text; if (submitBtn) submitBtn.textContent = 'Save Note'; @@ -822,6 +822,16 @@ function initReadingDetail() { } +function renderJournalEntry(container, journal) { + if (!container) return; + if (journal.entry_html) { + container.innerHTML = journal.entry_html; + return; + } + container.textContent = journal.entry || ''; +} + + /** Build a journal timeline node using safe DOM methods. */ function buildJournalNode(j) { const item = document.createElement('div'); @@ -868,9 +878,9 @@ function buildJournalNode(j) { body.appendChild(head); if (j.entry) { - const text = document.createElement('p'); + const text = document.createElement('div'); text.className = 'r-tl-text'; - text.textContent = j.entry; + renderJournalEntry(text, j); body.appendChild(text); } diff --git a/static/js/settings.js b/static/js/settings.js index 0bb7e36..5cb6a1a 100644 --- a/static/js/settings.js +++ b/static/js/settings.js @@ -528,4 +528,41 @@ document.addEventListener('DOMContentLoaded', function () { } setupDirtyCheck(); + + // BookFusion re-sync all button handler + var bfResyncBtn = document.getElementById('bf-resync-all-btn'); + if (bfResyncBtn) { + bfResyncBtn.addEventListener('click', function() { + if (!confirm('This will re-download all highlights from BookFusion. Continue?')) { + return; + } + bfResyncBtn.classList.remove('error', 'done'); + bfResyncBtn.disabled = true; + bfResyncBtn.textContent = 'Syncing...'; + fetch('/api/bookfusion/sync-highlights', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ full_resync: true }) + }) + .then(function(r) { return r.json(); }) + .then(function(data) { + if (data.success) { + bfResyncBtn.classList.remove('error'); + bfResyncBtn.textContent = 'Synced (' + data.new_highlights + ' new)'; + bfResyncBtn.classList.add('done'); + } else { + bfResyncBtn.classList.remove('done'); + bfResyncBtn.textContent = data.error || 'Failed'; + bfResyncBtn.classList.add('error'); + bfResyncBtn.disabled = false; + } + }) + .catch(function() { + bfResyncBtn.classList.remove('done'); + bfResyncBtn.textContent = 'Error'; + bfResyncBtn.classList.add('error'); + bfResyncBtn.disabled = false; + }); + }); + } }); diff --git a/templates/bookfusion.html b/templates/bookfusion.html deleted file mode 100644 index c32395f..0000000 --- a/templates/bookfusion.html +++ /dev/null @@ -1,81 +0,0 @@ - - - - - - {{ title_prefix }}BookFusion - PageKeeper - - - - - - - - - - - - - {% include 'partials/navbar.html' %} - -
-
-
-

BookFusion

-
-
- -
- - - -
- - -
-
- -
-
-
-
📚
-
Loading library catalog...
-
-
-
- - -
-
- - - -
-
-
-
📝
-
Sync your highlights
-
Click "Sync Highlights" to fetch highlights from BookFusion
-
-
-
- - -
-
- -
-
-
-
Search your Grimmory library
-
Type above to find books to upload
-
-
-
-
- - - - - - diff --git a/templates/partials/navbar.html b/templates/partials/navbar.html index c04f09b..b6dc7dd 100644 --- a/templates/partials/navbar.html +++ b/templates/partials/navbar.html @@ -82,9 +82,6 @@

PageKeeper

{% endif %} Dashboard Reading Log - {% if show_bookfusion_page %} - BookFusion Books - {% endif %} {% if get_bool('SUGGESTIONS_ENABLED') %} Suggestions{% if suggestion_count %} {{ suggestion_count }}{% endif %} {% endif %} diff --git a/templates/reading_detail.html b/templates/reading_detail.html index ec61ee0..1652a49 100644 --- a/templates/reading_detail.html +++ b/templates/reading_detail.html @@ -42,8 +42,12 @@
{% if book.cover_url %} - + @@ -286,15 +290,18 @@

{{ book.title }}{% if metadata.subtitle %}: {{ metada {% if integrations.bookfusion %}
BF - BookFusion + BookFusion Linked +
- {% else %} - + {% elif bookfusion_upload_eligible %} +
BF BookFusion - Link - + +
{% endif %} {% endif %}

@@ -501,7 +508,7 @@

Timeline

{% endif %}
{% if journal.entry %} -

{{ journal.entry }}

+
{{ journal.entry|markdown }}
{% endif %} {% if journal.event in ['note', 'highlight', 'progress', 'resumed', 'paused', 'dnf'] or (journal.event in ['started', 'finished'] and journal.id) %}
@@ -536,8 +543,12 @@

Timeline

BookFusion Highlights

{{ bf_highlights|length }} highlight{{ 's' if bf_highlights|length != 1 }}

- +
+ + +
{% set current_chapter = namespace(value='') %} @@ -798,6 +809,110 @@

Link Storyteller

var _rdTitle = {{ book.title | tojson }}; var _rdPendingJournalId = null; + // BookFusion upload button handler + (function() { + var uploadBtn = document.getElementById('bf-upload-btn'); + if (uploadBtn) { + uploadBtn.addEventListener('click', function() { + var absId = this.dataset.absId; + uploadBtn.disabled = true; + uploadBtn.textContent = 'Uploading...'; + fetch('/api/bookfusion/upload', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ abs_id: absId }) + }) + .then(function(r) { return r.json(); }) + .then(function(data) { + if (data.success) { + if (data.already_linked) { + uploadBtn.textContent = 'Already linked'; + } else { + uploadBtn.textContent = 'Uploaded'; + } + uploadBtn.classList.add('done'); + setTimeout(function() { window.location.reload(); }, 1500); + } else { + uploadBtn.textContent = data.error || 'Failed'; + uploadBtn.classList.add('error'); + uploadBtn.disabled = false; + } + }) + .catch(function() { + uploadBtn.textContent = 'Error'; + uploadBtn.classList.add('error'); + uploadBtn.disabled = false; + }); + }); + } + })(); + + // BookFusion refresh button handler (service row) + (function() { + var refreshBtn = document.getElementById('bf-refresh-btn'); + if (refreshBtn) { + refreshBtn.addEventListener('click', function() { + var absId = this.dataset.absId; + refreshBtn.disabled = true; + refreshBtn.textContent = 'Syncing...'; + fetch('/api/bookfusion/sync-book', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ abs_id: absId }) + }) + .then(function(r) { return r.json(); }) + .then(function(data) { + if (data.success) { + refreshBtn.textContent = 'Synced'; + setTimeout(function() { window.location.reload(); }, 1000); + } else { + refreshBtn.textContent = data.error || 'Failed'; + refreshBtn.classList.add('error'); + refreshBtn.disabled = false; + } + }) + .catch(function() { + refreshBtn.textContent = 'Error'; + refreshBtn.classList.add('error'); + refreshBtn.disabled = false; + }); + }); + } + })(); + + // BookFusion refresh highlights button handler (highlights tab) + (function() { + var refreshHighlightsBtn = document.getElementById('bf-refresh-highlights-btn'); + if (refreshHighlightsBtn) { + refreshHighlightsBtn.addEventListener('click', function() { + var absId = this.dataset.absId; + refreshHighlightsBtn.disabled = true; + refreshHighlightsBtn.textContent = 'Syncing...'; + fetch('/api/bookfusion/sync-book', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ abs_id: absId }) + }) + .then(function(r) { return r.json(); }) + .then(function(data) { + if (data.success) { + refreshHighlightsBtn.textContent = 'Synced'; + setTimeout(function() { window.location.reload(); }, 1000); + } else { + refreshHighlightsBtn.textContent = data.error || 'Failed'; + refreshHighlightsBtn.classList.add('error'); + refreshHighlightsBtn.disabled = false; + } + }) + .catch(function() { + refreshHighlightsBtn.textContent = 'Error'; + refreshHighlightsBtn.classList.add('error'); + refreshHighlightsBtn.disabled = false; + }); + }); + } + })(); + {% if active_tab != 'overview' %} switchTab({{ active_tab | tojson }}); {% endif %} @@ -811,10 +926,41 @@

Link Storyteller

} })(); + (function() { + var coverImage = document.getElementById('reading-cover-image'); + if (!coverImage) return; + + function showPlaceholder() { + coverImage.style.display = 'none'; + if (coverImage.nextElementSibling) { + coverImage.nextElementSibling.classList.remove('hidden'); + } + } + + coverImage.addEventListener('error', function() { + var fallbackCoverUrl = coverImage.dataset.fallbackCoverUrl; + var kosyncCoverUrl = coverImage.dataset.kosyncCoverUrl; + + if (fallbackCoverUrl && coverImage.src !== fallbackCoverUrl) { + coverImage.src = fallbackCoverUrl; + delete coverImage.dataset.fallbackCoverUrl; + return; + } + + if (kosyncCoverUrl && coverImage.src !== window.location.origin + kosyncCoverUrl) { + coverImage.src = kosyncCoverUrl; + delete coverImage.dataset.kosyncCoverUrl; + return; + } + + showPlaceholder(); + }); + })(); + // Hardcover re-link function openHcRelink(btn) { hardcoverModalState.bookId = btn.dataset.bookId; - hardcoverModalState.bookTitle = btn.dataset.title || '{{ book.title|e }}'; + hardcoverModalState.bookTitle = btn.dataset.title || _rdTitle; hardcoverModalState.bookData = null; hardcoverModalState.selectedEditionId = null; openHardcoverModal(); @@ -1077,26 +1223,32 @@

Link Storyteller

var importBtn = document.getElementById('bulk-import-highlights'); if (!importBtn) return; importBtn.addEventListener('click', function() { - var bookId = importBtn.dataset.bookId; + importBtn.classList.remove('error', 'done'); importBtn.disabled = true; importBtn.textContent = 'Importing...'; fetch('/api/bookfusion/save-journal', { method: 'POST', headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ abs_id: bookId }), + body: JSON.stringify({ abs_id: _rdAbsId }), }) .then(function(r) { return r.json(); }) .then(function(data) { if (data.success) { + importBtn.classList.remove('error'); + importBtn.classList.add('done'); importBtn.textContent = 'Imported ' + data.saved + ' highlights'; } else { + importBtn.classList.remove('done'); + importBtn.classList.add('error'); importBtn.textContent = data.error || 'Import failed'; importBtn.disabled = false; } }) .catch(function() { - importBtn.textContent = 'Import All to Journal'; + importBtn.classList.remove('done'); + importBtn.classList.add('error'); + importBtn.textContent = 'Error'; importBtn.disabled = false; }); }); diff --git a/templates/settings.html b/templates/settings.html index ffc75a4..952b4a5 100644 --- a/templates/settings.html +++ b/templates/settings.html @@ -591,7 +591,7 @@

Calibre-Web Automated

BookFusion → Settings → Calibre section. Used for book uploads.
-
- When linking BookFusion books, PageKeeper sets reading dates automatically - (from Hardcover if configured, otherwise from highlight timestamps) - and fetches cover images from Hardcover. +
+
+ +

Re-sync all BookFusion highlights from the server. This refreshes cached highlight data and does not re-import journal entries.

+
diff --git a/tests/conftest.py b/tests/conftest.py index 8553651..5379396 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -5,7 +5,7 @@ import sys import tempfile from pathlib import Path -from types import ModuleType +from types import ModuleType, SimpleNamespace from unittest.mock import Mock import pytest @@ -16,6 +16,9 @@ if _mod_name not in sys.modules: sys.modules[_mod_name] = ModuleType(_mod_name) +sys.modules.setdefault("nh3", SimpleNamespace(clean=lambda value, tags=None, attributes=None: value)) +sys.modules.setdefault("mistune", SimpleNamespace(html=lambda value: f"

{value}

" if value else "")) + # ── MockABSService ───────────────────────────────────────────────── # Lightweight stand-in for ABSService that avoids network calls. @@ -103,6 +106,7 @@ def __init__(self): # ── Paths (temp) ── self._tmp = Path(tempfile.gettempdir()) + self.config = {} # ── Accessors (match Container's callable interface) ── diff --git a/tests/test_app_runtime.py b/tests/test_app_runtime.py new file mode 100644 index 0000000..c05f809 --- /dev/null +++ b/tests/test_app_runtime.py @@ -0,0 +1,80 @@ +# pyright: reportMissingImports=false + +from unittest.mock import MagicMock, Mock, patch + +import schedule + +from src.app_runtime import apply_settings, reconcile_socket_listener + + +class TestApplySettingsHelpers: + def setup_method(self): + schedule.clear() + + def teardown_method(self): + schedule.clear() + + def test_apply_settings_updates_app_config_flags(self): + app = MagicMock() + app.config = { + "sync_manager": Mock(), + "abs_listener": None, + "_abs_listener_server": "", + "_abs_listener_key": "", + } + + with ( + patch.dict( + "os.environ", + { + "SYNC_PERIOD_MINS": "7", + "LOG_LEVEL": "INFO", + "ABS_COLLECTION_NAME": "Shelf Sync", + "SUGGESTIONS_ENABLED": "true", + "INSTANT_SYNC_ENABLED": "false", + "ABS_SOCKET_ENABLED": "false", + "TELEGRAM_ENABLED": "false", + }, + clear=False, + ), + patch("src.utils.logging_utils.reconcile_telegram_logging"), + ): + apply_settings(app) + + assert app.config["ABS_COLLECTION_NAME"] == "Shelf Sync" + assert app.config["SUGGESTIONS_ENABLED"] is True + jobs = schedule.get_jobs("sync_cycle") + assert len(jobs) == 1 + assert jobs[0].interval == 7 + + +class TestSocketListenerHelpers: + @patch("src.services.abs_socket_listener.ABSSocketListener") + @patch("threading.Thread") + def test_reconcile_socket_listener_starts_listener(self, mock_thread_cls, mock_listener_cls): + app = MagicMock() + app.config = { + "abs_listener": None, + "_abs_listener_server": "", + "_abs_listener_key": "", + "database_service": Mock(), + "sync_manager": Mock(), + } + mock_listener = Mock() + mock_listener_cls.return_value = mock_listener + mock_thread_cls.return_value = Mock() + + with patch.dict( + "os.environ", + { + "INSTANT_SYNC_ENABLED": "true", + "ABS_SOCKET_ENABLED": "true", + "ABS_SERVER": "http://abs:13378", + "ABS_KEY": "secret", + }, + clear=False, + ): + reconcile_socket_listener(app) + + assert app.config["abs_listener"] is mock_listener + mock_thread_cls.return_value.start.assert_called_once() diff --git a/tests/test_bookfusion_repository.py b/tests/test_bookfusion_repository.py new file mode 100644 index 0000000..9f8372c --- /dev/null +++ b/tests/test_bookfusion_repository.py @@ -0,0 +1,49 @@ +import tempfile +from pathlib import Path + +from src.db.database_service import DatabaseService +from src.db.models import Book, BookfusionBook + + +def test_unlink_bookfusion_by_book_id_clears_book_and_highlight_links(): + with tempfile.TemporaryDirectory() as temp_dir: + db_path = Path(temp_dir) / "test.db" + db_service = DatabaseService(str(db_path)) + + book = db_service.save_book( + Book( + abs_id="test-abs-id", + title="Test Book", + ebook_filename="test.epub", + kosync_doc_id="doc-1", + status="active", + ) + ) + + db_service.save_bookfusion_book( + BookfusionBook( + bookfusion_id="bf-123", + title="BF Book", + matched_book_id=book.id, + ) + ) + db_service.save_bookfusion_highlights( + [ + { + "bookfusion_book_id": "bf-123", + "highlight_id": "hl-1", + "content": "Quote", + "quote_text": "Quote", + "book_title": "BF Book", + "chapter_heading": "Chapter 1", + } + ] + ) + db_service.link_bookfusion_highlights_by_book_id("bf-123", book.id) + + db_service.unlink_bookfusion_by_book_id(book.id) + + assert db_service.get_bookfusion_book_by_book_id(book.id) is None + assert db_service.get_bookfusion_highlights_for_book_by_book_id(book.id) == [] + + db_service.db_manager.close() diff --git a/tests/test_bookfusion_routes.py b/tests/test_bookfusion_routes.py index 309e36a..e3ec74e 100644 --- a/tests/test_bookfusion_routes.py +++ b/tests/test_bookfusion_routes.py @@ -1,15 +1,18 @@ """Tests for BookFusion blueprint routes.""" from datetime import datetime -from unittest.mock import Mock +from unittest.mock import Mock, patch -from tests.conftest import MockContainer -# ── Helpers ──────────────────────────────────────────────────────── - - -def _make_mock_book(abs_id="test-abs-id", title="Test Book", book_id=1, status="active"): - """Return a Mock that behaves like a Book ORM instance.""" +def _make_mock_book( + abs_id="test-abs-id", + title="Test Book", + book_id=1, + status="active", + ebook_filename="book.epub", + original_ebook_filename=None, + author="Test Author", +): book = Mock() book.id = book_id book.abs_id = abs_id @@ -18,6 +21,9 @@ def _make_mock_book(abs_id="test-abs-id", title="Test Book", book_id=1, status=" book.started_at = None book.finished_at = None book.sync_mode = "audiobook" + book.ebook_filename = ebook_filename + book.original_ebook_filename = original_ebook_filename + book.author = author return book @@ -27,714 +33,300 @@ def _make_bf_book( authors="Author", filename="book.epub", highlight_count=3, - matched_abs_id=None, matched_book_id=None, - hidden=False, - series=None, - tags=None, ): - """Return a Mock that behaves like a BookfusionBook ORM instance.""" bf = Mock() bf.bookfusion_id = bookfusion_id bf.title = title bf.authors = authors bf.filename = filename bf.highlight_count = highlight_count - bf.matched_abs_id = matched_abs_id bf.matched_book_id = matched_book_id - bf.hidden = hidden - bf.series = series - bf.tags = tags - bf.frontmatter = None return bf def _make_bf_highlight( - hl_id=1, - highlight_id="hl-1", bookfusion_book_id="bf-123", book_title="BF Book", content="Some highlight text", quote_text=None, chapter_heading=None, - matched_abs_id=None, highlighted_at=None, ): - """Return a Mock that behaves like a BookfusionHighlight ORM instance.""" hl = Mock() - hl.id = hl_id - hl.highlight_id = highlight_id hl.bookfusion_book_id = bookfusion_book_id hl.book_title = book_title hl.content = content hl.quote_text = quote_text hl.chapter_heading = chapter_heading - hl.matched_abs_id = matched_abs_id hl.highlighted_at = highlighted_at return hl -# ── Grimmory Books ───────────────────────────────────────────────── +def test_upload_requires_data(client): + resp = client.post("/api/bookfusion/upload", json={}) + assert resp.status_code == 400 + assert resp.get_json()["error"] == "No data provided" -def test_grimmory_books_returns_supported_formats(client, mock_container): - mock_container.mock_grimmory_client.is_configured.return_value = True - mock_container.mock_grimmory_client.get_all_books.return_value = [ - {"id": 1, "title": "Book One", "authors": "Author A", "fileName": "book1.epub"}, - {"id": 2, "title": "Book Two", "authors": "Author B", "fileName": "book2.txt"}, - {"id": 3, "title": "Book Three", "authors": "Author C", "fileName": "book3.pdf"}, - ] - resp = client.get("/api/bookfusion/grimmory-books") - assert resp.status_code == 200 - data = resp.get_json() - assert len(data) == 2 - assert data[0]["title"] == "Book One" - assert data[1]["title"] == "Book Three" +def test_upload_book_not_found(client, mock_container): + mock_container.mock_bookfusion_client.upload_api_key = "test-key" + mock_container.mock_database_service.get_book_by_ref.return_value = None + resp = client.post("/api/bookfusion/upload", json={"abs_id": "nonexistent"}) -def test_grimmory_books_with_search_query(client, mock_container): - mock_container.mock_grimmory_client.is_configured.return_value = True - mock_container.mock_grimmory_client.search_books.return_value = [ - {"id": 5, "title": "Searched", "authors": "A", "fileName": "searched.epub"}, - ] - resp = client.get("/api/bookfusion/grimmory-books?q=searched") - assert resp.status_code == 200 - data = resp.get_json() - assert len(data) == 1 - mock_container.mock_grimmory_client.search_books.assert_called_once_with("searched") + assert resp.status_code == 404 + assert resp.get_json()["error"] == "Book not found" -def test_grimmory_books_not_configured(client, mock_container): - mock_container.mock_grimmory_client.is_configured.return_value = False - resp = client.get("/api/bookfusion/grimmory-books") - assert resp.status_code == 200 - assert resp.get_json() == [] +def test_upload_requires_ebook_file(client, mock_container): + mock_container.mock_bookfusion_client.upload_api_key = "test-key" + mock_container.mock_database_service.get_book_by_ref.return_value = _make_mock_book( + ebook_filename=None, + original_ebook_filename=None, + ) + resp = client.post("/api/bookfusion/upload", json={"abs_id": "test-abs-id"}) -def test_grimmory_books_exception_returns_empty(client, mock_container): - mock_container.mock_grimmory_client.is_configured.return_value = True - mock_container.mock_grimmory_client.get_all_books.side_effect = Exception("Grimmory down") - resp = client.get("/api/bookfusion/grimmory-books") - assert resp.status_code == 200 - assert resp.get_json() == [] + assert resp.status_code == 400 + assert resp.get_json()["error"] == "No ebook file associated with this book" -# ── Upload Book ──────────────────────────────────────────────────── +def test_upload_requires_upload_api_key(client, mock_container): + mock_container.mock_bookfusion_client.upload_api_key = None + resp = client.post("/api/bookfusion/upload", json={"abs_id": "test-abs-id"}) -def test_upload_book_success(client, mock_container): - mock_container.mock_bookfusion_client.upload_api_key = "key-123" - mock_container.mock_grimmory_client.is_configured.return_value = True - mock_container.mock_grimmory_client.download_book.return_value = b"file-bytes" - mock_container.mock_bookfusion_client.upload_book.return_value = {"id": "new-bf-id"} + assert resp.status_code == 400 + assert resp.get_json()["error"] == "BookFusion upload API key not configured" - resp = client.post( - "/api/bookfusion/upload", - json={ - "book_id": 10, - "title": "My Book", - "authors": "Auth", - "fileName": "my.epub", - }, - ) - assert resp.status_code == 200 - data = resp.get_json() - assert data["success"] is True - assert data["result"] == {"id": "new-bf-id"} +def test_upload_success_when_local_epub_missing(client, mock_container): + mock_container.mock_bookfusion_client.upload_api_key = "test-key" + mock_container.mock_database_service.get_book_by_ref.return_value = _make_mock_book(ebook_filename="test.epub") -def test_upload_book_no_data(client, mock_container): - resp = client.post("/api/bookfusion/upload", content_type="application/json", data="") - assert resp.status_code == 400 + with patch("src.utils.epub_resolver.get_local_epub", return_value=None): + resp = client.post("/api/bookfusion/upload", json={"abs_id": "test-abs-id"}) + assert resp.status_code == 500 + assert resp.get_json()["error"] == "Could not locate ebook file" -def test_upload_book_missing_book_id(client, mock_container): - resp = client.post("/api/bookfusion/upload", json={"title": "Missing ID"}) - assert resp.status_code == 400 - assert "book_id required" in resp.get_json()["error"] +def test_upload_saves_bookfusion_link(client, mock_container, tmp_path): + book = _make_mock_book(ebook_filename="test.epub", book_id=42) + mock_container.mock_bookfusion_client.upload_api_key = "test-key" + mock_container.mock_bookfusion_client.upload_book.return_value = {"id": "bf-123", "title": "Test Book"} + mock_container.mock_database_service.get_book_by_ref.return_value = book + mock_container.mock_database_service.get_bookfusion_book_by_book_id.return_value = None -def test_upload_book_no_api_key(client, mock_container): - mock_container.mock_bookfusion_client.upload_api_key = "" - resp = client.post("/api/bookfusion/upload", json={"book_id": 1}) - assert resp.status_code == 400 - assert "API key" in resp.get_json()["error"] + test_file = tmp_path / "test.epub" + test_file.write_bytes(b"fake epub content") + with patch("src.utils.epub_resolver.get_local_epub", return_value=test_file): + resp = client.post("/api/bookfusion/upload", json={"abs_id": "test-abs-id"}) -def test_upload_book_grimmory_not_configured(client, mock_container): - mock_container.mock_bookfusion_client.upload_api_key = "key" - mock_container.mock_grimmory_client.is_configured.return_value = False - resp = client.post("/api/bookfusion/upload", json={"book_id": 1}) - assert resp.status_code == 400 - assert "Grimmory not configured" in resp.get_json()["error"] + assert resp.status_code == 200 + data = resp.get_json() + assert data["success"] is True + assert data["already_linked"] is False + mock_container.mock_database_service.save_bookfusion_book.assert_called_once() -def test_upload_book_download_fails(client, mock_container): - mock_container.mock_bookfusion_client.upload_api_key = "key" - mock_container.mock_grimmory_client.is_configured.return_value = True - mock_container.mock_grimmory_client.download_book.return_value = None - resp = client.post("/api/bookfusion/upload", json={"book_id": 1}) - assert resp.status_code == 500 - assert "Failed to download" in resp.get_json()["error"] - - -def test_upload_book_upload_fails(client, mock_container): - mock_container.mock_bookfusion_client.upload_api_key = "key" - mock_container.mock_grimmory_client.is_configured.return_value = True - mock_container.mock_grimmory_client.download_book.return_value = b"bytes" - mock_container.mock_bookfusion_client.upload_book.return_value = None - resp = client.post( - "/api/bookfusion/upload", - json={ - "book_id": 1, - "fileName": "x.epub", - }, - ) - assert resp.status_code == 500 - assert "Upload to BookFusion failed" in resp.get_json()["error"] +def test_sync_highlights_requires_api_key(client, mock_container): + mock_container.mock_bookfusion_client.highlights_api_key = None + resp = client.post("/api/bookfusion/sync-highlights") -# ── Sync Highlights ──────────────────────────────────────────────── + assert resp.status_code == 400 + assert resp.get_json()["error"] == "BookFusion highlights API key not configured" def test_sync_highlights_success(client, mock_container): - mock_container.mock_bookfusion_client.highlights_api_key = "hlkey" + mock_container.mock_bookfusion_client.highlights_api_key = "test-key" mock_container.mock_bookfusion_client.sync_all_highlights.return_value = { "new_highlights": 5, "books_saved": 2, - "new_ids": ["a", "b"], + "new_ids": ["hl-1", "hl-2"], } - mock_container.mock_database_service.get_unmatched_bookfusion_highlights.return_value = [] resp = client.post("/api/bookfusion/sync-highlights") - assert resp.status_code == 200 - data = resp.get_json() - assert data["success"] is True - assert data["new_highlights"] == 5 - assert data["books_saved"] == 2 - assert data["auto_matched"] == 0 - -def test_sync_highlights_no_api_key(client, mock_container): - mock_container.mock_bookfusion_client.highlights_api_key = "" - resp = client.post("/api/bookfusion/sync-highlights") - assert resp.status_code == 400 - assert "API key" in resp.get_json()["error"] + assert resp.status_code == 200 + assert resp.get_json() == { + "success": True, + "new_highlights": 5, + "books_saved": 2, + "new_ids": ["hl-1", "hl-2"], + } -def test_sync_highlights_full_resync_clears_cursor(client, mock_container): - mock_container.mock_bookfusion_client.highlights_api_key = "hlkey" +def test_sync_highlights_full_resync(client, mock_container): + mock_container.mock_bookfusion_client.highlights_api_key = "test-key" mock_container.mock_bookfusion_client.sync_all_highlights.return_value = { - "new_highlights": 0, - "books_saved": 0, + "new_highlights": 10, + "books_saved": 3, "new_ids": [], } - mock_container.mock_database_service.get_unmatched_bookfusion_highlights.return_value = [] resp = client.post("/api/bookfusion/sync-highlights", json={"full_resync": True}) + assert resp.status_code == 200 mock_container.mock_database_service.set_bookfusion_sync_cursor.assert_called_once_with(None) -def test_sync_highlights_exception(client, mock_container): - mock_container.mock_bookfusion_client.highlights_api_key = "hlkey" - mock_container.mock_bookfusion_client.sync_all_highlights.side_effect = Exception("API error") - - resp = client.post("/api/bookfusion/sync-highlights") - assert resp.status_code == 500 - assert "failed" in resp.get_json()["error"].lower() - - -# ── Get Highlights ───────────────────────────────────────────────── - - -def test_get_highlights_empty(client, mock_container): - mock_container.mock_database_service.get_bookfusion_highlights.return_value = [] - mock_container.mock_database_service.get_bookfusion_sync_cursor.return_value = None - mock_container.mock_database_service.get_all_books.return_value = [] - - resp = client.get("/api/bookfusion/highlights") - assert resp.status_code == 200 - data = resp.get_json() - assert data["highlights"] == {} - assert data["has_synced"] is False - +def test_sync_book_requires_data(client): + resp = client.post("/api/bookfusion/sync-book", json={}) + assert resp.status_code == 400 + assert resp.get_json()["error"] == "No data provided" -def test_get_highlights_with_data(client, mock_container): - hl = _make_bf_highlight( - highlighted_at=datetime(2025, 1, 15, 10, 30, 0), - quote_text="A great quote", - matched_abs_id="book-1", - ) - mock_container.mock_database_service.get_bookfusion_highlights.return_value = [hl] - mock_container.mock_database_service.get_bookfusion_sync_cursor.return_value = "cursor-abc" - mock_container.mock_database_service.get_all_books.return_value = [ - _make_mock_book(abs_id="book-1", title="Dashboard Book"), - ] - resp = client.get("/api/bookfusion/highlights") - assert resp.status_code == 200 - data = resp.get_json() - assert data["has_synced"] is True - assert len(data["books"]) == 1 - # The highlight should be grouped under the book title - assert len(data["highlights"]) == 1 +def test_sync_book_book_not_found(client, mock_container): + mock_container.mock_database_service.get_book_by_ref.return_value = None + resp = client.post("/api/bookfusion/sync-book", json={"abs_id": "nonexistent"}) -# ── Link Highlight ───────────────────────────────────────────────── + assert resp.status_code == 404 + assert resp.get_json()["error"] == "Book not found" -def test_link_highlight_success(client, mock_container): +def test_sync_book_requires_api_key(client, mock_container): book = _make_mock_book() mock_container.mock_database_service.get_book_by_ref.return_value = book + mock_container.mock_bookfusion_client.highlights_api_key = None - resp = client.post( - "/api/bookfusion/link-highlight", - json={ - "bookfusion_book_id": "bf-123", - "abs_id": "test-abs-id", - }, - ) - assert resp.status_code == 200 - assert resp.get_json()["success"] is True - mock_container.mock_database_service.link_bookfusion_highlights_by_book_id.assert_called_once_with( - "bf-123", - book.id, - ) - + resp = client.post("/api/bookfusion/sync-book", json={"abs_id": book.abs_id}) -def test_link_highlight_unlink(client, mock_container): - resp = client.post( - "/api/bookfusion/link-highlight", - json={ - "bookfusion_book_id": "bf-123", - "abs_id": "", - }, - ) - assert resp.status_code == 200 - mock_container.mock_database_service.link_bookfusion_highlights_by_book_id.assert_called_once_with( - "bf-123", - None, - ) - - -def test_link_highlight_no_data(client, mock_container): - resp = client.post("/api/bookfusion/link-highlight", content_type="application/json", data="") assert resp.status_code == 400 + assert resp.get_json()["error"] == "BookFusion highlights API key not configured" -def test_link_highlight_missing_bookfusion_id(client, mock_container): - resp = client.post("/api/bookfusion/link-highlight", json={"abs_id": "x"}) - assert resp.status_code == 400 - assert "bookfusion_book_id required" in resp.get_json()["error"] +def test_sync_book_not_linked(client, mock_container): + book = _make_mock_book() + mock_container.mock_database_service.get_book_by_ref.return_value = book + mock_container.mock_database_service.get_bookfusion_books_by_book_id.return_value = [] + mock_container.mock_bookfusion_client.highlights_api_key = "test-key" + resp = client.post("/api/bookfusion/sync-book", json={"abs_id": book.abs_id}) -# ── Save Journal ─────────────────────────────────────────────────── + assert resp.status_code == 404 + assert resp.get_json()["error"] == "BookFusion link not found for this book" -def test_save_journal_success(client, mock_container): - book = _make_mock_book() +def test_sync_book_success(client, mock_container): + book = _make_mock_book(book_id=42) + bf_book = _make_bf_book(bookfusion_id="bf-123", matched_book_id=42) mock_container.mock_database_service.get_book_by_ref.return_value = book - mock_container.mock_database_service.cleanup_bookfusion_import_notes.return_value = {"deleted": 0} - - resp = client.post( - "/api/bookfusion/save-journal", - json={ - "abs_id": "test-abs-id", - "highlights": [ - {"quote": "Great quote", "chapter": "Ch 1", "highlighted_at": "2025-01-15 10:30:00"}, - {"quote": "Another quote"}, - ], - }, - ) - assert resp.status_code == 200 - data = resp.get_json() - assert data["success"] is True - assert data["saved"] == 2 + mock_container.mock_database_service.get_bookfusion_books_by_book_id.return_value = [bf_book] + mock_container.mock_bookfusion_client.highlights_api_key = "test-key" + mock_container.mock_bookfusion_client.sync_all_highlights.return_value = { + "new_highlights": 3, + "books_saved": 1, + } + resp = client.post("/api/bookfusion/sync-book", json={"abs_id": book.abs_id}) -def test_save_journal_no_data(client, mock_container): - resp = client.post("/api/bookfusion/save-journal", content_type="application/json", data="") - assert resp.status_code == 400 + assert resp.status_code == 200 + assert resp.get_json() == { + "success": True, + "new_highlights": 3, + "books_saved": 1, + "linked_books": 1, + } + mock_container.mock_database_service.link_bookfusion_highlights_by_book_id.assert_called_once_with("bf-123", 42) -def test_save_journal_missing_abs_id(client, mock_container): - resp = client.post("/api/bookfusion/save-journal", json={"highlights": []}) +def test_save_journal_requires_data(client): + resp = client.post("/api/bookfusion/save-journal", json={}) assert resp.status_code == 400 - assert "abs_id required" in resp.get_json()["error"] + assert resp.get_json()["error"] == "No data provided" def test_save_journal_book_not_found(client, mock_container): mock_container.mock_database_service.get_book_by_ref.return_value = None - mock_container.mock_database_service.get_bookfusion_highlights_for_book_by_book_id.return_value = [] resp = client.post("/api/bookfusion/save-journal", json={"abs_id": "nonexistent"}) - # No highlights provided and none server-side -> error - assert resp.status_code in (400, 404) - -def test_save_journal_skips_empty_quotes(client, mock_container): - book = _make_mock_book() - mock_container.mock_database_service.get_book_by_ref.return_value = book - mock_container.mock_database_service.cleanup_bookfusion_import_notes.return_value = {} - - resp = client.post( - "/api/bookfusion/save-journal", - json={ - "abs_id": "test-abs-id", - "highlights": [ - {"quote": "", "chapter": "Ch 1"}, - {"quote": " "}, - {"quote": "Valid quote"}, - ], - }, - ) - assert resp.status_code == 200 - assert resp.get_json()["saved"] == 1 + assert resp.status_code == 404 + assert resp.get_json()["error"] == "Book not found" -def test_save_journal_fetches_server_side_highlights(client, mock_container): +def test_save_journal_no_highlights(client, mock_container): book = _make_mock_book() mock_container.mock_database_service.get_book_by_ref.return_value = book - mock_container.mock_database_service.cleanup_bookfusion_import_notes.return_value = {} - - hl = _make_bf_highlight(quote_text="Server-side quote", highlighted_at=datetime(2025, 3, 1)) - mock_container.mock_database_service.get_bookfusion_highlights_for_book_by_book_id.return_value = [hl] - - resp = client.post("/api/bookfusion/save-journal", json={"abs_id": "test-abs-id"}) - assert resp.status_code == 200 - assert resp.get_json()["saved"] == 1 - - -# ── Library ──────────────────────────────────────────────────────── - - -def test_library_returns_books(client, mock_container): - bf = _make_bf_book(bookfusion_id="bf-1", title="Library Book") - mock_container.mock_database_service.get_bookfusion_books.return_value = [bf] - mock_container.mock_database_service.get_all_books.return_value = [] - - resp = client.get("/api/bookfusion/library") - assert resp.status_code == 200 - data = resp.get_json() - assert len(data["books"]) == 1 - assert data["books"][0]["title"] == "Library Book" - assert data["books"][0]["on_dashboard"] is False - - -def test_library_marks_on_dashboard(client, mock_container): - bf = _make_bf_book(bookfusion_id="bf-1", matched_abs_id="dash-1") - dashboard_book = _make_mock_book(abs_id="dash-1", title="Dashboard Book") - mock_container.mock_database_service.get_bookfusion_books.return_value = [bf] - mock_container.mock_database_service.get_all_books.return_value = [dashboard_book] - - resp = client.get("/api/bookfusion/library") - assert resp.status_code == 200 - data = resp.get_json() - assert data["books"][0]["on_dashboard"] is True - assert data["books"][0]["abs_id"] == "dash-1" - - -def test_library_merges_duplicate_titles(client, mock_container): - bf1 = _make_bf_book(bookfusion_id="bf-1", title="Same Title", filename="book.epub", highlight_count=5) - bf2 = _make_bf_book(bookfusion_id="bf-2", title="Same Title", filename="book.mobi", highlight_count=2) - mock_container.mock_database_service.get_bookfusion_books.return_value = [bf1, bf2] - mock_container.mock_database_service.get_all_books.return_value = [] - - resp = client.get("/api/bookfusion/library") - assert resp.status_code == 200 - data = resp.get_json() - # Duplicate titles should be merged into a single entry - assert len(data["books"]) == 1 - assert data["books"][0]["highlight_count"] == 7 - assert len(data["books"][0]["bookfusion_ids"]) == 2 - - -def test_library_hidden_books(client, mock_container): - bf = _make_bf_book(bookfusion_id="bf-1", title="Hidden Book", hidden=True) - mock_container.mock_database_service.get_bookfusion_books.return_value = [bf] - mock_container.mock_database_service.get_all_books.return_value = [] - - resp = client.get("/api/bookfusion/library") - assert resp.status_code == 200 - data = resp.get_json() - assert data is not None - assert data["books"][0]["hidden"] is True - - -def test_library_empty(client, mock_container): - mock_container.mock_database_service.get_bookfusion_books.return_value = [] - mock_container.mock_database_service.get_all_books.return_value = [] - - resp = client.get("/api/bookfusion/library") - assert resp.status_code == 200 - data = resp.get_json() - assert data["books"] == [] - - -# ── Add to Dashboard ─────────────────────────────────────────────── - - -def test_add_to_dashboard_success(client, mock_container): - bf = _make_bf_book(bookfusion_id="bf-new", title="New Book") - mock_container.mock_database_service.get_bookfusion_book.return_value = bf - - saved_book = _make_mock_book(abs_id="bf-bf-new", title="New Book", book_id=42) - saved_book.started_at = None - saved_book.finished_at = None - # First call: not yet on dashboard (None), second: after save, third: _estimate_reading_dates - mock_container.mock_database_service.get_book_by_ref.side_effect = [None, saved_book, saved_book] - mock_container.mock_database_service.get_hardcover_details.return_value = None - mock_container.mock_database_service.get_bookfusion_highlight_date_range.return_value = None - mock_container.mock_hardcover_client.is_configured.return_value = False - - resp = client.post( - "/api/bookfusion/add-to-dashboard", - json={ - "bookfusion_ids": ["bf-new"], - }, - ) - assert resp.status_code == 200 - data = resp.get_json() - assert data["success"] is True - assert data["abs_id"] == "bf-bf-new" - - -def test_add_to_dashboard_already_exists(client, mock_container): - existing = _make_mock_book(abs_id="bf-bf-1", title="Already There") - mock_container.mock_database_service.get_bookfusion_book.return_value = _make_bf_book() - mock_container.mock_database_service.get_book_by_ref.return_value = existing - - resp = client.post( - "/api/bookfusion/add-to-dashboard", - json={ - "bookfusion_ids": ["bf-1"], - }, - ) - assert resp.status_code == 200 - data = resp.get_json() - assert data["already_existed"] is True - - -def test_add_to_dashboard_no_data(client, mock_container): - resp = client.post("/api/bookfusion/add-to-dashboard", content_type="application/json", data="") - assert resp.status_code == 400 - + mock_container.mock_database_service.get_bookfusion_highlights_for_book_by_book_id.return_value = [] -def test_add_to_dashboard_missing_id(client, mock_container): - # Empty dict is falsy in Python, so this gets "No data provided" - resp = client.post("/api/bookfusion/add-to-dashboard", json={}) - assert resp.status_code == 400 + resp = client.post("/api/bookfusion/save-journal", json={"abs_id": book.abs_id}) - # With a key but no bookfusion_ids, we get "bookfusion_id required" - resp = client.post("/api/bookfusion/add-to-dashboard", json={"foo": "bar"}) assert resp.status_code == 400 - assert "bookfusion_id required" in resp.get_json()["error"] - - -def test_add_to_dashboard_book_not_in_catalog(client, mock_container): - mock_container.mock_database_service.get_bookfusion_book.return_value = None - resp = client.post( - "/api/bookfusion/add-to-dashboard", - json={ - "bookfusion_id": "nonexistent", - }, - ) - assert resp.status_code == 404 - assert "not found" in resp.get_json()["error"].lower() - + assert resp.get_json()["error"] == "No highlights found for this book" -def test_add_to_dashboard_single_id_fallback(client, mock_container): - """When bookfusion_ids is absent, falls back to bookfusion_id.""" - bf = _make_bf_book(bookfusion_id="bf-single", title="Single") - mock_container.mock_database_service.get_bookfusion_book.return_value = bf - existing = _make_mock_book(abs_id="bf-bf-single") - mock_container.mock_database_service.get_book_by_ref.return_value = existing - resp = client.post( - "/api/bookfusion/add-to-dashboard", - json={ - "bookfusion_id": "bf-single", - }, +def test_save_journal_success(client, mock_container): + book = _make_mock_book(book_id=42) + highlight = _make_bf_highlight( + quote_text="Test quote", + chapter_heading="Chapter 1", + highlighted_at=datetime(2026, 1, 15), ) - assert resp.status_code == 200 - - -# ── Match to Book ────────────────────────────────────────────────── - - -def test_match_to_book_link(client, mock_container): - book = _make_mock_book(abs_id="dash-1", title="Dashboard Book") - book.started_at = None - book.finished_at = None mock_container.mock_database_service.get_book_by_ref.return_value = book - mock_container.mock_database_service.get_hardcover_details.return_value = None - mock_container.mock_database_service.get_bookfusion_highlight_date_range.return_value = None - mock_container.mock_hardcover_client.is_configured.return_value = False - - resp = client.post( - "/api/bookfusion/match-to-book", - json={ - "bookfusion_ids": ["bf-1", "bf-2"], - "abs_id": "dash-1", - }, - ) - assert resp.status_code == 200 - data = resp.get_json() - assert data["success"] is True - assert data["abs_id"] == "dash-1" - assert mock_container.mock_database_service.set_bookfusion_book_match_by_book_id.call_count == 2 + mock_container.mock_database_service.get_bookfusion_highlights_for_book_by_book_id.return_value = [highlight] + mock_container.mock_database_service.get_reading_journal_entries_for_book.return_value = [] + resp = client.post("/api/bookfusion/save-journal", json={"abs_id": book.abs_id}) -def test_match_to_book_unlink(client, mock_container): - resp = client.post( - "/api/bookfusion/match-to-book", - json={ - "bookfusion_ids": ["bf-1"], - }, - ) assert resp.status_code == 200 - mock_container.mock_database_service.set_bookfusion_book_match_by_book_id.assert_called_once_with( - "bf-1", - None, - ) - - -def test_match_to_book_no_data(client, mock_container): - resp = client.post("/api/bookfusion/match-to-book", content_type="application/json", data="") - assert resp.status_code == 400 - - -def test_match_to_book_missing_bf_id(client, mock_container): - resp = client.post("/api/bookfusion/match-to-book", json={"abs_id": "x"}) - assert resp.status_code == 400 - assert "bookfusion_id required" in resp.get_json()["error"] - - -def test_match_to_book_book_not_found(client, mock_container): - mock_container.mock_database_service.get_book_by_ref.return_value = None - resp = client.post( - "/api/bookfusion/match-to-book", - json={ - "bookfusion_ids": ["bf-1"], - "abs_id": "nonexistent", - }, + assert resp.get_json() == {"success": True, "saved": 1, "skipped": 0} + mock_container.mock_database_service.cleanup_bookfusion_import_notes.assert_called_once_with(book.id) + mock_container.mock_database_service.add_reading_journal.assert_called_once_with( + 42, + "highlight", + entry="Test quote\n— *Chapter 1*", + created_at=datetime(2026, 1, 15), + abs_id=book.abs_id, ) - assert resp.status_code == 404 - assert "not found" in resp.get_json()["error"].lower() -def test_match_to_book_single_id_fallback(client, mock_container): - resp = client.post( - "/api/bookfusion/match-to-book", - json={ - "bookfusion_id": "bf-single", - }, +def test_save_journal_deduplicates_legacy_entry(client, mock_container): + book = _make_mock_book(book_id=42) + highlight = _make_bf_highlight( + quote_text="Duplicate quote", + chapter_heading="## Chapter 1", + highlighted_at=datetime(2026, 1, 15), ) - assert resp.status_code == 200 - mock_container.mock_database_service.set_bookfusion_book_match_by_book_id.assert_called_once() - - -# ── Hide / Unhide ────────────────────────────────────────────────── + existing_journal = Mock() + existing_journal.entry = "Duplicate quote\n— Chapter 1" + mock_container.mock_database_service.get_book_by_ref.return_value = book + mock_container.mock_database_service.get_bookfusion_highlights_for_book_by_book_id.return_value = [highlight] + mock_container.mock_database_service.get_reading_journal_entries_for_book.return_value = [existing_journal] + resp = client.post("/api/bookfusion/save-journal", json={"abs_id": book.abs_id}) -def test_hide_book_success(client, mock_container): - resp = client.post( - "/api/bookfusion/hide", - json={ - "bookfusion_ids": ["bf-1", "bf-2"], - "hidden": True, - }, - ) assert resp.status_code == 200 - assert resp.get_json()["success"] is True - mock_container.mock_database_service.set_bookfusion_books_hidden.assert_called_once_with( - ["bf-1", "bf-2"], - True, - ) + assert resp.get_json() == {"success": True, "saved": 0, "skipped": 1} + mock_container.mock_database_service.add_reading_journal.assert_not_called() -def test_unhide_book(client, mock_container): - resp = client.post( - "/api/bookfusion/hide", - json={ - "bookfusion_ids": ["bf-1"], - "hidden": False, - }, - ) - assert resp.status_code == 200 - mock_container.mock_database_service.set_bookfusion_books_hidden.assert_called_once_with( - ["bf-1"], - False, +def test_save_journal_with_existing_entries(client, mock_container): + book = _make_mock_book(book_id=42) + highlight = _make_bf_highlight( + quote_text="New quote", + chapter_heading="Chapter 2", + highlighted_at=datetime(2026, 2, 1), ) - - -def test_hide_book_no_data(client, mock_container): - resp = client.post("/api/bookfusion/hide", content_type="application/json", data="") - assert resp.status_code == 400 - - -def test_hide_book_missing_id(client, mock_container): - resp = client.post("/api/bookfusion/hide", json={"hidden": True}) - assert resp.status_code == 400 - assert "bookfusion_id required" in resp.get_json()["error"] - - -def test_hide_book_single_id_fallback(client, mock_container): - resp = client.post( - "/api/bookfusion/hide", - json={ - "bookfusion_id": "bf-single", - "hidden": True, - }, - ) - assert resp.status_code == 200 - mock_container.mock_database_service.set_bookfusion_books_hidden.assert_called_once_with( - ["bf-single"], - True, - ) - - -# ── Unlink Book ──────────────────────────────────────────────────── - - -def test_unlink_book_success(client, mock_container): - book = _make_mock_book(book_id=7) + existing_journal = Mock() + existing_journal.entry = "Old quote" mock_container.mock_database_service.get_book_by_ref.return_value = book + mock_container.mock_database_service.get_bookfusion_highlights_for_book_by_book_id.return_value = [highlight] + mock_container.mock_database_service.get_reading_journal_entries_for_book.return_value = [existing_journal] - resp = client.post("/api/bookfusion/unlink", json={"abs_id": "test-abs-id"}) - assert resp.status_code == 200 - assert resp.get_json()["success"] is True - mock_container.mock_database_service.unlink_bookfusion_by_book_id.assert_called_once_with(7) - - -def test_unlink_book_not_found_still_succeeds(client, mock_container): - mock_container.mock_database_service.get_book_by_ref.return_value = None - resp = client.post("/api/bookfusion/unlink", json={"abs_id": "missing"}) - assert resp.status_code == 200 - assert resp.get_json()["success"] is True - mock_container.mock_database_service.unlink_bookfusion_by_book_id.assert_not_called() - - -def test_unlink_book_no_data(client, mock_container): - resp = client.post("/api/bookfusion/unlink", content_type="application/json", data="") - assert resp.status_code == 400 - - -def test_unlink_book_missing_abs_id(client, mock_container): - # Empty dict is falsy → "No data provided" - resp = client.post("/api/bookfusion/unlink", json={}) - assert resp.status_code == 400 + resp = client.post("/api/bookfusion/save-journal", json={"abs_id": book.abs_id}) - # With a key but no abs_id → "abs_id required" - resp = client.post("/api/bookfusion/unlink", json={"foo": "bar"}) - assert resp.status_code == 400 - assert "abs_id required" in resp.get_json()["error"] - - -# ── BookFusion Page ──────────────────────────────────────────────── - - -def test_bookfusion_page_renders(client, mock_container): - resp = client.get("/bookfusion") assert resp.status_code == 200 + assert resp.get_json() == {"success": True, "saved": 1, "skipped": 0} + mock_container.mock_database_service.add_reading_journal.assert_called_once_with( + 42, + "highlight", + entry="New quote\n— *Chapter 2*", + created_at=datetime(2026, 2, 1), + abs_id=book.abs_id, + ) diff --git a/tests/test_bookfusion_save_journal_regressions.py b/tests/test_bookfusion_save_journal_regressions.py new file mode 100644 index 0000000..ca554c8 --- /dev/null +++ b/tests/test_bookfusion_save_journal_regressions.py @@ -0,0 +1,65 @@ +from datetime import datetime +from unittest.mock import Mock + + +def _make_book(book_id=42, abs_id="test-abs-id"): + book = Mock() + book.id = book_id + book.abs_id = abs_id + return book + + +def _make_highlight(quote_text, chapter_heading, highlighted_at=None): + highlight = Mock() + highlight.quote_text = quote_text + highlight.content = quote_text + highlight.chapter_heading = chapter_heading + highlight.highlighted_at = highlighted_at + return highlight + + +def _make_existing_journal(entry): + journal = Mock() + journal.entry = entry + return journal + + +def test_save_journal_dedupes_legacy_plain_text_entry(client, mock_container): + book = _make_book() + highlight = _make_highlight("Duplicate quote", "## Chapter 1", datetime(2026, 1, 15)) + existing = _make_existing_journal("Duplicate quote\n— Chapter 1") + + mock_db = mock_container.mock_database_service + mock_db.get_book_by_ref.return_value = book + mock_db.get_bookfusion_highlights_for_book_by_book_id.return_value = [highlight] + mock_db.get_reading_journal_entries_for_book.return_value = [existing] + + resp = client.post("/api/bookfusion/save-journal", json={"abs_id": book.abs_id}) + + assert resp.status_code == 200 + assert resp.get_json() == {"success": True, "saved": 0, "skipped": 1} + mock_db.cleanup_bookfusion_import_notes.assert_called_once_with(book.id) + mock_db.add_reading_journal.assert_not_called() + + +def test_save_journal_saves_markdown_formatted_chapter(client, mock_container): + book = _make_book() + highlight = _make_highlight("Fresh quote", "# Chapter 2", datetime(2026, 2, 1)) + + mock_db = mock_container.mock_database_service + mock_db.get_book_by_ref.return_value = book + mock_db.get_bookfusion_highlights_for_book_by_book_id.return_value = [highlight] + mock_db.get_reading_journal_entries_for_book.return_value = [] + + resp = client.post("/api/bookfusion/save-journal", json={"abs_id": book.abs_id}) + + assert resp.status_code == 200 + assert resp.get_json() == {"success": True, "saved": 1, "skipped": 0} + mock_db.cleanup_bookfusion_import_notes.assert_called_once_with(book.id) + mock_db.add_reading_journal.assert_called_once_with( + book.id, + "highlight", + entry="Fresh quote\n— *Chapter 2*", + created_at=datetime(2026, 2, 1), + abs_id=book.abs_id, + ) diff --git a/tests/test_cache_cleanup_service.py b/tests/test_cache_cleanup_service.py new file mode 100644 index 0000000..f94aa0a --- /dev/null +++ b/tests/test_cache_cleanup_service.py @@ -0,0 +1,28 @@ +from pathlib import Path +from unittest.mock import Mock + +from src.services.cache_cleanup_service import CacheCleanupService + + +def test_cache_cleanup_service_removes_only_orphaned_files(tmp_path: Path): + cache_dir = tmp_path / "epub_cache" + cache_dir.mkdir() + + keep_book = cache_dir / "keep-book.epub" + keep_suggestion = cache_dir / "keep-suggestion.epub" + orphan = cache_dir / "orphan.epub" + + keep_book.write_text("book") + keep_suggestion.write_text("suggestion") + orphan.write_text("orphan") + + db = Mock() + db.get_all_books.return_value = [Mock(ebook_filename="keep-book.epub")] + db.get_all_actionable_suggestions.return_value = [Mock(matches=[{"filename": "keep-suggestion.epub"}])] + + service = CacheCleanupService(db, cache_dir) + service.cleanup() + + assert keep_book.exists() + assert keep_suggestion.exists() + assert not orphan.exists() diff --git a/tests/test_kosync_progress_service.py b/tests/test_kosync_progress_service.py new file mode 100644 index 0000000..087e337 --- /dev/null +++ b/tests/test_kosync_progress_service.py @@ -0,0 +1,44 @@ +from unittest.mock import Mock + +from src.services.kosync_progress_service import KosyncProgressService + + +class _ServiceStub: + def __init__(self): + self._db = Mock() + self._container = Mock() + self._manager = Mock() + self.start_discovery_if_available = Mock(return_value=False) + self.run_put_auto_discovery = Mock() + self.run_get_auto_discovery = Mock() + self.resolve_book_by_sibling_hash = Mock(return_value=None) + self.register_hash_for_book = Mock() + self.serialize_progress = Mock(return_value={"document": "doc"}) + + +def test_handle_put_progress_rejects_invalid_percentage(): + service = _ServiceStub() + progress = KosyncProgressService(service) + + body, status = progress.handle_put_progress( + { + "document": "doc-123", + "percentage": "not-a-number", + }, + remote_addr="127.0.0.1", + ) + + assert status == 400 + assert body["error"] == "Invalid percentage value" + + +def test_handle_get_progress_returns_502_for_unknown_document_without_discovery(): + service = _ServiceStub() + service._db.get_kosync_document.return_value = None + service._db.get_book_by_kosync_id.return_value = None + progress = KosyncProgressService(service) + + body, status = progress.handle_get_progress("missing-doc", remote_addr="127.0.0.1") + + assert status == 502 + assert body["message"] == "Document not found on server" diff --git a/tests/test_markdown_utils.py b/tests/test_markdown_utils.py new file mode 100644 index 0000000..088f140 --- /dev/null +++ b/tests/test_markdown_utils.py @@ -0,0 +1,16 @@ +from pathlib import Path + +from src.utils.markdown import render_markdown_html, sanitize_html + + +def test_render_markdown_html_returns_block_html(): + assert render_markdown_html("Hello world") == "

Hello world

" + + +def test_sanitize_html_returns_markup(): + assert str(sanitize_html("

safe

")) == "

safe

" + + +def test_reading_detail_uses_block_container_for_markdown(): + template = (Path(__file__).resolve().parent.parent / "templates" / "reading_detail.html").read_text() + assert '
{{ journal.entry|markdown }}
' in template diff --git a/tests/test_reading_journal_markdown.py b/tests/test_reading_journal_markdown.py new file mode 100644 index 0000000..3576f25 --- /dev/null +++ b/tests/test_reading_journal_markdown.py @@ -0,0 +1,59 @@ +from datetime import datetime +from unittest.mock import Mock + + +def _make_book(book_id=42, abs_id="test-abs-id"): + book = Mock() + book.id = book_id + book.abs_id = abs_id + return book + + +def _make_journal(journal_id=7, event="note", entry="Line 1\n\nLine 2"): + journal = Mock() + journal.id = journal_id + journal.book_id = 42 + journal.event = event + journal.entry = entry + journal.percentage = 0.5 + journal.created_at = datetime(2026, 1, 15) + return journal + + +def test_add_journal_returns_rendered_entry_html(client, mock_container): + book = _make_book() + journal = _make_journal(entry="A **bold** note") + + mock_db = mock_container.mock_database_service + mock_db.get_book_by_ref.return_value = book + mock_db.get_states_for_book.return_value = [] + mock_db.add_reading_journal.return_value = journal + + resp = client.post( + "/api/reading/book/test-abs-id/journal", + json={"entry": journal.entry}, + ) + + assert resp.status_code == 200 + data = resp.get_json() + assert data["journal"]["entry"] == journal.entry + assert data["journal"]["entry_html"] == f"

{journal.entry}

" + + +def test_update_note_returns_rendered_entry_html(client, mock_container): + existing = _make_journal(entry="Old") + updated = _make_journal(entry="Updated note") + + mock_db = mock_container.mock_database_service + mock_db.get_reading_journal.side_effect = [existing] + mock_db.update_reading_journal.return_value = updated + + resp = client.patch( + "/api/reading/journal/7", + json={"entry": updated.entry}, + ) + + assert resp.status_code == 200 + data = resp.get_json() + assert data["journal"]["entry"] == updated.entry + assert data["journal"]["entry_html"] == f"

{updated.entry}

"