Skip to content

Fix TikTok artifacts SQL parsing and add robust error handling#1396

Open
AdityaS207 wants to merge 2 commits intoabrignoni:mainfrom
AdityaS207:fix/errorparsing/tiktokpy-tiktok_repliedpy
Open

Fix TikTok artifacts SQL parsing and add robust error handling#1396
AdityaS207 wants to merge 2 commits intoabrignoni:mainfrom
AdityaS207:fix/errorparsing/tiktokpy-tiktok_repliedpy

Conversation

@AdityaS207
Copy link

Summary

This pull request fixes runtime failures when parsing TikTok artifacts by ensuring constructed SQL is syntactically valid and by adding robust exception handling around SQLite operations. The change prevents artifact crashes caused by missing contact tables or schema differences across TikTok database samples.

Previously observed errors

  • sqlite3.OperationalError: near ")": syntax error (caused by empty/invalid subquery e.g., LEFT JOIN ())

  • sqlite3.OperationalError: no such table: TIMMessageKVORM

  • These raised unhandled exceptions that stopped artifact processing and sometimes terminated the overall run.

Changes Included

1. tikTok Artifact

  • Use ' UNION ALL '.join(...) when assembling contact subqueries so multiple AwemeContactsV% tables are combined correctly.

  • Add a safe fallback subquery when no contact tables are found (a valid SELECT that returns zero rows) to prevent LEFT JOIN () syntax errors.

  • Wrap main SQL executions in try/except sqlite3.OperationalError and log errors via logfunc(...) rather than letting exceptions crash the parser.

2. tikTokReplied Artifact

  • Mirror the same robust subquery assembly and fallback behavior for contact tables.
  • Add import sqlite3 and wrap execution of the CTE + SELECT in a try/except sqlite3.OperationalError block; on error the artifact logs the issue and continues with an empty result.
  • Ensure the deduplication CTE receives a guaranteed-valid contacts_subquery (either the UNION ALL of tables or the zero-row fallback).

General improvements

  • Standardized, non-intrusive error handling for SQLite operations: Error messages now follow the pattern: Reading had SQL error:

  • All fixes preserve original parsing logic and output formats where possible; changes prioritize stability over failing the entire run.

Testing Performed

1. Local verification:

ran ileapp.py with the provided profile limiting to tikTok and tiktok_replied modules (example command used during testing was the same command the user runs with 13-4-1.tar).

2. Observed behavior:

  • Prior crashes caused by malformed subqueries or missing tables no longer abort artifact processing.

  • When contact tables exist, artifacts still return results as before.

  • When contact tables are absent or queries fail due to schema differences, the artifacts log the SQL error and continue; the run finishes and report generation completes.

Results

1. No unhandled sqlite3 OperationalErrors for the tikTok and tiktok_replied artifacts in tested samples.

2. Artifacts are resilient to:

  • missing AwemeContactsV% tables,

  • missing TIMMessageKVORM table,

  • SQL differences that would previously crash the module.

3. Report generation completes without interruption; missing data is handled gracefully with logged diagnostics.

Notes / Caveats

  • The fallback subquery intentionally returns zero rows. That means contact-related columns will be empty when contact tables are genuinely absent; this is a deliberate tradeoff to maintain run continuity.

  • If the SQLite build used in a DB does not support window functions (e.g., ROW_NUMBER() in the CTE), the try/except will catch the error and log it. If preferred, we can provide a compatibility fallback (group-by/aggregate-based deduplication) instead of relying on CTE/window functions.

  • These changes are defensive and do not add new external dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant