Skip to content

Fix database schema compatibility issues in multiple artifacts ( deviceHealthServices_Battery.py, FacebookMessenger.py, googleDuo.py, googleMessages.py, keepNotes.py , notificationHistory.py ) by 5025231128 #639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vokalishoregmelankolis wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix database schema compatibility issues in multiple artifacts ( deviceHealthServices_Battery.py, FacebookMessenger.py, googleDuo.py, googleMessages.py, keepNotes.py , notificationHistory.py ) by 5025231128 #639

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 87 additions & 41 deletions scripts/artifacts/FacebookMessenger.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,28 +233,52 @@ def get_FacebookMessenger(files_found, report_folder, seeker, wrap_text):
''')
snippet = 1
except:
cursor.execute('''
select
case messages.timestamp_ms
when 0 then ''
else datetime(messages.timestamp_ms/1000,'unixepoch')
End as Datestamp,
(select json_extract (messages.sender, '$.name')) as "Sender",
substr((select json_extract (messages.sender, '$.user_key')),10) as "Sender ID",
messages.thread_key,
messages.text,
(select json_extract (messages.attachments, '$[0].filename')) as AttachmentName,
(select json_extract (messages.shares, '$[0].name')) as ShareName,
(select json_extract (messages.shares, '$[0].description')) as ShareDesc,
(select json_extract (messages.shares, '$[0].href')) as ShareLink,
message_reactions.reaction as "Message Reaction",
datetime(message_reactions.reaction_timestamp/1000,'unixepoch') as "Message Reaction Timestamp",
messages.msg_id
from messages, threads
left join message_reactions on message_reactions.msg_id = messages.msg_id
where messages.thread_key=threads.thread_key and generic_admin_message_extensible_data IS NULL and msg_type != -1
order by messages.thread_key, datestamp;
''')
try:
cursor.execute('''
select
case messages.timestamp_ms
when 0 then ''
else datetime(messages.timestamp_ms/1000,'unixepoch')
End as Datestamp,
(select json_extract (messages.sender, '$.name')) as "Sender",
substr((select json_extract (messages.sender, '$.user_key')),10) as "Sender ID",
messages.thread_key,
messages.text,
(select json_extract (messages.attachments, '$[0].filename')) as AttachmentName,
(select json_extract (messages.shares, '$[0].name')) as ShareName,
(select json_extract (messages.shares, '$[0].description')) as ShareDesc,
(select json_extract (messages.shares, '$[0].href')) as ShareLink,
message_reactions.reaction as "Message Reaction",
datetime(message_reactions.reaction_timestamp/1000,'unixepoch') as "Message Reaction Timestamp",
messages.msg_id
from messages, threads
left join message_reactions on message_reactions.msg_id = messages.msg_id
where messages.thread_key=threads.thread_key and generic_admin_message_extensible_data IS NULL and msg_type != -1
order by messages.thread_key, datestamp;
''')
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause without specifying exception type. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception to avoid catching SystemExit and KeyboardInterrupt.

Copilot uses AI. Check for mistakes.
# Fallback for databases without reaction_timestamp column
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment "Fallback for databases without reaction_timestamp column" is misleading. This fallback actually handles databases that are missing the entire message_reactions table (note the LEFT JOIN is removed). If only the reaction_timestamp column is missing but the message_reactions table exists, this query would still fail when trying to access message_reactions.reaction on line 251-252 in the previous try block. Consider clarifying the comment to: "Fallback for databases without message_reactions table".

Suggested change
# Fallback for databases without reaction_timestamp column
# Fallback for databases without message_reactions table

Copilot uses AI. Check for mistakes.
cursor.execute('''
select
case messages.timestamp_ms
when 0 then ''
else datetime(messages.timestamp_ms/1000,'unixepoch')
End as Datestamp,
(select json_extract (messages.sender, '$.name')) as "Sender",
substr((select json_extract (messages.sender, '$.user_key')),10) as "Sender ID",
messages.thread_key,
messages.text,
(select json_extract (messages.attachments, '$[0].filename')) as AttachmentName,
(select json_extract (messages.shares, '$[0].name')) as ShareName,
(select json_extract (messages.shares, '$[0].description')) as ShareDesc,
(select json_extract (messages.shares, '$[0].href')) as ShareLink,
NULL as "Message Reaction",
NULL as "Message Reaction Timestamp",
messages.msg_id
from messages, threads
where messages.thread_key=threads.thread_key and generic_admin_message_extensible_data IS NULL and msg_type != -1
order by messages.thread_key, datestamp;
''')

all_rows = cursor.fetchall()
usageentries = len(all_rows)
Expand Down Expand Up @@ -323,25 +347,47 @@ def get_FacebookMessenger(files_found, report_folder, seeker, wrap_text):
else:
logfunc(f'No Facebook{typeof}- Calls{usernum} - threads_db2 data available')

cursor.execute('''
select
substr(user_key,10),
first_name,
last_name,
username,
(select json_extract (profile_pic_square, '$[0].url')) as profile_pic_square,
case is_messenger_user
when 0 then ''
else 'Yes'
end is_messenger_user,
case is_friend
when 0 then 'No'
when 1 then 'Yes'
end is_friend,
friendship_status,
contact_relationship_status
from thread_users
''')
try:
cursor.execute('''
select
substr(user_key,10),
first_name,
last_name,
username,
(select json_extract (profile_pic_square, '$[0].url')) as profile_pic_square,
case is_messenger_user
when 0 then ''
else 'Yes'
end is_messenger_user,
case is_friend
when 0 then 'No'
when 1 then 'Yes'
end is_friend,
friendship_status,
contact_relationship_status
from thread_users
''')
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause without specifying exception type. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception to avoid catching SystemExit and KeyboardInterrupt.

Suggested change
except:
except sqlite3.OperationalError:

Copilot uses AI. Check for mistakes.
# Fallback for databases without friendship_status or contact_relationship_status columns
cursor.execute('''
select
substr(user_key,10),
first_name,
last_name,
username,
(select json_extract (profile_pic_square, '$[0].url')) as profile_pic_square,
case is_messenger_user
when 0 then ''
else 'Yes'
end is_messenger_user,
case is_friend
when 0 then 'No'
when 1 then 'Yes'
end is_friend,
NULL as friendship_status,
NULL as contact_relationship_status
from thread_users
''')

all_rows = cursor.fetchall()
usageentries = len(all_rows)
Expand Down
4 changes: 2 additions & 2 deletions scripts/artifacts/deviceHealthServices_Battery.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ def Turbo_Battery(files_found, report_folder, seeker, wrap_text):
if timestamp is None:
pass
else:
timestamp = convert_utc_human_to_timezone(convert_ts_human_to_utc(timestamp),time_offset)
timestamp = convert_utc_human_to_timezone(convert_ts_human_to_utc(timestamp), row[4])
data_list.append((timestamp,row[1],row[2],row[3],row[4],file_found))

db.close()
Expand Down Expand Up @@ -121,7 +121,7 @@ def Turbo_Bluetooth(files_found, report_folder, seeker, wrap_text):
if timestamp is None:
pass
else:
timestamp = convert_utc_human_to_timezone(convert_ts_human_to_utc(timestamp),time_offset)
timestamp = convert_utc_human_to_timezone(convert_ts_human_to_utc(timestamp), row[5])
data_list.append((timestamp,row[1],row[2],row[3],row[4],row[5],file_found))
db.close()

Expand Down
80 changes: 53 additions & 27 deletions scripts/artifacts/googleDuo.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,31 +99,57 @@ def get_googleDuo(files_found, report_folder, seeker, wrap_text):
else:
logfunc('No Google Duo - Contacts data available')

cursor.execute('''
select
case sent_timestamp_millis
when 0 then ''
else datetime(sent_timestamp_millis/1000,'unixepoch')
end as 'Sent Timestamp',
case received_timestamp_millis
when 0 then ''
else datetime(received_timestamp_millis/1000,'unixepoch')
end as 'Received Timestamp',
case seen_timestamp_millis
when 0 then ''
else datetime(seen_timestamp_millis/1000,'unixepoch')
end as 'Viewed Timestamp',
sender_id,
recipient_id,
content_uri,
replace(content_uri, rtrim(content_uri, replace(content_uri, '/', '')), '') as 'File Name',
content_size_bytes,
case saved_status
when 0 then ''
when 1 then 'Yes'
end as 'File Saved'
from messages
''')
try:
cursor.execute('''
select
case sent_timestamp_millis
when 0 then ''
else datetime(sent_timestamp_millis/1000,'unixepoch')
end as 'Sent Timestamp',
case received_timestamp_millis
when 0 then ''
else datetime(received_timestamp_millis/1000,'unixepoch')
end as 'Received Timestamp',
case seen_timestamp_millis
when 0 then ''
else datetime(seen_timestamp_millis/1000,'unixepoch')
end as 'Viewed Timestamp',
sender_id,
recipient_id,
content_uri,
replace(content_uri, rtrim(content_uri, replace(content_uri, '/', '')), '') as 'File Name',
content_size_bytes,
case saved_status
when 0 then ''
when 1 then 'Yes'
end as 'File Saved'
from messages
''')
has_saved_status = True
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause without specifying exception type. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception to avoid catching SystemExit and KeyboardInterrupt.

Suggested change
except:
except sqlite3.OperationalError:

Copilot uses AI. Check for mistakes.
# Fallback for databases without saved_status column
cursor.execute('''
select
case sent_timestamp_millis
when 0 then ''
else datetime(sent_timestamp_millis/1000,'unixepoch')
end as 'Sent Timestamp',
case received_timestamp_millis
when 0 then ''
else datetime(received_timestamp_millis/1000,'unixepoch')
end as 'Received Timestamp',
case seen_timestamp_millis
when 0 then ''
else datetime(seen_timestamp_millis/1000,'unixepoch')
end as 'Viewed Timestamp',
sender_id,
recipient_id,
content_uri,
replace(content_uri, rtrim(content_uri, replace(content_uri, '/', '')), '') as 'File Name',
content_size_bytes
from messages
''')
has_saved_status = False

all_rows = cursor.fetchall()
usageentries = len(all_rows)
Expand All @@ -140,7 +166,7 @@ def get_googleDuo(files_found, report_folder, seeker, wrap_text):
content_uri = row[5]
content_name = row[6]
content_size = row[7]
file_saved = row[8]
file_saved = row[8] if has_saved_status else ''
thumb = ''

for match in files_found:
Expand All @@ -149,7 +175,7 @@ def get_googleDuo(files_found, report_folder, seeker, wrap_text):
data_file_name = os.path.basename(match)
thumb = f'<img src="{report_folder}/{data_file_name}" width="300"></img>'

data_list.append((row[0],row[1],row[2],row[3],row[4],thumb,row[7],row[8]))
data_list.append((row[0],row[1],row[2],row[3],row[4],thumb,row[7],file_saved))

report = ArtifactHtmlReport('Google Duo - Notes')
report.start_artifact_report(report_folder, 'Google Duo - Notes')
Expand Down
76 changes: 57 additions & 19 deletions scripts/artifacts/googleMessages.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,63 @@ def get_googleMessages(files_found, report_folder, seeker, wrap_text):

db = open_sqlite_db_readonly(file_found)
cursor = db.cursor()
cursor.execute('''
SELECT
datetime(parts.timestamp/1000,'unixepoch') AS "Timestamp (UTC)",
parts.content_type AS "Message Type",
conversations.name AS "Other Participant/Conversation Name",
participants.display_destination AS "Message Sender",
parts.text AS "Message",
CASE
WHEN parts.file_size_bytes=-1 THEN "N/A"
ELSE parts.file_size_bytes
END AS "Attachment Byte Size",
parts.local_cache_path AS "Attachment Location"
FROM
parts
JOIN messages ON messages._id=parts.message_id
JOIN participants ON participants._id=messages.sender_id
JOIN conversations ON conversations._id=parts.conversation_id
ORDER BY "Timestamp (UTC)" ASC
''')
try:
cursor.execute('''
SELECT
datetime(parts.timestamp/1000,'unixepoch') AS "Timestamp (UTC)",
parts.content_type AS "Message Type",
conversations.name AS "Other Participant/Conversation Name",
participants.display_destination AS "Message Sender",
parts.text AS "Message",
CASE
WHEN parts.file_size_bytes=-1 THEN "N/A"
ELSE parts.file_size_bytes
END AS "Attachment Byte Size",
parts.local_cache_path AS "Attachment Location"
FROM
parts
JOIN messages ON messages._id=parts.message_id
JOIN participants ON participants._id=messages.sender_id
JOIN conversations ON conversations._id=parts.conversation_id
ORDER BY "Timestamp (UTC)" ASC
''')
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause without specifying exception type. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception to avoid catching SystemExit and KeyboardInterrupt.

Copilot uses AI. Check for mistakes.
# Fallback for databases without file_size_bytes column
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment "Fallback for databases without file_size_bytes column" is incomplete. Looking at the query, this fallback omits file_size_bytes but still uses local_cache_path. A more accurate comment would be: "Fallback for databases without file_size_bytes column (but with local_cache_path)".

Suggested change
# Fallback for databases without file_size_bytes column
# Fallback for databases without file_size_bytes column (but with local_cache_path)

Copilot uses AI. Check for mistakes.
try:
cursor.execute('''
SELECT
datetime(parts.timestamp/1000,'unixepoch') AS "Timestamp (UTC)",
parts.content_type AS "Message Type",
conversations.name AS "Other Participant/Conversation Name",
participants.display_destination AS "Message Sender",
parts.text AS "Message",
"N/A" AS "Attachment Byte Size",
parts.local_cache_path AS "Attachment Location"
FROM
parts
JOIN messages ON messages._id=parts.message_id
JOIN participants ON participants._id=messages.sender_id
JOIN conversations ON conversations._id=parts.conversation_id
ORDER BY "Timestamp (UTC)" ASC
''')
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause without specifying exception type. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception to avoid catching SystemExit and KeyboardInterrupt.

Suggested change
except:
except sqlite3.OperationalError:

Copilot uses AI. Check for mistakes.
# Final fallback for databases without both file_size_bytes and local_cache_path columns
cursor.execute('''
SELECT
datetime(parts.timestamp/1000,'unixepoch') AS "Timestamp (UTC)",
parts.content_type AS "Message Type",
conversations.name AS "Other Participant/Conversation Name",
participants.display_destination AS "Message Sender",
parts.text AS "Message",
"N/A" AS "Attachment Byte Size",
"N/A" AS "Attachment Location"
FROM
parts
JOIN messages ON messages._id=parts.message_id
JOIN participants ON participants._id=messages.sender_id
JOIN conversations ON conversations._id=parts.conversation_id
ORDER BY "Timestamp (UTC)" ASC
''')

all_rows = cursor.fetchall()
usageentries = len(all_rows)
Expand Down
49 changes: 38 additions & 11 deletions scripts/artifacts/keepNotes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,44 @@ def get_keepNotes(files_found, report_folder, seeker, wrap_text):
if filename.endswith('keep.db'):
db = open_sqlite_db_readonly(file_found)
cursor = db.cursor()
cursor.execute('''
SELECT
datetime(tree_entity.time_created/1000, 'unixepoch') AS "Time Created",
datetime(tree_entity.time_last_updated/1000, 'unixepoch') AS "Time Last Updated",
datetime(tree_entity.user_edited_timestamp/1000, 'unixepoch') AS "User Edited Timestamp",
tree_entity.title AS Title,
text_search_note_content_content.c0text AS "Text",
tree_entity.last_modifier_email AS "Last Modifier Email"
FROM text_search_note_content_content
INNER JOIN tree_entity ON text_search_note_content_content.docid = tree_entity._id
''')
try:
cursor.execute('''
SELECT
datetime(tree_entity.time_created/1000, 'unixepoch') AS "Time Created",
datetime(tree_entity.time_last_updated/1000, 'unixepoch') AS "Time Last Updated",
datetime(tree_entity.user_edited_timestamp/1000, 'unixepoch') AS "User Edited Timestamp",
tree_entity.title AS Title,
text_search_note_content_content.c0text AS "Text",
tree_entity.last_modifier_email AS "Last Modifier Email"
FROM text_search_note_content_content
INNER JOIN tree_entity ON text_search_note_content_content.docid = tree_entity._id
''')
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clauses are being used without specifying exception types. This is a best practice violation as it catches all exceptions including SystemExit and KeyboardInterrupt, which can hide bugs and make debugging difficult. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception.

Copilot uses AI. Check for mistakes.
# Fallback for databases without text_search_note_content_content table
try:
cursor.execute('''
SELECT
datetime(tree_entity.time_created/1000, 'unixepoch') AS "Time Created",
datetime(tree_entity.time_last_updated/1000, 'unixepoch') AS "Time Last Updated",
datetime(tree_entity.user_edited_timestamp/1000, 'unixepoch') AS "User Edited Timestamp",
tree_entity.title AS Title,
tree_entity.text_content AS "Text",
tree_entity.last_modifier_email AS "Last Modifier Email"
FROM tree_entity
WHERE tree_entity.type = 0
''')
except:
Copy link

Copilot AI Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause without specifying exception type. Consider catching specific exceptions like sqlite3.OperationalError or at minimum Exception to avoid catching SystemExit and KeyboardInterrupt.

Copilot uses AI. Check for mistakes.
# Final fallback - just tree_entity with minimal columns
cursor.execute('''
SELECT
datetime(tree_entity.time_created/1000, 'unixepoch') AS "Time Created",
datetime(tree_entity.time_last_updated/1000, 'unixepoch') AS "Time Last Updated",
datetime(tree_entity.user_edited_timestamp/1000, 'unixepoch') AS "User Edited Timestamp",
tree_entity.title AS Title,
NULL AS "Text",
tree_entity.last_modifier_email AS "Last Modifier Email"
FROM tree_entity
''')

all_rows = cursor.fetchall()
usageentries = len(all_rows)
Expand Down
Loading
Loading