Skip to content

[Bug] unstable JSON parser and record count in forwarding #1197

@tgnine5

Description

@tgnine5

Describe the bug

JSON parsing issues with official exports

I suggest using a more robust JSON parser. Currently, when importing files from the official client using tdl from "./temp.json", the parser often only picks up a few records (e.g., num: 16 in debug logs) and stops prematurely. It usually takes several manual attempts to get it to read the full file.

Suggestions:

  • Enhance the JSON parser for better stability.
  • Print the total count of detected records at the beginning of the process for quick manal verification.

This would help with automation and potential WebUI integration with Python for those including me.
(I have a Python background but no on Go). Thanks!

json解析器容易出错,解析不全消息
建议使用更稳健的json解析器,更好地支持官方客户端导出的json

使用tdl from "./temp.json" --to CHAT --mode clone --delay 15s

其中 temp.json使用官方客户端导出的。

经常发现最开始几次只转发几条就停了,添加 --debug参数发现 Parse file {"file": "./temp.json", "num": 16} 只读取了几条记录,里面的“16”可能是其他值,最常见的是9和11,一般不超过20。多手动修改几次后,一般能全部读取。

因此建议:

使用更稳健的json解析器,以更好地支持json文件;

在开始时,显示读取了多少条记录,方便判断是否真的读取完毕;

这个也有助于自己调用实现定期自动运行,还有自己尝试

To Reproduce

tdl from "./temp.json" --to CHAT --mode clone --delay 15s --debug

tail ~/.tdl/log/latest.log -f | grep Parse

Sorry, I will try to provide a json when I find a suitable one

Expected behavior

The value of "num" of Parse file is at least close to the nubmer of expected messages in json

Version

tdl version
Version: 0.20.1
Commit: 742da1b
Date: 2025-12-14T16:19:18Z

go1.23.10 linux/amd64

Which OS are you running tdl on?

Windows

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions