Optimize record size by ItosEO · Pull Request #57 · Itosang/BatteryRecorder

ItosEO · 2026-04-17T06:53:11Z

No description provided.

- **新增** `RecordFileIO` 与 `RecordFileNames` 统一处理 `.txt` 和 `.txt.gz` 文件的读写与命名逻辑。 - **Server 端**：引入后台压缩机制，自动将已完成的历史分段压缩为 `.txt.gz` 以节省空间。 - **App 端**： - 支持透明读取压缩后的记录文件。 - 统一使用“逻辑记录名”（不带 `.gz` 后缀）进行缓存、导出与删除操作。 - 在数据同步（Sync）过程中增加缓存预热逻辑。 - **数据同步**：优化 `PfdFileSender` 支持批量发送指定文件列表，并在同步后正确清理 Server 侧的物理文件及其压缩副本。 - **缓存**：提升 `HISTORY_STATS_CACHE_VERSION` 版本至 14。

- 在非 root 模式下才执行 `writer.awaitCompressionBlocking()`，优化 root 模式下的同步效率。

- 更新文档，明确 `sync()` 现在支持同步活跃分段文件，并说明了 shell 侧的清理规则。 - 移除同步前的压缩等待阻塞逻辑，提升同步响应速度。

- **doc**: 简化 `AGENTS.md` 中的目录索引，仅保留根模块结构，并微调关键路径表格格式。 - **refactor**: 在 `HistoryViewModel` 中引入 `logicalRecordName()`，统一使用逻辑记录名进行文件过滤和索引查找，解决因 `.txt` 与 `.txt.gz` 后缀不一致导致的列表状态同步问题。

Copilot

Pull request overview

This PR reduces record storage footprint by introducing gzip-compressed historical record segments while keeping a stable “logical record name” (timestamp.txt) across the app/server/sync pipeline.

Changes:

Add background gzip compression for closed record segments on the server; keep active segment as plain text.
Introduce RecordFileNames (logical vs physical naming) and RecordFileIO (plain/gzip unified reader/copy) and migrate parsing/export/cache keys to logical names.
Update sync protocol usage to send a selected file list and adjust shell-side cleanup + client-side cache prewarming.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
shared/src/main/java/yangfentuozi/batteryrecorder/shared/sync/PfdFileSender.kt	Add `sendFiles()` API and refactor single-file sending logic for selected file lists.
shared/src/main/java/yangfentuozi/batteryrecorder/shared/data/RecordsFile.kt	Normalize `RecordsFile.name` to logical record name derived from physical file names.
shared/src/main/java/yangfentuozi/batteryrecorder/shared/data/RecordFileParser.kt	Switch to unified reader to support parsing `.txt` and `.txt.gz`.
shared/src/main/java/yangfentuozi/batteryrecorder/shared/data/RecordFileNames.kt	New: define record naming rules, logical naming, and “stable file” selection (prefer gzip).
shared/src/main/java/yangfentuozi/batteryrecorder/shared/data/RecordFileIO.kt	New: unify reading/copying for plain vs gzip record files.
server/src/main/java/yangfentuozi/batteryrecorder/server/writer/PowerRecordWriter.kt	Add compression thread, bootstrap historical compression, and schedule compression on segment close.
server/src/main/java/yangfentuozi/batteryrecorder/server/Server.kt	Use `sendFiles()` for sync and adjust post-sync deletion to handle `.txt`/`.txt.gz` variants.
app/src/main/java/yangfentuozi/batteryrecorder/ui/viewmodel/HistoryViewModel.kt	Compare/update list state using logical record names to avoid `.txt` vs `.txt.gz` churn.
app/src/main/java/yangfentuozi/batteryrecorder/data/history/SyncUtil.kt	After receiving, prewarm `power_stats` cache keyed by logical record name.
app/src/main/java/yangfentuozi/batteryrecorder/data/history/HistoryRepository.kt	Use stable file listing, logical-name cache keys, gzip-aware export, and variant-aware deletes.
app/src/main/java/yangfentuozi/batteryrecorder/data/history/HistoryCacheVersions.kt	Bump cache version due to cache key/name changes.
app/src/main/java/yangfentuozi/batteryrecorder/data/history/HistoryCacheNaming.kt	Rename power-stats cache naming parameter to logical record name.
app/src/main/java/yangfentuozi/batteryrecorder/data/history/DischargeRecordScanner.kt	Handle `.gz` end-timestamp fallback by scanning via parser.
AGENTS.md	Document gzip compression behavior, logical naming, and updated sync/cache behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-17T06:58:31Z

+    fun awaitCompressionBlocking() {
+        val latch = CountDownLatch(1)
+        compressionHandler.post { latch.countDown() }
+        latch.await()
+    }


awaitCompressionBlocking() 通过 CountDownLatch 等待 compressionHandler 执行一个 post；如果该方法在 compressionHandler 所在线程调用，会造成自锁/死等。建议在进入时检测 Looper.myLooper() == compressionHandler.looper 则直接返回/直接执行，或至少抛出明确异常以避免潜在死锁。

Copilot · 2026-04-17T06:58:32Z

    /**
     * 列出指定类型目录下的所有物理记录文件。
     *
     * @param context 应用上下文。
     * @param type 历史类型。
     * @return 返回目录内全部文件，不做文件名与内容合法性过滤。
     */
    private fun listAllRecordFiles(context: Context, type: BatteryStatus): List<File> =
-        dataDir(context, type)
-            .listFiles()
-            ?.filter { it.isFile }
-            ?.toList()
-            ?: emptyList()
+        listSortedRecordFiles(dataDir(context, type))


listAllRecordFiles() 的注释声明“返回目录内全部文件，不做文件名与内容合法性过滤”，但实现已改为调用 listSortedRecordFiles()，会过滤掉无法解析的文件名并去重 plain/gzip。建议同步更新注释（或改回真正的“列出全部物理文件”实现），避免后续调用方误判行为。

Copilot · 2026-04-17T06:58:32Z

    private fun listSortedRecordFiles(dir: File): List<File> =
-        dir.listFiles()
-            ?.asSequence()
-            ?.filter { it.isFile }
-            ?.mapNotNull { file ->
-                val timestamp = recordFileTimestampOrNull(file)
-                if (timestamp == null) {
-                    LoggerX.w(TAG, "[记录] 跳过非法记录文件: ${file.absolutePath}")
-                    return@mapNotNull null
+        RecordFileNames.listStableFiles(dir).also {
+            dir.listFiles()
+                ?.asSequence()
+                ?.filter { it.isFile }
+                ?.forEach { file ->
+                    if (
+                        RecordFileNames.parse(file.name) == null &&
+                        !RecordFileNames.isTempFileName(file.name)
+                    ) {
+                        LoggerX.w(TAG, "[记录] 跳过非法记录文件: ${file.absolutePath}")
+                    }
                }
-                file to timestamp
-            }
-            ?.sortedByDescending { it.second }
-            ?.map { it.first }
-            ?.toList()
-            ?: emptyList()
+        }


listSortedRecordFiles() 目前会对同一目录做两次 listFiles() 扫描：一次在 RecordFileNames.listStableFiles(dir) 内部，另一次在 also { dir.listFiles()... } 用于告警非法文件。目录中文件数较多时会带来不必要的 IO/分配开销。建议只扫描一次（例如把 listFiles() 结果复用，或让 listStableFiles 返回稳定列表的同时产出/记录非法文件）。

Copilot · 2026-04-17T06:58:32Z

                val currChargeDataPath =
                    if (writer.chargeDataWriter.needStartNewSegment(writer.chargeDataWriter.hasPendingStatusChange)) null
                    else writer.chargeDataWriter.segmentFile?.toPath()

                val currDischargeDataPath =
                    if (writer.dischargeDataWriter.needStartNewSegment(writer.dischargeDataWriter.hasPendingStatusChange)) null
                    else writer.dischargeDataWriter.segmentFile?.toPath()
+                val protectedPaths = setOfNotNull(
+                    currChargeDataPath?.toAbsolutePath()?.toString(),
+                    currDischargeDataPath?.toAbsolutePath()?.toString()
+                )
+                val filesToSync = writer.listStableHistoryFiles()


在 sync() 中计算 protectedPaths 时，如果 needStartNewSegment(...) 为 true 会把当前 segmentFile 当作“不需要保护”，但该文件此时仍可能被 AdvancedWriter 持有并继续写入。随后在 sendFiles 回调里会对已发送文件执行 delete()，可能把仍在写入的当前分段删掉（Linux 下会导致 FD 指向已 unlink 文件，后续采样数据丢失且文件不可见）。建议 protectedPaths 直接基于当前 writer.*DataWriter.segmentFile（或其 canonicalPath）构建，不要依赖 needStartNewSegment 的判断；或者在发送/清理前显式 closeCurrentSegment() 并确保不再写入后再删除。

ItosEO added 5 commits April 17, 2026 10:04

fix: 移除历史记录同步时的文件保护路径过滤逻辑

01a5535

refactor: 调整 Server.kt 中同步逻辑的执行顺序

65aaee5

- 在非 root 模式下才执行 `writer.awaitCompressionBlocking()`，优化 root 模式下的同步效率。

feat: 优化文件同步逻辑，支持同步当前活跃分段文件

42c9f76

- 更新文档，明确 `sync()` 现在支持同步活跃分段文件，并说明了 shell 侧的清理规则。 - 移除同步前的压缩等待阻塞逻辑，提升同步响应速度。

Copilot AI review requested due to automatic review settings April 17, 2026 06:53

ItosEO marked this pull request as draft April 17, 2026 06:53

Copilot started reviewing on behalf of ItosEO April 17, 2026 06:53 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize record size#57

Optimize record size#57
ItosEO wants to merge 5 commits intomainfrom
Optimize-record-size

ItosEO commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ItosEO commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants