Main dev #51

ShuhaoZhangTony · 2025-10-08T14:38:01Z

No description provided.

- Remove unused Callable and glob imports from sage_flow.py - Improve TYPE_CHECKING imports in sage_flow_service.py - Code quality improvements from sage dev quality

- 检测conda环境中的libstdc++是否支持C++20/GCC 11+所需的GLIBCXX_3.4.30+ - 如果版本过低，自动通过conda安装最新版本的libstdcxx-ng - 如果自动更新失败，设置LD_LIBRARY_PATH使用系统libstdc++ - 解决'GLIBCXX_3.4.31' not found错误

Python bindings are now maintained in the main SAGE repository at: packages/sage-middleware/src/sage/middleware/components/sage_flow/python/ This submodule now only contains the C++ core implementation.

Examples are now maintained in the main SAGE repository at: packages/sage-middleware/src/sage/middleware/components/sage_flow/examples/

- Change project name from CANDY to sageFlow - Rename library target from candy to sageflow - Update all library references in CMakeLists.txt files - Keep candy namespace in C++ code unchanged - This aligns library naming with the sageFlow project name

Python bindings are now exclusively built in the main SAGE repository. The submodule only provides the C++ core library (sageflow interface). Changes: - Remove BUILD_PYTHON_BINDINGS option - Remove all pybind11-related code - Remove _sage_flow target and related configurations - Python bindings are built by main repo's build.sh script

Unified namespace across entire codebase: - Replaced 'namespace candy' with 'namespace sageFlow' (178 files) - Updated all header files, source files, and tests - Changed 'using namespace candy' to 'using namespace sageFlow' - Updated namespace closing comments This improves code consistency and aligns with the project name.

Previous commit only replaced 'namespace candy' declarations but missed the scope resolution operators (candy::) in function implementations. This completes the namespace renaming by replacing all occurrences of: - candy:: -> sageFlow:: in all .cpp, .h files across src/, include/, and test/ directories.

- 将 sageflow 从 INTERFACE 库改为 SHARED 动态库 - 使用 GLOB_RECURSE 收集所有源文件（排除测试和示例） - 添加 install 规则，输出到 lib/ 目录 - 与 sageDB 保持一致的架构，简化 Python 绑定

问题: - BUILD_GTEST 被强制设为 ON - 即使 BUILD_TESTING=OFF，仍会尝试安装 gtest - 导致 Python 绑定构建时安装失败修复: - gtest 构建遵循 BUILD_TESTING 选项 - BUILD_TESTING=OFF 时跳过 googletest 的 FetchContent - externalTestLibs 也设为条件编译 - 允许 cmake --install 正常工作

Copilot

Pull Request Overview

This pull request performs a comprehensive refactoring to rename the project from "candy" to "sageFlow" throughout the entire codebase. The changes include namespace updates, library name changes, build system modifications, and removal of Python bindings. The refactoring maintains functional equivalence while establishing a new project identity.

Systematic namespace change from candy to sageFlow across all C++ source files
Build system updates to create sageflow shared library instead of candy interface library
Complete removal of Python bindings and examples
Conditional build improvements for test dependencies

Reviewed Changes

Copilot reviewed 151 out of 151 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
CMakeLists.txt	Updated project name and removed Python bindings
third-party/CMakeLists.txt	Added conditional building for test dependencies
src/CMakeLists.txt	Changed from interface to shared library target
All C++ files	Renamed namespace from `candy` to `sageFlow`
Python files	Removed all Python bindings and service files
Examples	Removed all example applications

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

test/CMakeLists.txt

CMakeLists.txt

ZeroJustMe

LGTM

* refactor third-party dependency setup * fix examples build bug

…ta persistence, join data sources, configuration-based management, and multi-mode data loading tests (#53) * refactor third-party dependency setup * fix examples build bug * Initial plan * Implement data source framework with random and dataset sources Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add comprehensive tests and documentation for data source framework Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add runnable example and implementation summary Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add verification document for critical tests Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add comprehensive test execution results for critical test files Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add data persistence framework with FVECS and JSON support Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Refactor logging macros to use consistent naming convention - Updated all instances of `sageFlow_LOG_*` to `SAGEFLOW_LOG_*` across multiple source files for consistency and adherence to naming conventions. - This change affects various components including join operators, sink operators, storage manager, stream sources, and performance monitoring. - Adjusted related test files to reflect the updated logging macros. * Merge main-dev namespace changes (sageFlow_LOG to SAGEFLOW_LOG) Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add join data source framework for flexible join test data generation Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Refactor test_pipeline_basic and test_join_perf_scaling to use JoinTestHelper Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add comprehensive test refactoring documentation Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * 修改 CI 工作流中的分支名称，将 'join_pre_experiment' 更改为 'main-dev' * Refactor join data source: move VectorListSource to data_source folder and simplify modes Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add comprehensive documentation for code review improvements Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Replace std::cout/cerr with SAGEFLOW_LOG and add data source configuration support Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Add documentation for remaining work on test_join_perf_scaling enhancements Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * 修复缩进 * Add test_join_datasource_modes with 3 data loading modes (generate-save-load, direct-load, generate-direct-use) Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Fix compilation error: remove duplicate code in test_join_perf_scaling.cpp Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Translate all documentation to Chinese (batch 1: CODE_REVIEW_IMPROVEMENTS, IMPLEMENTATION_SUMMARY) Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Translate remaining documentation to Chinese (batch 2-4: all remaining files) Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com> * Refactor and remove obsolete documentation files; update test cases for data source integration - Deleted REMAINING_WORK.md, TEST_EXECUTION_RESULTS.md, TEST_REFACTORING_SUMMARY.md, translate_all_docs.sh, translate_docs.py, and translate_remaining.py as they are no longer needed. - Updated test_join_perf_scaling.cpp to include new test cases for RandomDataSource and DatasetDataSource, ensuring performance metrics are comparable. - Verified that all critical test files pass successfully with the new data source framework changes, maintaining backward compatibility. - Improved code maintainability by refactoring test cases to use JoinTestHelper, reducing duplication and enhancing readability. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: ZeroJustMe <13223093+ZeroJustMe@users.noreply.github.com>

* 更新 .gitignore 文件以包含实验输出文件类型，并添加 NotoSansSC-Regular 字体文件以支持报告中文显示 * 添加项目相对路径解析功能，更新数据源和存储配置路径的处理逻辑 * Update test/result/analyze.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * 更新 .gitignore 文件，添加对 .otf 文件的忽略规则 * Delete test/result/.fonts/NotoSansSC-Regular.otf --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

ShuhaoZhangTony added 12 commits September 30, 2025 11:17

Clean unused imports and improve type annotations

e506f86

- Remove unused Callable and glob imports from sage_flow.py - Improve TYPE_CHECKING imports in sage_flow_service.py - Code quality improvements from sage dev quality

code quality fix

36a0db6

Update SageFlow import error message to recommend CLI install

71b6a91

chore: remove python bindings (moved to main SAGE repo)

6374a63

Python bindings are now maintained in the main SAGE repository at: packages/sage-middleware/src/sage/middleware/components/sage_flow/python/ This submodule now only contains the C++ core implementation.

chore: move examples to main SAGE repo

fa7c6b0

Examples are now maintained in the main SAGE repository at: packages/sage-middleware/src/sage/middleware/components/sage_flow/examples/

refactor: 改为动态库以匹配 sageDB 架构

aa8879b

- 将 sageflow 从 INTERFACE 库改为 SHARED 动态库 - 使用 GLOB_RECURSE 收集所有源文件（排除测试和示例） - 添加 install 规则，输出到 lib/ 目录 - 与 sageDB 保持一致的架构，简化 Python 绑定

ShuhaoZhangTony requested review from ZeroJustMe and Copilot October 8, 2025 14:38

ShuhaoZhangTony self-assigned this Oct 8, 2025

ShuhaoZhangTony added the enhancement New feature or request label Oct 8, 2025

ShuhaoZhangTony added this to SAGE Oct 8, 2025

Copilot AI reviewed Oct 8, 2025

View reviewed changes

test/CMakeLists.txt Show resolved Hide resolved

CMakeLists.txt Show resolved Hide resolved

ShuhaoZhangTony added 5 commits October 9, 2025 09:59

将例子推送回c++项目

88bf3a7

update namespaces and add examples

63f3895

fix examples to link against sageflow

f1c1ae4

updated: refs/heads/main-dev

af22e8a

quick fix

8f43967

ZeroJustMe previously approved these changes Oct 9, 2025

View reviewed changes

清理.gitignore文件，移除不必要的注释和空行

a6839ac

ShuhaoZhangTony dismissed ZeroJustMe’s stale review via a6839ac October 9, 2025 07:00

ZeroJustMe and others added 3 commits October 9, 2025 16:32

fix streaming examples build bug (#52)

f0827ee

* refactor third-party dependency setup * fix examples build bug

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Main dev #51

Main dev #51

Uh oh!

ShuhaoZhangTony commented Oct 8, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

ZeroJustMe left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Main dev #51

Are you sure you want to change the base?

Main dev #51

Uh oh!

Conversation

ShuhaoZhangTony commented Oct 8, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

ZeroJustMe left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants