Skip to content

117 structure refactor#118

Merged
thientc merged 24 commits intomainfrom
117-structure-refactor
Apr 3, 2026
Merged

117 structure refactor#118
thientc merged 24 commits intomainfrom
117-structure-refactor

Conversation

@thientc
Copy link
Copy Markdown
Collaborator

@thientc thientc commented Apr 3, 2026

No description provided.

thientc and others added 24 commits March 22, 2026 08:25
Move root-level documentation files into docs/ for better organization:
- How-to-work-with-Futag.md → docs/how-to-work-with-futag.md
- Project-summary.md → docs/project-summary.md
- References.md → docs/references.md
- futag-work.mmd → docs/diagrams/futag-work.mmd
- futag-work.png → docs/diagrams/futag-work.png

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rename to better communicate that these are Docker-based integration
tests, distinct from the unit tests in the Python package.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- src/python/futag-package/ → futag-package/ (reduces 2 levels of nesting)
- src/python/*.py → scripts/
- src/python/requirements.txt → scripts/
- Update build scripts, CI workflows, .gitignore, and package metadata

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- src/Checkers/ → analyzers/checkers/
- src/clang/ → analyzers/clang-patches/
- Update all build scripts to use new paths
- Update .gitignore patterns

The analyzers/ directory now clearly represents the analysis pipeline
stage, grouping checker plugins and clang infrastructure together.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clearer name for the LLVM toolchain build infrastructure directory.
Uses build-llvm/ (not build/) to avoid .gitignore collision with the
CMake build output directory.

- Update all build script self-references
- Update .gitignore patterns
- Update Dockerfile WORKDIR paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update documentation to reflect the restructured repository:
- src/Checkers/ → analyzers/checkers/
- src/clang/ → analyzers/clang-patches/
- src/python/futag-package/ → futag-package/
- custom-llvm/ → build-llvm/
- product-tests/ → integration-tests/
- src/python/template-script.py → scripts/template-script.py
- How-to-work-with-Futag.md → docs/how-to-work-with-futag.md

Files updated: CONTRIBUTING.md, README.md, README.en.md,
docs/architecture.md, docs/checkers.md, docs/generators.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update futag-package/README.md and docs/how-to-work-with-futag.md
with new directory paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce futag.toolchain.ToolchainConfig with three factory methods:
- from_futag_llvm() — backward-compatible, from compiled toolchain dir
- from_system() — discover tools via PATH (enables system clang)
- for_generation_only() — no tools needed, source generation only

This is the foundation for decoupling the Python package from the
build infrastructure, enabling alternative analysis backends (CodeQL).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add toolchain parameter to BaseGenerator.__init__()
- Replace hardcoded futag_llvm_package / "bin/..." paths in
  compile_targets() with self.toolchain.clang/clangpp
- Pass toolchain through Generator, FuzzDataProviderGenerator,
  ContextGenerator, and NatchGenerator constructors
- Existing API (positional futag_llvm_package) fully preserved

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace all hardcoded futag_llvm_package / "bin/..." paths in
  preprocessor.py with self.toolchain.* references
- Replace llvm-profdata, llvm-cov, llvm-symbolizer, svres template
  paths in fuzzer.py with self.toolchain.* references
- Add intercept_build to ToolchainConfig for build system integration
- Add toolchain parameter to Builder, ConsumerBuilder, Fuzzer,
  NatchFuzzer constructors
- Add graceful None handling for svres template

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- docs/analysis-schema.json — formal JSON Schema (draft-07) for
  futag-analysis-result.json, the interface between analysis backends
  and the Python generator layer
- docs/analysis-backend.md — guide for producing compatible JSON from
  alternative tools (CodeQL, Joern, etc.) including gen_list type
  decomposition rules, param_usage heuristics, and a minimal example

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Mark build infrastructure as optional/one-time in architecture.md
- Add Usage Modes section with three modes (full, pre-built JSON, gen-only)
- Add ToolchainConfig documentation to python-api.md
- Add toolchain.py to module structure in CLAUDE.md
- Reference analysis-schema.json and analysis-backend.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The ToolchainConfig refactoring incorrectly changed futag_llvm_package
and library_root from required to optional (default=""), breaking
backward compatibility. User scripts passing library_root as the first
positional arg had it misinterpreted as futag_llvm_package.

Restore original API contract: futag_llvm_package and library_root
are required positional parameters in all constructors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generator classes never use futag_llvm_package after construction — they
only used it to build a ToolchainConfig. Now library_root is the first
positional arg and toolchain is an optional keyword parameter.

New API:
  Generator(lib_path)                          # generation-only
  Generator(lib_path, toolchain=tc)            # with compiler
  tc = ToolchainConfig.from_futag_llvm(path)   # construct toolchain

Builder still accepts futag_llvm_package as before (it needs it).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oolchainConfig

- Remove futag_llvm_package from BaseFuzzer, Fuzzer, NatchFuzzer
- Add llvm_config field to ToolchainConfig (populated in from_futag_llvm
  and from_system)
- Fix bug in preprocessor.py:524 where _make_build_env received a Path
  instead of ToolchainConfig
- Replace all self.futag_llvm_package / 'bin/...' in preprocessor.py
  with self.toolchain.* fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All documentation, examples, and workshop scripts now use the new API
where library_root is the first positional arg and toolchain is an
optional keyword parameter:

  tc = ToolchainConfig.from_futag_llvm(FUTAG_PATH)
  generator = Generator(lib_path, toolchain=tc)
  fuzzer = Fuzzer("futag-fuzz-drivers", toolchain=tc)

Builder API is unchanged (still takes futag_llvm_package first).

Updated files: __init__.py, python-api.md, generators.md,
architecture.md, how-to-work-with-futag.md, analysis-backend.md,
template-script.py, workshop/json-c/*, workshop/libvirt/*,
README.md, README.en.md, CLAUDE.md, futag-package/README.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
_validate_common() stored self.futag_llvm_package but no code ever
read it — all build methods already use self.toolchain.* fields.
Remove the 3 dead assignments. Builder.__init__ still accepts
futag_llvm_package as a parameter (used to construct ToolchainConfig).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove futag_llvm_package from pydoc-style signatures for Generator
and Fuzzer in futag-package/README.md. Only Builder signatures retain
the parameter (it still needs it).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add missing `from futag.toolchain import ToolchainConfig` imports
- Add `tc = ToolchainConfig.from_futag_llvm(FUTAG_PATH)` before
  ContextGenerator usage in consumer examples

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Combine the separate Builder and Generator examples into single
continuous scripts so imports and variables are shared. Also fix
missing commas in gen_targets() arguments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Builder and ConsumerBuilder now use toolchain as the primary config
source, consistent with Generator and Fuzzer. library_root is the
first positional arg; futag_llvm_package is an optional keyword
(backward compat).

New API:
  tc = ToolchainConfig.from_futag_llvm(FUTAG_PATH)
  builder = Builder(lib_path, clean=True, toolchain=tc)
  consumer = ConsumerBuilder(lib_path, consumer_path, toolchain=tc)

Update all docs, examples, and workshop scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ppers

All generated fuzz wrappers now use:
- extern "C" int LLVMFuzzerTestOneInput(...) signature
- .cpp file extension (regardless of library source language)
- clang++ / afl-clang-fast++ compiler

This ensures proper C++ linkage for LibFuzzer and eliminates the
C/C++ branching logic in harness generation and compilation.

Merge LIBFUZZER_PREFIX_C and LIBFUZZER_PREFIX_CXX into single
LIBFUZZER_PREFIX constant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The harness_preamble property was defined in BaseGenerator and
overridden by FuzzDataProviderGenerator to return the FDP
initialization line, but was never called in the code path that
writes fuzz target files. FDP-generated targets were missing the
`FuzzedDataProvider provider(Fuzz_Data, Fuzz_Size);` declaration.

Insert the call at all 4 harness-writing sites across base_generator,
context_generator, and natch_generator.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thientc thientc linked an issue Apr 3, 2026 that may be closed by this pull request
@thientc thientc merged commit ca534f8 into main Apr 3, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Structure refactor

1 participant