chore: add tutorial for python gen-build-spec and support more build tools

Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>

Author: behnazh-w

docs/source/pages/cli_usage/command_gen_build_spec.rst

@@ -39,5 +39,4 @@ Options
 
 .. option:: --output-format OUTPUT_FORMAT
 
-    The desired output format for the build specification. The default format is `rc-buildspec`, which is the Reproducible-Central build specification.
-    Other formats may be available depending on your configuration.
+    The output format. Can be `default-buildspec` (default) or `rc-buildspec` (Reproducible-central build spec)

docs/source/pages/output_files.rst

@@ -21,6 +21,7 @@ Top level structure
 
     output/
         ├── build_log/
+        ├── buildspec/
         ├── git_repos/
         ├── reports/
         ├── debug.log
@@ -43,8 +44,8 @@ The report files of Macaron (from using the :ref:`analyze command <analyze-comma
 Unique result path
 ''''''''''''''''''
 
-For each target software component, Macaron creates a directory under ``reports`` to store the report files. This directory
-path is formed from the PURL string of that component. The final path is created using the following template:
+For each target software component, Macaron creates a directory under ``reports`` to store the report. These directory
+paths are formed from the PURL string of that component. The final path is created using the following template:
 
 .. code-block::
 
@@ -131,6 +132,25 @@ to the directory:
 
 .. note:: Please see :ref:`pages/using:analyzing a repository on the local file system` to know how to set the directory for analyzing local repositories.
 
+.. _output_files_macaron_build_spec-Gen:
+
+--------------------------------------
+Output files of macaron gen-build-spec
+--------------------------------------
+
+As part of the ``gen-build-spec`` command, Macaron generates build spec files to help rebuilding artifacts from source. For each target software component, Macaron creates a dedicated directory under ``buildspec`` to store the generated build specification file. These directory paths are derived from the component's PURL (Package URL) string. The resulting path structure follows this template:
+
+.. code-block::
+
+    <path_to_output>/buildspec/<purl_type>/<purl_namespace>/<purl_name>
+
+Depending on the chosen output format, the following files may be generated in each directory:
+- ``macaron.buildspec`` (default format)
+- ``reproducible_central.buildspec`` (when run with the ``rc-buildspec`` output format for Maven artifacts)
+
+Each file contains the build specification for the corresponding software component.
+
+
 .. _output_files_macaron_verify_policy:
 
 -------------------------------------

docs/source/pages/supported_technologies/index.rst

@@ -29,7 +29,8 @@ such as GitHub Actions workflows.
 Build Specification Generation
 ------------------------------
 
-* Maven and Gradle builds for Java artifacts
+* Maven and Gradle builds for Java packages
+* The built-in ``build`` module and various build tools, like Poetry for Python packages
 
 .. _supported_git_services:
 

docs/source/pages/tutorials/rebuild_third_party_artifacts.rst

@@ -16,6 +16,7 @@ These buildspecs help document and automate the build process for packages, enab
 
    * - Currently Supported packages
    * - Maven packages built with Gradle or Maven
+   * - Python packages built with the built-in ``build`` module and various build tools, like Poetry
 
 .. contents:: :local:
 
@@ -31,9 +32,9 @@ Addressing this lack of transparency is critical for improving supply chain secu
 Background
 **********
 
-A build specification is a file that describes all necessary information to rebuild a package from source. This includes metadata such as the build tool, the specific build command to run, the language version, e.g., JDK for Java, and artifact coordinates. Macaron can now generate this file automatically for supported ecosystems, greatly simplifying build from source.
+A build specification is a file that describes all necessary information to rebuild a package from source. This includes metadata such as the build tool, the specific build command to run, the language version, e.g., Python or JDK for Java, and artifact coordinates. Macaron can now generate this file automatically for supported ecosystems, greatly simplifying build from source.
 
-The generated buildspec will be stored in an ecosystem- and PURL-specific path under the ``output/`` directory (see more under :ref:`Output Files Guide <output_files_guide>`).
+The generated buildspec will be stored in an ecosystem- and PURL-specific path under the ``output/`` directory (see more under :ref:`Output Files Guide <output_files_macaron_build_spec-Gen>`).
 
 ******************************
 Installation and Prerequisites
@@ -101,7 +102,46 @@ In the example above, the buildspec is located at:
 Step 3: Review and Use the Buildspec File
 *****************************************
 
-The generated buildspec uses the `Reproducible Central buildspec <https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/doc/BUILDSPEC.md>`_ format, for example:
+By default we generate the buildspec in JSON format as follows:
+
+.. code-block:: ini
+
+    {
+        "macaron_version": "0.18.0",
+        "group_id": "org.apache.hugegraph",
+        "artifact_id": "computer-k8s",
+        "version": "1.0.0",
+        "git_repo": "https://github.com/apache/hugegraph-computer",
+        "git_tag": "d2b95262091d6572cc12dcda57d89f9cd44ac88b",
+        "newline": "lf",
+        "language_version": [
+            "11"
+        ],
+        "ecosystem": "maven",
+        "purl": "pkg:maven/org.apache.hugegraph/computer-k8s@1.0.0",
+        "language": "java",
+        "build_tool": "maven",
+        "build_commands": [
+            [
+            "mvn",
+            "-DskipTests=true",
+            "-Dmaven.test.skip=true",
+            "-Dmaven.site.skip=true",
+            "-Drat.skip=true",
+            "-Dmaven.javadoc.skip=true",
+            "clean",
+            "package"
+            ]
+        ]
+    }
+
+If you use the ``rc-buildspec`` output format, the generated buildspec follows the `Reproducible Central buildspec <https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/doc/BUILDSPEC.md>`_ format. For example, you can generate it with:
+
+.. code-block:: shell
+
+    ./run_macaron.sh gen-build-spec -purl pkg:maven/org.apache.hugegraph/computer-k8s@1.0.0 --database output/macaron.db --output-format rc-buildspec
+
+The resulting file will be saved as ``output/buildspec/maven/org_apache_hugegraph/computer-k8s/reproducible_central.buildspec``, and will look like this:
 
 .. code-block:: ini
 
@@ -136,18 +176,18 @@ The ``gen-build-spec`` works as follows:
 
 - Extracts metadata and build information from Macaron’s local SQLite database.
 - Parses and modifies build commands from CI/CD configurations to ensure compatibility with rebuild systems.
-- Identifies the JDK version by parsing CI/CD configurations or extracting it from the ``META-INF/MANIFEST.MF`` file in Maven Central artifacts.
+- Identifies the language version, e.g., JDK version by parsing CI/CD configurations or extracting it from the ``META-INF/MANIFEST.MF`` file in Maven Central artifacts.
 - Ensures that only the major JDK version is included, as required by the build specification format.
 
 
-This feature is described in more detail in our accepted ASE 2025 Industry ShowCase paper: `Unlocking Reproducibility: Automating the Re-Build Process for Open-Source Software <https://arxiv.org/pdf/2509.08204>`_.
+The Java support for this feature is described in more detail in our accepted ASE 2025 Industry ShowCase paper: `Unlocking Reproducibility: Automating the Re-Build Process for Open-Source Software <https://arxiv.org/pdf/2509.08204>`_.
 
 ***********************************
 Frequently Asked Questions (FAQs)
 ***********************************
 
 *Q: What formats are supported for buildspec output?*
-A: Currently, only ``rc-buildspec`` is supported.
+A: Currently, a default JSON spec and optional ``rc-buildspec`` are supported.
 
 *Q: Do I need to analyze the package every time before generating a buildspec?*
 A: No, you only need to analyze the package once unless you want to update the database with newer information.

src/macaron/build_spec_generator/common_spec/core.py

@@ -57,6 +57,9 @@ class MacaronBuildToolName(str, Enum):
     GRADLE = "gradle"
     PIP = "pip"
     POETRY = "poetry"
+    FLIT = "flit"
+    HATCH = "hatch"
+    CONDA = "conda"
 
 
 def format_build_command_info(build_command_info: list[GenericBuildCommandInfo]) -> str:

src/macaron/build_spec_generator/common_spec/pypi_spec.py

@@ -67,6 +67,12 @@ def get_default_build_command(
                 default_build_command = "python -m build".split()
             case "poetry":
                 default_build_command = "poetry build".split()
+            case "flit":
+                default_build_command = "flit build".split()
+            case "hatch":
+                default_build_command = "hatch build".split()
+            case "conda":
+                default_build_command = "conda build".split()
             case _:
                 pass
 
@@ -95,8 +101,7 @@ def resolve_fields(self, purl: PackageURL) -> None:
         registry.load_defaults()
 
         registry_info = PackageRegistryInfo(
-            build_tool_name="pip",
-            build_tool_purl_type="pypi",
+            ecosystem="pypi",
             package_registry=registry,
             metadata=[],
         )

src/macaron/config/defaults.ini

@@ -293,11 +293,13 @@ packager =
     pip3
     flit
     conda
+    hatch
 publisher =
     twine
     flit
     conda
     tox
+    hatch
 # These are the Python interpreters that may be used to load modules.
 interpreter =
     python
@@ -322,6 +324,11 @@ package_lock = poetry.lock
 builder =
     poetry
     poetry-core
+# build-system information.
+build_requires =
+    poetry-core
+build_backend =
+    poetry.core.masonry.api
 # These are the Python interpreters that may be used to load modules.
 interpreter =
     python
@@ -336,6 +343,82 @@ deploy_arg =
 [builder.poetry.ci.deploy]
 github_actions = pypa/gh-action-pypi-publish
 
+# This is the spec for Flit packaging tool.
+[builder.flit]
+entry_conf =
+build_configs =
+    pyproject.toml
+    flit.ini
+builder =
+    flit
+# build-system information.
+build_requires =
+    flit_core
+build_backend =
+    flit_core.buildapi
+# These are the Python interpreters that may be used to load modules.
+interpreter =
+    python
+    python3
+interpreter_flag =
+    -m
+build_arg =
+    build
+deploy_arg =
+    publish
+
+[builder.flit.ci.deploy]
+github_actions = pypa/gh-action-pypi-publish
+
+# This is the spec for the Hatch packaging tool.
+[builder.hatch]
+entry_conf =
+build_configs =
+    pyproject.toml
+    hatch.toml
+builder =
+    hatch
+# build-system information.
+build_requires =
+    hatchling
+build_backend =
+    hatchling.build
+# These are the Python interpreters that may be used to load modules.
+interpreter =
+    python
+    python3
+interpreter_flag =
+    -m
+build_arg =
+    build
+deploy_arg =
+    publish
+
+[builder.hatch.ci.deploy]
+github_actions = pypa/gh-action-pypi-publish
+
+# This is the spec for the Conda packaging tool.
+[builder.conda]
+entry_conf =
+build_configs =
+    environment.yml
+    meta.yaml
+builder =
+    conda
+# These are the Python interpreters that may be used to load modules.
+interpreter =
+    python
+    python3
+interpreter_flag =
+    -m
+build_arg =
+    build
+deploy_arg =
+    publish
+
+[builder.conda.ci.deploy]
+github_actions = pypa/gh-action-pypi-publish
+
 # This is the spec for trusted Docker build tool usages.
 [builder.docker]
 entry_conf =

src/macaron/provenance/provenance_finder.py

@@ -554,11 +554,7 @@ def get_artifact_hash(
             return None
 
         registry_info = next(
-            (
-                info
-                for info in package_registries_info
-                if info.package_registry == pypi_registry and info.build_tool_name in {"pip", "poetry"}
-            ),
+            (info for info in package_registries_info if info.package_registry == pypi_registry),
             None,
         )
         if not registry_info:

src/macaron/repo_finder/repo_finder_pypi.py

@@ -37,11 +37,7 @@ def find_repo(
     if package_registries_info:
         # Find the package registry info object that contains the PyPI registry and has the pypi build tool.
         pypi_info = next(
-            (
-                info
-                for info in package_registries_info
-                if isinstance(info.package_registry, PyPIRegistry) and info.build_tool_name in {"poetry", "pip"}
-            ),
+            (info for info in package_registries_info if isinstance(info.package_registry, PyPIRegistry)),
             None,
         )
         if not pypi_info:

src/macaron/slsa_analyzer/analyzer.py

@@ -379,7 +379,7 @@ def run_single(
             )
 
         # Pre-populate all package registries so assets can be stored for later.
-        package_registries_info = self._populate_package_registry_info()
+        package_registries_info = self._populate_package_registry_info(parsed_purl) if parsed_purl else []
 
         provenance_is_verified = False
         provenance_asset = None
@@ -1127,18 +1127,14 @@ def _determine_ci_services(self, analyze_ctx: AnalyzeContext, git_service: BaseG
                 "[red]Not Found[/]",
             )
 
-    def _populate_package_registry_info(self) -> list[PackageRegistryInfo]:
+    def _populate_package_registry_info(self, parsed_purl: PackageURL) -> list[PackageRegistryInfo]:
         """Add all possible package registries to the analysis context."""
         package_registries = []
         for package_registry in PACKAGE_REGISTRIES:
-            for build_tool in BUILD_TOOLS:
-                build_tool_name = build_tool.name
-                if build_tool_name not in package_registry.build_tool_names:
-                    continue
+            if package_registry.ecosystem == parsed_purl.namespace:
                 package_registries.append(
                     PackageRegistryInfo(
-                        build_tool_name=build_tool_name,
-                        build_tool_purl_type=build_tool.purl_type,
+                        ecosystem=parsed_purl.namespace,
                         package_registry=package_registry,
                     )
                 )
@@ -1149,14 +1145,10 @@ def _determine_package_registries(
         analyze_ctx: AnalyzeContext,
         package_registries_info: list[PackageRegistryInfo],
     ) -> None:
-        """Determine the package registries used by the software component based on its build tools."""
-        build_tools = (
-            analyze_ctx.dynamic_data["build_spec"]["tools"] or analyze_ctx.dynamic_data["build_spec"]["purl_tools"]
-        )
-        build_tool_names = {build_tool.name for build_tool in build_tools}
+        """Determine the package registries used by the software component."""
         relevant_package_registries = []
         for package_registry in package_registries_info:
-            if package_registry.build_tool_name not in build_tool_names:
+            if not package_registry.ecosystem == analyze_ctx.component.namespace:
                 continue
             relevant_package_registries.append(package_registry)