Failed to Generate Vulnerability Signature

[aixiao0621]

Hello, I would like to add support for GitHub commit signature generation by enhancing the code_extractor. Before proceeding, I tested the command:

$ ./bazel-bin/sign_generator_runner --vulnerability_file_name=/mnt/d/vanir/test/tty1.json --signature_file_name=/mnt/d/vanir/test/op.json

The content of tty1.json is:

[
    {
    "id": "ASB-A-244395411",
    "details": "In pipe_resize_results of pipe.c, there is a possible UAF bug caused by a race condition. This could lead to local denial of service and local escalation of privilege with no additional execution privileges needed. User interaction is not needed for exploitation. ",
    "aliases": [
        "A-244395411",
        "CVE-2022-2959"
    ],
    "modified": "2024-11-06T12:16:03.231308Z",
    "published": "2023-01-01T00:00:00Z",
    "references": [
        {
            "type": "ADVISORY",
            "url": "https://source.android.com/security/bulletin/2023-01-01"  
        },
        {
            "type": "FIX",
            "url": "https://android.googlesource.com/kernel/common/+/a2c2b6c91475908e2ac90a3d70f5d12ae86b8033"  
        }
    ],
    "affected": [
        {
            "package": {
                "name": ":linux_kernel:",
                "ecosystem": "Android"
            },
            "ranges": [
                {
                    "type": "ECOSYSTEM",
                    "events": [
                        {
                            "introduced": ":0"
                        },
                        {
                            "fixed": ":2023-01-05"
                        }
                    ]
                }
            ],
            "versions": [
                "Kernel"
            ],
            "database_specific": {
                "source": "https://storage.googleapis.com/android-osv/ASB-A-244395411.json"  
            }
        }
    ],
    "schema_version": "1.6.0"
}]

However, this operation did not output the signature for the vulnerability. Instead, it only added an empty "ecosystem_specific": {} under affected. Could this be due to an incorrect usage on my part?

[baekhyunwook]

Thank you for your interest in contributing to Vanir!

The Vanir signature generator currently takes the fix URL from affected[].ecosystem_specific.fixes
(I added more verbose reason on why we do this at the bottom of this comment).

You would be able to generate the signature by adding the fix link in the field as follow:

[
    {
    "id": "ASB-A-244395411",
    "details": "In pipe_resize_results of pipe.c, there is a possible UAF bug caused by a race condition. This could lead to local denial of service and local escalation of privilege with no additional execution privileges needed. User interaction is not needed for exploitation. ",
    "aliases": [
        "A-244395411",
        "CVE-2022-2959"
    ],
    "modified": "2024-11-06T12:16:03.231308Z",
    "published": "2023-01-01T00:00:00Z",
    "references": [
        {
            "type": "ADVISORY",
            "url": "https://source.android.com/security/bulletin/2023-01-01"  
        },
        {
            "type": "FIX",
            "url": "https://android.googlesource.com/kernel/common/+/a2c2b6c91475908e2ac90a3d70f5d12ae86b8033"  
        }
    ],
    "affected": [
        {
            "package": {
                "name": ":linux_kernel:",
                "ecosystem": "Android"
            },
            "versions": ["SoCVersion"],
            "ecosystem_specific": {
              "fixes": [
                "https://android.googlesource.com/kernel/common/+/a2c2b6c91475908e2ac90a3d70f5d12ae86b8033"
              ]
            },
            "ranges": [
                {
                    "type": "ECOSYSTEM",
                    "events": [
                        {
                            "introduced": ":0"
                        },
                        {
                            "fixed": ":2023-01-05"
                        }
                    ]
                }
            ],
            "versions": [
                "Kernel"
            ],
            "database_specific": {
                "source": "https://storage.googleapis.com/android-osv/ASB-A-244395411.json"  
            }
        }
    ],
    "schema_version": "1.6.0"
}]

Why not directly taking fixes from references?

We used to directly retrieve patches from the references, but, with the introduction of version-aware signature generation, we decided to always associate fixes with their target versions in the signature generation time. Simply speaking, this was in order to prevent the signature publishers from generating false-positive-causing signatures.

We will try adding more detailed documentation on the signature generation process soon, but please feel free to let me know if you have more questions meanwhile.

[aixiao0621]

Thank you for your response.
If I wish to incorporate GitHub commit or local Git diff information to generate custom vulnerability signatures, how should I proceed?
Could you provide me with some guidance and assistance?

[baekhyunwook]

Sure! I think it might be a multi-stage project, so I tried dumping my high-level idea below. Please take a look and feel free to let me know what you think.

Support different sources in the Android ecosystem

The current commit extraction mechanisms are mostly implemented in code_extractor_android. Thus, for simplicity, it would be easier to start with extending the functionality in the Android ecosystem.

For now, I would simply assume that we would not need any additional flags for this feature since we would be able to make Vanir transparently handle different commit URLs such as GitHub commits and local commits. For it, we would need some modifications in class AndroidCommit, especially in the logic that handle the given fix URL with assumption that it uses Gitiles. E.g., the current AndroidCommit class extracts the plain text version of the patch by accessing the TEXT version of the commit from Gitiles, implemented in AndroidCommit._extract_patch().

One important thing that we should have in mind is that Vanir requires three different pieces of information for creating signatures -- 1) patch, 2) affected sources before applying the patch, 3) affected sourced after applying the patch (technically, 3) can be driven from 1) and 2) but the current behavior pulls 3) directly from the source). These pieces of information is supposed to be extracted through three different methods in Commit class -- _extract_patch(), _extract_patched_files() and _extract_unpatched_files().

Github commit

For supporting Github commit, we may first need to update the AndroidCommit class to identify the type of the given fix commit (Github commit, Googlesource commit, ...) and make the _extract_XYZ() method behave differently depending on the type of fix commit. Presumably using [Github REST API]https://docs.github.com/en/rest/quickstart?apiVersion=2022-11-28&tool=curl() or the corresponding Python wrapper such as PyGithub would be preferred.

Local patch

Compared to the Github support, I think supporting the local patch could be somewhat tricky due to several design problems.

Safety control

For signatures generated from local patches, it would be important to clarify the sources of the signatures to prevent potential abuse of using this type of signatures for hiding vulnerabilities (e.g., by generating signatures for vuln-introducing commits). Thus, most of signatures generated from the local patches would not be acceptable for public-use without strong justification on the authenticity of the fix sources. No clear design decision has been made regarding this yet on our side, but, when implementing this, at least some safety assurance mechanism (such as a flag for tolerating signatures from unknown sources) would be necessary.

Local Git commit VS Local raw patches

On the technical side, supporting local raw patch would require more work than supporting local commit.

Local Git commit

As mentioned above, you would need to tell Vanir the pointers to the three different pieces of information somehow - patch, patched version of affected files and unpatched version of affected files. If you would use local Git, you would be able to provide local git repository information together with the fix commit through the "fixes" field, and you may make Vanir to leverage existing Python git lib such as GitPython for extracting corresponding information.

Local raw patches

If you also want to support local patches (i.e., just source files and patch files without any version control system), you should first make a design decision on how we should tell Vanir the pointers to the different pieces of information. I can imagine multiple ways (e.g., breaking the assumption that "fixes" are URL strings and allowing nested dictionary VS just introduce experimental flags allowing Vanir to accept file paths for patches and related files) but I don't have a clear idea on what would be systematically clean and also easy to use for the actual users. So, further design discussion / decision would be needed for this.

Support other ecosystem

I think it would be better to ultimately refactor the existing AndroidCommit to factor out the part more related to the source type. It could be multi-inheritance model or mix-in mod model or something else, but I think the design direction can be more easily clarified once the stage suggested above is completed.