Skip to content

feat(heuristics): add whitespace check to detect excessive spacing and invisible characters for malware check #1086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

feat(heuristics): add whitespace check to detect excessive spacing and invisible characters for malware check #1086

wants to merge 16 commits into from
+51 −1

Conversation

AmineRaouane
Copy link
Member

@AmineRaouane AmineRaouane commented May 19, 2025 •
edited
Loading

Summary

This PR adds a new heuristic that analyzes code to detect suspicious use of excessive spaces and invisible characters. It checks whether the amount of spacing and invisible Unicode characters exceeds a defined threshold.

Description of changes

  • Implemented the WhiteSpaces heuristic in a new Python module.
  • Registered the new heuristic inside the main heuristics.py file.
  • Created unit tests to verify the behavior of the WhiteSpacesAnalyzer heuristic.
  • Updated detect_malicious_metadata_check.py to integrate and execute the new heuristic logic during analysis.
  • The heuristic scans the codebase for abnormal invisible characters and spaces in the code.
  • This heuristic is combined with ForceSetup to justify high confidence in detection, as the presence of extra spaces alone could be due to poor formatting rather than malicious intent.

Related issues

None

Checklist

  • I have reviewed the contribution guide.
  • My PR title and commits follow the Conventional Commits convention.
  • My commits include the "Signed-off-by" line.
  • I have signed my commits following the instructions provided by GitHub. Note that we run GitHub's commit verification tool to check the commit signatures. A green verified label should appear next to all of your commits on GitHub.
  • I have updated the relevant documentation, if applicable.
  • I have tested my changes and verified they work as expected.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label May 19, 2025
@behnazh-w behnazh-w changed the title feat(heuristics): add Whitespace Check to detect excessive spacing and invisible characters. feat(malware-check): add whitespace Check to detect excessive spacing and invisible characters. May 20, 2025
@behnazh-w behnazh-w changed the title feat(malware-check): add whitespace Check to detect excessive spacing and invisible characters. feat(malware-check): add whitespace check to detect excessive spacing and invisible characters. May 20, 2025
@behnazh-w behnazh-w changed the title feat(malware-check): add whitespace check to detect excessive spacing and invisible characters. feat(malware-check): add whitespace check to detect excessive spacing and invisible characters May 20, 2025
@behnazh-w behnazh-w requested a review from art1f1c3R May 26, 2025 05:15
@AmineRaouane
feat(heuristics): add Whitespace Check to detect excessive spacing an…
2815028 
…d invisible characters

Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@AmineRaouane
chore: add config variable to defaults.ini and minor cleanup
6978bd7 
Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@AmineRaouane AmineRaouane force-pushed the white-spaces-heuristic branch from db0e35c to 6978bd7 Compare May 26, 2025 10:58
art1f1c3R
art1f1c3R reviewed Jun 2, 2025
View reviewed changes
art1f1c3R
art1f1c3R reviewed Jun 2, 2025
View reviewed changes
benmss and others added 12 commits June 17, 2025 15:45
@benmss @AmineRaouane
chore: store provenance asset info (oracle#975)
76bc6dc 
Signed-off-by: Ben Selwyn-Smith <benselwynsmith@googlemail.com>
@AmineRaouane
feat(security): add package name typosquatting detection (oracle#1059)
8dad2ae 
Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@art1f1c3R @AmineRaouane
refactor: improve experimental source code pattern analysis of pypi p…
a470123 
…ackages (oracle#965)

Include support for using Semgrep for analysis of source code to detect malicious code patterns, specified using Semgrep's YAML files.

Signed-off-by: Carl Flottmann <carl.flottmann@oracle.com>
@benmss @AmineRaouane
feat: add GitHub attestation discovery (oracle#1020)
d6134ad 
This PR allows Macaron to discover GitHub attestation. To retrieve these attestations, the SHA256 hash of the related artefact is required. Hashes are computed from local artefact files if available, or from downloaded ones otherwise.

Signed-off-by: Ben Selwyn-Smith <benselwynsmith@googlemail.com>
@behnazh-w @AmineRaouane
build: replace shared library with standalone cuevalidator binary (or…
279a001 
…acle#1096)

This PR replaces the Go shared library previously used via C-bindings in Python with a standalone binary for the cuevalidator component. The binary can now be invoked as a subprocess, simplifying integration and improving portability.

Signed-off-by: behnazh-w <behnaz.hassanshahi@oracle.com>
@art1f1c3R @AmineRaouane
fix: include inspector links with information on if they are reachabl…
540cf2d 
…e. (oracle#1102)

The detail info containing inspector links now contains links as keys regardless of whether they are reachable, and includes a boolean value for reachability.

Signed-off-by: Carl Flottmann <carl.flottmann@oracle.com>
@art1f1c3R @AmineRaouane
docs: include source code analysis subsection in malicious package tu…
d943e7b 
…torial (oracle#1101)

Signed-off-by: Carl Flottmann <carl.flottmann@oracle.com>
@AmineRaouane
fix(pypi): update get_maintainers_of_package to avoid request blocking (
f28a8cd 
oracle#1097)

Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@AmineRaouane
feat(heuristics): add Whitespace Check to detect excessive spacing an…
c2b016a 
…d invisible characters

Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@AmineRaouane
chore: add config variable to defaults.ini and minor cleanup
2cbfd7b 
Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@AmineRaouane
Merge upstream/main into white-spaces-heuristic
d2b5af5 
Signed-off-by: Amine <amine.raouane@enim.ac.ma>
@AmineRaouane
feat(heuristics): add Whitespace Check to Semgrep
697106a 
Signed-off-by: Amine <amine.raouane@enim.ac.ma>
art1f1c3R
art1f1c3R reviewed Jun 19, 2025
View reviewed changes
art1f1c3R
art1f1c3R reviewed Jun 19, 2025
View reviewed changes
@behnazh-w behnazh-w changed the title feat(malware-check): add whitespace check to detect excessive spacing and invisible characters feat(heuristics): add whitespace check to detect excessive spacing and invisible characters for malware check Aug 8, 2025
@art1f1c3R
Copy link
Member

art1f1c3R commented Aug 8, 2025 •
edited
Loading

The CI test seems to be failing due to detecting excessive whitespace in django@5.0.6 for django/utils/html.py:149 and tests/gis_tests/distapp/tests.py:384.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@art1f1c3R art1f1c3R art1f1c3R approved these changes

@behnazh-w behnazh-w Awaiting requested review from behnazh-w behnazh-w is a code owner

@tromai tromai Awaiting requested review from tromai tromai is a code owner

Assignees
No one assigned
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@AmineRaouane @art1f1c3R @benmss @behnazh-w