Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,15 +239,15 @@ The result are presented in a table that contains the following information.

To have detailed information about each test category run, [see here.](modules/securibench/src/docs-metrics/jsvfa/jsvfa-metrics-v0.3.0.md) (*computed in June 2023.*)

#### New metrics (v0.6.1)
#### New metrics (v0.6.2)

> failed: 47, passed: 75 of 122 tests - (61.48%)
> failed: 46, passed: 76 of 122 tests - (62.3%)

| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F-score | Pass Rate |
|:--------------:|:-----:|:--------:|:------:|:--:|:--:|:---|:---------:|:------:|:-------:|:---------:|
| Aliasing | 10 | 12 | 2/6 | 8 | 1 | 3 | 0.89 | 0.73 | 0.80 | 33.33% |
| Arrays | 11 | 9 | 5/10 | 5 | 4 | 2 | 0.56 | 0.71 | 0.63 | 50% |
| Basic | 57 | 60 | 38/42 | 55 | 1 | 4 | 0.98 | 0.93 | 0.95 | 90.48% |
| Basic | 60 | 60 | 38/42 | 55 | 2 | 2 | 0.96 | 0.96 | 0.96 | 90.48% |
| Collections | 8 | 15 | 5/14 | 5 | 1 | 8 | 0.83 | 0.38 | 0.52 | 35.71% |
| Datastructures | 5 | 5 | 4/6 | 4 | 1 | 1 | 0.80 | 0.80 | 0.80 | 66.67% |
| Factories | 4 | 3 | 2/3 | 2 | 1 | 0 | 0.67 | 1.00 | 0.80 | 66.67% |
Expand All @@ -257,9 +257,9 @@ To have detailed information about each test category run, [see here.](modules/s
| Pred | 8 | 5 | 6/9 | 5 | 3 | 0 | 0.63 | 1.00 | 0.77 | 66.67% |
| Reflection | 0 | 4 | 0/4 | 0 | 0 | 4 | 0.00 | 0.00 | 0.00 | 0% |
| Sanitizers | 2 | 6 | 2/6 | 1 | 0 | 4 | 1.00 | 0.20 | 0.33 | 33.33% |
| TOTAL | 120 | 141 | 75/122 | 95 | 14 | 35 | 0.87 | 0.73 | 0.79 | 61.48% |
| TOTAL | 124 | 141 | 76/122 | 96 | 15 | 32 | 0.86 | 0.75 | 0.80 | 62.3% |

To have detailed information about each test category run, [see here.](modules/securibench/src/docs-metrics/jsvfa/jsvfa-metrics-v0.6.1.md) (*computed in November 2025.*)
To have detailed information about each test category run, [see here.](modules/securibench/src/docs-metrics/jsvfa/jsvfa-metrics-v0.6.2.md) (*computed in December 2025.*)

#### Running Securibench Tests

Expand All @@ -279,9 +279,10 @@ sbt "testOnly br.unb.cic.securibench.deprecated.SecuribenchTestSuite"
From the 47 tests, we have categorized nine (9) issues.

[i] **Wrong counting**: Some tests from the Securibench benchmark are incorrectly labeled, leading to wrong expected values.
We have mapped four cases: `(8.51%)`
We have mapped four cases: `(10.64%)`
- Aliasing2
- Aliasing4
- Basic31
- Inter4
- Inter5

Expand All @@ -295,21 +296,19 @@ We have mapped six cases: `(12.77%)`
- Arrays10

[iii] Support Class Missing: Some tests use methods from securibench that are not mocked.
We have mapped seven cases: `(14.89%)`
- Basic31
- Basic36
- Basic38
We have mapped seven cases: `(6.38%)`
- Session1
- Session2
- Session3
- Sanitizers5

[iv] Missing Context: The logic for handling context is not entirely flawless, resulting in certain edge cases that lead to bugs such as:
[a] Nested structures as HashMap, LinkedList, and others,
[b] Loop statement as "for" or "while",
[c] Parameters passed in the constructor.
We have mapped 16 cases: `(34.04%)`
We have mapped 16 cases: `(38.3%)`
- Aliasing5
- Basic36
- Basic38
- Basic42
- Collections3
- Collections5
Expand Down Expand Up @@ -347,9 +346,10 @@ We have mapped three cases: `(6.38%)`
- Pred7

[viii] Sanitizer method: The current implementation fails to deal with the intermediary method utilized by the sanitizer.
We have mapped three cases: `(6.38%)`
We have mapped three cases: `(8.51%)`
- Sanitizers2
- Sanitizers4
- Sanitizers5
- Sanitizers6

[ix] Flaky
Expand Down Expand Up @@ -405,7 +405,7 @@ To have detailed information about each group of tests run, [see here.](modules/
| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F-score | Pass Rate |
|:------------:|:-----:|:--------:|:------:|:--:|:--:|:---|:---------:|:------:|:-------:|----------:|
| JSVFA v0.3.0 | 102 | 139 | 63/122 | 80 | 11 | 50 | 0.88 | 0.62 | 0.72 | 51.64% |
| JSVFA v0.6.1 | 120 | 141 | 75/122 | 95 | 14 | 35 | 0.87 | 0.73 | 0.79 | 61.48% |
| JSVFA v0.6.2 | 124 | 141 | 76/122 | 96 | 15 | 32 | 0.86 | 0.75 | 0.80 | 62.3% |
| Flowdroid | 98 | 126 | 67/103 | 77 | 9 | 37 | 0.90 | 0.68 | 0.77 | 65.05% |
| Joana | 123 | 138 | 85/122 | 86 | 19 | 34 | 0.82 | 0.72 | 0.77 | 69.67% |

Expand Down
36 changes: 26 additions & 10 deletions modules/core/src/main/scala/br/unb/cic/soot/svfa/jimple/JSVFA.scala
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,14 @@ abstract class JSVFA
val methodRules = languageParser.evaluate(code())

/*
* Create an edge from the definition of the local argument
* Create an edge from the definition of the local argument
* to the definitions of the base object of a method call. In
* more details, we should use this rule to address a situation
* like:
*
* - virtualinvoke r3.<java.lang.StringBuffer: java.lang.StringBuffer append(java.lang.String)>(r1);
*
* Where we wanto create an edge from the definitions of r1 to
* Where we want to create an edge from the definitions of r1 to
* the definitions of r3.
*/
trait CopyFromMethodArgumentToBaseObject extends RuleAction {
Expand Down Expand Up @@ -130,10 +130,14 @@ abstract class JSVFA
* In more details, we should use this rule to address
* a situation like:
*
* - $r6 = virtualinvoke r3.<java.lang.StringBuffer: java.lang.String toString()>();
* [i] $r6 = virtualinvoke r3.<java.lang.StringBuffer: java.lang.String toString()>();
* [ii] virtualinvoke r3.<java.lang.StringBuffer: java.lang.String toString()>();
*
* Where we want to create an edge from the definitions of r3 to
* this statement.
* the current statement.
*
* CONDITIONS:
* - For [i] case, the left operation ($r6) must be a local variable
*/
trait CopyFromMethodCallToLocal extends RuleAction {
def apply(
Expand All @@ -142,10 +146,18 @@ abstract class JSVFA
localDefs: SimpleLocalDefs
) = {
val expr = invokeStmt.getInvokeExpr
if (hasBaseObject(expr) && invokeStmt.isInstanceOf[jimple.AssignStmt]) {
val base = getBaseObject(expr)
var isLocalLeftOpFromAssignStmt = true

if (invokeStmt.isInstanceOf[jimple.AssignStmt]) {
val local = invokeStmt.asInstanceOf[jimple.AssignStmt].getLeftOp
if (base.isInstanceOf[Local] && local.isInstanceOf[Local]) {
if (! local.isInstanceOf[Local]) {
isLocalLeftOpFromAssignStmt = false
}
}

if (hasBaseObject(expr) && isLocalLeftOpFromAssignStmt) {
val base = getBaseObject(expr)
if (base.isInstanceOf[Local]) {
val localBase = base.asInstanceOf[Local]
localDefs
.getDefsOfAt(localBase, invokeStmt)
Expand All @@ -162,7 +174,11 @@ abstract class JSVFA
/* Create an edge from the definitions of a local argument
* to the assignment statement. In more details, we should use this rule to address
* a situation like:
* $r12 = virtualinvoke $r11.<java.lang.StringBuilder: java.lang.StringBuilder append(java.lang.String)>(r6);
*
* [i] $r12 = virtualinvoke $r11.<java.lang.StringBuilder: java.lang.StringBuilder append(java.lang.String)>(r6);
* [ii] virtualinvoke $r11.<java.lang.StringBuilder: java.lang.StringBuilder append(java.lang.String)>(r6);
*
* Where we want to create an edge from the definitions of r6 to the current statement.
*/
trait CopyFromMethodArgumentToLocal extends RuleAction {
def from: Int
Expand All @@ -173,9 +189,9 @@ abstract class JSVFA
localDefs: SimpleLocalDefs
) = {
val srcArg = invokeStmt.getInvokeExpr.getArg(from)
if (invokeStmt.isInstanceOf[JAssignStmt] && srcArg.isInstanceOf[Local]) {
if (srcArg.isInstanceOf[Local]) {
val local = srcArg.asInstanceOf[Local]
val targetStmt = invokeStmt.asInstanceOf[jimple.AssignStmt]
val targetStmt = invokeStmt // current statement
localDefs
.getDefsOfAt(local, targetStmt)
.forEach(sourceStmt => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,31 @@ trait DSL {
if NamedMethodRule(className: "java.lang.StringBuffer", methodName: "toString")
then CopyFromMethodCallToLocal()

rule cookieMethods =
if NamedMethodRule(className: "javax.servlet.http.Cookie", methodName: "getName")
then CopyFromMethodCallToLocal()

rule cookieMethods =
if NamedMethodRule(className: "javax.servlet.http.Cookie", methodName: "getValue")
then CopyFromMethodCallToLocal()

rule cookieMethods =
if NamedMethodRule(className: "javax.servlet.http.Cookie", methodName: "getComment")
then CopyFromMethodCallToLocal()

rule sessionMethods =
if NamedMethodRule(className: "javax.servlet.http.HttpSession", methodName: "setAttribute")
then [
CopyFromMethodArgumentToLocal(from: 1),
CopyFromMethodCallToLocal()
]

rule sessionMethods =
if NamedMethodRule(className: "javax.servlet.http.HttpSession", methodName: "getAttribute")
then [
CopyFromMethodCallToLocal()
]

rule skipNativeMethods = if NativeRule() then DoNothing()

rule skipMethodsWithoutActiveBody =
Expand Down
Loading