Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
2f7332b
set version to 0.6.1-SNAPSHOT
Jclavo Oct 20, 2025
01594af
remove time computation (partial)
Jclavo Oct 27, 2025
4cd5b35
metrics in v0.6.1
Jclavo Oct 26, 2025
73fb395
metrics in v0.3.0
Jclavo Oct 27, 2025
5d40053
refactor logic in method reportConflicts
Jclavo Nov 4, 2025
ec8e43f
enable test Aliasing6
Jclavo Nov 4, 2025
4a61c6e
add missing tests for Aliasing
Jclavo Nov 5, 2025
88784ed
merge all tests in a single file
Jclavo Nov 5, 2025
66ac38f
enable Sanitizers1
Jclavo Nov 5, 2025
46b61ee
enable StrongUpdates4 and Inter11
Jclavo Nov 5, 2025
06a8370
set StrongUpdates4 as FLAKY
Jclavo Nov 5, 2025
a13d6df
set Inter11 as FLAKY
Jclavo Nov 5, 2025
35e4ab9
Merge pull request #14 from PAMunb/task/enable_tests
Jclavo Nov 5, 2025
f879ad7
Merge branch 'develop' of https://github.com/PAMunb/svfa into task/al…
Jclavo Nov 5, 2025
924d5e1
fix tests in AndroidTaintBenchSuiteExperiment2Test
Jclavo Nov 5, 2025
42394e4
update metrics for aliasing
Jclavo Nov 5, 2025
cc8e745
Add comments for aliasing tests
Jclavo Nov 5, 2025
e414323
Merge pull request #15 from PAMunb/task/aliasing_6
Jclavo Nov 6, 2025
3bb8367
add comments about logic
Jclavo Nov 6, 2025
e27b701
to-do for method isValidContext
Jclavo Nov 6, 2025
f73a7bb
remove extra aliasing tests
Jclavo Nov 7, 2025
184c264
recompute metrics
Jclavo Nov 7, 2025
2f264e6
Merge pull request #19 from PAMunb/issue/remove_extra_aliasing_tests
Jclavo Nov 7, 2025
2f6b3b5
update total for metrics v0.3.0
Jclavo Nov 21, 2025
f39f125
update metrics for Basic category
Jclavo Nov 21, 2025
24ac939
update metrics for Inter category
Jclavo Nov 21, 2025
5c2cac7
add info to readme
Jclavo Nov 21, 2025
0c25e64
update joana metrics
Jclavo Nov 22, 2025
9bc25d6
update securibench section information
Jclavo Nov 22, 2025
9d7d7cd
update tainbench section information
Jclavo Nov 22, 2025
7a1b2e9
update flowdroid metrics
Jclavo Nov 22, 2025
177a09c
update securibench section information
Jclavo Nov 22, 2025
66644a6
update readme
Jclavo Nov 24, 2025
39de5cb
wip: info about issues
Jclavo Nov 24, 2025
409e562
map error in collections category
Jclavo Nov 24, 2025
8e0a3ae
Datastructures
Jclavo Nov 24, 2025
40375a9
map error in factories category
Jclavo Nov 24, 2025
66ee8d8
map error in inter category
Jclavo Nov 24, 2025
0e080b0
map error in session category
Jclavo Nov 24, 2025
ec5e9b0
map error in strong_updates category
Jclavo Nov 24, 2025
4cd3d53
map error in pred category
Jclavo Nov 24, 2025
23f467c
map error in reflection category
Jclavo Nov 24, 2025
0451ca3
map error in sanitizers category
Jclavo Nov 24, 2025
99f2fc6
typo
Jclavo Nov 24, 2025
60077b2
Common issues
Jclavo Nov 24, 2025
1122faf
typo
Jclavo Nov 24, 2025
4f6adf8
add issue ix
Jclavo Nov 24, 2025
ebf5edd
Add section about common issues
Jclavo Nov 24, 2025
ae0a4b0
update error classification
Jclavo Nov 26, 2025
0c970ee
v0.6.1 updates
Jclavo Dec 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ authors:
given-names: "Eric"
orcid: "https://orcid.org/0000-0003-3470-3647"
title: "SVFA-Scala: an implementation of SVFA for Java"
version: 0.6.0
version: 0.6.1-SNAPSHOT
date-released: 2025-09-06
url: "https://github.com/PAMunb/svfa"
365 changes: 220 additions & 145 deletions README.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

ThisBuild / scalaVersion := "2.12.20"
ThisBuild / organization := "br.unb.cic"
ThisBuild / version := "0.6.0"
ThisBuild / version := "0.6.1-SNAPSHOT"

// Global settings
ThisBuild / publishConfiguration := publishConfiguration.value.withOverwrite(true)
Expand Down
22 changes: 7 additions & 15 deletions modules/core/src/main/scala/br/unb/cic/soot/graph/Graph.scala
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,12 @@ class Graph() {

val csOpenAndClose = csOpen ++ csClose

/**
TO-DO: Implement a better way to calculate the right csOpen and Close
because the right one can lead to a bug in some edges cases.
*/


csOpenAndClose.foreach(open => {
if (open.value.context.nonEmpty) {
cs = cs + open.value.context.head
Expand Down Expand Up @@ -568,21 +574,7 @@ class Graph() {
f(stmt)
)

def reportConflicts(
useUniquePaths: Boolean = false
): scala.collection.Set[String] = {
val conflicts = findConflictingPaths()

if (useUniquePaths) {
var conflictsByUniquePaths: Set[String] = Set.empty[String]
conflicts.foreach(path => {
conflictsByUniquePaths += s"source: ${path.head.show()} - sink: ${path.last.show()}"
})
conflictsByUniquePaths
} else {
conflicts.map(p => p.toString)
}
}
def reportConflicts(): scala.collection.Set[List[GraphNode]] = findConflictingPaths()

def findConflictingPaths(): scala.collection.Set[List[GraphNode]] = {
if (fullGraph) {
Expand Down
6 changes: 2 additions & 4 deletions modules/core/src/main/scala/br/unb/cic/soot/svfa/SVFA.scala
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,8 @@ abstract class SVFA extends SootConfiguration {
svg.toDotModel()
}

def reportConflictsSVG(
useUniquePaths: Boolean = false
): collection.Set[String] = {
svg.reportConflicts(useUniquePaths)
def reportConflictsSVG(): scala.collection.Set[List[GraphNode]] = {
svg.reportConflicts()
}

def executionTime(): Double = {
Expand Down
61 changes: 41 additions & 20 deletions modules/core/src/main/scala/br/unb/cic/soot/svfa/jimple/JSVFA.scala
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,7 @@ abstract class JSVFA
case (p: Local, q: ArrayRef) => // p = q[i]
loadArrayRule(assignStmt.stmt, q, method, defs)
case (p: Local, q: InvokeExpr) =>
invokeRule(assignStmt, q, method, defs) // call a method
invokeRule(assignStmt, q, method, defs) // p = myObject.method() : call a method and assign its return value to a local variable
case (p: Local, q: Local) => copyRule(assignStmt.stmt, q, method, defs)
case (p: Local, _) =>
copyRuleInvolvingExpressions(assignStmt.stmt, method, defs)
Expand Down Expand Up @@ -367,10 +367,18 @@ abstract class JSVFA
})
}

/** Handles invocation rules for a call statement by traversing the call
* graph. This method avoids infinite recursion and limits the traversal
* depth for performance.
*/
/**
* Handles invocation rules for a call statement by traversing the call
* graph. This method avoids infinite recursion and limits the traversal
* depth for performance.
*
* i.e:
*
* myObject.method()
* myObject.method(q)
* this.method()
* this.method(q)
*/
private def invokeRule(
callStmt: Statement,
exp: InvokeExpr,
Expand Down Expand Up @@ -1039,22 +1047,36 @@ abstract class JSVFA
case v => v
}

/** CASE 1: UPDATE EDGE(S) "FROM" each stmt where the variable, passed as an
* argument, is defined. "TO" stmt where the method is called (call-site
* stmt).
*
* CASE 2: ???
*
* CASE 2: ???
*/
/** CASE #1: UPDATE EDGE(S) "FROM" each stmt where the variable, passed as an
* argument, is defined. "TO" stmt where the method is called (call-site
* stmt). i.e: [s1 -> s2]
*
* -------------------
* s1: p = ...
* s2: myObj.method(p)
* -------------------
*
* CASE #2:
* TO-DO
*
* CASE #3: UPDATE EDGE(S) "FROM" from definition of base object "TO" where it
* calls any of its methods. The expression must be type (invoke).
* i.e: [s1 -> s2]
*
* -------------------
* s1: myObj = new Object()
* s2: myObj.method()
* -------------------
*
*/
private def defsToCallOfSinkMethod(
stmt: Statement,
exp: InvokeExpr,
caller: SootMethod,
defs: SimpleLocalDefs
) = {

// CASE 1
// CASE #1
exp.getArgs
.stream()
.filter(a => a.isInstanceOf[Local])
Expand All @@ -1069,22 +1091,21 @@ abstract class JSVFA
updateGraph(
source,
target
) // update 'edge(s)' FROM "declaration stmt(s) for args" TO "call-site stmt" (current stmt)
)
})

// CASE 2
// CASE #2
if (local.getType.isInstanceOf[ArrayType]) {
val stores = arrayStores.getOrElseUpdate(local, List())
stores.foreach(sourceStmt => {
val source = createNode(caller, sourceStmt)
val target = createNode(caller, targetStmt)
updateGraph(source, target) // add comment
updateGraph(source, target)
})
}
})

// CASE 3
// edges from definition to base object of an invoke expression
// CASE #3
if (isFieldSensitiveAnalysis() && exp.isInstanceOf[InstanceInvokeExpr]) {
if (exp.asInstanceOf[InstanceInvokeExpr].getBase.isInstanceOf[Local]) {
val local =
Expand All @@ -1095,7 +1116,7 @@ abstract class JSVFA
.forEach(sourceStmt => {
val source = createNode(caller, sourceStmt)
val target = createNode(caller, targetStmt)
updateGraph(source, target) // add comment
updateGraph(source, target)
})
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -191,9 +191,9 @@ trait TestResult {
)

val header =
"| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F-score | Execution Time |"
"| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F-score |"
val sep =
"|:--------------:|:-----:|:--------:|:------:|:--:|:--:|:---|:---------:|:------:|:-------:|:--------------:|"
"|:--------------:|:-----:|:--------:|:------:|:--:|:--:|:---|:---------:|:------:|:-------:|"
println(header)
println(sep)
var totalFound = 0
Expand Down Expand Up @@ -229,7 +229,7 @@ trait TestResult {
val totalF1 = f1Score()
val totalStatus = s"${totalPassed}/${totalTests}"
println(
f"| TOTAL | ${totalFound}%5d | ${totalExpected}%8d | ${totalStatus}%6s | ${totalTP}%2d | ${totalFP}%2d | ${totalFN}%3d | ${totalPrec}%9.2f | ${totalRec}%6.2f | ${totalF1}%7.2f | ${totalExecutionTime}%9.2f ms |"
f"| TOTAL | ${totalFound}%5d | ${totalExpected}%8d | ${totalStatus}%6s | ${totalTP}%2d | ${totalFP}%2d | ${totalFN}%3d | ${totalPrec}%9.2f | ${totalRec}%6.2f | ${totalF1}%7.2f |"
)
}
}
Expand Down
35 changes: 19 additions & 16 deletions modules/securibench/src/docs-metrics/flowdroid/flowdroid-metrics.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
#### FLOWDROID metrics

### SUMMARY
### SUMMARY (*computed in November 2025.*)

> failed: 36, passed: 67 of 103 tests. `(65.05%)`

| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F1 |
|----------------|-------|----------|--------|----|----|----|-----------|--------|------|
| Aliasing | 11 | 11 | 4/6 | 9 | 1 | 1 | 0.90 | 0.90 | 0.90 |
| Arrays | 14 | 9 | 6/10 | 6 | 5 | 0 | 0.55 | 1.00 | 0.71 |
| Basic | 38 | 61 | 26/42 | 33 | 1 | 24 | 0.97 | 0.58 | 0.73 |
| Collections | 14 | 15 | 11/14 | 12 | 1 | 2 | 0.92 | 0.86 | 0.89 |
| Datastructures | 5 | 5 | 4/6 | 3 | 1 | 1 | 0.75 | 0.75 | 0.75 |
| Factories | 1 | 3 | 1/3 | 1 | 0 | 2 | 1.00 | 0.33 | 0.50 |
| Inter | 15 | 18 | 11/14 | 13 | 0 | 3 | 1.00 | 0.81 | 0.90 |
| Session | 0 | 3 | 0/3 | 0 | 0 | 3 | 0.00 | 0.00 | 0.00 |
| StrongUpdates | 0 | 1 | 4/5 | 0 | 0 | 1 | 0.00 | 0.00 | 0.00 |
| TOTAL | 98 | 126 | 67/103 | 77 | 9 | 37 | 0.90 | 0.68 | 0.77 |
| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F1 | Pass Rate |
|----------------|-------|----------|--------|----|----|----|-----------|--------|------|-----------|
| Aliasing | 11 | 11 | 4/6 | 9 | 1 | 1 | 0.90 | 0.90 | 0.90 | 66.67% |
| Arrays | 14 | 9 | 6/10 | 6 | 5 | 0 | 0.55 | 1.00 | 0.71 | 60% |
| Basic | 38 | 61 | 26/42 | 33 | 1 | 24 | 0.97 | 0.58 | 0.73 | 61.90% |
| Collections | 14 | 15 | 11/14 | 12 | 1 | 2 | 0.92 | 0.86 | 0.89 | 78.57% |
| Datastructures | 5 | 5 | 4/6 | 3 | 1 | 1 | 0.75 | 0.75 | 0.75 | 66.67% |
| Factories | 1 | 3 | 1/3 | 1 | 0 | 2 | 1.00 | 0.33 | 0.50 | 33.33% |
| Inter | 15 | 18 | 11/14 | 13 | 0 | 3 | 1.00 | 0.81 | 0.90 | 78.57% |
| Session | 0 | 3 | 0/3 | 0 | 0 | 3 | 0.00 | 0.00 | 0.00 | 0% |
| StrongUpdates | 0 | 1 | 4/5 | 0 | 0 | 1 | 0.00 | 0.00 | 0.00 | 80% |
| Pred | - | - | - | - | - | - | - | - | - | - |
| Reflection | - | - | - | - | - | - | - | - | - | - |
| Sanitizers | - | - | - | - | - | - | - | - | - | - |
| **TOTAL** | 98 | 126 | 67/103 | 77 | 9 | 37 | 0.90 | 0.68 | 0.77 | 65.05% |


According to Flowdroid Paper (https://www.bodden.de/pubs/far+14flowdroid.pdf), they computed the next values.
Expand All @@ -38,7 +41,7 @@ According to Flowdroid Paper (https://www.bodden.de/pubs/far+14flowdroid.pdf), t
| StrongUpdate | 0/0 | 0 |
| **TOTAL** | 117/121 | 9 |

### Details
### DETAILS

- ✅ : PASSED; ❌ : FAIL

Expand All @@ -55,7 +58,7 @@ According to Flowdroid Paper (https://www.bodden.de/pubs/far+14flowdroid.pdf), t
| TOTAL | 11 | 11 | 4/6 | 9 | 1 | 1 | 0.90 | 0.90 | 0.90 |


- **ArraysTest** - failed: 4, passed: 6 of 10 tests. `(60.00%)`
- **ArraysTest** - failed: 4, passed: 6 of 10 tests. `(60%)`

| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F1 |
|----------|-------|----------|--------|----|----|----|-----------|--------|------|
Expand Down Expand Up @@ -197,7 +200,7 @@ According to Flowdroid Paper (https://www.bodden.de/pubs/far+14flowdroid.pdf), t
| TOTAL | 0 | 3 | 0/3 | 0 | 0 | 3 | 0.00 | 0.00 | 0.00 |


- **StrongUpdateTest** - failed: 1, passed: 4 of 5 tests. `(80.00%)`
- **StrongUpdateTest** - failed: 1, passed: 4 of 5 tests. `(80%)`

| Test | Found | Expected | Status | TP | FP | FN | Precision | Recall | F1 |
|----------------|-------|----------|--------|----|----|----|-----------|--------|------|
Expand Down
Loading