strongly connected components are not strongly connected?

[arekfu]

I generate strongly connected components of random graphs and subsequently check if they are strongly connected. The result is systematically false. Am I missing something?

import LightGraphs: strongly_connected_components, is_strongly_connected,
                    induced_subgraph, erdos_renyi

function scc_ok(graph)
  scc = strongly_connected_components(graph)
  scc_as_subgraphs = map(i -> induced_subgraph(graph, i), scc)
  return all(is_strongly_connected, scc_as_subgraphs)
end

println(all(scc_ok, [erdos_renyi(400, 0.01, is_directed=true) for i in 1:100]))

[sbromberger]

Yup, this definitely smells like a bug, though it's not "systematically" false. I've got one graph that does it.

The graph: https://gist.github.com/sbromberger/af5912dd34190a8a2ed6e0ff9d0c0974

The subgraph that fails the check: https://gist.github.com/sbromberger/21bc112240f0308fc63de8e198fe2715

[CarloLucibello]

Could this be a bug introduced in #332 ?

Can someone try the same test on a previous commit? I don't have time at the moment, but eventually I can get to it in the next days

[sbromberger]

I can, but there are tons of warnings thrown on the new build: see https://gist.github.com/sbromberger/6a81432250b093b09d442a008177ab94. I'll file a new issue.

[sbromberger]

@CarloLucibello looks like you may be off the hook - this problem existed in 0.5.2.

[sbromberger]

sccfail.jgz.zip

Unzip this, then load sccfail.jgz. The big graph is g, the one that fails the scc test is bad.

[arekfu]

Incidentally, I think it would be a good idea to make extensive use of the testing strategy in the example above (asserting invariants on several instances of random graphs) in the package tests. Do you currently do this?

[CarloLucibello]

a smaller graph failing the test:

julia> println(scc_ok(erdos_renyi(20, 0.1, is_directed=true,seed=17)))
false

[CarloLucibello]

Incidentally, I think it would be a good idea to make extensive use of the testing strategy in the example above (asserting invariants on several instances of random graphs) in the package tests. Do you currently do this?

We mostly do tests on handmade small graphs, and we certainly could check for more invariants. Would be nice to see more functions like scc_ok in the testsuite

[CarloLucibello]

smaller

julia> ok=scc_ok(erdos_renyi(11, 0.12, is_directed=true,seed=17))
false

[sbromberger]

Incidentally, I think it would be a good idea to make extensive use of the testing strategy in the example above (asserting invariants on several instances of random graphs) in the package tests. Do you currently do this?

No, because by definition they are not deterministic tests and can yield varying results. See #323 for more discussion.

[arekfu]

They are deterministic if you specify the seeds, aren't they?

[sbromberger]

I'm not sure determinism crosses system bounds.

[arekfu]

Still (last comment about this side issue), testing on random graphs might be nondeterministic and you might run into false negatives now and then, but at the same time it would enormously improve your coverage. If your code works on thousands on random graphs, the probability that it contains a bug becomes negligible.

[sbromberger]

That's certainly true, but the set of functions that we could run these on is limited to those that return boolean (or can be coerced to boolean) since we'd use any / all. Centrality measures, for example, won't be able to be tested this way. Balance this against the time it takes to run tests and it's not a straightforward "let's definitely do this".

That said, I do like the idea as long as we're not imposing too much of a burden on the testing framework. It might be worthwhile to see what functions can benefit from tests of random graphs and then make sure they don't take forever to run.

(...and I hope this isn't really your last comment on this issue, since it's worthy of discussion - please reconsider.)