Stochastic segfault

[Juul]

Yeah, it's as bad as it sounds.

I think it happens in the clustGating function within the fcs-analysis R program. Difficult to debug since it's not always reproducible, but it's possible that it only happens for small datasets.

The segfault takes down the entire daemon that runs scheduled jobs, so no other jobs get run after such a segfault.

Note: This is actually a bug in the BioConductor flowClust library.

[vivekmutalik]

This is crazy.

There should be something better than Bioconductor.

Vivek

Sent from my cell phone. Please excuse any inevitable typos.

On Nov 28, 2011, at 4:49 PM, Marc Juul Christoffersen <reply+i-2375614-
c6c5fdd0bdd1e890f189b9530a3377c5c1140ace-1214310@reply.github.com>
wrote:

Yeah, it's as bad as it sounds.

I think it happens in the clustGating function within the fcs-
analysis R program. Difficult to debug since it's not always
reproducible, but it's possible that it only happens for small
datasets.

The segfault takes down the entire daemon that runs scheduled jobs,
so no other jobs get run after such a segfault.

Note: This is actually a bug in the BioConductor flowClust library.

---

Reply to this email directly or view it on GitHub:
#7

[Juul]

Yeah. I'm definitely looking for something better.

[vivekmutalik]

Hey Marc,

Lets keep tomorrow's lunch time at Market place for chatting about your
next steps....What do you say ?

Vivek

On Tue, Nov 29, 2011 at 11:25 AM, Marc Juul Christoffersen <
reply@reply.github.com

wrote:

Yeah. I'm definitely looking for something better.

---

Reply to this email directly or view it on GitHub:
#7 (comment)

[Juul]

Example of error:

Now analyzing FCS file: /var/rails/biofab-web/public/flow_cytometer_input_data/BD_GOII_PlateM5-M11_TetR/BD_GOII_M5-M11-TetR-gfp_102611_plate2_w_GC/BD_GOII_M5-M11-TetR-gfp_102611_plate2_w_GC-13.fcs

Events: 12831
Well name: A12
Cleaning...
Recapitulate rectangular gating...
Clustered Gating...
Using the serial version of flowClust

*** caught segfault ***
address (nil), cause 'memory not mapped'

Traceback:
1: .C("flowClust", as.double(t(y)), as.integer(ly), as.integer(py), as.integer(K[i]), w = rep(0, K[i]), mu = rep(0, K[i] * py), precision = rep(0, K[i] * py * py), lambda = as.double(rep(lambda, length.out = (if (trans > 1) K[i] else 1))), nu = as.double(rep(nu, K[i])), z = rep(0, ly * K[i]), u = rep(0, ly * K[i]), as.integer(if (M == 0) label else maxLabel[[M]]), uncertainty = double(ly), as.double(rep(u.cutoff, K[i])), as.double(z.cutoff), flagOutliers = integer(ly), as.integer(B), as.double(tol), as.integer(trans), as.integer(nu.est), logLike = as.double(0), as.integer(control$B.lambda), as.integer(control$B.brent), as.double(control$tol.brent), as.double(control$xLow), as.double(control$xUp), as.double(control$nuLow), as.double(control$nuUp), PACKAGE = "flowClust")
2: doTryCatch(return(expr), name, parentenv, handler)
3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
4: tryCatchList(expr, classes, parentenv, handlers)
5: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L msg <- conditionMessage(e) sm <- strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste(prefix, "\n ", sep = "") } else prefix <- "Error : " msg <- paste(prefix, conditionMessage(e), "\n", sep = "") .Internal(seterrmessage(msg[1L])) if (!silent && identical(getOption("show.error.messages"), TRUE)) { cat(msg, file = stderr()) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))})
6: try(.C("flowClust", as.double(t(y)), as.integer(ly), as.integer(py), as.integer(K[i]), w = rep(0, K[i]), mu = rep(0, K[i] * py), precision = rep(0, K[i] * py * py), lambda = as.double(rep(lambda, length.out = (if (trans > 1) K[i] else 1))), nu = as.double(rep(nu, K[i])), z = rep(0, ly * K[i]), u = rep(0, ly * K[i]), as.integer(if (M == 0) label else maxLabel[[M]]), uncertainty = double(ly), as.double(rep(u.cutoff, K[i])), as.double(z.cutoff), flagOutliers = integer(ly), as.integer(B), as.double(tol), as.integer(trans), as.integer(nu.est), logLike = as.double(0), as.integer(control$B.lambda), as.integer(control$B.brent), as.double(control$tol.brent), as.double(control$xLow), as.double(control$xUp), as.double(control$nuLow), as.double(control$nuUp), PACKAGE = "flowClust"))
7: FUN(X[[1L]], ...)
8: lapply(as.list(1:length(K)), .flowClustK, y, expName = expName, varNames = varNames, K = K, B = B, tol = tol, nu = nu, lambda = lambda, nu.est = nu.est, trans = trans, min.count = min.count, max.count = max.count, min = min, max = max, level = level, u.cutoff = u.cutoff, z.cutoff = z.cutoff, randomStart = randomStart, B.init = B.init, tol.init = tol.init, seed = seed, criterion = criterion, control = control, include = include, rm.max, rm.min)
9: flowClust(flowframe, varNames = c("FSC-HLog", "SSC-HLog"), K = 1, level = levels[1])
10: clustGating(flowframe, fluo.channel = fluo.channel, scale.gating = scale.gating, scale.analysis = scale.analysis, out.path.plot = out.path.plot, levels = clust.levels, min.cells)
11: run(cur.out.dir, out.plot.path, fcs.paths[i], fluo.channel = fluo.channel, well.channels = well.channels, clean = clean, cluster = cluster, init.gate = init.gate, output.filename = out.fcs.c1, output.filename.cluster2 = out.fcs.c2, clust.levels = clust.levels, scale.gating = scale.gating, scale.analysis = scale.analysis, verbose = verbose)
12: pAGbyeZxOMh3Py1(...)
13: withCallingHandlers(pAGbyeZxOMh3Py1(...), error = function(e) { dump.frames() n <- length(last.dump) calls <- names(last.dump) msgs = paste(" ", 1L:n, ": ", calls, sep = "") msg = "" for (i in 1:length(msgs)) { msg = paste(msg, msgs[i], sep = "\n") } very_random_string = "9pAGbyeZxOMh3Py1tO5YrsrYj7pHyWP5bNRrI5p6Z0MKAPVAoH3pjpUFvy0ewG98" stop(paste(conditionMessage(e), very_random_string, msg, sep = ""))})
14: function (pAGbyeZxOMh3Py1, ...) { withCallingHandlers(pAGbyeZxOMh3Py1(...), error = function(e) { dump.frames() n <- length(last.dump) calls <- names(last.dump) msgs = paste(" ", 1L:n, ": ", calls, sep = "") msg = "" for (i in 1:length(msgs)) { msg = paste(msg, msgs[i], sep = "\n") } very_random_string = "9pAGbyeZxOMh3Py1tO5YrsrYj7pHyWP5bNRrI5p6Z0MKAPVAoH3pjpUFvy0ewG98" stop(paste(conditionMessage(e), very_random_string, msg, sep = "")) })}(function (out.dir, fcs.paths, fluo.channel = "", well.channels = list(), clean = TRUE, cluster = TRUE, init.gate = "ellipse", clust.levels = c(0.95, 0.95), scale.gating = "Lin", scale.analysis = "Log", min.cells = 100, verbose = FALSE) { data.set = list() for (i in 1:length(fcs.paths)) { cur.out.dir = file.path(out.dir, paste("fcs", i, sep = "_")) dir.create(cur.out.dir, showWarnings = FALSE, recursive = TRUE) out.plot.path = file.path(cur.out.dir, "plot.svg") out.fcs.c1 = "c1.fcs" out.fcs.c2 = "c2.fcs" data = run(cur.out.dir, out.plot.path, fcs.paths[i], fluo.channel = fluo.channel, well.channels = well.channels, clean = clean, cluster = cluster, init.gate = init.gate, output.filename = out.fcs.c1, output.filename.cluster2 = out.fcs.c2, clust.levels = clust.levels, scale.gating = scale.gating, scale.analysis = scale.analysis, verbose = verbose) if (is.null(data)) { unlink(cur.out.dir) next } if (!is.null(data["error"])) { unlink(cur.out.dir)