Problems with sample programs in the Nifty documentation

[urakubo]

First of all, thank you for your efforts to develop nice APIs. To understand lifted multicut, I am now trying to execute two example programs in the Nifty documentation:
‘plot_agglomerative_clustering.py / Agglomerative Clustering on RAG
‘plot_isbi_2012_multicut_2D_simple.py / ISBI 2012 Simple 2D Multicut Pipeline'
http://derthorsten.github.io/nifty/docs/python/html/index.html

I first found that nifty.graph.rag.gridRag should have an additional argument, n_labels, such as:
n_labels = overseg.max()
n_labels = int(n_labels + 1)
rag = nifty.graph.rag.gridRag(overseg, n_labels)

Then, I am stucked at the commands:
nifty.graph.rag.accumulateStandartFeatures
nifty.graph.rag.accumulateMeanAndLength
Their arguments seems to be revised, and I have the errors something like follows:

---

Traceback (most recent call last):
File "plot_isbi_2012_multicut_2D_simple.py", line 183, in
features = computeFeatures(raw=raw, pmap=pmap, rag=rag)
File "plot_isbi_2012_multicut_2D_simple.py", line 121, in computeFeatures
minVal=0.0, maxVal=255.0, numberOfThreads=1)
TypeError: accumulateStandartFeatures(): incompatible function arguments. The following argument types are supported:
1. (rag: nifty.graph.rag._rag.ExplicitLabelsGridRag2D, data: numpy.ndarray[float32], minVal: float, maxVal: float, blockShape: List[int[2]]=[100, 100], numberOfThreads: int=-1) -> Tuple[numpy.ndarray[float32], numpy.ndarray[float32]]
2. (rag: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, data: numpy.ndarray[float32], minVal: float, maxVal: float, blockShape: List[int[3]]=[100, 100, 100], numberOfThreads: int=-1) -> Tuple[numpy.ndarray[float32], numpy.ndarray[float32]]
3. (rag: nifty.graph.rag._rag.GridRag3DHdf532, data: nifty::hdf5::Hdf5Array, minVal: float, maxVal: float, blockShape: List[int[3]]=[100, 100, 100], numberOfThreads: int=-1) -> Tuple[numpy.ndarray[uint8], numpy.ndarray[uint8]]

---

So far, I cannot solve this problem. Could someone kindly give me suggestion or workable/updated sample programs? The nifty module was downloaded from conda-forge repo (v1.0.5, py37hef89d9c_1001).

Sincerely,
Hidetoshi Urakubo

[constantinpape]

I think the issue is that accumulateStandartFeatures was not compiled for uint8 input data.
Can you try casting raw to float32?

[urakubo]

Thanks you for prompt reply. Indeed, they both worked by just adding “.astype(‘float32’)” !
Then, in the case of plot_isbi_2012_multicut_2D_simple.py, I faced the following error:

Traceback (most recent call last):
File "plot_isbi_2012_multicut_2D_simple.py", line 201, in
gt = nifty.segmentation.seededWatersheds(growMap, seeds=seeds)
File "/home/urakubo/miniconda3/envs/py37/lib/python3.7/site-packages/nifty/segmentation/init.py", line 78, in seededWatersheds
seg = seg.reshape(shape)
ValueError: cannot reshape array of size 1 into shape (512,512)

This occurred at the following part:
growMap = nifty.filters.gaussianSmoothing(1.0-gtImage, 1.0)
growMap += 0.1*nifty.filters.gaussianSmoothing(1.0-gtImage, 6.0)
gt = nifty.segmentation.seededWatersheds(growMap, seeds=seeds)

The contents of the arguments were:
growMap.shape: (512, 512)
growMap.dtype: float32
seeds.shape: (512, 512)
seeds.dtype: uint64

Also, in the case of plot_agglomerative_clustering.py, I saw the following:

Traceback (most recent call last):
File "plot_agglomerative_clustering.py", line 75, in
seg = nifty.graph.rag.projectScalarNodeDataToPixels(rag, nodeSeg)
TypeError: projectScalarNodeDataToPixels(): incompatible function arguments. The following argument types are supported:
1. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[uint32], numberOfThreads: int=-1) -> numpy.ndarray[uint32]
2. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[uint64], numberOfThreads: int=-1) -> numpy.ndarray[uint64]
3. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[float32], numberOfThreads: int=-1) -> numpy.ndarray[float32]
4. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[float64], numberOfThreads: int=-1) -> numpy.ndarray[float64]
5. (graph: nifty.graph.rag._rag.GridRag3DZ564, nodeData: numpy.ndarray[uint64], pixelData: nifty::nz5::DatasetWrapper, blockShape: List[int[3]], numberOfThreads: int=-1) -> None

Invoked with: <nifty.graph.rag.ExplicitLabelsGridRag2D object at 0x7fe6d2a3cbf0>, array([99, 99, 99, ..., 99, 99, 99], dtype=uint64)

Excuse me for the flood of thoughtless questions, but I believe they help someone else.

Hidetoshi

[constantinpape]

About the first issue:
I am not sure how to fix this one in nifty; I would suggest to use the watershed from another library, for example skimage.
You should be able to just use

gt = skimage.segmentation.watershed(growMap, markers=seeds)

About the second issue:
For some reason the function is not compiled for the 2d rag.
Fortunately there is another nifty function you can use here instead

seg = nifty.tools.take(nodeSeg, overseg)

[urakubo]

Thanks! They almost work. Additionally, I saw the following
at Line 347: result = nifty.graph.rag.projectScalarNodeDataToPixels(rag, arg)
in "plot_isbi_2012_multicut_2D_simple.py."
The object rag "ExplicitLabelsGridRag2D" seems to be different from the required "ExplicitLabelsGridRag3D32."

Traceback (most recent call last):
File "plot_isbi_2012_multicut_2D_simple.py", line 351, in
result = nifty.graph.rag.projectScalarNodeDataToPixels(rag, arg)
TypeError: projectScalarNodeDataToPixels(): incompatible function arguments. The following argument types are supported:
1. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[uint32], numberOfThreads: int=-1) -> numpy.ndarray[uint32]
2. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[uint64], numberOfThreads: int=-1) -> numpy.ndarray[uint64]
3. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[float32], numberOfThreads: int=-1) -> numpy.ndarray[float32]
4. (graph: nifty.graph.rag._rag.ExplicitLabelsGridRag3D32, nodeData: numpy.ndarray[float64], numberOfThreads: int=-1) -> numpy.ndarray[float64]
5. (graph: nifty.graph.rag._rag.GridRag3DZ564, nodeData: numpy.ndarray[uint64], pixelData: nifty::nz5::DatasetWrapper, blockShape: List[int[3]], numberOfThreads: int=-1) -> None

Invoked with: <nifty.graph.rag.ExplicitLabelsGridRag2D object at 0x7fc1ba7d90f0>, array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 2, 11, 12,
13, 4, 4, 16, 17, 18, 24, 20, 21, 27, 23, 24, 25,
27, 27, 27, 29, 30, 29, 42, 35, 34, 35, 25, 27, 48,
...
205, 205, 210, 211, 206, 213, 215, 215, 216, 187, 187, 215, 220,
107, 222, 223, 224, 225, 107, 227, 187, 222, 230], dtype=uint64)

I also saw the following at Line 77 in "plot_agglomerative_clustering.py."

print('dir(rag): ', dir(rag))
seg = nifty.tools.take(rag, nodeSeg)

dir(rag): ['EdgeContractionGraph', 'LiftedMulticutObjective', 'MincutObjective', 'MulticutObjective', 'class', 'delattr', 'dir', 'doc', 'eq', 'format', 'ge', 'getattribute', 'gt', 'hash', 'init', 'init_subclass', 'le', 'lt', 'module', 'ne', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'bfsEdges', 'deserialize', 'edgeIdUpperBound', 'edges', 'edgesFromNodeList', 'extractSubgraphFromNodes', 'findEdge', 'findEdges', 'insertEdge', 'insertEdges', 'nodeAdjacency', 'nodeIdUpperBound', 'nodes', 'numberOfEdges', 'numberOfNodes', 'serialize', 'shape', 'shrinkToFit', 'u', 'uv', 'uvIds', 'v']

Traceback (most recent call last):
File "plot_agglomerative_clustering.py", line 77, in
seg = nifty.tools.take(rag, nodeSeg)
File "/home/urakubo/miniconda3/envs/py37/lib/python3.7/site-packages/nifty/tools/init.py", line 62, in take
return _tools._take(relabeling, toRelabelFlat).reshape(shape)
TypeError: _take(): incompatible function arguments. The following argument types are supported:
1. (relabeling: numpy.ndarray[uint32], toRelabel: numpy.ndarray[uint32]) -> numpy.ndarray[uint32]
2. (relabeling: numpy.ndarray[uint64], toRelabel: numpy.ndarray[uint64]) -> numpy.ndarray[uint64]
3. (relabeling: numpy.ndarray[int32], toRelabel: numpy.ndarray[int32]) -> numpy.ndarray[int32]
4. (relabeling: numpy.ndarray[int64], toRelabel: numpy.ndarray[int64]) -> numpy.ndarray[int64]

Invoked with: <nifty.graph.rag.ExplicitLabelsGridRag2D object at 0x7f0c6025e4f0>, array([99, 99, 99, ..., 99, 99, 99], dtype=uint64)

The object "rag" ExplicitLabelsGridRag2D seems to be converted into numpy.ndarray[uint32]. I guess, some attribute function provides this functionality ...

Probably they are last questions. Thank you for your patience. I will upload the revised programs if they work!

Sincerely,
Hidetoshi

[constantinpape]

Thanks! They almost work. Additionally, I saw the following
at Line 347: result = nifty.graph.rag.projectScalarNodeDataToPixels(rag, arg)
in "plot_isbi_2012_multicut_2D_simple.py."
The object rag "ExplicitLabelsGridRag2D" seems to be different from the required "ExplicitLabelsGridRag3D32."

Yes, that is the same issue that you brought up in the question before. The problem here is that projectScalarNodeDataToPixels was not compiled for the 2d version of the rag. As I have suggested before, you can use nifty.tools.take instead.
However, for this, you need to pass the segmentation as argument, not the rag.

Traceback (most recent call last):
File "plot_agglomerative_clustering.py", line 77, in
seg = nifty.tools.take(rag, nodeSeg)

This is the wrong function call. It needs to be take(nodeSeg, segmentation), where segmentation is the segmentation array that was passed to gridRag earlier.

[urakubo]

Thank you very much! Finally, they worked (attached blow) . I am then checking variables such as nodeSeg and segmentation. The variable "segmentation" is pixel-wised, over-segmented labels, and the object variable "nodeSeg" should contain their connections. But the "nodeSeg" seems to be just one dimensional python list of integers. What do those values stand for?

Anyway, thank you for your kind help and guidance!

plot_agglomerative_clustering.py.txt
plot_isbi_2012_multicut_2D_simple.py.txt

[constantinpape]

I am then checking variables such as nodeSeg and segmentation. The variable "segmentation" is pixel-wised, over-segmented labels, and the object variable "nodeSeg" should contain their connections.

There 2 different segmentation images (they might be called a bit different in the different examples):

• over_segmenation: contains an over-segmentation of the image
• segmentation: contains the segmentation that results from applying multicut or clustering

nodeSeg is a 1d array that contains the assignment of ids in over_segmentation to ids in segmentation.

[urakubo]

Thank you very much for your help! Anyway, I have succeeded in the labeling of a probability map of cellular boundaries (in a EM image) that was produced by a CNN. Now I am seriously considering using such outputs for our study. How should I cite this API?
　I would ask a few additional questions, but this public place may not be appropriate for this. Could you tell me your email address? My personal email address is urakubo@nips.ac.jp.

Sincerely yours,
Hidetoshi Urakubo
https://researchmap.jp/urakubo?lang=en