Java SDK requre additional additional update due to 'Path invalid' error

[b3y0urs3lf]

Tested locally using java SDK commit 0667d48 and GO aggregator commit 11a5d3a879978df9470b0d1ed54276c3f6d255a3.

Steps to reproduce:

1. Setup GO aggregator either in single node mode or in HA mode, but not in sharding mode
2. Prepare few commitments i.e. mint commitments from TokenUtils.mintToken
3. Send them to aggregators
   Actual result: inclusion body from time to time get PATH INVALID error, which also was mentioned initially in Add "parent mode" to the aggregator aggregator-go#59 (comment) example:

Request body: {"id":"07fb8230-5439-4a93-b34d-ae8d2081eb16","method":"get_inclusion_proof","params":{"requestId":"000045e44a1ca813c623647caf86ddf55ca5fc86a6323587c30f5a20aeec239b8bb4"},"jsonrpc":"2.0"}
Request body: {"id":"bc96e375-0f6b-495a-baae-3511082f3720","method":"get_inclusion_proof","params":{"requestId":"0000618e4c9e84a7d0f7507bd23322bc4bf65b1615844a36439db2599092103b825b"},"jsonrpc":"2.0"}
[08:10:15.307] Response from http://localhost:3000: 200
[08:10:15.307] Response from http://localhost:3000: 200
Response body: {"jsonrpc":"2.0","result":{"inclusionProof":{"authenticator":null,"merkleTreePath":{"root":"000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4","steps":[{"path":"474287331788400978591636042461498062939101149043977414635986793121620136454275449","data":"000060431a4acdb4d188faca64705cb82376957da3422ed7e876bf41f9f3415d9c27"},{"path":"3","data":"8ec18db93f36d6630acdb12c3a0ada0f6687524ae159d88104413cce51418866"},{"path":"2","data":"946b2bf83a4646b7b65a1e0a18827e611f1b7f5802c43cd4eeba0a81505a6ab5"},{"path":"3","data":"02b60a0b24f202bf84264eebab5a12d01bfe7de10d613ad682f7e558c8f67bcf"},{"path":"3","data":"bc3d3219d13c4e0e0efb89f0205fa950c5f9963257c42f1ac3e642ca3beb51cb"},{"path":"1","data":"787bfb5f10e90d864423c6780aa1e0ba16cb9cf2ac501535b47d4f620881ed09"}]},"transactionHash":null,"unicityCertificate":"d903ef8701d903f08a01192d92005822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c45822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4401a69157642f600f65820d2d3ea603250776a6843a369912d130cb575a2d564df5813d7b1045a3d962b73582003e917968f869e74174271e9359a69e957e79dd5c871a8cc34f3bdc6f10385f982418080d903f683010780d903e98801031a00011166001a69157645582085e1c966ecb54ca7ec589dc83130969418d4252db2f23710956259efba1620915820a465bc08d1d035a19f2ed028443430314cdcdb5357b7bc36e8454f1f4c8aa26fa4783531365569753248416d484734726b59767a6f6668425a357061564e6450676d41704653375659417537457367676250384a70454b5658418f44c71cc4167ca7c484eb4f73ea5d5fb6e52aa052c44b68282a02d5a1ab566254e3f59b4373d393605fe1b268ce15d6383801b0d323015031f538c9524d42c801783531365569753248416d4a7a416166384b7438757961374453714a69783963456b346379474a724a356b754e355154634e5164574b745841fe2cc7619c207979db1a3eebaeb7f2bf62e4bfe35626899b9766e3d9269ae87273b994a24f01ded8adea6a038cc308c0aab3a2cf9c14f9eaaaec39db7daacfd201783531365569753248416d504c6a764b56396834734c74376b6433523837646d3232366239797a50636e453476546e62686f34764741755841a13854bf8be244b4aa8d90df15cffb296252ce46f44d3c9897b8fc83f7c9a1ff62c74ad900cff332c9aff6894c4ac508d5681c3711bc0b5dc23f47f0a93162c900783531365569753248416d5272334b66376d6d3367345252566f5165364d70423456736153684a666a3750796d4a734c67646345394d425841096d89f063525313a4da4f133e79868e7c330aef84c0f0cd34c59b325b13acdc2612a04b1803a879cb2adfd393937e2955860333ae265dd774022ef6c6c4d08001"}},"id":"bc96e375-0f6b-495a-baae-3511082f3720"}

Response body: {"jsonrpc":"2.0","result":{"inclusionProof":{"authenticator":null,"merkleTreePath":{"root":"000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4","steps":[{"path":"0","data":"7fa0289fbcc9f287e475b5526849df077bf528a2f36cccade159965bb7ce7314"},{"path":"8","data":"23b680874e82688db2f5ad27219fd933325ed8456943e82f6c9c8fcd91b9d7ee"},{"path":"3","data":"dff4aa4e66d3cc2fa794fafde69dc3bec7ebdc8921cf415b33d65343a8c4d22d"},{"path":"2","data":"91727047413e541f20cf74b01705ebd6dfa4023db596f66173faaf737d36d20b"},{"path":"2","data":"c8430dbf7da28cc0aa560e0bb49b323b5e2a14459d863611329d49855d13db5b"},{"path":"1","data":"913cb295cf89d8880effcf79e04c9449de7c616699e443c0e0dcd21c518d1ed1"}]},"transactionHash":null,"unicityCertificate":"d903ef8701d903f08a01192d92005822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c45822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4401a69157642f600f65820d2d3ea603250776a6843a369912d130cb575a2d564df5813d7b1045a3d962b73582003e917968f869e74174271e9359a69e957e79dd5c871a8cc34f3bdc6f10385f982418080d903f683010780d903e98801031a00011166001a69157645582085e1c966ecb54ca7ec589dc83130969418d4252db2f23710956259efba1620915820a465bc08d1d035a19f2ed028443430314cdcdb5357b7bc36e8454f1f4c8aa26fa4783531365569753248416d484734726b59767a6f6668425a357061564e6450676d41704653375659417537457367676250384a70454b5658418f44c71cc4167ca7c484eb4f73ea5d5fb6e52aa052c44b68282a02d5a1ab566254e3f59b4373d393605fe1b268ce15d6383801b0d323015031f538c9524d42c801783531365569753248416d4a7a416166384b7438757961374453714a69783963456b346379474a724a356b754e355154634e5164574b745841fe2cc7619c207979db1a3eebaeb7f2bf62e4bfe35626899b9766e3d9269ae87273b994a24f01ded8adea6a038cc308c0aab3a2cf9c14f9eaaaec39db7daacfd201783531365569753248416d504c6a764b56396834734c74376b6433523837646d3232366239797a50636e453476546e62686f34764741755841a13854bf8be244b4aa8d90df15cffb296252ce46f44d3c9897b8fc83f7c9a1ff62c74ad900cff332c9aff6894c4ac508d5681c3711bc0b5dc23f47f0a93162c900783531365569753248416d5272334b66376d6d3367345252566f5165364d70423456736153684a666a3750796d4a734c67646345394d425841096d89f063525313a4da4f133e79868e7c330aef84c0f0cd34c59b325b13acdc2612a04b1803a879cb2adfd393937e2955860333ae265dd774022ef6c6c4d08001"}},"id":"07fb8230-5439-4a93-b34d-ae8d2081eb16"}

Thread 0 failed: java.lang.RuntimeException: Inclusion proof verification failed: PATH_INVALID
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Inclusion proof verification failed: PATH_INVALID
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
	at org.unicitylabs.sdk.utils.TokenUtils.mintToken(TokenUtils.java:99)
	at org.unicitylabs.sdk.utils.TokenUtils.mintToken(TokenUtils.java:33)
	at org.unicitylabs.sdk.common.CommonTestFlow.testTransferFlowSimplified(CommonTestFlow.java:416)
	at org.unicitylabs.sdk.e2e.TokenE2EWorker.executeTransferFlow(TokenE2EWorker.java:19)
	at org.unicitylabs.sdk.e2e.TokenE2EConcurrentTest.lambda$runConcurrentTransfers$0(TokenE2EConcurrentTest.java:45)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.RuntimeException: Inclusion proof verification failed: PATH_INVALID
	at org.unicitylabs.sdk.util.InclusionProofUtils.lambda$checkInclusionProof$1(InclusionProofUtils.java:100)
	at java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
	at org.unicitylabs.sdk.jsonrpc.JsonRpcHttpTransport$1.onResponse(JsonRpcHttpTransport.java:88)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	... 3 more
[08:10:16.366] Sending request to http://localhost:3000
Request body: {"id":"3a0232de-f560-43e6-bd4c-500b348e088f","method":"get_inclusion_proof","params":{"requestId":"0000618e4c9e84a7d0f7507bd23322bc4bf65b1615844a36439db2599092103b825b"},"jsonrpc":"2.0"}
[08:10:16.371] Response from http://localhost:3000: 200
Response body: {"jsonrpc":"2.0","result":{"inclusionProof":{"authenticator":null,"merkleTreePath":{"root":"000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4","steps":[{"path":"474287331788400978591636042461498062939101149043977414635986793121620136454275449","data":"000060431a4acdb4d188faca64705cb82376957da3422ed7e876bf41f9f3415d9c27"},{"path":"3","data":"8ec18db93f36d6630acdb12c3a0ada0f6687524ae159d88104413cce51418866"},{"path":"2","data":"946b2bf83a4646b7b65a1e0a18827e611f1b7f5802c43cd4eeba0a81505a6ab5"},{"path":"3","data":"02b60a0b24f202bf84264eebab5a12d01bfe7de10d613ad682f7e558c8f67bcf"},{"path":"3","data":"bc3d3219d13c4e0e0efb89f0205fa950c5f9963257c42f1ac3e642ca3beb51cb"},{"path":"1","data":"787bfb5f10e90d864423c6780aa1e0ba16cb9cf2ac501535b47d4f620881ed09"}]},"transactionHash":null,"unicityCertificate":"d903ef8701d903f08a01192d92005822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c45822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4401a69157642f600f65820d2d3ea603250776a6843a369912d130cb575a2d564df5813d7b1045a3d962b73582003e917968f869e74174271e9359a69e957e79dd5c871a8cc34f3bdc6f10385f982418080d903f683010780d903e98801031a00011166001a69157645582085e1c966ecb54ca7ec589dc83130969418d4252db2f23710956259efba1620915820a465bc08d1d035a19f2ed028443430314cdcdb5357b7bc36e8454f1f4c8aa26fa4783531365569753248416d484734726b59767a6f6668425a357061564e6450676d41704653375659417537457367676250384a70454b5658418f44c71cc4167ca7c484eb4f73ea5d5fb6e52aa052c44b68282a02d5a1ab566254e3f59b4373d393605fe1b268ce15d6383801b0d323015031f538c9524d42c801783531365569753248416d4a7a416166384b7438757961374453714a69783963456b346379474a724a356b754e355154634e5164574b745841fe2cc7619c207979db1a3eebaeb7f2bf62e4bfe35626899b9766e3d9269ae87273b994a24f01ded8adea6a038cc308c0aab3a2cf9c14f9eaaaec39db7daacfd201783531365569753248416d504c6a764b56396834734c74376b6433523837646d3232366239797a50636e453476546e62686f34764741755841a13854bf8be244b4aa8d90df15cffb296252ce46f44d3c9897b8fc83f7c9a1ff62c74ad900cff332c9aff6894c4ac508d5681c3711bc0b5dc23f47f0a93162c900783531365569753248416d5272334b66376d6d3367345252566f5165364d70423456736153684a666a3750796d4a734c67646345394d425841096d89f063525313a4da4f133e79868e7c330aef84c0f0cd34c59b325b13acdc2612a04b1803a879cb2adfd393937e2955860333ae265dd774022ef6c6c4d08001"}},"id":"3a0232de-f560-43e6-bd4c-500b348e088f"}

[08:10:17.386] Sending request to http://localhost:3000
Request body: {"id":"905e299e-88cd-4f75-980f-309b180ea744","method":"get_inclusion_proof","params":{"requestId":"0000618e4c9e84a7d0f7507bd23322bc4bf65b1615844a36439db2599092103b825b"},"jsonrpc":"2.0"}
[08:10:17.397] Response from http://localhost:3000: 200
Response body: {"jsonrpc":"2.0","result":{"inclusionProof":{"authenticator":null,"merkleTreePath":{"root":"000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4","steps":[{"path":"474287331788400978591636042461498062939101149043977414635986793121620136454275449","data":"000060431a4acdb4d188faca64705cb82376957da3422ed7e876bf41f9f3415d9c27"},{"path":"3","data":"8ec18db93f36d6630acdb12c3a0ada0f6687524ae159d88104413cce51418866"},{"path":"2","data":"946b2bf83a4646b7b65a1e0a18827e611f1b7f5802c43cd4eeba0a81505a6ab5"},{"path":"3","data":"02b60a0b24f202bf84264eebab5a12d01bfe7de10d613ad682f7e558c8f67bcf"},{"path":"3","data":"bc3d3219d13c4e0e0efb89f0205fa950c5f9963257c42f1ac3e642ca3beb51cb"},{"path":"1","data":"787bfb5f10e90d864423c6780aa1e0ba16cb9cf2ac501535b47d4f620881ed09"}]},"transactionHash":null,"unicityCertificate":"d903ef8701d903f08a01192d92005822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c45822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4401a69157642f600f65820d2d3ea603250776a6843a369912d130cb575a2d564df5813d7b1045a3d962b73582003e917968f869e74174271e9359a69e957e79dd5c871a8cc34f3bdc6f10385f982418080d903f683010780d903e98801031a00011166001a69157645582085e1c966ecb54ca7ec589dc83130969418d4252db2f23710956259efba1620915820a465bc08d1d035a19f2ed028443430314cdcdb5357b7bc36e8454f1f4c8aa26fa4783531365569753248416d484734726b59767a6f6668425a357061564e6450676d41704653375659417537457367676250384a70454b5658418f44c71cc4167ca7c484eb4f73ea5d5fb6e52aa052c44b68282a02d5a1ab566254e3f59b4373d393605fe1b268ce15d6383801b0d323015031f538c9524d42c801783531365569753248416d4a7a416166384b7438757961374453714a69783963456b346379474a724a356b754e355154634e5164574b745841fe2cc7619c207979db1a3eebaeb7f2bf62e4bfe35626899b9766e3d9269ae87273b994a24f01ded8adea6a038cc308c0aab3a2cf9c14f9eaaaec39db7daacfd201783531365569753248416d504c6a764b56396834734c74376b6433523837646d3232366239797a50636e453476546e62686f34764741755841a13854bf8be244b4aa8d90df15cffb296252ce46f44d3c9897b8fc83f7c9a1ff62c74ad900cff332c9aff6894c4ac508d5681c3711bc0b5dc23f47f0a93162c900783531365569753248416d5272334b66376d6d3367345252566f5165364d70423456736153684a666a3750796d4a734c67646345394d425841096d89f063525313a4da4f133e79868e7c330aef84c0f0cd34c59b325b13acdc2612a04b1803a879cb2adfd393937e2955860333ae265dd774022ef6c6c4d08001"}},"id":"905e299e-88cd-4f75-980f-309b180ea744"}

[08:10:18.407] Sending request to http://localhost:3000
Request body: {"id":"65e48300-f604-4175-bb7e-b8e691e968de","method":"get_inclusion_proof","params":{"requestId":"0000618e4c9e84a7d0f7507bd23322bc4bf65b1615844a36439db2599092103b825b"},"jsonrpc":"2.0"}
[08:10:18.413] Response from http://localhost:3000: 200
Response body: {"jsonrpc":"2.0","result":{"inclusionProof":{"authenticator":{"algorithm":"secp256k1","publicKey":"0309a34985e3386b442039047c8bc850e2e6bfa3409e831ee530ad546f1cabbd02","signature":"b58d77e3ea89b2dc34d2b696f32b1c5c37daf33b5db524e8ffa9267a34ad19b673695b2d2267ff5e0488cbc9d6d56da097578cf1b37c5cae580ea7016c0a33fc01","stateHash":"000090b61f95c841fb1606591da0204ebf5fe76f96be93927035fd8099f2ef7b2b6f"},"merkleTreePath":{"root":"00007d9522d141c7f4ad9dcd7f8351eb7a1d00e06988d7c9791fd8841cababe12fa3","steps":[{"path":"118571788844127442544328106003613387982847629213831709850899587478540711240986121","data":"0000fb570d70655f57c83a0cdfb6d88630a3800099f8793cec77b71a972d1e000c74"},{"path":"5","data":"6af1a22954836aa45faf2242a8849edafc384221037103b25cf2b8363d4f076a"},{"path":"3","data":"8ec18db93f36d6630acdb12c3a0ada0f6687524ae159d88104413cce51418866"},{"path":"2","data":"946b2bf83a4646b7b65a1e0a18827e611f1b7f5802c43cd4eeba0a81505a6ab5"},{"path":"3","data":"02b60a0b24f202bf84264eebab5a12d01bfe7de10d613ad682f7e558c8f67bcf"},{"path":"3","data":"bc3d3219d13c4e0e0efb89f0205fa950c5f9963257c42f1ac3e642ca3beb51cb"},{"path":"1","data":"35e816e10e372b34fb69354b71aa58154e4426d38d91a8646cdbab9b0571eb84"}]},"transactionHash":"00009cdb62bcaad4baa51d3a8330c31f89c29d113571f12b32c659eb2078c74df7a6","unicityCertificate":"d903ef8701d903f08a01192d93005822000000316d054c54841de6d090c8e964af6dfbcc6ab42675e373ae4de8d520ff16c4582200007d9522d141c7f4ad9dcd7f8351eb7a1d00e06988d7c9791fd8841cababe12fa3401a69157645582200007d9522d141c7f4ad9dcd7f8351eb7a1d00e06988d7c9791fd8841cababe12fa300f6582037816ec4eee58e9a140796563a26c5fb0a9d9e40f73f8ecf2878083b5b964e37582003e917968f869e74174271e9359a69e957e79dd5c871a8cc34f3bdc6f10385f982418080d903f683010780d903e98801031a0001116c001a6915764858206eacbc5afe88491470782892e1f2753d18038059ccb5b2941769904aacec8b75582001d784e173af397f96ac6ec8013aeac8970d87919e38d731f409b9baba144696a4783531365569753248416d484734726b59767a6f6668425a357061564e6450676d41704653375659417537457367676250384a70454b5658413072e6b1afb639421463f8d5890a14265ea4358d9db183554435f82e4e2157776db8786a0289657beb5fb7913b2dc9f2b18da48c417df37d89e7ff392d73232b00783531365569753248416d4a7a416166384b7438757961374453714a69783963456b346379474a724a356b754e355154634e5164574b745841db654010f5e5219089dd681b80e0ee91b82fb705230afe1fd9d5a7cf07477e345b201e5edbdd2524996de63ddb09745f92def425d1fe20d16394c01ab274d68500783531365569753248416d504c6a764b56396834734c74376b6433523837646d3232366239797a50636e453476546e62686f34764741755841b4e883bb7d79bd3d04d4d9302700ac2fdb233d0aec9b09898cece7d37ba5ee000116677a5b1b054c62726e9df36e664eb21f824cf495a15a5548d78f12f765cf01783531365569753248416d5272334b66376d6d3367345252566f5165364d70423456736153684a666a3750796d4a734c67646345394d425841e6191194493b59929aea006f6dd4428809d9d6fcf535b19f963c566669d8e8691b3e1f92f098c32c02757729bc5259b6235e8852c2bd4a17661c79665cdf2f1201"}},"id":"65e48300-f604-4175-bb7e-b8e691e968de"}

Worker 1: flow completed successfully
Iteration 1 completed.

Expected result: Java SDK should be updated so inclusion proof check would be able to work with latest go aggregator changes.

[b3y0urs3lf]

Tried to test current sum-tree hashing functionality on Java SDK commit ad333be and GO aggregator commit 11a5d3a879978df9470b0d1ed54276c3f6d255a3.

Current PR #34 according to developer contains also functionality:

From #35 (get inclusion proof for specific transactions == atm implemented for tokens)
And fix for PATH_INVALID #36

Moving current task back to do

AS functionality added with regards to task 35 not being finalized yet, it is impossible to merge PR 34 as developer will make separate PR that will target current PR 34.
PATH_INVALID issue from task #36 related not only to Java SDK , but also to State Transition SDK, and in order to check that sum-tree hashing functionality interoperable between both SDK fix should be done on state transition sdk side also, developer will do it in unicitynetwork/state-transition-sdk-js#73.

[b3y0urs3lf]

Checked on go aggregator commit 4052d57dad5dde14493ec023394dd991216eed01 and Java SDK commit 7980e68 .

With implemented code fix, current PATH_INVALID issue is not reproducible anymore.

Moving task to done.