Review of the Fulib Solution

[arturboronat]

The Fulib solution features:

• Table classes to enable concise, efficient OCL-like queries.
• An efficient way to load models against a hybrid object model.

The solution is presented in plain Java and, despite Java's verbosity, reads well and clear using an imperative approach. So this is a good point in favour of Fulib. The performance smashes the performance of the reference solution and of all other solutions that work with separate object models.

Metamodel changes were worth it in order to explore possible optimizations in performance and the case study has succeeded in demonstrating this.

However, I could not find the Java classes implementing the solution at: https://github.com/sekassel/ttc2021fuliblabworkflow/tree/master/src/main/java/fulib/labworkflow. Changing the metamodels also means that the baseline for comparing the different solutions no longer applies.

Overall, an interesting achievement in performance results! Could similar performance results be achieved without modifying the given object models?

[fjouault]

The Fulib solution is imperative, relatively compact, and readable. It seems to be quite scalable.

We noticed the following issues in the result models. Because the solution cannot be run by the benchmark script, we cannot confirm at this time whether they would be detected by the NMF-based validation program or not.

• Major issues:
  • Incubation jobs do not seem to be translated, although there is some code that should process them. The reason why it seems to fail is unclear to us at this time.
  • There is no support for adding new samples.
    • This is reported in the paper.
  • There is no status update for jobs or tips after change01, and therefore no removal of failed samples from following jobs, but statuses are updated after change02.
• Minor issues:
  • LiquidTransferJobs with trough as source should not have sourceCavityIndexes. This will probably be ignored by the robot.
  • The previous and next references link individual jobs instead of linking to the jobs corresponding to the previous and next steps.
    • This may hinder parallelization of some tasks by a robot that could do it, but should not otherwise break the process.
    • This is reported in the paper.

The solution runner (FulibSolution.java) seems to incorrectly apply some changes:

• The first change seems to be never applied, which may explain the issue about change01 having no apparent effect, as mentioned above.
• Job status should be failed if any tip is failed, but this is not the case with the current code.
• It ignores plate names, which means it will not work correctly when they are multiple plates.

Some issues may be explained by the fact that some changes were performed on the source and target metamodels, as reported in the paper. The authors wrote that they did not realize that it would break things when they applied these changes.

Remark: this review is based on the code from branch “main” of the Fulib solution repository, because branch “master” seemed obsolete.

This review was written in collaboration with @TheoLeCalvar.