Clarification Regarding Time Required to Finish the Search

[seunghwan6960]

Hello. I am currently conducting research on indoor gas source localization algorithms, with a particular focus on evaluating them against existing methods, including your work.

While running the publicly released code from your project, I noticed a significant discrepancy in performance that I hope you can clarify. Specifically, when testing the A2 environment, the search took approximately 1000 seconds to complete. This is considerably longer than the average of 87 seconds reported in your paper.

Considering the physical travel time of the robot and the time required for sensing — particularly given that the current code appears to perform five measurements and update the hit map at each sensing location, which also incurs a substantial time cost — the reported duration of 87 seconds seems practically unattainable when factoring in the computational time required by your algorithm. I am wondering whether there are any differences between the termination conditions or time measurement procedures used in the released code versus those used in the experiments reported in the paper.

If there are any details I may have missed, I would greatly appreciate your clarification. This information is crucial to ensuring a fair and accurate comparison for my own research.

Thank you very much for your time and consideration.

[PepeOjeda]

Hello. It seems like you probably are running into some issue with the algorithm execution, and it is simply not working (1000s is just the hard time limit for the execution, meaning the source is never found). I would need more information to help you figure out what is happening -- what algorithm are you using? do you see the robot move around in Rviz? etc.

[seunghwan6960]

I would like to clarify that the search duration I referred to in my previous message was not exactly 1000 seconds, but rather close to 1000 seconds. I have attached a video demonstrating the execution of your algorithm in the PMFS setting to support my observation.

In order to run the code, I made only two minor modifications to the publicly released repository:

The gsl_actionserver package could not recognize the gmrf_mapping package during the build process, so I modified the CMakeLists.txt file to add it as a dependency.

In the launch file, the topic amcl_pose_with_offset could not be resolved, presumably due to the absence of the offset_pose package. Therefore, I changed the robot_location_topic parameter from amcl_pose_with_offset to amcl_pose.

As far as I understand, the offset_pose package appears to apply a static offset to the robot pose. However, since the offset values in your launch files are all zero (0,0,0), I assume this change should not affect the algorithm's behavior.

Aside from these two adjustments, I have made no other modifications to your code.

Could you please confirm whether the absence of the offset_pose node and the use of amcl_pose instead could affect the execution time or results in any significant way?

Thank you again for your time and support.

Video link:
https://unistackr0-my.sharepoint.com/:v:/g/personal/kevin6960_unist_ac_kr/EauzllgQ-_BDg86toFfh_Q4Bngjap4sJ_8IjpUFMW6Bfjw?nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJPbmVEcml2ZUZvckJ1c2luZXNzIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXciLCJyZWZlcnJhbFZpZXciOiJNeUZpbGVzTGlua0NvcHkifX0&e=AMmwH2

[PepeOjeda]

Well, looking at the video, the algorithm does seem to be executing correctly, but everything is running surprisingly slow. I don't know if this is simply a matter of the computational power of your machine (for reference, the testing was done in a desktop computer with an i5-9600K CPU), or if there is some other issue with the setup.

I ran some tests today as a sanity check (as there have been some updates to the code since the experiments in the article), and got times in the 80-100s range for this scenario.

[seunghwan6960]

I truly appreciate your quick and thoughtful reply.

Since my current PC is equipped with an i7-10700 CPU, I assume that this issue is unlikely to stem from hardware limitations. I will check my setup further and reach out again if necessary.

I’d like to ask an additional question that is not directly related to this issue.
I plan to run the PMFS algorithm on an NVIDIA Jetson Orin AGX onboard computer. I was wondering if you have ever tested PMFS on such onboard platforms?

If so, I’d greatly appreciate it if you could share any approximate execution times for key parts of the algorithm, especially:

the update time for the hit map
the update time for the source probability map

[seunghwan6960]

Hello. To address the issue, I have conducted several tests.

First, I tested on another computer equipped with an i7-1360P CPU, but observed no significant improvement compared to previous results. However, I discovered an interesting detail: the robot traveled approximately 186 meters until the source was detected.

Please allow me to express my thoughts, although they may sound a bit assertive—I mean no offense.

error_over_time.csv
results.csv
path.csv

Since the maximum velocity of the robot in your simulation environment using move_base is set to 0.4 m/s, the current behavior cannot be fully explained by simply attributing it to a slow environment or algorithm performance. I believe it is more accurate to say that the robot’s trajectory exhibited inefficient behavior compared to the results presented in your paper.

This raises the question: is this inefficiency due to the algorithm setup?

It seems unlikely. The hitmap update and source probability update appear to function quite well. In fact, the estimated source location has a remarkably low error of just 0.65 meters. To be honest, if there were an issue in the setup, I would expect problems during build or execution, not during runtime behavior. Unless your algorithm depends on some version-sensitive libraries, I find the current behavior quite difficult to understand.

For now, I plan to reset and rebuild your algorithm and ROS environment from scratch.

One request I would like to make is whether you might be willing to share your algorithm's execution environment through a Docker image. Your work represents one of the most feasible and novel solutions currently available for indoor GSL, and sharing the Docker environment could greatly contribute to reproducibility and, ultimately, to the advancement of this field.

Of course, I understand that preparing and sharing a Docker environment takes time and effort. If you are unable to provide it, I completely understand and have no intention of criticizing you for that.

[shuaidongwang334-web]

Are you using code for the ROS2 Humble version?