QUEENS 2025 usage survey

[gilrrei]

Survey

Thanks to everyone who participated in the QUEENS survey. Your input is much appreciated ❤️

Answering this survey is an important work for the community as it allows us to identify problems together.

For the next survey, I think adding some postive question like: what is your favorite feature, how does QUEENS help your research might also be of interest, if nothing else just to enjoy the progress :)

Answers

Some answers were grouped into one when possible. Sometimes anonymized. Some AI was use to generate these tables.

Question 1: Answers to Which iterator do you use?

Iterator	Number of Users
Grid	4
Sobol Sequence	3
Monte Carlo	2
No iterator	2
Sobol Index	2
Sobol Index with GP	2
LHS	1
Adaptive sampling	1
Points	1
Variational inference	1
MCMC	1
SMC	1

Question 2: Which simulation tool do you use QUEENS with?

Tool	number of users
4C	5
Python functions	5
Liggghts	1

Question 3: Which model do you use?

Model	number of users
Simulation	7
Gaussian Processes	3
Likelihood models	1
logpdfgaussianprocess	1

Question 4: Which driver do you use?

Driver	number of users
Function	2
Jobscript	8
Jobscript with mods	3
Jobscript with restart mod	2
Mpi	1
FourC with custom restart	1

Question 5: Which scheduler do you use?

Scheduler	number of users
Pool	2
Cluster	8
Local	5
Modified Cluster	1
Modified Cluster for local runs on cluster	1

Question 6: Answers to Do you use a remote connection to execute simulation jobs on a different machine?

Answer	number of users
Yes	9
No	2

Question 7: Answers to What is super annoying for you when using QUEENS?

Summarized answers with AI.

Response	Category	Key Issues
1	Documentation	Bad documentation of iterator classes; unclear output dictionary from process_outputs function
2	Data Handling	Only numpy arrays accepted for simulation models; working directory handling difficulties
3	Robustness	Need for restart feature for failed simulations; difficulty separating data generation from post-processing
4	Dependencies	Lots of dependencies
5	Robustness	Having to restart entire runs due to low-level errors (post-processing, iteration failures)
6	Learning Curve	Requires expert help for fast progress
7	Performance	Long startup times due to large imports
8	Documentation & Features	Generally satisfied; minor improvements: better data processor helpers (e.g., PyVista integration); more tutorials for different problem types and iterators
9	Usability	Results functionality is unintuitive; user unaware of full functionality
10	Architecture	Rigid structure tailored to 4C; restrictive drivers; poor plotting and result handling

Question 8: Do you need to customize drivers or any other QUEENS object in order for it to work? If yes, which ones, and what did you modify?

Summarized answers with AI.

#	Customization Type	Purpose
1	Custom drivers	Pass arbitrary argument types and keyword arguments; enable MPI execution; enable sequential model execution; inject into 4C input files; modify JobScriptDriver; remove/overwrite existing functionality; run cluster computations without remote connection
2	Custom Sobol sequence & data processor	Write test/training data as results for GP retraining; extract 4C results
3	Custom data processors	Extract 4C results; handle non-numpy data types
4	Custom schedulers	Pass non-floating-point numpy objects; run cluster computations without remote connection
5	Sometimes	No reason given

Some remarks

Some additional points that do not fit directly into one category:

• Some comments mentioned they don't know how to use different drivers, schedulers, models, ... So here I'm glad the tutorials are advancing
• Some people mentioned by name who helped them :) For anonymity reasons, I removed them, but I am glad to see interactions between QUEENS people
• Most of the customization and frustration lies in the drivers and dataprocessors. Oftentimes needing more flexibility
• Some users mainly use the backend of QUEENS
• The usage of QUEENS is quite heterogeneous within the community

Thanks for the answers, and I hope this picture helps to indetify some improvements, which can be tackled by the community

[sbrandstaeter]

Very interesting. Thank you for organising the survey and also postprocessing it!