Absence of evaluation tools for Personal Agent Track scores

[zlh-source]

Excellent work!
However, it appears that the openclaw-test directory contains only the code for simulating dialogues, while the code for evaluating model performance is missing.
In other words, the evaluation tools corresponding to the scores presented in Table 3 of the paper are currently unavailable.

One indication of this is that I was unable to locate the specific prompts mentioned in Appendix C.3 of the paper—"Personal Agent: Evaluative Prompt from Simulator"—within this repository.

[yinjjiew]

You are correct. The script to obtain the evaluation score is not included in this repo. I think it's more straightfoward to take look at the first output of the policy in each conversation session (in our student/teacher evaluation settings). The change of response pattern is a more stable and promising judgement than score provided by llm as judge.

For score evaluation, we find the score variance is large due to llm as judge. So we are looking for a more stable quantitative metric.

[zlh-source]

Looking forward to the latest developments.