About the CT-Prompt code

[Veluriyam]

Hi authors, thank you for open-sourcing the excellent code for your paper "Context Tuning for In-Context Optimization"! I have successfully run the CT-KV experiments and the results are impressive.
I am also very interested in the CT-Prompt variant described in the paper. I've looked through the codebase but couldn't find its implementation. Could you please indicate if:
There is a plan to release the code for CT-Prompt in the future?
Or, if the current codebase supports CT-Prompt through a specific configuration flag that I might have missed?
Any guidance or pointers would be greatly appreciated! Thank you for your time and contribution.

[Jacklu0831]

Thank you for taking the time to study the paper and explore the codebase!

The public repository currently only includes the implementation for CT-KV, which we recommend for its superior accuracy and efficiency in comparison to CT-Prompt (and baselines). If you still want to implement CT-Prompt, here are two practical pointers from the paper:

• Initialize the tunable prompt from the demo context embeddings (i.e., set the trainable soft prompt P_{CT} to the input embeddings of the concatenated demonstrations C = [x_1; y_1; ... ; x_k; y_k].
• Use leave-one-out masking + token dropout during optimization. When optimizing on example i, mask the (x_i, y_i) segment inside P_{CT} to prevent leakage and apply token dropout to the prompt tokens as regularization.

I hope this helps, and best of luck with your experiments!