关于“gt_type”字段

[CTZhou-byte]

首先，感谢作者的工作。我想咨询下，代码里“gt_type”字段是起什么作用？我看脚步默认positive，但数据集并没有这个字段

[likaixin2000]

The gt_type field is intended to distinguish between different types of instructions, such as positive (an actionable instruction) and negative (an instruction that cannot be completed in the screenshot). This field is not explicitly included in the ScreenSpot-Pro dataset yet.

We’re currently working on improving support for this, and have already included such annotations in the ScreenSpot-v2-variants dataset. It includes:

• instruction: the original instruction
• action: a clarified action to perform generated by GPT-4o
• description: description of the target UI element generated by GPT-4o
• negative: an operation that cannot be completed in the given screenshot