Reformulate Questions to clarify the required value to be extracted

[FrancescoCasalegno]

Context

• In Question-Answering: collect example questions + run first analysis with pre-trained QA models #612 we analyzed the predicted outputs of some QA models used to extract parameter values.
• Typically, to extract the attribute ATTR of some entity ENT we formulated the question as What is the ATTR of ENT?
  • Example: What is the resting membrane potential of pyramidal cell? (ATTR is resting membrane potential, ENT is pyramidal cell`)
• Depending on the attribute we want to extract, sometimes the expected answer is a string / list of strings but sometimes we are expecting a numerical value.
  • What are the cell types of hippocampus? — expecting a list of a strings
  • What is the resting membrane potential of pyramidal cell? — expecting a numerical value followed by some unit
• However, even if we are expecting a numerical value, sometimes the model returns a string, corresponding to the definition of the attribute found in the context. Conceptually, this is not a wrong answer per se, but it is not what we are looking for. Here are some examples of this kind of "errors".
  
  
• Can we clarify in the question that we are looking for values rather than definitions?
• Should this approach fail, we can still hope to fine-tune the QA model on our samples (Fine-tune Question-Answering model on our own data #622), so that the model will learn that we are looking for values rather than strings.

Actions

• Reformulate questions aimed at extracting numerical values as What is the value of ATTR of ENT? and check if predicted answers are better.