Handling of combined questions and intents

[svenseeberg]

Messages with multiple questions/intents are not handled in a good way. We should concept solutions. This can also improve the mixed messages when merging the history.

Example:

"I need to learn German and a job" would be 2 different intents:

Especially the word "intent" is often used in the context of chat messages and probably causes the LLM to produce good results.

[dasgoutam]

Option 1 - Changing the prompt

Background-
The current prompt for checking the message/question summarizes the current/last 3 messages to create a new question. Changing the prompt to - https://github.com/digitalfabrik/integreat-chat/blob/feature/multiple-intents/integreat_chat/chatanswers/static/prompts.py#L48-L92 has some interesting results.

The new prompt does the following

1. Checks for distinct "intents" across multiple messages
2. Separates a single question into multiple intents
3. Summarizes if required

Results -
("messages" are the 3 questions/messages provided. "extracted question" has the resultant multiple intents separated by | )

  "messages": [
    {
      "content": "Hello",
      "role": "user"
    },
    {
      "content": "I need help to learn german and a job",
      "role": "user"
    }
  ],
  "extracted_question": "learn German | find a job",
  "rag_language": "en",
  "gui_language": "en",
  "region": "muenchen"

  "messages": [
    {
      "content": "Hello. I need some help",
      "role": "user"
    },
    {
      "content": "I need help to learn german and find a job",
      "role": "user"
    },
    {
      "content": "and need help with my visa. I want to apply for PR. My name is Alan and I am 21 years old",
      "role": "user"
    }
  ],
  "extracted_question": "learn German | find a job | visa help | apply for PR",
  "rag_language": "en",
  "gui_language": "en",
  "region": "muenchen"

[dasgoutam]

If multiple intents are identified, each of them can be answered separately and joined together (with a limit on the number of intents that can be answered together)

One downside right now is that some personal information can be lost in the summarization (age, country of origin). This can be relevant for document extraction. There can be an option to extract this information separately if provided by the user, and add it to the document extraction prompt