elevate the chatbot experience

[duck-master]

A lot of websites offer chatbots, but they often don't stand out. Right now, if I understand correctly, we're using an off-the-shelf model with no system prompt. This is bad! So to make ARK 2.0 work, especially if we want to emphasize the AI part, we'll really need to make it stand out.

This would require the following tasks:

• Decide on the bot persona
• Pick a model and write a draft system prompt
• Run automated evals
  • Pick and install an LLM automated evals framework
  • Write test cases
  • Run test cases
  • Commit test cases + results for future reference
• Rewrite the model_module folder to deploy the best model/prompt combination in the API (ideally determined through automated evals)

[duck-master]

By the way, here is a massive list of LLM evaluation tools. The biggest ones, at least if we're going by GitHub stars, are MLflow, OpenAI Evals, Langfuse, Opik, Promptfoo, Ragas, and TraceLoop.