Chatbot pilot implementation

[JoeProgrammer88]

Client would like to try a pilot implementation of a chatbot service. This would use an LLM service that can be trained on data from the current website/database

• Needs to support multiple languages
• After the pilot runs, we would need to get a good estimate of costs to run this and give client an estimate
• Need to test with some users and get feedback

[JoeProgrammer88]

📌 Summary
The client is requesting a multilingual AI chatbot pilot to evaluate whether an LLM‑powered assistant can reduce support load, improve user self‑service, and provide a scalable foundation for future automation. This pilot should demonstrate feasibility, gather usage data, and produce a reliable cost estimate for a full deployment.

🎯 Problem Statement
The client currently handles a high volume of user questions manually. They want to explore whether an AI chatbot can:

Answer common questions accurately

Support multiple languages

Reduce staff workload

Improve user experience through faster responses

The pilot should validate technical feasibility, user satisfaction, and operational cost.

🧩 Scope of the Pilot
In Scope
Build a prototype chatbot using an LLM (Azure OpenAI preferred unless otherwise noted)

Implement retrieval‑augmented generation (RAG) using selected website content and/or database fields

Support multiple languages (auto‑detect + respond)

Deploy to a test environment or standalone prototype UI

Log interactions for analysis

Conduct user testing with a small group

Produce a cost estimate based on real usage metrics

Out of Scope
Full production deployment

Integration with authentication systems

Full‑site ingestion or enterprise‑wide automation

Custom fine‑tuning (unless later approved)

📚 Data Sources
Specify the exact content to include in the pilot:

Website pages (list URLs or sections)

Internal documents (if any)

Database fields or structured content

Any multilingual content already available

Add the list here once confirmed.

🛠️ Technical Requirements
LLM & Infrastructure
Preferred: Azure OpenAI (GPT‑4o or GPT‑4.1)

RAG pipeline using embeddings + vector search

Hosting in Azure App Service or equivalent test environment

Logging of prompts, responses, and language detection

Privacy & Compliance
No storage of personal user data

All logs must be anonymized

Data must remain within approved hosting regions

🌍 Multilingual Support Requirements
Auto‑detect user language

Respond in the same language

Support at least the top N languages (define which)

Evaluate accuracy across languages during testing

🧪 User Testing Plan
Participants: small group of internal or client‑selected users

Tasks: ask typical questions, test multilingual queries, attempt edge cases

Feedback collection: survey + error logs

Success metrics: accuracy, satisfaction, latency, cost per interaction

💰 Cost Estimation Deliverables
The pilot must produce a cost analysis including:

Model inference cost

Embedding + vector DB cost

Hosting cost

Estimated monthly cost at scale

Comparison of at least two model options (e.g., GPT‑4o vs GPT‑4.1‑mini)

🚧 Risks & Unknowns
Accuracy variance across languages

Data quality issues

Latency for multilingual queries

Cost unpredictability under heavy load

Need for ongoing content updates

✅ Acceptance Criteria
Chatbot prototype deployed to a test environment

RAG pipeline connected to selected data sources

Multilingual support functioning and tested

Interaction logging implemented

User testing completed with documented results

Cost estimate delivered to client

Summary report outlining findings, risks, and next steps

📄 Definition of Done
[ ] Pilot chatbot running in test environment

[ ] Supports multilingual Q&A

[ ] Uses RAG with approved content

[ ] Logs interactions for analysis

[ ] User testing completed

[ ] Cost estimate created

[ ] Final report delivered to client