AI based Multi-Modal Route Planning App with Langchain

This project integrates LangChain to enable natural language route planning.
Instead of manually entering waypoints, users can simply describe their travel needs in plain language, and the system extracts structured steps automatically.

🌍 Open Source Integrations

This project leverages powerful open-source tools:

• OpenStreetMap (OSM) → Provides detailed and community-driven map data.
• OpenTripPlanner (OTP) → Used for multi-modal route planning and optimization.

By combining LangChain for natural language understanding with OSM + OTP for geospatial intelligence, the system delivers a seamless, AI-powered travel planning experience.

📹 Demo

vid.mp4

Features

Natural Language Voice Input

Users can input complex, multi-stop travel plans through voice. Speech is transcribed using OpenAI Whisper, allowing for flexible, conversational inputs.
Example:
"I want to go from Sardar Patel Institute of Technology to TSG Sports Arena in Borivali, and then towards Greeshma Residency in Thane. On the way to TSG, I want to stop at a library. While heading home from TSG, I want to stop for petrol."

NLP-Powered Intent Extraction

The transcribed text is processed using Gemini to extract:

• The ordered sequence of main destinations.
• Intermediate preferences, such as specific stopovers.
• Categorization of each location as Primary destination or Auxiliary Point of Interest (POI) (e.g., petrol pump, library).

The output is structured into a clean JSON format for further processing.

Multimodal Route Generation (Car and Public Transit)

• If the user selects car, routes are computed using Ola Maps Directions API.
• If the user prefers public transit, routes are generated using OpenTripPlanner, backed by manually compiled GTFS data (supports metro, bus, and train schedules).
• The system can dynamically reprocess the route if preferences change.

Dynamic Point of Interest (POI) Integration and Rerouting

• For each auxiliary stop (e.g., petrol, library), nearby suggestions are fetched using Ola Maps Places API.
• The user can select from these suggestions.
• The route is then automatically recalculated based on the selected POIs and overall path logic.

In-App Public Transit Ticket Booking

• If the final journey involves public transport, users can directly book their ticket within the app.
• The booking process utilizes Google Pay deep linking, allowing a smooth transition from route planning to booking.

AI-powered extraction (via LangChain):

• Start: Sardar Patel Institute of Technology
• Stop 1: Library (on the way to TSG Sports Arena)
• Destination 1: TSG Sports Arena, Borivali
• Stop 2: Petrol Pump (on the way back)
• Destination 2: Greeshma Residency, Thane

🔧 How it Works

• LangChain parses natural language into structured intents.
• Entities like locations, stops, and directions are extracted dynamically.
• The parsed data is then passed to the route optimization module for execution.

This makes the route planning process conversational, intuitive, and AI-driven.

---

Run Instruction

Follow these instructions to run the application locally:

1️. Start Native App (Client)

cd native/
npm install
npm start

2️. Start Flask API (Server)

cd server/
pip install -r requirements.txt
flask run