Skip to main content

AI — Semantic Search & RAG Assistant

Two AI surfaces, built on a shared embeddings + vector-store foundation in the NestJS API (apps/api/src/ai). Both are env-gated and degrade gracefully: with no Qdrant/provider configured the store falls back to keyword search and the assistant to extractive snippets — nothing breaks.

Replaces the storefront's substring title.includes(q) with vector search: typo-tolerant, synonym-aware ("inverter" ⇄ "PCU"), Hinglish- and intent-friendly ("battery for frequent power cuts" → tubular/PCU). Query-time synonym expansion (solar-synonyms.ts) adds domain recall on top.

  • Index: ProductIndexService embeds each Medusa product (title + categories + option values + spec metadata + cleaned description) into the utl_products Qdrant collection.
  • Query: GET /api/ai/search/products?q=… → { enabled, results: [{id, handle, title, score}] }.
  • Storefront: app/store/page.tsx calls lib/aiSearch.ts; if enabled and there are hits, it filters + orders the catalog by relevance, else falls back to the keyword filter. Facets and price ranges are unchanged.

2. RAG support assistant (site + WhatsApp)​

Answers pre-sales ("which PCU for a 3-BHK with cuts?") and after-sales (warranty/AMC) questions from the docs + product specs.

  • Index: KnowledgeIndexService chunks the Docusaurus docs (read from KNOWLEDGE_DOCS_DIR) and product specs into the utl_knowledge collection.
  • Answer: POST /api/ai/assistant { question } → RagService retrieves the top chunks, then answers with Claude (ANTHROPIC_API_KEY) or, without a key, an extractive answer from the top snippets. Returns { answer, sources[], mode }.
  • Site: the floating AssistantWidget (above the WhatsApp FAB) on every page.
  • WhatsApp: GET/POST /api/webhooks/whatsapp — Meta verification handshake + inbound messages, replied through the existing NotificationService (same WhatsApp credentials as order notifications).

RAG is the answer layer. Every customer-facing chat surface now goes through the guided sequence below, which calls RAG only for free-text questions — order tracking, service requests and human handoff are handled by the flow itself.

Enabling it​

  1. Vector store — Qdrant is in infra/docker-compose.yml:
    docker compose -f infra/docker-compose.yml up -d qdrant # → http://localhost:6333
    Set QDRANT_URL=http://localhost:6333 in apps/api/.env (+ QDRANT_API_KEY for Qdrant Cloud).
  2. Embeddings provider — auto-detected: VOYAGE_API_KEY → Voyage, OPENAI_API_KEY → OpenAI, else a deterministic local hashed embedding so everything works in dev with no keys. Force with EMBEDDINGS_PROVIDER (voyage|openai|local). Vultr is chat-only — its Serverless Inference API has no embeddings endpoint, so it is not an embeddings option; pair Vultr chat with local / OpenAI / Voyage embeddings.
  3. LLM answers (optional) — the chat layer (ChatService) is provider-agnostic, auto-detected in order ANTHROPIC_API_KEY → OPENAI_API_KEY → VULTR_API_KEY (force with AI_CHAT_PROVIDER=anthropic|openai|vultr). Models: AI_CHAT_MODEL (Anthropic, default claude-haiku-4-5), OPENAI_CHAT_MODEL (default gpt-4o-mini), VULTR_CHAT_MODEL (default llama-3.3-70b-instruct). Without any key the assistant/summary/triage stay extractive/heuristic.
  4. Index (SUPER_ADMIN):
    TOKEN=$(curl -s localhost:4000/api/auth/login -H 'content-type: application/json' \
    -d '{"email":"admin@utlsolar.test","password":"demo-admin-pass"}' | jq -r .access_token)
    curl -s -XPOST localhost:4000/api/ai/index/products -H "authorization: Bearer $TOKEN"
    curl -s -XPOST localhost:4000/api/ai/index/knowledge -H "authorization: Bearer $TOKEN"
    curl -s localhost:4000/api/ai/health
    Re-run the index endpoints after catalog/docs changes (or wire them to product.* events).

Switching providers changes the vector dimension; QdrantService recreates the collection automatically on the next index — just re-index.

The guided chat sequence​

POST /ai/chat runs one server-driven flow behind every chat surface (storefront widget, customer app, WhatsApp): greeting chips → pick one of your orders → pick the issue → resolution (live status, a policy answer from the knowledge base, a real service ticket, or a human). GET /ai/chat/:sessionId polls for replies a human agent has since written.

The session is the isolation boundary. It is bound to an account on the first authenticated turn and after that refuses anonymous callers and other accounts (403); guest sessions get no personal data at all; order ids are re-derived from the session owner rather than trusted from the client; and the LLM sees only the last 10 turns of that one conversation plus the allow-listed knowledge base.

Staff work the handoffs at /account/admin/inbox (GET/POST /admin/chat/…, SUPER_ADMIN · MANAGER · SERVICE_HEAD). An agent reply mutes the bot for that thread until it is handed back.

WhatsApp setup​

Point the Meta WhatsApp Cloud API webhook at https://<api-host>/api/webhooks/whatsapp with verify token WHATSAPP_VERIFY_TOKEN. Replies use the existing WHATSAPP_TOKEN + WHATSAPP_PHONE_ID. The POST handler acks immediately (200) and replies out of band so a slow LLM call never triggers Meta retries.

WhatsApp runs the same sequence as the widget, rendered for a channel with no buttons: options are printed as a numbered list and a reply of "2" is understood as the second one. Handing off to a human puts the thread in the same admin inbox, and the agent's reply is pushed back out over WhatsApp.

WHATSAPP_APP_SECRET is required in production. It is the app secret from the Meta dashboard, and every inbound POST is verified against X-Hub-Signature-256 before it is processed:

  • signed → the sender's number is trusted, so it may resolve to a registered account and unlock that customer's own orders;
  • unset → webhooks are still served, but every WhatsApp conversation stays a guest one (no order data), because an unsigned webhook is forgeable.

Environment reference​

VarPurpose
QDRANT_URL / QDRANT_API_KEYVector store (unset → AI disabled, keyword/extractive fallback)
EMBEDDINGS_PROVIDERvoyage | openai | local (default: auto-detect; Vultr has no embeddings)
VOYAGE_API_KEY / VOYAGE_MODEL / VOYAGE_DIMVoyage embeddings (default voyage-3.5, 1024)
OPENAI_API_KEY / OPENAI_EMBED_MODEL / OPENAI_EMBED_DIMOpenAI embeddings (default text-embedding-3-small, 1536)
AI_CHAT_PROVIDERanthropic | openai | vultr (default: auto-detect by key)
ANTHROPIC_API_KEY / AI_CHAT_MODELAnthropic chat (default claude-haiku-4-5)
OPENAI_CHAT_MODELOpenAI chat model (default gpt-4o-mini; uses OPENAI_API_KEY)
VULTR_API_KEY / VULTR_CHAT_MODEL / VULTR_INFERENCE_URLVultr chat (chat-only) — default model deepseek-ai/DeepSeek-V4-Flash, base https://api.vultrinference.com/v1. Set VULTR_CHAT_MODEL to a model from your Vultr catalog.
QDRANT_PRODUCTS_COLLECTION / QDRANT_KNOWLEDGE_COLLECTIONCollection names (default utl_products / utl_knowledge)
KNOWLEDGE_DOCS_DIR / DOCS_URLDocs source dir for RAG + public docs base for source links
MEDUSA_URL / MEDUSA_PUBLISHABLE_API_KEY / MEDUSA_REGION_IDCatalog source for indexing
WHATSAPP_VERIFY_TOKENMeta webhook verification token
WHATSAPP_APP_SECRETMeta app secret — verifies X-Hub-Signature-256. Unset → WhatsApp chats stay guest-only

Built on this foundation​

All of these reuse the embeddings + Qdrant + (optional) Claude layer above, env-gated with heuristic/keyword fallbacks:

CapabilityEndpoint(s)Surface
Semantic product searchGET /ai/search/productsStorefront /store
Guided support chatPOST /ai/chat, GET /ai/chat/:sessionIdStorefront widget, customer app, WhatsApp
Support inbox (staff)GET/POST /admin/chat/sessions…/account/admin/inbox
RAG assistant (single-shot)POST /ai/assistant, GET/POST /webhooks/whatsappLegacy/one-off answers
RecommendationsGET /ai/recommend/productsPDP related / frequently-bought
Review intelligenceGET /ai/reviews/summaryPDP "What buyers say"
Service-ticket triageGET /ai/tickets/triage, /ai/tickets/recent (staff)/account/admin/triage
Supplier / RFQ matchingGET /ai/suppliers/match (staff)/account/admin/suppliers
Blog search + relatedGET /ai/blog/search, /ai/blog/relatedBlog post related-articles

Re-index endpoints (SUPER_ADMIN): POST /ai/index/{products,knowledge,tickets,suppliers,blog}. Extra collections: utl_products, utl_knowledge, utl_tickets, utl_suppliers, utl_blog (override names via QDRANT_*_COLLECTION). Blog indexing reads Payload at PAYLOAD_URL.