Sage AI- A Chatbot with short and longterm memory
Project Details
a long term memory chatbot
Skills
Tools
AI-Powered Chatbot for Freelancer Market Research
Project Overview
I am excited to propose the development of an advanced AI-powered chatbot tailored for freelancers, specifically designed to revolutionize their market research capabilities. This project will leverage cutting-edge AI technologies to provide freelancers with a powerful tool for understanding market trends, identifying opportunities, and gaining a competitive edge. The chatbot will be built on a sophisticated architecture that prioritizes both short-term and long-term memory, ensuring a seamless and informative user experience.
Problem Identification
Freelancers frequently struggle with the time-consuming and often complex process of market research. Gathering relevant data, analyzing trends, and staying informed about industry changes requires significant effort and resources. Existing tools may lack the intuitive interface, the ability to retain context, or the comprehensive data integration needed to efficiently support a freelancer's research needs. The primary gap identified is the lack of a market research chatbot that can maintain both short-term conversational context and long-term memory of user preferences and previously discussed topics. This deficiency leads to wasted time, inefficient research processes, and missed opportunities for freelancers to optimize their strategies and secure projects.
Goal & Process
The primary goal is to create a full-fledged, internet-connected, and intelligent Sage-AI chatbot that empowers freelancers with comprehensive market research capabilities. This chatbot will serve as a critical support system for understanding the GetMeDesign platform, navigating its features, and providing general conversational assistance. It's designed to deliver personalized responses based on user profiles and conversation history, using real-time information grounded by Google Search.
The architecture follows a modular service design, with these key components:
Modular Service Architecture: The system is built using a modular architecture, using separate services for various functionalities.
`chatbot.sage.ts`: Orchestrates all the other services.
`llm.sage.ts`: Handles interactions with LLMs.
`rag.sage.ts`: Manages Retrieval-Augmented Generation for platform documentation.
`tools.sage.ts`: Provides tool-calling capabilities.
`redis.sage.ts`: Handles caching, rate limiting, and session management.
`db.sage.ts`: Manages database operations for chats and messages.
`short-term-memory.sage.ts`: Manages conversation-specific memory.
`long-term-memory.sage.ts`: Handles cross-conversation memory persistence.
Multi-Layer Memory System: The system implements a sophisticated memory architecture.
Short-term Memory (STM):
Stores conversation-specific "crux points" in Redis.
Uses vector similarity search for retrieval.
Automatically expires after 24 hours.
Limited to 100 items per chat.
Long-term Memory (LTM):
Stores important facts across conversations using MongoDB Atlas Vector Search.
Uses AI to determine what should be saved long-term.
Automatically cleans up old memories (max 1000 per user).
Persists user preferences, personal details, and important context.
Advanced RAG Implementation: The RAG system uses multiple strategies for optimal retrieval.
MongoDB Atlas Vector Search for semantic similarity.
Query rewriting with abbreviations and synonyms.
Multi-query search with different query variations.
High-relevance filtering (80-100% similarity threshold).
Deduplication and reranking of results.
Intelligent Context Management: The system maintains rich context through:
User Profile Integration: Automatically includes user data from Redis cache.
Conversation History: Maintains up to 40 recent messages in context.
Memory Context: Combines STM and LTM for personalized responses.
RAG Context: Includes relevant platform documentation.
Robust Error Handling and Monitoring: Comprehensive error handling includes:
Rate Limiting: 20 requests per minute per user.
Token Limits: 100,000 tokens per day per user.
Health Checks: Monitors all service dependencies.
Graceful Degradation: Continues operation even if some services fail.
Detailed Logging: Comprehensive logging for debugging and monitoring.
Performance Optimizations: Several performance optimizations were implemented:
Redis Caching: User profiles and chat contexts cached in Redis.
Connection Pooling: Efficient database connection management.
Token Estimation: Smart token counting for cost management.
Lazy Loading: Services initialize only when needed.
Background Cleanup: Automatic cleanup of old data.
Security and Privacy: Security measures include:
Input Validation: Message content validation and sanitization.
Access Control: User ownership verification for all operations.
Rate Limiting: Prevents abuse and ensures fair usage.
Data Isolation: User data properly isolated and secured.
Scalability Considerations: The architecture supports horizontal scaling:
Stateless Services: Most services are stateless and can be scaled.
Database Indexing: Proper indexing for efficient queries.
Redis Clustering: Can be extended to Redis cluster.
MongoDB Sharding: Vector search can be sharded by user.
Integration Points: The service integrates with:
Google Gemini API: For LLM capabilities and search grounding.
OpenAI Embeddings: For vector embeddings.
MongoDB Atlas: For vector search and data persistence.
Redis: For caching and session management.
GetMeDesign Platform: For user profile data and portfolio links.
Development Process: The implementation follows these principles:
TypeScript: Full type safety throughout the codebase.
Interface-Driven Design: Clear interfaces for all services.
Dependency Injection: Services are loosely coupled.
Configuration Management: Environment-based configuration.
Testing Considerations: Health checks and monitoring built-in.
Project Impact
This project will have a significant impact on how freelancers conduct market research. By providing a sophisticated, AI-powered chatbot, freelancers will gain access to:
Improved Efficiency: Streamlined research processes, saving time and effort.
Enhanced Insights: Access to comprehensive data and trend analysis.
Competitive Advantage: Better-informed decision-making based on accurate and up-to-date information.
Personalized Experience: A chatbot that learns and adapts to individual needs and preferences.
The resulting Sage-AI chatbot will empower freelancers to make data-driven decisions, identify lucrative opportunities, and ultimately, achieve greater success in their respective fields. This is a full fledged internet connected sage-ai chatbot that helps you.
