Vector Database
A vector database is a database system designed to store, index, and query high-dimensional vector embeddings at scale, enabling fast similarity search across large collections of embedded data.
Understanding Vector Database
Traditional databases store structured data in tables and query it with exact-match filters. Vector databases work differently: they store floating-point vectors (embeddings) and query them by similarity using distance metrics like cosine similarity or Euclidean distance. This makes them essential infrastructure for AI applications that need semantic search, recommendation, or memory. The core challenge vector databases solve is the 'nearest neighbor' problem at scale. Finding the closest vectors to a query vector among millions of stored embeddings requires specialized indexing algorithms. Approximate Nearest Neighbor (ANN) algorithms like HNSW and IVF make this fast by trading a small amount of accuracy for a massive speed improvement. Popular vector databases include ChromaDB, Pinecone, Weaviate, Qdrant, and pgvector (a PostgreSQL extension). They differ in deployment model, scalability, filtering capabilities, and ease of use. ChromaDB is particularly popular for local and self-hosted deployments due to its simplicity. In RAG systems, the vector database stores embeddings of your knowledge base. At query time, the database finds the most relevant embeddings and returns the original documents for the LLM to use as context. This allows AI systems to access specific knowledge without including everything in the LLM's context window.
How GAIA Uses Vector Database
GAIA uses ChromaDB as its vector database to store and query embeddings of your emails, tasks, documents, and calendar events. When GAIA needs to find relevant context for a task or answer a search query, ChromaDB performs a fast similarity search across all embedded content. This gives GAIA a persistent, searchable memory of your entire digital workspace that grows smarter as more data is indexed.
Related Concepts
Embeddings
Embeddings are dense numerical vector representations of data, such as text, images, or audio, that capture semantic meaning and relationships in a high-dimensional space.
Vector Embeddings
Vector embeddings are numerical representations of text, images, or other data that capture semantic meaning, enabling machines to understand similarity and relationships between pieces of information.
Retrieval-Augmented Generation (RAG)
Retrieval-Augmented Generation (RAG) is a technique that enhances LLM responses by first retrieving relevant documents or data from an external knowledge base and injecting that context into the model's prompt.
Semantic Search
Semantic search is a search technique that understands the meaning and intent behind a query, returning results based on conceptual relevance rather than exact keyword matches.
Graph-Based Memory
Graph-based memory is an AI memory architecture that stores information as interconnected nodes and relationships, enabling rich contextual understanding and persistent knowledge across interactions.
