Local LLM
A local LLM is a large language model that runs entirely on your own hardware — a laptop, workstation, or self-hosted server — without sending data to external API providers.
Understanding Local LLM
Cloud-based LLMs (GPT-4, Claude, Gemini) process your prompts on external infrastructure. Every query you send includes your data — email content, task descriptions, document text — which travels to and is processed by the provider's servers. For sensitive data, this creates privacy and compliance concerns. Local LLMs eliminate this data exposure. Models like Llama 3, Mistral, Gemma, and Phi run entirely on your own hardware using tools like Ollama, LM Studio, or llama.cpp. Your data never leaves your machine. The tradeoff is capability and speed: local models are generally less capable than frontier cloud models, and running large models requires significant GPU hardware. The gap between local and cloud LLMs is narrowing rapidly. Llama 3 70B approaches GPT-4 quality on many tasks. Quantization techniques reduce model sizes dramatically — a 70B model can run on consumer hardware when quantized to 4-bit precision. For specific domains and tasks (especially those requiring privacy), local LLMs are increasingly viable. Hybrid approaches are emerging: use a local LLM for sensitive, personal data processing, and a cloud LLM for tasks requiring maximum capability where the data is less sensitive.
How GAIA Uses Local LLM
GAIA supports local LLM configurations via Ollama and compatible local model servers. When configured with a local LLM, GAIA processes all personal data (emails, tasks, calendar events) entirely on your infrastructure — no data leaves your environment. This is the maximum-privacy configuration for users handling sensitive information.
Related Concepts
Self-Hosting
Self-hosting is the practice of running software on your own servers or infrastructure instead of using a cloud-hosted service, giving you complete control over your data, configuration, and availability.
Data Sovereignty
Data sovereignty is the principle that data is subject to the laws and governance of the jurisdiction where it is stored, and that individuals and organizations have the right to control where their data resides and who has access to it.
Large Language Model (LLM)
A Large Language Model (LLM) is a deep learning model trained on massive text datasets that can understand, generate, and reason about human language across a wide range of tasks.
Open Source AI
Open source AI refers to artificial intelligence software whose source code is publicly available, allowing anyone to inspect, modify, distribute, and contribute to the project.
Foundation Model
A foundation model is a large AI model trained on broad data at scale that can be adapted to a wide range of downstream tasks through fine-tuning, prompting, or integration into application architectures.
