Temperature (AI)
Temperature is a parameter in language model inference that controls the randomness of token selection — higher temperatures produce more varied and creative outputs, while lower temperatures produce more consistent and deterministic responses.
Understanding Temperature (AI)
When an LLM generates text, it calculates a probability distribution over possible next tokens. Temperature scales this distribution before sampling. At temperature 0, the model always picks the highest-probability token (greedy decoding), producing deterministic output. At higher temperatures (0.7-1.0), the model samples from a wider distribution, introducing variety and occasionally surprising — or incorrect — outputs. Temperature is a dial between creativity and consistency. For tasks requiring factual accuracy — summarizing an email, extracting tasks from a meeting — low temperature (0.0-0.3) is preferred. For tasks requiring creativity — brainstorming, writing varied copy, generating names — higher temperature (0.7-1.0) produces more interesting results. Related parameters include top-p (nucleus sampling), which limits sampling to the top probability mass, and top-k, which limits sampling to the top k tokens. Together, these parameters give developers fine-grained control over output characteristics. Most AI assistant APIs expose temperature as a configurable parameter. GAIA uses different temperature settings for different task types — lower for factual retrieval and task creation, higher for creative writing and brainstorming.
How GAIA Uses Temperature (AI)
GAIA uses different temperature settings depending on the task. For factual tasks like reading your inbox, extracting action items, or reporting on calendar events, GAIA uses low temperatures for consistent accuracy. For creative tasks like drafting replies, generating summaries, or brainstorming, it uses higher temperatures for more natural, varied output.
Related Concepts
Large Language Model (LLM)
A Large Language Model (LLM) is a deep learning model trained on massive text datasets that can understand, generate, and reason about human language across a wide range of tasks.
Prompt Engineering
Prompt engineering is the practice of designing and refining inputs to AI language models to reliably elicit desired outputs, shaping model behavior without modifying the underlying weights.
Inference
Inference is the process of running a trained AI model on new input data to generate predictions, responses, or decisions, as opposed to training, which is the process of building the model from data.
Hallucination
AI hallucination is the phenomenon where a language model generates confident-sounding but factually incorrect, fabricated, or nonsensical information that is not grounded in the input or training data.
Structured Output
Structured output is a technique that constrains an LLM to respond in a predefined format — typically JSON or XML — enabling reliable programmatic parsing of model responses rather than free-form text.
