Machine Learning
SentencePiece
SentencePiece is an open-source tokenizer library from Google that trains subword models directly on raw text, without language-specific pre-tokenization. It supports both BPE and Unigram algorithms and underpins LLaMA, T5, mT5, and Google's Gemini family.
Subword Tokenization
Subword tokenization splits text into pieces smaller than words but larger than characters. It solves the open-vocabulary problem for neural language models and is the dominant approach in modern LLMs, with BPE, WordPiece, and Unigram as the main algorithm families.
Byte-Pair Encoding (BPE)
Byte-Pair Encoding is a subword tokenization algorithm that learns a vocabulary by iteratively merging the most frequent adjacent symbol pairs in a corpus. Originally a 1994 compression trick, it became the dominant tokenization method for GPT, LLaMA, and most modern LLMs after a 2016 NLP adaptation.
Curse of Dimensionality
The curse of dimensionality is a cluster of phenomena that make geometric, statistical, and search algorithms degrade as the number of features grows. Volume grows exponentially, samples become sparse, and pairwise distances concentrate, so nearest-neighbor and density-based methods lose discriminative power.
GSM-Symbolic Benchmark
Mathematical reasoning benchmark introduced by Mirzadeh et al. (Apple, 2024) that builds symbolic templates from GSM8K problems so names, numbers, and irrelevant clauses can be varied. Designed to test whether LLMs reason or pattern-match; frontier models show large accuracy drops under simple perturbations.
Reinforcement Learning from Human Feedback (RLHF)
RLHF is a three-stage fine-tuning pipeline — supervised demos, a learned reward model trained on human preference comparisons, and policy optimization against that reward — that turned base LLMs into modern chat assistants. It made models far more helpful but introduced reward-model exploitation, sycophancy, and over-refusal as recurring failure modes.
Brier Score
Strictly proper scoring rule introduced by Glenn W. Brier in 1950 to evaluate probabilistic forecasts. Computes the mean squared error between predicted probabilities and realized outcomes; lower is better, with 0 indicating a perfect forecast.
Calibration (Machine Learning)
A probabilistic classifier is calibrated when its predicted probabilities match empirical frequencies: among predictions labeled "p," roughly p of them are correct. Measured by reliability diagrams, Expected Calibration Error, and proper scoring rules like the Brier score and log loss.
Vector Embeddings
Vector embeddings are dense numeric representations of items — words, sentences, images, users, products — in a continuous space where geometric proximity reflects semantic similarity. They turn similarity into distance computations, enabling search, clustering, recommendation, and retrieval over unstructured data.
C4 (Colossal Clean Crawled Corpus)
C4 is a roughly 750-gigabyte English text dataset released by Google in 2019 alongside the T5 model, derived from a single April 2019 snapshot of Common Crawl after heuristic deduplication and quality filtering. It has been used to train T5, LaMDA, and many open-source LLMs, and is the subject of a widely-cited 2021 audit by Dodge et al.
HNSW (Hierarchical Navigable Small World)
HNSW is a graph-based algorithm for {{approximate nearest neighbor}} search introduced by Malkov and Yashunin in 2016. It builds a multi-layer proximity graph in which higher layers contain exponentially fewer nodes with longer-range edges. Searches start at the top layer and descend greedily, giving logarithmic expected complexity at high recall.
Product Quantization
Product quantization (PQ) is a lossy vector compression technique that splits each vector into subvectors and encodes each with a small learned codebook. It shrinks high-dimensional embeddings by 90–97% while still supporting approximate distance computation in the compressed domain, making it a backbone of large-scale ANN indexes.
Locality-Sensitive Hashing
Locality-sensitive hashing (LSH) is a family of probabilistic techniques for {{approximate nearest neighbor}} search and near-duplicate detection. Hash functions are chosen so that similar items collide with high probability and dissimilar items collide rarely, turning a similarity search into a bucket lookup with sublinear query time.
Common Crawl
Common Crawl is a US 501(c)(3) nonprofit founded in 2007 by Gil Elbaz that publishes monthly open web-crawl datasets. As of 2026 the archive exceeds 10 petabytes and is the single largest source of training data for major large language models including GPT-3, LLaMA, and Claude precursors.
Deep Learning: The Neural Network Revolution That Enabled Modern AI
Deep learning uses multi-layered neural networks trained on large datasets to learn hierarchical representations — the foundation of modern AI from image recognition to language models.
Large Language Models: How Next-Token Prediction Creates General Intelligence
Large language models are transformer-based neural networks trained on massive text corpora via next-token prediction, developing broad capabilities as emergent properties of scale.
Constitutional AI
Constitutional AI is Anthropic's training method that replaces most harmlessness labels in RLHF with model self-critique against an explicit written list of principles, plus a reward model trained on AI-generated preferences (RLAIF). It makes the intended values legible and is the training backbone of the Claude assistants.
Approximate Nearest Neighbor Search
Approximate nearest neighbor (ANN) search finds vectors close to a query in high-dimensional space without guaranteeing the exact best match. Exact k-NN suffers from the curse of dimensionality, so ANN trades a small loss of recall for major gains in latency and memory. Major families include tree-based methods, locality-sensitive hashing, graph indexes such as HNSW, and product quantization, implemented in libraries like FAISS, Annoy, hnswlib, and ScaNN. ANN underpins semantic search, recommendation, deduplication, and RAG retrieval.
Structured Outputs (OpenAI)
OpenAI's Structured Outputs feature, introduced August 2024 with gpt-4o-2024-08-06, lets developers supply a {{JSON Schema}} via the response_format parameter with strict: true. The model's decoder is gated server-side so output is guaranteed to match the schema. On OpenAI's internal eval, the feature scored 100% schema adherence vs under 40% for prompt-only GPT-4.
Constrained Decoding
Constrained decoding (also called grammar-guided or structured generation) forces an {{LLM}}'s output to match a target grammar by masking invalid tokens at each generation step. The model still picks the highest-probability token — but only among tokens that keep the output syntactically valid. This is how {{Outlines}}, {{guidance}}, XGrammar, and llguidance enforce {{JSON Schema}} or regex constraints.
Reasoning Models (LLM)
Class of large language models trained to spend substantial test-time compute generating internal chain-of-thought before answering. Pioneered by OpenAI's o1 (Sept 2024) and replicated by DeepSeek-R1, QwQ, and others; shifts the scaling axis from train-time to test-time compute on reasoning-heavy tasks.
Outlines (Library)
Outlines is an open-source Python library for {{constrained decoding}} of {{LLM}} outputs. It compiles a {{JSON Schema}}, regex, or context-free grammar into a finite-state machine and uses it to mask invalid next tokens at each generation step, guaranteeing the output parses. Outlines pioneered the FSM-based approach later adopted in commercial structured-output APIs.
Function Calling (LLM)
Function calling is the provider-supported mechanism by which a large language model returns a structured request to invoke a named tool with typed arguments, leaving the host application to actually execute the function and feed the result back into the conversation.
System Prompt
A system prompt is text prepended to every conversation with a {{large language model}} that establishes role, rules, available tools, and context the model would otherwise lack — including the current date. It is the primary place applications shape model behavior at inference time.
Self-Consistency Decoding
Decoding strategy that samples multiple reasoning paths from an LLM at non-zero temperature and aggregates them — typically by majority vote on the final answer — instead of taking a single greedy chain-of-thought. Improves accuracy on reasoning tasks and yields a cheap confidence proxy via vote share.
ReAct Prompting
ReAct is a prompting framework, introduced by Yao et al. in 2022, that interleaves explicit reasoning traces with discrete tool actions so that a language model both thinks aloud and acts step by step against an external environment.
Beam Search: The AI Decoding Strategy That Balances Quality and Speed
Beam search is a heuristic decoding algorithm that maintains the top k candidate sequences at each step, trading compute for output quality compared to greedy decoding.
AI Debate
AI Debate, proposed by Irving, Christiano, and Amodei (OpenAI, 2018), is a scalable oversight technique in which two AI agents argue opposing sides before a judge, exploiting adversarial incentives to surface errors. It is studied as a way to verify outputs beyond human expertise and to reduce sycophancy by removing the lone-assistant-pleases-judge dynamic.
Tool Use (LLM)
Tool use is the broad capability by which a large language model interacts with external systems, such as calculators, search engines, code interpreters, or domain APIs, instead of relying solely on parametric knowledge in its weights.
RWKV: Recurrent Architecture with Constant State Size for Parallel Inference
{{RWKV}} is a recurrent language-model architecture whose internal state has a fixed size independent of context length, making large-batch inference and parallel-perturbation training dramatically cheaper than for {{transformer}}-based models with their growing {{KV-cache}}.
Evolution Strategies for LLM Fine-Tuning: A Revival of a Pre-Deep-Learning Optimizer
Two 2025 papers revive {{evolution strategies}} (ES) as a credible alternative to {{reinforcement learning}} for fine-tuning large language models, exploiting the fact that RL fine-tuning rewards are already scalar at the sequence level — the regime where ES is naturally competitive.
EGGROLL: Low-Rank Perturbations Make Evolution Strategies 100x Faster at Hyperscale
{{EGGROLL}} (Evolution Guided GeneRal Optimisation via Low-rank Learning), from an Oxford/MILA/NVIDIA collaboration in November 2025, structures each {{evolution strategies}} perturbation as a low-rank matrix so that thousands of perturbations can be computed in a single batched forward pass — yielding a claimed 100-fold training-speed increase over naive ES at billion-parameter scale.
Evolution Strategies at Scale (Cognizant 2025): First Full-Parameter ES on Billion-Parameter LLMs
A September 2025 paper from {{Cognizant AI Lab}} demonstrated full-parameter {{evolution strategies}} fine-tuning of billion-parameter {{LLMs}} using a population of just 30 perturbations, breaking the prior assumption that ES could not scale past roughly a million parameters.