Quantization

STANDARD DEFINITION · CACHED

is a technique used to reduce the precision of a 's weights and activations to decrease memory footprint and increase inference speed. By mapping high-precision values, typically 32-bit floating-point (), to lower-precision formats such as 8-bit integers (), , or even 4-bit representations, it enables the deployment of on resource-constrained hardware like devices or mobile processors. Common strategies include (PTQ) and (QAT), both aiming to minimize the loss in caused by discretization errors. Major industry players like , , and utilize this technique within frameworks like and to optimize performance on and architectures.

[PREVIEW MODE] Definitions streamed at five depths, references, and related terms are available to signed-in readers — [SIGN IN]

Contextual terminology map

Active Learning AI Alignment AI Audit AI Bubble AI Capex AI Watermarking Algorithmic Bias Anthropic Attention Mechanism Backpropagation Bayesian Inference Big Data Content Credentials Convolutional Neural Network CUDA Data Broker Data Lake Data Normalization Data Poisoning Decision Trees Deepfake Differential Privacy Diffusion Models Digital Divide Disparate Impact End-to-End Encryption EU AI Act Explainable AI Export Controls Feature Engineering Fine-tuning Frontier Model GDPR (General Data Protection Regulation)Generative AI Google DeepMind GPU Computing Gradient Boosting Gradient Descent HBM (High-Bandwidth Memory)Hugging Face Hyperscaler Inference Interpretability Job Displacement Large Language Model Mistral AI Model Card Model Distillation Neocloud Neural Network NIST AI RMF (AI Risk Management Framework)NVIDIA OpenAI Overfitting Power Purchase Agreement Principal Component Analysis Prompt Engineering Prompt Injection PUE (Power Usage Effectiveness)PyTorch Quantization RAG (Retrieval-Augmented Generation)Random Forest Recurrent Neural Network Red Teaming Reinforcement Learning Small Modular Reactor Sovereign AI Stochastic Gradient Descent Supervised Learning TensorFlow TPU (Tensor Processing Unit)Transformer Architecture Underfitting Unsupervised Learning Vector Database WCAG (Web Content Accessibility Guidelines)Zero-Day

KEEP IN VIEW

Infrastructure, markets & the economy

The compute, power and capital behind the boom

Concept inspired by Dev Valladares’s Infinite Wiki. Independently built; not affiliated with or endorsed by the original.

Quantization

Contextual terminology map

KEEP IN VIEW

Ethics, safety & society

Infrastructure, markets & the economy

Keyboard shortcuts

Go to

On this page