Large Language Model

A Large Language Model (LLM) is a type of Artificial Intelligence that uses large Deep Learning models trained on extensive datasets to process, generate, and manipulate human language. Most modern LLMs use the Transformer architecture, whose Self-Attention mechanisms help model long-range dependencies in text. Autoregressive LLMs generate text by predicting a Probability Distribution over the next Token in a sequence, whereas other LLMs may use Masked Language Modeling or other training objectives; some modern systems also process non-text inputs through Multimodal Learning. Notable examples include GPT-4 by OpenAI, Claude by Anthropic, Llama by Meta AI, and Gemini by Google DeepMind. Many LLMs undergo Pretraining using Self-Supervised Learning and may subsequently receive Supervised Fine-Tuning, Preference Optimization, Reinforcement Learning from Human Feedback (RLHF), or other post-training methods to improve performance and align outputs with human preferences; RLHF is common but not universal. Large-scale training and Inference often require substantial computational resources, commonly supplied by GPU or other AI Accelerator clusters. NVIDIA GPUs are widely used but are not required: alternatives include Google Tensor Processing Unit hardware, AMD accelerators, and custom Application-Specific Integrated Circuit hardware, while smaller LLMs can run on consumer devices with much less compute.