Recurrent Neural Network

A Recurrent Neural Network (RNN) is a class of Artificial Neural Networks designed to process Sequential Data, including Time Series, by maintaining an internal Hidden State that acts as memory. Unlike Feedforward Neural Networks, recurrent networks update this state using both the current input and the previous state, allowing earlier inputs to influence later outputs. They are commonly trained using Backpropagation Through Time, which computes gradients across successive steps rather than providing the memory mechanism itself. RNNs are used in Natural Language Processing, Speech Recognition, and Time Series Forecasting, but standard architectures often struggle with the Vanishing Gradient Problem when learning long-range dependencies. Variants designed to mitigate this limitation include Long Short-Term Memory, introduced by Sepp Hochreiter and Jürgen Schmidhuber in 1997, and Gated Recurrent Units, introduced by Kyunghyun Cho and colleagues in 2014. Although largely superseded by Transformers in modern Large Language Models, RNNs remain foundational to the development of Deep Learning, with historical advances contributed by researchers across multiple institutions.