You control both networks' hyperparameters — lag length, hidden units, training epochs — and can flip the input data between its raw form and a log-transformed, differenced version, then retrain and compare test MAPE and RMSE side by side to see firsthand why the recurrent model needs stationary data and the dense model largely shrugs it off.
Understanding RNN vs DNN for Time Series
This interactive visualization demonstrates a critical concept in time series forecasting:
Dense Neural Networks (DNNs) can work well on raw time series data by treating lags as independent features,
while Recurrent Neural Networks (RNNs) require data preprocessing to effectively capture temporal patterns.
Using the classic airline passenger dataset, you'll see how DNNs achieve good performance on raw data,
while RNNs initially struggle but excel after log transformation and differencing (making data stationary).
Key Learning: Model choice matters, but so does understanding what preprocessing each architecture needs!
🤔 What's Going On? Is RNN Failing?
Note that the Dense Neural Network (DNN) model works well on the unprocessed airline passenger data,
even without preprocessing steps like differencing or logarithmic transformation, while the Recurrent Neural
Network (RNN) requires these preprocessing steps to perform effectively.
This can be attributed to several factors related to the nature of DNNs and how they process information compared to RNNs:
📊 DNNs and Feature Representation
Independent Features: Treat lags as independent features, capturing trend and seasonality without understanding sequential data.
Pattern Mapping: Learn patterns from raw data without requiring stationarity.
No Temporal Dynamics: Perform well if input-output relationships are straightforward, even without sequential understanding.
🔄 RNNs and Sequential Data
Sequential Processing: Capture temporal dependencies but can struggle with non-stationary data.
Need for Stationarity: Preprocessing (e.g., differencing) helps reveal patterns by stabilizing mean and variance.
💡 Conclusion
DNNs excel with raw data by leveraging absolute lag values.
RNNs require preprocessing to focus on temporal relationships effectively.
Model choice and preprocessing depend on data characteristics and forecasting goals.
Configuration
🔧 Shared Configuration
Raw Data
Log + Differencing
100
🔷 DNN Configuration
12
32
16
🔶 RNN Configuration
12
16
Results
🔷 DNN Performance
Test MAPE:-
Test RMSE:-
Training Time:-
🔶 RNN Performance
Test MAPE:-
Test RMSE:-
Training Time:-
🏆 Winner
-
Model Architectures
🔷 DNN Architecture
🔶 RNN Architecture
Training Loss Curves
Time Series Predictions Comparison
🎓 Key Insights
DNNs treat each lag as an independent feature, so they can learn patterns directly from absolute values (levels). This works well even without preprocessing.
RNNs process sequences temporally, expecting to learn from changes and patterns over time. Non-stationary data (with trends/seasonality) confuses the sequential learning process.
Preprocessing (log + differencing) makes data stationary by removing trends and stabilizing variance, which helps RNNs focus on temporal dependencies rather than absolute levels.
After preprocessing, RNNs can match or exceed DNN performance because they properly capture the sequential nature of time series data.
Trade-off: DNNs are simpler and work out-of-the-box, while RNNs require more careful data preparation but offer better sequence modeling capabilities.