A Comparative study of Neural Networks vs. Mechanistic ODE Models for Health Data Forecasting
Résumé
Neural networks are know as universal approximators i.e. with sufficent data they can model any function. They have achieved exceptional results in a variety of domains including image classification, speech-processing, and natural language generation, serving as state-of-the-art models for classification and regression tasks. However, one of the main limitations of such models are that they are require a large amount of data to accomplish any given task. The availability of data becomes even more challenging in the domain of health care. In our study we highlight this limitation by comparing the performance of 3 distinct neural network architectures: an MLP, an ESN model with a reservoir, and an LSTM model against a mechanistic ODE model on a simple data forecasting task, using simulated patient data enriched with random missing values to mimic real-world scenarios. Given the sparse nature of the data, we performed various pre-processing steps such as imputation using SAITS and smoothing with a moving average to fill in the gaps. We also used group information to further enhance the data. Reservoir and LSTM models were trained auto-regressively, while the MLP model leveraged initial data to make predictions at specific future points. The evaluation metrics—bias, relative bias, MSE, and relative RMSE—revealed the superior performance of the Mechanistic model in handling sparse data, registering a minimal bias of 0.003 and an RMSE of 0.09 on the 180th day. In conclusion, the study illustrates that Mechanistic ODE models hold a comparative advantage in forecasting under conditions of limited and sparse data, offering insights into potential enhancements for Neural Network models in data-restricted environments.