A comparison of methods for quantifying prediction uncertainty in systems biology
Contents
Citation
Villaverde, Alejandro F., et al. "A comparison of methods for quantifying prediction uncertainty in systems biology." IFAC-PapersOnLine 52.26 (2019): 45-51.
Summary
Three methods for quantifying prediction uncertainty in ODE models are assessed. Here, prediction uncertainty does not refer to estimated parameters, but to the uncertatinty of state trajectories. The three methods are: Fisher Information Matrix (FIM), Prediction Posetrior (PP), Ensemble Consensus (ENS).
Study outcomes
Outcome O1
For a small, fully-observed ODE model (α-pinene, Box et al.), all three methods yield nearly same results consistent with the known true trajectories. For a larger, only-partially observed ODE model (JAK2/STAT5, Bachmann et al.), PP and ENS yield better accuracy than FIM. However, even for PP and ENS, confidence levels do not cover the truth.
Outcome O2
The computational cost of the three models is differing, especially for large problems: FIM (small), ENS (intermediate), PP (high)
Study design and evidence level
General aspects
Synthetic data is generated for two examplary ODE models given a true parameter set. One model is smaller (5 parameter, 5 states, 5 observables) and is rather used as sanity check, the other is larger (27 parameter, 25 states, 20 observables) and considered more realistic. For FIM and ENS the MATLAB version of the MEIGO toolbox (Egea et al.) was used for parameter estimation, whereas for PP it was used MATLAB parameter estimation toolbox PESTO (Stapor et al.).
Design for Outcome O1
The sample correlation coefficient is used to quantify the agreement between predicted and true state trajectories / trajectory errors. The different methods are compared for the different ODE models, respectively.
Design for Outcome O2
Computation time is compared for the different models.
Further comments and aspects
The so far missing assessment of Prediction Profile Likelihood (Kreutz et al., Hass et al.) as further prediction uncertainty quantification method is planned to be presented "in the short term".
