Approximately 2 years ago, my lab was firmly entrenched in what we now refer to as ‘calibration hell’. Each of us, myself included, was caught in a near-endless cycle of attempting to calibrate new mathematical models to diverse temporal datasets. This process ended up taking months (yes, in some cases months) of manual effort. We changed model inputs, sampled parameter space, ran the model, compared model outcomes against experimental data, changed model inputs, sampled parameter space, ran the model, …. etc. over and over again.
Personally, I grew extremely frustrated with manual calibration — even with the advantage of stratified sampling schemes, this required tons of effort. Surely there was a way to automate this process? Scanning the literature, I was not unique in this thought. There are countless calibration methods that employ optimization schemes to calibrate mathematical models to experimental data trend lines. Unfortunately, we wanted to capture a range of outcomes across time, so these methods were not quite right for our calibration goals.
Of course, I also identified a suite of techniques that fit under the ‘Bayesian Calibration’ methods umbrella. Techniques like sample importance resampling (SIR) do a phenomenal job of relating parameter distributions to experimental outcome distributions in order to identify a calibrated parameter space. However, for our datasets, we were not comfortable assigning distributions to the experimental data at each timepoint. Additionally, and as a matter of personal preference, I did not like the discretization of parameter space that bayesian methods provide as a calibrated space. Essentially they identify pockets within the original parameter space where your model outcomes agree with experimental outcomes, and sample those regions heavily over and over again. For my purposes, I wanted a continuous, smooth calibrated parameter space.
To automate/quicken calibration given these types of constraints, I developed CaliPro. Published in Cellular and Molecular Bioengineering, CaliPro is a model-agnostic calibration protocol to find a continuous region of parameter space wherein model outcomes recapitulate a range of experimental data outcomes across time.
In addition to publishing the new method, I gave a presentation at the Department of Computational Medicine and Bioinformatic’s Tools and Technology Seminar Series. You can watch the recording below:
Please reach out if you have any questions about the method or implementation!
Louis R. Joslyn 