Date Published: June 3, 2019
Publisher: Public Library of Science
Author(s): Daniel Lill, Jens Timmer, Daniel Kaschek, Timon Idema.
When non-linear models are fitted to experimental data, parameter estimates can be poorly constrained albeit being identifiable in principle. This means that along certain paths in parameter space, the log-likelihood does not exceed a given statistical threshold but remains bounded. This situation, denoted as practical non-identifiability, can be detected by Monte Carlo sampling or by systematic scanning using the profile likelihood method. In contrast, any method based on a Taylor expansion of the log-likelihood around the optimum, e.g., parameter uncertainty estimation by the Fisher Information Matrix, reveals no information about the boundedness at all. In this work, we present a geometric approach, approximating the original log-likelihood by geodesic coordinates of the model manifold. The Christoffel Symbols in the geodesic equation are fixed to those obtained from second order model sensitivities at the optimum. Based on three exemplary non-linear models we show that the information about the log-likelihood bounds and flat parameter directions can already be contained in this local information. Whereas the unbounded case represented by the Fisher Information Matrix is embedded in the geometric framework as vanishing Christoffel Symbols, non-vanishing constant Christoffel Symbols prove to define prototype non-linear models featuring boundedness and flat parameter directions of the log-likelihood. Finally, we investigate if those models could allow to approximate and replace computationally expensive objective functions originating from non-linear models by a surrogate objective function in parameter estimation problems.
Parameter estimation by the maximum-likelihood method has numerous applications in different fields of physics, engineering, and other quantitative sciences. In systems biology, e.g., ordinary differential equation (ODE) models are used to describe cell-biological processes [1, 2]. Parameter estimation in these non-linear models can easily become time-consuming. Solving the ODEs and computing model sensitivities for numerical optimization is computationally demanding. The difficulty is further increased if many experimental conditions contribute to the evaluation of the likelihood function because the model ODE needs to be solved independently for each condition.
The methods described above can be used to approximate χ2 in a new way to allow for regions in P where χ2 is bounded.
We established a connection between the Christoffel Symbols of a model and its boundedness. While models with globally vanishing Christoffel Symbols, i.e. linear models, are representatives of unbound models, models with constant Christoffel Symbols are bound in certain parameter directions and can therefore be considered as simple representatives of bounded models. We explored a possible application of this class of models as an approximation to non-linear least-squares, for which information about boundedness can be of importance in parameter uncertainty assessment. We now discuss various additional remarks concerning the quality, applicability and use of the approximation.
Parameter estimation in non-linear models is based on the optimization of an objective function, here denoted as χ2, which is non-quadratic, possibly non-convex, and features certain directions in which its values are bounded. This means that the position of the optimum in parameter space or the Hessian matrix around the optimum represent only a fraction of the information necessary to determine confidence intervals and relationships between model parameters in the case of practical non-identifiability.