.. _ScipyLeastSquares: ================================================ ScipyLeastSquares ================================================ Contents ========= :`Purpose`_: The purpose of the driver. :`Tutorials`_: Tutorials demonstrating the application of this driver. :`Driver Interface`_: Driver-specific methods of the Matlab interface. :`Configuration`_: Configuration of the driver. Purpose ======= The driver solves the least squares problem based on the `scipy` function ``scipy.optimize.least_squares``. It is suited to search locally for a solution of a least-squares problem, which consists of a target vector :math:`\mathbf{t} \in \mathbb{R}^K` and a vectorial function :math:`\mathbf{f}_{\rm bb}(\mathbf{p}_{\rm design}) \in \mathbb{R}^K`. The goal is to find design parameters :math:`\mathbf{p}_{\rm design}` that minimize the chi-squared deviation .. math:: \chi^2 = \sum_{i=1}^K \frac{\left(t_i - y_i\right)^2}{\eta_i^2}. between `K` outputs :math:`y_i = f_i(\mathbf{p}_{\rm design})` to the `K` targets :math:`t_i` scaled by the target uncertainties :math:`\eta_i`. If the target uncertainties are not independent but correlated, it is also possible to specify the covariance matrix :math:`G` of the targets. In this case, the chi-squared deviation is given as .. math:: \chi^2 = \sum_{i,j=1}^K (t_i - y_i) G_{i j}^{-1} (t_j - y_j). In order to perform search for the global minimum, the driver allows to run local minimizations starting from multiple initial points in parallel. A global minimization with this driver is advisable if the Jacobian of the inexpensive vectorial function is known. Available optimization methods are the Trust Region Reflective algorithm (trf) and the dogleg algorithm with rectangular trust regions (dogbox) which are described in more detail in the `reference `_ of ``scipy.optimize.least_squares``. For **expensive** vectorial functions with evaluation times larger than a few seconds, the :ref:`BayesianLeastSquares` driver is recommended. Tutorials ========= .. toctree:: ../tutorials/scipy_least_squares Driver Interface ================ The driver instance can be obtained by :attr:`.Study.driver`. .. currentmodule:: jcmoptimizer .. autoclass:: ScipyLeastSquares best_sample """"""""""""""""""""""""""""""""""" .. automethod:: LeastSquaresDriver.best_sample :noindex: describe """"""""""""""""""""""""""""""""""" .. automethod:: Driver.describe :noindex: get_state """"""""""""""""""""""""""""""""""" .. automethod:: Driver.get_state :noindex: historic_parameter_values """"""""""""""""""""""""""""""""""" .. automethod:: Driver.historic_parameter_values :noindex: min_objective """"""""""""""""""""""""""""""""""" .. automethod:: LeastSquaresDriver.min_objective :noindex: uncertainties """"""""""""""""""""""""""""""""""" .. automethod:: LeastSquaresDriver.uncertainties Configuration ============= The configuration parameters can be set by calling, e.g. .. code-block:: matlab study.configure('example_parameter1',[1,2,3], 'example_parameter2',true); This driver requires a definition of a :ref:`target_vector` :math:`[t_1,\cdots, t_K]` to find a least-squares solution that minimizes :math:`\chi^2 = \sum_{i=1}^K \left(t_i - y_i\right)^2` where :math:`y_i = f_i(\mathbf{p}_{\rm design})` is the :math:`i`-th entry of the observed black-box function. To perform a weighted least-squares minimization one can specify an :ref:`uncertainty_vector` or more generally a :ref:`covariance_matrix` between the :math:`K` target-vector entries. The driver allows to run :ref:`num_initial` minimizations starting from multiple :ref:`initial samples ` in parallel. Only continous design parameters are minimized while discrete and categorial parameters are fixed to the values of the initial samples. .. _ScipyLeastSquares.max_iter: max_iter (int) """""""""""""" Maximum number of evaluations of the studied system. Default: Infinite number of evaluations. .. _ScipyLeastSquares.max_time: max_time (float) """""""""""""""" Maximum run time of study in seconds. The time is counted from the moment, that the parameter is set or reset. Default: ``inf`` .. _ScipyLeastSquares.num_parallel: num_parallel (int) """""""""""""""""" Number of parallel evaluations of the studied system. Default: ``1`` .. _ScipyLeastSquares.min_val: min_val (float) """"""""""""""" The minimization is stopped when the chi-squared deviation :math:`\chi^2` is below the specified minimum value. Default: ``0.0`` .. _ScipyLeastSquares.min_step: min_step (float) """""""""""""""" The minimization is stopped when the Eucledian distance between consecutive sampling points in the design parameter space is below the specified value. Default: ``0.0`` .. _ScipyLeastSquares.target_vector: target_vector (cell{float}) """"""""""""""""""""""""""" Vector of targets :math:`t_i` for :math:`i=1,\dots, K`. Default: ``{0.0}`` .. _ScipyLeastSquares.uncertainty_vector: uncertainty_vector (cell{float}) """""""""""""""""""""""""""""""" Vector of target uncertainties :math:`\eta_i` such that :math:`\chi^2 = \sum_{i=1}^K \frac{(t_i - y_i)^2}{\eta_i^2}`. Default: vector of ones. .. _ScipyLeastSquares.covariance_matrix: covariance_matrix (cell{cell{float}}) """"""""""""""""""""""""""""""""""""" Covariance matrix :math:`G` such that :math:`\chi^2 = \sum_{i,j=1}^K (t_i - y_i) G_{i j}^{-1} (t_j - y_j).`. Default: diagonal identity matrix. .. _ScipyLeastSquares.num_initial: num_initial (int) """"""""""""""""" Number of independent initial optimizers. Default: ``1`` .. _ScipyLeastSquares.max_num_optimizers: max_num_optimizers (int) """""""""""""""""""""""" If an optimizer has converged, it is restarted at another position. If ``max_num_optimizers`` have converged, the optimization is stopped. Default: Infinite number of optimizers. .. _ScipyLeastSquares.initial_samples: initial_samples (cell{cell}) """""""""""""""""""""""""""" List of initial samples taken as starting points. If ``num_initial > len(initial_samples)`` the rest of the starting points is chosen randomly. Default: ``{}`` .. _ScipyLeastSquares.sobol_sequence: sobol_sequence (bool) """"""""""""""""""""" If true, all initial samples are taken from a Sobol sequence. This typically improves the coverage of the parameter space. Default: ``true`` .. _ScipyLeastSquares.method: method (str) """""""""""" Algorithm to perform minimization. :trf: Trust Region Reflective algorithm, particularly suitable for large sparse problems with bounds. Generally robust method. :dogbox: dogleg algorithm with rectangular trust regions, typical use case is small problems with bounds. Not recommended for problems with rank-deficient Jacobian. For more details on the algorithms, see the `scipy documentation `_. Default: ``"trf"`` Choices: ``"trf"``, ``"dogbox"``. .. _ScipyLeastSquares.jac: jac (bool) """""""""" If true, the Jacobian is used for optimization. Default: ``false`` .. admonition:: Example If set to true, the full Jacobian must be added to the observations. That is, for each continuous parameter one has to bass a list of :math:`K` derivatives by calling: .. code-block:: matlab observation.add('value',deriv_vec, 'derivative','param_name') .. _ScipyLeastSquares.diff_step: diff_step (cell{float}) """"""""""""""""""""""" Determines the relative step size for the finite difference approximation of the Jacobian. The actual step is computed as ``x * diff_step``. Default: list with all entries ``1e-6``. .. _ScipyLeastSquares.ftol: ftol (float) """""""""""" Tolerance for termination by the change of the cost function. The optimization process is stopped when ``dF < ftol * F``, and there was an adequate agreement between a local quadratic model and the true model in the last step. Default: ``1e-08`` .. _ScipyLeastSquares.xtol: xtol (float) """""""""""" Tolerance for termination by the change of the independent variables. The optimization process is stopped when ``norm(dx) < xtol * (xtol + norm(x))``. Default: ``1e-08`` .. _ScipyLeastSquares.gtol: gtol (float) """""""""""" Tolerance for termination by the norm of the gradient. Default is 1e-8. The exact condition depends on a `method` used: * For 'trf' : ``norm(g_scaled, ord=np.inf) < gtol``, where ``g_scaled`` is the value of the gradient scaled to account for the presence of the bounds. * For 'dogbox' : ``norm(g_free, ord=np.inf) < gtol``, where ``g_free`` is the gradient with respect to the variables which are not in the optimal state on the boundary. For more details see the `scipy documentation `_. Default: ``1e-08``