.. _BayesianLeastSquares: ================================================ BayesianLeastSquares ================================================ Contents ========= :`Purpose`_: The purpose of the driver. :`Tutorials`_: Tutorials demonstrating the application of this driver. :`Driver Interface`_: Driver-specific methods of the Python interface. :`Configuration`_: Configuration of the driver. Purpose ======= The driver solves the least squares problem based on the Bayesian optimization approach. It is suited to search globally for a solution of a least-squares problem, which consists of a target vector :math:`\mathbf{t} \in \mathbb{R}^K` and a vectorial function :math:`\mathbf{f}_{\rm bb}(\mathbf{p}_{\rm design}) \in \mathbb{R}^K`. The goal is to find design parameters :math:`\mathbf{p}_{\rm design}` that minimize the chi-squared deviation .. math:: \chi^2 = \sum_{i=1}^K \frac{\left(t_i - y_i\right)^2}{\eta_i^2}. between `K` outputs :math:`y_i = f_i(\mathbf{p}_{\rm design})` to the `K` targets :math:`t_i` scaled by the target uncertainties :math:`\eta_i`. If the target uncertainties are not independent but correlated, it is also possible to specify the covariance matrix :math:`G` of the targets. In this case, the chi-squared deviation is given as .. math:: \chi^2 = \sum_{i,j=1}^K (t_i - y_i) G_{i j}^{-1} (t_j - y_j). For **inexpensive** vectorial functions with evaluation times smaller than a seconds, and for functions with known Jacobian and moderate evaluation times, the :ref:`ScipyLeastSquares` driver is recommended. Tutorials ========= .. toctree:: ../tutorials/bayesian_least_squares Driver Interface ================ The driver instance can be obtained by :attr:`.Study.driver`. .. currentmodule:: jcmoptimizer .. autoclass:: BayesianLeastSquares :members: :inherited-members: Configuration ============= The configuration parameters can be set by calling, e.g. .. code-block:: python study.configure(example_parameter1 = [1,2,3], example_parameter2 = True) This driver requires a definition of a :ref:`target_vector` :math:`[t_1,\cdots, t_K]` to find a least-squares solution that minimizes :math:`\chi^2 = \sum_{i=1}^K \left(t_i - y_i\right)^2` where :math:`y_i = f_i(\mathbf{p}_{\rm design})` is the :math:`i`-th entry of the observed black-box function. To perform a weighted least-squares minimization one can specify an :ref:`uncertainty_vector` or more generally a :ref:`covariance_matrix` between the :math:`K` target-vector entries. .. _BayesianLeastSquares.max_iter: max_iter (int) """""""""""""" Maximum number of evaluations of the studied system. Default: Infinite number of evaluations. .. _BayesianLeastSquares.max_time: max_time (float) """""""""""""""" Maximum run time of study in seconds. The time is counted from the moment, that the parameter is set or reset. Default: ``inf`` .. _BayesianLeastSquares.num_parallel: num_parallel (int) """""""""""""""""" Number of parallel evaluations of the studied system. Default: ``1`` .. _BayesianLeastSquares.target_vector: target_vector (list[float]) """"""""""""""""""""""""""" Vector of targets :math:`t_i` for :math:`i=1,\dots, K`. Default: ``[0.0]`` .. _BayesianLeastSquares.uncertainty_vector: uncertainty_vector (list[float]) """""""""""""""""""""""""""""""" Vector of target uncertainties :math:`\eta_i` such that :math:`\chi^2 = \sum_{i=1}^K \frac{(t_i - y_i)^2}{\eta_i^2}`. Default: vector of ones. .. _BayesianLeastSquares.covariance_matrix: covariance_matrix (list[list[float]]) """"""""""""""""""""""""""""""""""""" Covariance matrix :math:`G` such that :math:`\chi^2 = \sum_{i,j=1}^K (t_i - y_i) G_{i j}^{-1} (t_j - y_j).`. Default: diagonal identity matrix. .. _BayesianLeastSquares.parameter_distribution: parameter_distribution (dict) """"""""""""""""""""""""""""" Probability distribution of design and environment parameters. Default: ``{'include_study_constraints': False, 'distributions': [], 'constraints': []}`` Probability distribution of design and environment parameters defined by distribution functions and constraints. The definition of the parameter distribution can have several effects: * In a call to the method ``get_statistics`` of the driver interface the value of interest is averaged over samples drawn from the space distribution. * In a call to the method ``run_mcmc`` of the driver interface the space distribution acts as a prior distribution. * In a call to the method ``get_sobol_indices`` of the driver interface the space distribution acts as a weighting factor for determining expectation values. * In an :ref:`ActiveLearning` driver, one can access the value of the log-probability density (up to an additive constant) by the name ``'log_prob'`` in any expression, e.g. in :ref:`ExpressionVariable`, :ref:`LinearCombinationVariable`. See :ref:`parameter_distribution configuration ` for details. .. toctree:: :maxdepth: 100 :hidden: SpaceDistribution2 .. _BayesianLeastSquares.scaling: scaling (float) """"""""""""""" Scaling parameter of the model uncertainty. For scaling :math:`\gg 1.0` (e.g. ``scaling=10.0``) the search is more explorative. For scaling :math:`\ll 1.0` (e.g. ``scaling=0.1``) the search becomes more greedy (e.g. any local minimum is intensively exploited). Default: ``1.0`` .. _BayesianLeastSquares.optimization_step_min: optimization_step_min (int) """"""""""""""""""""""""""" Minimum number of observations of the objective before the hyperparameters are optimized. Default: Automatic choice according to number of dimensions. .. note:: Each derivative observation is counting as an independent observation. .. _BayesianLeastSquares.optimization_step_max: optimization_step_max (int) """"""""""""""""""""""""""" Maximum number of observations of the objective after which no more hyperparameter optimization is performed. Default: ``100`` .. note:: Each derivative observation is counting as an independent observation. .. _BayesianLeastSquares.min_optimization_interval: min_optimization_interval (int) """"""""""""""""""""""""""""""" Minimum number of observations of the objective after which the hyperparameters are optimized. Default: ``2`` .. note:: Each derivative observation is counting as an independent observation. .. _BayesianLeastSquares.max_optimization_interval: max_optimization_interval (int) """"""""""""""""""""""""""""""" Maximum number of observations of the objective after which the hyperparameters are optimized. Default: ``20`` .. note:: Each derivative observation is counting as an independent observation. .. _BayesianLeastSquares.optimization_level: optimization_level (float) """""""""""""""""""""""""" Controls how often the hyper-parameters are optimized. Small values (e.g. 0.01) lead to more frequent optimizations. Large values (e.g. 1) lead to less frequent optimizations. Default: ``0.2`` .. _BayesianLeastSquares.approximate: approximate (bool) """""""""""""""""" If true, the generalized chi-squared variable with different uncertainties of the predictions of each channel `K` is approximated by a chi-squared variable with averaged uncertainties. This allows to analytically compute probability densities and any acquisition function that is directly based on the variable. Default: ``True`` .. _BayesianLeastSquares.effective_DOF: effective_DOF (float) """"""""""""""""""""" Number of effective degrees of freedom (DOF) used for stochastic variable of the chi-squared distribution. This number roughly indicates how many output channels of the forward model are statistically independent. Default: If not specified, the value us determined automatically. .. note:: If ``approximate`` is false, this parameter has no effect. .. _BayesianLeastSquares.effective_DOF_bounds: effective_DOF_bounds (list[float]) """""""""""""""""""""""""""""""""" The number of effective degrees of freedom (DOF) is determined by a maximum likelihood estimate within the given lower and upper bounds. Default: ``[20.0,50.0]`` .. note:: If ``approximate`` is false or the ``effective_DOF`` is set manually, this parameter has no effect. .. _BayesianLeastSquares.min_val: min_val (float) """"""""""""""" The minimization of the objective is stopped when the observed objective value is below the specified minimum value. Default: ``-inf`` .. _BayesianLeastSquares.min_uncertainty: min_uncertainty (float) """"""""""""""""""""""" The study is stopped if the uncertainty (square root of variance) of the objective at the last 5 sampling points was below ``min_uncertainty``. Default: ``0.0`` .. _BayesianLeastSquares.num_training_samples: num_training_samples (int) """""""""""""""""""""""""" Number of pseudo-random initial samples before the samples are drawn according to the acquisition function. Default: Automatic choice depending depending on dimensionality of design space. .. _BayesianLeastSquares.min_PoI: min_PoI (float) """"""""""""""" The study is stopped if the maximum probability of improvement (PoI) of the last 5 iterations is below ``min_PoI``. Default: ``0.0`` .. _BayesianLeastSquares.adaptive_local_search: adaptive_local_search (bool) """""""""""""""""""""""""""" The maximization of the acquisition function consists of a global heuristic search followed by a local convergence of the best samples. By default, the local search effort adapts to the value of ``max_num_model_evals``. In some cases far better samples are computed, if the local search is only stopped when convergence to a local minimum was obtained. Default: ``False`` .. _BayesianLeastSquares.max_num_model_evals: max_num_model_evals (int) """"""""""""""""""""""""" Maximum number of evaluations of the surrogates for finding the maximum of the acquisition function. Default: ``1500`` .. _BayesianLeastSquares.num_iterative_solver_steps: num_iterative_solver_steps (int) """""""""""""""""""""""""""""""" Gaussian process regression requires to solve systems of linear equations. In order to improve the accuracy of direct solution methods, an iterative refinement can be performed with the given number of iterations. Default: ``5``