.. _ErrorModel:


Error model
-----------


An analytic model for the error of the target vector entries. The model :math:`\eta_i({\rm RSE}, y_i, \epsilon_i)` for each entry :math:`i=1,\dots,K` of the target vector can depend on the residual standard error (RSE), the model prediction :math:`y_i` and the corresponding initially assumed measurement error :math:`\epsilon_i`.   

The RSE is defined as :math:`\sqrt{\chi_\text{min}^2/\mathrm{DOF}}` where the number of degrees of freedom (DOF) is defined as the difference between the target-vector dimension :math:`K` and the number parameters to reconstruct (the dimension of the design space).  

Moreover, :math:`M` other random parameters :math:`\mathbf{d}` can be defined in the :ref:`parameter distribution<ActiveLearning.NegLogProbability.error_model.distributions>` and :ref:`initial_parameters<ActiveLearning.NegLogProbability.error_model.initial_parameters>`.  

The log-probability density of the random parameters :math:`\mathbf{d}` and the design space parameters :math:`p` is given as (up to constant additive terms)  

.. math::  

   \log[P(\mathbf{p}, \mathbf{d})] = -\frac{1}{2} \cdot    \sum_i\left(\frac{(t_i-y_i(\mathbf{p}))^2}{\eta_i^2(\mathbf{d})} +    \log[\eta_i^2(\mathbf{d})]\right) + \log[P_\text{prior}(\mathbf{p})] +    \log[P_\text{prior}(\mathbf{d})].

.. _ActiveLearning.NegLogProbability.error_model.expression:


expression (str)
""""""""""""""""
   String defining the error model as a function of the model values ``y_model`` (``y_model`` = :math:`y_i` ) and the initially assumed error ``err0`` (``err0`` = :math:`\epsilon_i` or ``err0`` = :math:`\sqrt{G_{ii}}`).   
   
   .. note::    If not specified, the default error model is ``RSE*err0`` is    equivalent to the standard least-squares error estimate.  
   
   The expression can contain numbers as either integers or floating-point reals in standard or exponential notation (``0.0012`` or ``1.2E-3``) as well as the constants ``E`` (Euler constant) and ``PI`` (:math:`\pi`). Parameter names may not start with the capital letters "``E``" nor "``PI``". Supported operations are ``+``, ``-``, ``*``, ``/``, ``^``, supported single-argument functions are ``sqrt``, ``sinh``, ``sin``, ``asinh``, ``asin``, ``cosh``, ``cos``, ``acosh``, ``acos``, ``tanh``, ``tan``, ``atanh``, ``atan``, ``exp``, ``log10``, ``log``, ``erfinv``, ``erf``, ``abs``, ``trunc``, ``round``, ``sign``, and supported multi-argument functions are ``max``, ``min``. 

   Default: ``'RSE*err0'``

   .. admonition:: Example

      Example with error contribution proportional  'to model value.

      .. code-block:: python

         'sqrt((a*y_model)^2 + b^2)'


.. _ActiveLearning.NegLogProbability.error_model.distributions:


distributions (list[dict])
""""""""""""""""""""""""""
   List of distributions for all error model parameters. 

   Default: ``[]``

   .. admonition:: Example

      A multivariate normal distribution over two parameters ``'a'``, ``'b'`` and a uniform distribution over a third parameter ``'c'``.

      .. code-block:: python

         [{'type': 'mvn', 'parameters': ['a','b'], 'mean': [0.0,0.0], 'covariance': [[1.0,0.4],
          [0.4,2.0]]},
          {'type': 'uniform', 'parameter': 'c', 'domain': [-2.0,2.0]}]

   Each element of the list must be a dict. The dict entry ``type`` specifies the type of the element. The remaining entries  specify its properties. In the following, all possible  list element types are described:

      **Uniform distribution** (type ``'uniform'``): Uniform distribution in a specific domain.      
      
      See :ref:`uniform configuration <Uniform>` for details.

      **Normal distribution** (type ``'normal'``): Normal distribution of specified mean and standard deviation.      
      
      See :ref:`normal configuration <Normal>` for details.

      **Gamma distribution** (type ``'gamma'``): Gamma distribution defined by rate and shape parameter.      
      
      See :ref:`gamma configuration <Gamma>` for details.

      **Distribution over discrete values** (type ``'discrete'``): Distribution over discrete values of a discrete or categorial parameter.      
      
      See :ref:`discrete configuration <DiscreteProbabilities>` for details.

      **Multivariate normal distribution** (type ``'mvn'``): Multivariate normal defined my mean vector and covariance matrix.      
      
      See :ref:`mvn configuration <MultivariateNormal>` for details.

.. toctree::
   :maxdepth: 100 
   :hidden:
   
   Uniform
   Normal
   Gamma
   DiscreteProbabilities
   MultivariateNormal



.. _ActiveLearning.NegLogProbability.error_model.initial_parameters:


initial_parameters (list[float])
""""""""""""""""""""""""""""""""
   Initial values of the random model parameters to perform the optimization. 

   Default: If not specified, the value with maximum prior probability is chosen as initial sample.

   .. admonition:: Example

      The initial value of :math:`M=2` parameters.

      .. code-block:: python

         [0.5,0.2]


.. _ActiveLearning.NegLogProbability.error_model.parameter_bounds:


parameter_bounds (list[list[float]])
""""""""""""""""""""""""""""""""""""
   List of tuples of lower and upper bounds used for the optimization of the model parameters. It is recommended to specify bound in order to avoid invalid values of the model value. 

   Default: If not specified, an unbounded optimization is performed.

   .. admonition:: Example

      Bounds for :math:`M=3` parameters.

      .. code-block:: python

         [[-1.0,1.0],
          [-2.0,2.0],
          [-3.0,1.0]]


.. _ActiveLearning.NegLogProbability.error_model.xtol:


xtol (float)
""""""""""""
   The model parameters are fitted by maximizing the log-probability density using ``scipy.optimize.minimize``. The average relative error parameter `xtol` is one of the convergence criteria of the 'Newton-CG' method.  Smaller values correspond to more accurate error model parameters, but longer fitting times that lead to longer sample computation times. 

   Default: ``1e-05``

.. _ActiveLearning.NegLogProbability.error_model.maxiter:


maxiter (int)
"""""""""""""
   Maximum number of iterations before the termination of the error model parameter fit. 

   Default: :math:`200\cdot M` where :math:`M` is the number of random error model parameters.