.. _bayesian_least_squares:

Solution of least-square problem using Bayesian optimization
==========================================================================================


:Driver: :ref:`BayesianLeastSquares`


:Download script: :download:`bayesian_least_squares.py`


The target of the study is to showcase the solution of a non-linear least-squares problem from the `NIST statistical reference datasets <https://www.itl.nist.gov/div898/strd/>`_. As an example, the `MGH17 problem <https://www.itl.nist.gov/div898/strd/nls/data/mgh17.shtml>`_ is considered which consists of fitting a vectorial model :math:`\mathbf{f}(\mathbf{b}) \in \mathbb{R}^{33}` with

.. math::
   f_i(\mathbf{b}) = b_1 + b_2 \exp(-i \cdot b_4) + b_3 \exp(-i \cdot b_5),\, i = 0, \dots, 32

to a target vector with 33 entries. The certified best-fit values are

.. math::
   b_1 &= 0.3754100521 \pm 2.0723153551 \cdot 10^{-3}&

   b_2 &= 1.9358469127 \pm 0.22031669222&
   
   b_3 &= -1.464687136 \pm 0.22175707739&
 
   b_4 &= 0.1286753464 \pm 4.4861358114\cdot 10^{-3}&
 
   b_5 &= 0.2212269966 \pm 8.9471996575 \cdot 10^{-3}&



.. literalinclude:: ./bayesian_least_squares.py
   :language: python
   :linenos:


 

.. figure:: images/bayesian_least_squares/corner_analytic.svg
   :alt: MCMC sampling based on surrogate

   Markov-Chain Monte-Carlo (MCMC) sampling of the probability density of the
   parameters :math:`b_1,\dots,b_5` based on the analytic model function.

.. figure:: images/bayesian_least_squares/corner_surrogate.svg
   :alt: MCMC sampling based on surrogate

   Markov-Chain Monte-Carlo (MCMC) sampling of the probability density of the
   parameters :math:`b_1,\dots,b_5` based on the trained surrogate of the study.
   A comparison between the analytic and the surrogate model function shows
   a good quantitative agreement.