Interface for Python

The following section contains the documentation for using the Python interface that allows to set up, run studies and analyze their results.

Overview of Python classes

The interface consists of the following Python classes

Class	Purpose
`Server`	Class for for starting and stopping the optimization server.
`Client`	Class for connecting to server, creating studies, and creating benchmarks.
`Study`	Class for configuring, running and analyzing results of a study
`Observation`	Container for observations to be sent to the server.
`Suggestion`	Suggestion retrieved by a study to be evaluated.
`Driver`	Interface to driver-specific methods.
`Benchmark`	Class for benchmarking different studies against each other

Documentation

class jcmoptimizer.Server(jcm_optimizer_path=None, port=None, persistent=False, timeout=40.0, max_retries=1)[source]

This class provides methods for starting and stopping a self-hosted optimization server on the local computer. Example:

server = Server()
client = Client(host=server.host)
study = client.create_study(...)

Parameters:

jcm_optimizer_path (Optional[str]) – The path of the JCMoptimizer installation.
port (Optional[int]) – The port that the optimization server is listening to. If not specified, the port is chosen automatically.
persistent (bool) –
If true, the server continues to run even after the Python script has finished. To shutdown a local server later on, one can reconnect to it:
```
client = Client(host="http://localhost:4554")
client.shutdown_server()
```
timeout (float) – The maximum amount of time to wait for the server startup.
max_retries (int) – The maximum number of attempts to start the server after a timeout.

property host: str: The host name of the server

property pid: int: The process id of the server.

property port: int: The port that the server is listening on

shutdown(force=False)[source]

Shuts down the optimization server.

Parameters:: force (bool) – If true, the optimization server is closed even if a study is not yet finished.
Return type:: None

class jcmoptimizer.Client(host, verbose=True, check=True)[source]

This class provides methods for creating new optimization studies. Example:

client = Client("http://localhost:4554")
design_space = [
   {'name': 'x1', 'type': 'continuous', 'domain': (-1.5,1.5)},
   {'name': 'x2', 'type': 'continuous', 'domain': (-1.5,1.5)},
]
study = client.create_study(design_space=design_space, name='example')

Parameters:

host (str) – The host name under which the optimization server can be contacted. For example, 'http://localhost:4554'
verbose (bool) – If true, messages from the optimization server are printed out.
check (bool) – If true, check if the optimization server can be contacted via the given host name.

check_server()[source]

Checks if the optimization server is running. Example:

>>> client.check_server()
Polling server at http://localhost:4554
Optimization server is running

Return type:: None

create_benchmark(benchmark_id=None, num_average=6, remove_completed_studies=False)[source]

Creates a new Benchmark object for benchmarking different optimization studies against each other. Example:

benchmark = client.create_benchmark(num_average=6)

Parameters:

benchmark_id (Optional[str]) – A unique identifier of the benchmark.
num_average (int) – Number of study runs to determine average study performance
remove_completed_studies (bool) – If true, studies that are completed (i.e. some stopping criterion was met) are removed from the server as long as no other client holds a handle to the study.

Return type:

Benchmark

create_study(design_space=None, environment=None, constraints=None, name=None, study_id=None, driver='ActiveLearning', save_dir=None, output_precision=1e-10, dashboard=True, open_browser=True, **kwargs)[source]

Creates a new Study instance. Example:

study = client.create_study(
    design_space=design_space,
    name='example',
    study_id="example_01"
)

Parameters:

design_space (Optional[list[dict[str, Any]]]) –
List of domain definitions for each parameter. A design space definition consists of a list of dictionary with the entries

name:

Name of the parameter. E.g. 'x1'. The name should contain no spaces and must not be equal to function names or mathematical constants like 'sin', 'cos', 'exp', 'pi' etc.

type:

Type of the parameter. Either 'continuous', 'discrete', or 'categorial'.

domain:

The domain of the parameter. For continuous parameters this is a tuple (min, max). For discrete parameters this is a list of values, e.g. [1.0,2.5,3.0]. For categorial inputs it is a list of strings, e.g. ['cat1','cat2','cat3']. Note, that categorial values are internally mapped to integer representations, which are allowed to have a correlation. The categorial values should therefore be ordered according to their similarity. For fixed parameters the domain is a single parameter value.

Example:
```
design_space = [
    {'name': 'x1', 'type': 'continuous', 'domain': (-2.0,2.0)},
    {'name': 'x2', 'type': 'continuous', 'domain': (-2.0,2.0)},
    {'name': 'x3', 'type': 'discrete', 'domain': [-1.0,0.0,1.0]},
    {'name': 'x4', 'type': 'categorial', 'domain': ['a','b']}
]
```
If not specified, the design space configuration from the study history is used. If no historic information exists, an error is raised.
environment (Optional[list[dict[str, Any]]]) –
Environment parameters are those which influence the behaviour of the system, but are not design parameters. Examples are uncontrollable environment parameters (e.g. temperature, time) or parameters that are scanned in each evaluation (e.g. wavelength, angle) for each run of the system can be described by environment parameters. Alternatively, scans can be described by surrogates that are trained on multiple inputs (one for each scan value). The environment definition consists of a list of dictionary with the entries:

name:

Name of the parameter. E.g. 'x1'. The name should contain no spaces and must not be equal to function names or mathematical constants like 'sin', 'cos', 'exp', 'pi' etc.

type:

Type of the parameter. Either 'variable' or 'fixed'. Fixed parameters can be used in the constraint functions or other expressions.

domain:

The domain of the parameter. For fixed parameters, this is a single value, for variable parameters this can be a tuple (min, max). If no bounds are specified for variable parameters, the environment is considered to be unconstrained. In this case the scale has to be set.

scale:

The scale at which environment parameter changes affect the system. If the environment parameter describes unknown drifts and aging effects, the length scale is equal to the timescale at which the system behaviour changes due to drifts or aging.

Example:
```
environment = [
    {'name': 'wavelength', 'type': 'continuous', 'domain': (300, 600)},
    {'name': 'time', 'type': 'continuous', 'scale': 0.1},
    {'name': 'radius', 'type': 'fixed', 'domain': 1.5}
]
```
If not specified, the environment configuration from the study history is used. If no historic information exists, the environment is assumed to be empty (environment = []).
constraints (Optional[list[dict[str, Any]]]) –
List of constraints on the design space. Each list element is a dictionary with the entries

name:

Name of the constraint.

constraint:

A string defining an inequality constraint in the for a <= b or a >= b. The following operations and functions may be for example used: +,-,*,/,^,sqrt,sin,cos,tan,exp,log,log10,abs,round,sign,trunc. E.g. 'x1^2 + x2^2 <= sin(x1+x2)'. For a more complete list of supported functions, see the expression variable reference.

Example:
```
constraints = [
    {'name': 'circle', 'expression': 'x1^2 + x2^2 <= radius^2'},
    {'name': 'triangle', 'expression': 'x1 >= x2'},
]
```
If not specified, the constraints configuration from the study history is used. If no historic information exists, the list of constraints is assumed to be empty (constraints = []).
study_id (Optional[str]) – A unique identifier of the study. All relevant information on the study are saved in a file named study_id+’.jcmo’ If the study already exists, the design_space, environment, and constraints do not need to be provided. If not set, the study_id is set to a random unique string.
name (Optional[str]) – The name of the study that will be shown in the dashboard.
save_dir (Optional[str]) – The path to a directory, where the study file (jcmo-file) is saved. Default is a directory in the system’s temporary directory.
driver (str) – Driver used for the study. Default is ‘ActiveLearning’. For a list of drivers, see the driver reference.
output_precision (float) –
Precision level for output of parameters.

Note

Rounding the output can potentially lead to a slight breaking of constraints.
dashboard (bool) – If true, a dashboard will be served for the study.
open_browser (bool) – If true, a browser window with the dashboard is started.

Return type:

Study

shutdown_server(force=False)[source]

Shuts down the optimization server. Example:

client.shutdown_server()

Parameters:: force (bool) – If true, the optimization server is closed even if a study is not yet finished.
Return type:: None

class jcmoptimizer.Study(host, study_id, session, driver)[source]

This class provides methods for controlling a numerical optimization study. Example:

def evaluate(study: Study, x1: float, x2: float) -> Observation:
    observation = study.new_observation()
    observation.add(x1**2+x2**2)
    return observation
study.configure(max_iter=30, num_parallel=3)

#Start optimization loop
study.set_evaluator(evaluate)
study.run()

#Alternatively, one can explicitly define an optimization loop
def acquire(suggestion: Suggestion) -> None:
   try: observation = evaluator(study, **suggestion.kwargs)
   except: study.clear_suggestion(suggestion.id, 'Evaluator failed')
   else: study.add_observation(observation, suggestion.id)

while (not study.is_done()):
    suggestion = study.get_suggestion()
    t = Threading.thread(target=acquire, args=(suggestion,))
    t.start()

The constructor should not be used directly since it does not create a study on the server side. Instead, one should use Client.create_study().

add_many(observations, design_values, environment_values=None, acquisition_times=None, check=True)[source]

Adds many observations to the study. Example:

study.add_many(observations, design_values)

Parameters:

observations (list[Observation]) – List of Observation objects for each sample (see new_observation())
design_values (list[list[Union[float, str]]]) – List of design values. E.g. [[0.1, 1.0, 'cat1'], [0.2, 2.0, 'cat2']]
environment_values (Optional[list[list[float]]]) – Optional list of environment values. If not specified, the last known environment values are taken. E.g. [[1.0, 2.0], [1.1, 2.3]]
acquisition_times (Optional[list[float]]) – Optional list of times required to acquire each observation.
check (bool) – If true, the validity of the observation is checked

Return type:

None

Warning

The purpose of this function is to speed up the process of adding many observations to the study. To this end, the intermediate driver states are not computed. This means that all driver-specific historic data (any path of historic_parameter_values() starting with driver…) is incorrect. The same holds for most of the data shown on the dashboard. To avoid this, one has to add the observations one by one using add_observation().

add_observation(observation, suggestion_id=None, design_value=None, environment_value=None, acquisition_time=None, check=True)[source]

Adds an observation to the study. Example:

study.add_observation(observation, suggestion.id)

Parameters:

observation (Observation) – Observation object with added values (see new_observation())
suggestion_id (Optional[int]) – Id of the corresponding suggestion if it exists.
design_value (Optional[list[Union[float, str]]]) – If the observation does not belong to an open suggestion, the corresponding design value must be provided as a list of floats for continuous and discrete parameters or string for categorial parameters. E.g. [0.1, 2.0, 'cat1'].
environment_value (Optional[list[float]]) – If an environment parameters are specified, this specifies the value of variable environment parameters as a list of floats that is valid for all values added to the observation. E.g. [1.0, 2.0]. Alternatively, one can also set different environment values for each entry of the observation (see add()).
acquisition_time (Optional[float]) – If the observation does not belong to an open suggestion, it is possible to specify the time it took to retrieve the observation (e.g. the computation time). This information can be used to adapt the effort of computing the next suggestions.
check (bool) – If true, the validity of the observation is checked

Return type:

None

add_observation_data(data)[source]

Add data from another study to the study. Example:

obs_data = study.get_observation_data()
other_study.add_observation_data(obs_data)

Parameters:: data (dict[str, Any]) – Dict with observation data. See get_observation_data() for the details.
Return type:: None

clear_all_suggestions()[source]

Clear all open suggestions. Example:

study.clear_all_suggestions()

Return type:: None

Note

The study only creates num_parallel suggestions (see configure()) until it waits for an observation to be added (see add_observation()) or a suggestion to be cleared.

clear_suggestion(suggestion_id, message='')[source]

If the evaluation for a certain suggestion fails, the suggestion can be cleared from the study. Example:

study.clear_suggestion(suggestion.id, 'Computation failed')

Note

The study only creates num_parallel suggestions (see configure()) until it waits for an observation to be added (see add_observation()) or a suggestion to be cleared.

Parameters:

suggestion_id (int) – Id of the suggestion to be cleared.
message (str) – An optional message that is printed out.

Return type:

None

property configuration: dict[str, Any]

Return the current configuration for the driver. Example:

config = study.configuration
study2.configure(**config)

configure(num_parallel=1, max_iter=None, max_time=None, **kwargs)[source]

Configures the study for its run. Example:

study.configure(max_iter=100, num_parallel=5)

Parameters:

num_parallel (int) – Number of parallel observations of the evaluator function (default: 1).
max_iter (Optional[int]) – Maximum number of evaluations of the evaluator 2 function (default: inf).
max_time (Optional[float]) – Maximum optimization time in seconds (default: inf).

Return type:

None

Note

The full list of parameters depends on the chosen driver. For a parameter description, see the the corresponding driver in the driver reference.

describe()[source]

Get description of all modules and their parameters that are used by the study. Example:

description = study.describe()
print(description["observation"]["acquisition_time"])
print(description["driver"]["members"]["surrogates"]["0"])

Returns: A dictionary with the root entries: :rtype: dict[str, Any]

driver:

Nested dictionary with description of submodules consisting of a name and a descriptive text. If the entry describes a module, it has an additional "members" entry with dictionaries describing submodules and parameters.

observation:

Dictionary with a description of the parameters of an observation.

suggestion:

Dictionary with a description of the parameters of a suggestion of the driver.

property driver: Driver: The driver of the study. For a documentation see the Driver Reference of the corresponding driver.

get_observation_data()[source]

Get table with data of the observations. This can be used to copy the data manually to another study. Example:

obs_data = study.get_observation_data()
other_study.add_observation_data(obs_data)

Return type:: dict[str, Any]

Returns: Dictionary, where each entry holds an equally long list of observation

data. The keys of the dictionary are:

value:: Observed value of black-box function
derivative:: Name of derivative parameter or None for non-derivative observation
uncertainty:: Uncertainty of observed value or None if no uncertainty
model_name:: Name of the surrogate model that is trained on the data or None
design_value:: Value of design parameters
environment_value:: Value of environment parameters or None if no environment is specified

get_state(path=None)[source]

Get state of the study. Example:

acquisition_time = study.get_state(
    path="observation.acquisition_time"
)

Parameters:: path (Optional[str]) – A dot-separated path to a submodule or parameter. If none, the full state is returned.
Return type:: dict[str, Any]

Returns: If path is None, a dictionary with the following entries

is returned:

driver:: Dictionary with information of driver state.
observation:: Dictionary with information of the latest observation.
suggestion:: Dictionary with information about the suggestion that corresponds to the last observation

Note

A description of the meaning of each entry in the state can be retrieved by describe().

get_suggestion(environment_value=None)[source]

Get a new suggestion to be evaluated by the user. Example:

def evaluate(study: Study, x1: float, x2: float) -> Observation:
   obs = study.new_observation()
   obs.add(x1**2 + x2**2)
   return obs

suggestion = study.get_suggestion()
obs = evaluate(study, **suggestion.kwargs)
study.add_observation(observation=obs, suggestion_id=suggestion.id)

Parameters:: environment_value (Optional[list[float]]) – If an environment is specified, this optional argument specifies the list of variable environment parameter values, for which a suggestion should be computed. E.g. [0.1, 1.2]. If an environment exists and no values are specified, the last known environment values are used.
Return type:: Suggestion

Warning

The function has to wait until the number of open suggestions is smaller than num_parallel before receiving a new suggestion. This can cause a deadlock if no observation is added by an independent thread.

historic_parameter_values(path)[source]

Get the values of an internal parameter for each iteration of the study. Example:

acquisition_times = study.historic_parameter_values(
    path="observation.acquisition_time")

Parameters:: path (str) – A dot-separated path to the parameter.
Return type:: list[Any]

Note

A description of the meaning of each parameter can be retrieved by describe().

is_done()[source]

Checks if the study has finished. Example:

if study.is_done(): break

Return type:: bool

Returns: True if some stopping criterion set by: configure() was met.

Note

Before returning true, the function call waits until all open suggestions have been added to the study.

new_observation()[source]

Create a new Observation object that allows to add data via add(). Example:

observation = study.new_observation()
observation.add(1.2)
observation.add(0.1, derivative='x1')

Return type:: Observation

run()[source]

Run the acquisition loop after the evaluator has been set (see set_evaluator()). The acquisition loop stops after a stopping criterion has been met (see configure()). Example:

study.run()

Return type:: None

set_evaluator(evaluator)[source]

Set the function that maps design parameters to an Observation. Example:

def evaluate(study: Study, x1: float, x2: float) -> Observation:
    observation = study.new_observation()
    observation.add(x1**2 + x2**2)
    return observation
study.set_evaluator(evaluate)

Parameters:: evaluator (Callable[..., Observation]) – Function handle for a function of the variable parameters that returns a corresponding Observation object. The function must accept a "study" argument as well as an argument with the name of each design parameter and fixed environment parameter.
Return type:: None

start_clock()[source]

The optimization stops after the time max_time (see configure()). This function resets the clock to zero. Example:

study.start_clock()

Return type:: None

Note

The clock is also set to zero by calling configure().

class jcmoptimizer.Observation[source]

This class provides a container to collect data to be sent to the optimizer. This includes scalar or vectorial inputs for the objective or surrogate models of an active learning-based driver.

add(value, derivative=None, uncertainty=None, model_name=None, environment_value=None)[source]

Add data to observation. Example:

obs.add(val)
obs.add(dval_dx1, derivative='x1')
obs.add(dval_dx2, derivative='x2')

Parameters:

value (Union[float, list[float]]) – Numerical value or list of values.
derivative (Optional[str]) – If the values are derivatives w.r.t. a design or environment parameter, this should be the name of the parameter.
uncertainty (Union[float, list[float], None]) – The uncertainty of the provided values, i.e. the estimated standard deviation of multiple observations for same parameter.
model_name (Optional[str]) – Name of the surrogate model that is trained on the data. Can be also None for drivers that do not support multiple models.
environment_value (Optional[list[float]]) – List of environment value for all entries of the environment of the study. Optionally, entries for different environment values can be added consecutively, such that also scans of environment values can be collected into one observation.

Return type:

None

class jcmoptimizer.Suggestion(sample, id)[source]

This class provides the sample to be evaluated and the id which is required to identify results for this suggestion. Example:

def evaluate(study: Study, x1: float, x2: float) -> Observation:
   obs = study.new_observation()
   obs.add(x1**2 + x2**2)
   return obs

suggestion = study.get_suggestion()
obs = evaluate(study, **suggestion.kwargs)
study.add_observation(observation=obs, suggestion_id=suggestion.id)

The constructor should not be used directly. Instead, one creates suggestions using Study.get_suggestion().

property id: int: The id of the suggestion.

property kwargs: dict[str, str | float]: Dictionary which maps names of design parameters to their values for which an observation is suggested.

class jcmoptimizer.Benchmark(host, benchmark_id, session, num_average)[source]

This class provides methods for benchmarking different studies against each other that try to minimize the same objective. Example:

benchmark = client.create_benchmark(num_average=6)
benchmark.add_study(study1)
benchmark.add_study(study2)
benchmark.set_evaluator(evaluate)
benchmark.run()
data = benchmark.get_data(x_type='num_evaluations',y_type='objective',
           average_type='mean')
fig = plt.figure(figsize=(8,4))
for idx,name in enumerate(data['names']):
    X = data['X'][idx]
    Y = np.array(data['Y'][idx])
    std_error = np.array(data['sdev'][idx])/np.sqrt(6)
    p = plt.plot(X,Y,linewidth=2.0, label=name)
    plt.fill_between(X, Y-std_error, Y+std_error, alpha=0.2, color = p[0].get_color())
plt.legend(loc='upper right',ncol=1)
plt.grid()
plt.ylim([0.1,10])
plt.rc('font',family='serif')
plt.xlabel('number of iterations',fontsize=12)
plt.ylabel('average objective',fontsize=12)
plt.show()

The constructor should not be used directly since it does not create a benchmark on the server side. Instead, one should use Client.create_benchmark().

add_study(study)[source]

Adds a study to the benchmark. Example:

benchmark.add_study(study1)

Parameters:: study (Study) – A Study object for which a benchmark should be run.
Return type:: None

add_study_results(study)[source]

Adds the results of a benchmark study at the end of an optimization run. Example:

benchmark.add_study_results(study1)

Parameters:: study (Study) – A Study object after the study was run.
Return type:: None

get_data(x_type='num_evaluations', y_type='objective', average_type='mean', invert=False, log_scale=False, minimum=None, scales=None, norm=None, num_samples=100)[source]

Get benchmark data. Example:

data = benchmark.get_data( x_type='num_evaluations', y_type='objective',
     average_type='mean')
plt.plot(data['X'][0],data['Y'][0])

Parameters:

x_type (Literal['num_evaluations', 'time']) – Data on x-axis. Can be either ‘num_evaluations’ or ‘time’. The time data is given in units of seconds.
y_type (Literal['objective', 'distance']) – Data type on y-axis. Can be either ‘objective’, ‘distance’, (i.e. accumulated minimum distance off all samples to overall minimum), or ‘min_distance’ (i.e. distance of current minimum to overall minimum).
average_type (Literal['mean', 'median']) – Type of averaging over study runs. Can be either ‘mean’ w.r.t. x-axis data or ‘median’ w.r.t. y-axis data
invert (bool) – If True, the objective is multiplied by -1. (Parameter not available for distance average types)
log_scale (bool) – If True, the output of Y and sdev are determined as mean and standard deviations of the natural logarithm of the considered y_type.
minimum (Optional[list[float]]) – Vector with minimum position. (Only available for distance average types)
scales (Optional[list[float]]) – Vector with positive weights for scaling distance in different directions. (Only available for distance average types)
norm (Union[str, int, None]) – Order of distance norm as defined in numpy.linalg.norm. (Only available for distance average types)
num_samples (int) – Number of samples on y-axis. (Only available for median average type or time on x-axis)

Return type:

dict[str, list[Union[str, float]]]

run()[source]

Run the benchmark after the evaluator has been set (see set_evaluator()). Example:

benchmark.run()

Return type:: None

set_evaluator(evaluator)[source]

Set the function that maps design parameters to an Observation. Example:

def evaluate(study: Study, x1: float, x2: float) -> Observation:
    observation = study.new_observation()
    observation.add(x1**2 + x2**2)
    return observation
benchmark.set_evaluator(evaluate)

Note

Call this function only after all studies have been added to the benchmark.

Parameters:: evaluator (Callable[..., Observation]) – Function handle for a function of the variable parameters that returns a corresponding Observation object. The function must accept a "study" argument as well as an argument with the name of each design parameter and fixed environment parameter.
Return type:: None

property studies: list[Study]: A list of studies to be run for the benchmark.

class jcmoptimizer.Driver(host, study_id, session)[source]

This class provides methods for retrieving and setting driver-specific information of the study. Depending on the chosen driver of Client.create_study() the driver provides specific methods. More details for each driver are available in the driver reference. Example:

study.run()
driver = study.driver
min_objective_values = driver.historic_parameter_values(
    path="acquisition_function.min_objective")

The constructor should not be used directly since it does not create a driver on the server side. Instead, one should use Study.driver.

describe()[source]

Get description of all modules and their parameters that are used by the driver. Example:

description = driver.describe()
print(description["members"]["surrogates"]["0"])

Return type:: dict[str, Any]

Returns: A nested dictionary with description of submodules consisting: of a name and a descriptive text. If the entry describes a module, it has an additional "members" entry with dictionaries describing submodules and parameters.

get_state(path=None)[source]

Get state of the driver. Example:

best_sample = driver.get_state(path="best_sample")

Parameters:: path (Optional[str]) – A dot-separated path to a submodule or parameter. If none, the full state is returned.
Return type:: dict[str, Any]

Returns: If path is None, a dictionary with information of driver state.

Note

A description of the meaning of each entry in the state can be retrieved by describe().

historic_parameter_values(path)[source]

Get the values of an internal parameter for each iteration of the study. Example:

min_objective_values = driver.historic_parameter_values(
    path="acquisition_function.min_objective")

Parameters:: path (str) – A dot-separated path to the parameter.
Return type:: list[Any]

Note

A description of the meaning of each parameter can be retrieved by describe().