profit.sur.gp.custom_surrogate

Module Contents

Classes

GPSurrogate	Custom GP model made from scratch.
MultiOutputGPSurrogate	This is the base class for all Gaussian Process models.

class profit.sur.gp.custom_surrogate.GPSurrogate

Bases: profit.sur.gp.GaussianProcess

Custom GP model made from scratch.

Supports custom Python and Fortran kernels with analytic derivatives and advanced Active Learning.

hess_inv

Inverse Hessian matrix which is required for active learning. It is calculated during the hyperparameter optimization.

Type:

ndarray

property Ky

Full training covariance matrix as defined in the kernel including data noise as specified in the hyperparameters.

property alpha

Convenient matrix-vector product of the inverse training matrix and the training output data. The equation is solved either exactly or with a least squares approximation.

\[ \begin{equation} \alpha = K_y^{-1} y_{train} \end{equation} \]

train(X, y, kernel=defaults['kernel'], hyperparameters=defaults['hyperparameters'], fixed_sigma_n=base_defaults['fixed_sigma_n'], eval_gradient=True, return_hess_inv=False)

After initializing the model with a kernel function and initial hyperparameters, it can be trained on input data X and observed output data y by optimizing the model’s hyperparameters. This is done by minimizing the negative log likelihood.

Parameters:

• X (ndarray) – (n, D) array of input training data.

• y (ndarray) – (n, 1) array of training output. Currently, only scalar output is supported.

• kernel (str/object) – Identifier of kernel like ‘RBF’ or directly the kernel object of the specific surrogate.

• hyperparameters (dict) – Hyperparameters such as length_scale, variance and noise. Taken either from given parameter, config file or inferred from the training data. The hyperparameters can be different depending on the kernel. E.g. The length_scale can be a scalar, a vector of the size of the training data, or for the custom LinearEmbedding kernel a matrix.

• fixed_sigma_n (bool) – Indicates if the data noise should be optimized or not.

• eval_gradient (bool) – Whether the gradients of the kernel and negative log likelihood are explicitly used in the scipy optimization or numerically calculated inside scipy.

• return_hess_inv (bool) – Whether to the attribute hess_inv after optimization. This is important for active learning.

post_train()

predict(Xpred, add_data_variance=True)

Predicts the output at test points Xpred.

Parameters:

• Xpred (ndarray/list) – Input points for prediction.

• add_data_variance (bool) – Adds the data noise \(\sigma_n^2\) to the prediction variance. This is especially useful for plotting.

Returns:

a tuple containing:

• ymean (ndarray) Predicted output values at the test input points.

• yvar (ndarray): Diagonal of the predicted covariance matrix.

Return type:

tuple

add_training_data(X, y)

Add training points to existing data. This is important for active learning.

Only the training dataset is updated, but the hyperparameters are not optimized yet.

Parameters:

• X (ndarray) – Input points to add.

• y (ndarray) – Observed output to add.

set_ytrain(ydata)

Set the observed training outputs. This is important for active learning.

Parameters:

ydata (np.array) – Full training output data.

get_marginal_variance(Xpred)

save_model(path)

Saves the model as dict to a .hdf5 file.

Parameters:

path (str) – Path including the file name, where the model should be saved.

classmethod load_model(path)

Loads a saved model from a .hdf5 file and updates its attributes.

Parameters:

path (str) – Path including the file name, from where the model should be loaded.

Returns:

Instantiated surrogate model.

Return type:

profit.sur.gaussian_process.GPSurrogate

select_kernel(kernel)

Convert the name of the kernel as string to the kernel class object of the surrogate. First search the kernels implemented in python, then the Fortran kernels.

Parameters:

kernel (str) – Kernel string such as ‘RBF’. Only single kernels are supported currently.

Returns:

Kernel object of the class. This is the function which builds the kernel and not the calculated covariance matrix.

Return type:

object

optimize(fixed_sigma_n=base_defaults['fixed_sigma_n'], eval_gradient=False, return_hess_inv=False)

Optimize the hyperparameters length_scale \(l\), scale \(\sigma_f\) and noise \(\sigma_n\).

As a backend, the scipy minimize optimizer is used.

Parameters:

• fixed_sigma_n (bool) – Indication if the data noise should also be optimized or not.

• eval_gradient (bool) – Flag if the gradients of the kernel and negative log likelihood should be used explicitly or numerically calculated inside the optimizer.

• return_hess_inv (bool) – Whether to set the inverse Hessian attribute hess_inv which is used to calculate the marginal variance in active learning.

_set_hyperparameters_from_model(model_hyperparameters)

class profit.sur.gp.custom_surrogate.MultiOutputGPSurrogate(child=GPSurrogate)

Bases: profit.sur.gp.GaussianProcess

This is the base class for all Gaussian Process models.

trained

Flag that indicates if the model is already trained and ready to make predictions.

Type:

bool

fixed_sigma_n

Indicates if the data noise should be optimized or not. If an ndarray is given, its length must match the training data.

Type:

bool/float/ndarray

Xtrain

Input training points.

Type:

ndarray

ytrain

Observed output data. Vector output is supported for independent variables only.

Type:

ndarray

ndim

Dimension of input data.

Type:

int

output_ndim

Dimension of output data.

Type:

int

kernel

Kernel identifier such as ‘RBF’ or directly the (surrogate specific) kernel object. Defaults to ‘RBF’.

Type:

str/object

hyperparameters

Parameters like length-scale, variance and noise which can be optimized during training. As default, they are inferred from the training data.

Type:

dict

Default parameters:

surrogate: GPy kernel: RBF

Default hyperparameters:

\(l\) … length scale \(\sigma_f\) … scaling \(\sigma_n\) … data noise

\[\begin{split} \begin{align} l &= \frac{1}{2} \overline{\lvert x - x' \rvert} \\ \sigma_f &= \overline{std(y)} \\ \sigma_n &= 0.01 \cdot \overline{max(y) - min(y)} \end{align} \end{split}\]

train(X, y, kernel=defaults['kernel'], hyperparameters=defaults['hyperparameters'], fixed_sigma_n=base_defaults['fixed_sigma_n'], return_hess_inv=False)

Trains the model on the dataset.

After initializing the model with a kernel function and initial hyperparameters, it can be trained on input data X and observed output data y by optimizing the model’s hyperparameters. This is done by minimizing the negative log likelihood.

Parameters:

• X (ndarray) – (n, d) array of input training data.

• y (ndarray) – (n, D) array of training output.

• kernel (str/object) – Identifier of kernel like ‘RBF’ or directly the kernel object of the surrogate.

• hyperparameters (dict) – Hyperparameters such as length scale, variance and noise. Taken either from given parameter, config file or inferred from the training data. The hyperparameters can be different depending on the kernel. E.g. The length scale can be a scalar, a vector of the size of the training data, or for the custom LinearEmbedding kernel a matrix.

• fixed_sigma_n (bool) – Indicates if the data noise should be optimized or not.

• return_hess_inv (bool) – Whether to the attribute hess_inv after optimization. This is important for active learning.

_set_hyperparameters_from_model()

predict(Xpred, add_data_variance=True)

Predicts the output at test points Xpred.

Parameters:

• Xpred (ndarray/list) – Input points for prediction.

• add_data_variance (bool) – Adds the data noise \(\sigma_n^2\) to the prediction variance. This is especially useful for plotting.

Returns:

a tuple containing:

• ymean (ndarray) Predicted output values at the test input points.

• yvar (ndarray): Diagonal of the predicted covariance matrix.

Return type:

tuple

add_training_data(X, y)

set_ytrain(y)

save_model(path)

Saves the model as dict to a .hdf5 file.

Parameters:

path (str) – Path including the file name, where the model should be saved.

classmethod load_model(path)

Loads a saved surrogate from a file. The file format can vary between surrogates.

Identifies the surrogate by its class label in the file name.

Parameters:

path (str) – Path including the file name, from where the model should be loaded.

Returns:

Instantiated surrogate model.

Return type:

profit.sur.Surrogate

optimize(**opt_kwargs)

Find optimized hyperparameters of the model. Optional kwargs for tweaking optimization.

Parameters:

opt_kwargs – Keyword arguments for optimization.

special_hyperparameter_decoding(key, value)

get_marginal_variance(Xpred)