surrogate

Class holding the surrogate model.

The surrogate model consists of different parts, some of them optional, acting in descending order:

• Preprocessors: defined using kwargs preprocessing_X|y, they define the transformation between the input space and the space in which the GP Regressor and the infinities classifiers are trained on. See the documentation of the preprocessing module.

• Infinities classifier(s): a class handling one or more classifiers that divide the space into regions where the log-posterior is expected to be finite or not. Its purpose is both to focus the GP Regressor into areas where the posterior is more regular, and to save time by avoiding evaluating the GP Regressor in uninsteresting regions. See its documentation.

• GP Regressor: It is the heart of the surrogate model and thus the only non-optionally trivial part. It is described in more detail in the documentation of its module.

• Clipper: an optional post-processor that clips the output of the GP Regressor whenever it is above some factor of the maximum log-posterior difference in the training set.

The SurrogateModel class provides the append() method to add new data to the model and optionally fit its different components (including their hyperparameters if present).

Surrogate model predictions can be obtained using the logp() method, or more generally the predict() method. Classification of finite vs infinite log-posterior values can be obtained with is_finite_X() (for predicting with the classifier(s) based on some input) or is_finite_y() (comparison against a set threshold). Do not call methods of the GP Regressor or the classifiers directly, since they need to be passed preprocessed input.

To extract the current training set there are a number of class properties defined, depending on whether one wants all the training points, just the last ones added, just the ones used (or not used) by the GP Regressor, etc. Alternatively, one can get a list of all the points and their properties with training_set_as_df().

class gpry.surrogate.SurrogateModel(bounds=None, preprocessing_X=None, preprocessing_y=None, regressor=None, infinities_classifier=None, clip_factor=1.1, random_state=None, verbose=1)

Bases: object

Object holding the Gaussian Process Regressor, and, if applicable, the input/output preprocessing layer and the infinities classifier.

Attributes ending with an underscore possess values in the respective preprocessed spaces.

Parameters:

• bounds (array-like, shape=(n_dims,2)) – Array of bounds of the prior [lower, upper] along each dimension.

• preprocessing_X (X-preprocessor, Pipeline_X, optional (default: None)) – Single preprocessor or pipeline of preprocessors for X. If None is passed the data is not preprocessed.

• preprocessing_y (y-preprocessor or Pipeline_y, optional (default: None)) – Single preprocessor or pipeline of preprocessors for y. If None is passed the data is not preprocessed.

• regressor (dict) – Dictionary of options to iniitalise the gpr.GaussianProcessRegressor`.

• clip_factor (float, optional (default: 1.1)) – Factor for upper clipping of the regressor’s predictions, to avoid overshoots.

• infinities_classifier (str, Sequence, dict) – Dictionary of options to initialise the infinities_classifier.InfinitiesClassifiers, used for selecting a subset of points to be used to train the regressor, based on their target value. Alternative, the name of a single classifier, or a list of names, can be passed.

• random_state (int or numpy.random.Generator, optional) – The generator used to perform random operations of the GPR. If an integer is given, it is used as a seed for the default global numpy random number generator.

• verbose (1, 2, 3, optional (default: 1)) – Level of verbosity of the GP. 3 prints Infos, Warnings and Errors, 2 Warnings and Errors, and 1 only Errors. Should be set to 2 or 3 if problems arise.

property d

Dimensionality of the training data.

property bounds

Copy of the problem’s prior bounds.

property trust_bounds

The bounds of a smaller trust region where the log-posterior is expected to be finite, possibly defined by a classifier (in particular if infinities_classifier.TrustRegion is being used). Otherwise returns a copy of the original prior bounds.

property fitted

True whenever the the surrogate model has already been fitted to some points.

property n_total

Number of points/features in the training set of the model, including points with target values classified as infinite.

property X

Coordinates of all the points used to train the model (returns a copy). Intended to be used when one wants to access the training data for any purpose.

property y

Log-posterior of all the points used to train the model (returns a copy). Intended to be used when one wants to access the training data for any purpose.

property X_last_appended

Coordinates of the training points added to the surrogate model in the last call to append(), regardless of classification (returns a copy).

property y_last_appended

Log-posterior of the training points added to the surrogate model in the last call to append(), regardless of classification (returns a copy).

property n_init

Number of points passed at initialisation.

property X_init

Coordinates of the points passed at initialisaion (returns a copy).

property y_init

Log-posterior of the points passed at initialisaion (returns a copy).

property n_regress

Number of points currently in use to train the GP Regressor.

property X_regress

Coordinates of the points currently in use to train the GP Regressor (returns a copy).

property y_regress

Log-posterior of the points currently in use to train the GP Regressor (returns a copy).

property X_last_appended_regress

Coordinates of the training points added to the GP Regressor in the last call to append() (returns a copy).

property y_last_appended_regress

Log-posterior of the training points added to the GP Regressor in the last call to append() (returns a copy).

property X_infinite

Coordinates of the training points classified as infinitely small (returns a copy).

property y_infinite

Log-posterior of the training points classified as infinitely small (returns a copy).

property y_max

The maximum log-posterior value in the training set (returns a copy).

Returns -inf (regularized if applicable) if not points have been added.

property scales

GPR Kernel scales as (output_scale, (length_scale_1, ...)) in non-transformed coordinates.

property noise_level

The noise level (square-root of the variance) of the uncorrelated training data, in non-transformed coordinates.

property abs_finite_threshold

Absolute threshold(s) for y values to be considered finite.

is_finite_y(y, validate=True)

Returns the classification of y (target) values as finite (True) or not, by comparing them with the current threshold(s).

Notes

Use this method instead of the equivalent one of the ‘infinities_classifier’ attribute, since the arguments of that one may need to be transformed first.

If calling with an argument which is not either the training set or a subset of it results may be inconsistent, since new values may modify the threshold.

is_finite_X(X, validate=True)

Returns a prediction for the classification of the target value at some given parameters.

Notes

Use this method instead of the equivalent one of the ‘infinities_classifier’ attribute, since the arguments of that one may need to be transformed first.

training_set_as_df(param_names=None)

Returns the training set as a pandas DataFrame (created on-the-fly and not saved).

If param_names are not passed, generic ones will be used.

set_random_state(random_state)

(Re)sets the random state.

append(X, y, noise_level=None, fit_gpr=True, fit_classifier=True, validate=True, i_iter=None, properties=None)

Append newly acquired data to the GPR and the infinities classifier, and updates them and the preprocessors.

Here updating of the GPR refers to the re-calculation of the the GPR inverse matrix \((K(X,X)+\sigma_n^2 I)^{-1}\) which is needed for predictions.

The highest cost incurred by this method is the refitting of the GPR kernel hyperparameters \(\theta\). It can be useful to disable it (fit_gpr=False) in cases where it is worth saving the computational expense in exchange for a loss of information, such as when performing parallelized active sampling (NB: this is only possible when the GPR hyperparameters have been fit at least once).

For an additional speed boost, the refitting of the infinities classifier (if present) can be disabled with fit_classifier=False (if a GPR refit is requested this value is overridden).

If called with X=None, y=None, it re-fits the model without adding new points.

The following calls should then be equivalent:

fit_gpr_kwargs = {"n_restarts": 10}
# A
gpr.append(new_X, new_y, fit_gpr=fit_gpr_kwargs)
# B
gpr.append(new_X, new_y, fit_gpr=False)
gpr.fit_gpr_hyperparameters(**fit_gpr_kwargs)
# C
gpr.append(new_X, new_y, fit_gpr=False, fit_classifier=False)
gpr.append(None, None, fit_gpr=fit_gpr_kwargs)

Parameters:

• X (array-like, shape = (n_samples, n_features), or None) – Training data to append to the model.

• y (array-like, shape = (n_samples), or None) – Target values to append to the data

• noise_level (number, array-like, shape = (n_samples)) – Uncorrelated standard deviation(s) to add to the diagonal part of the covariance matrix. If an array is passed, it needs to have the same number of entries as y. If None, the noise_level set in the instance is used (notice that passing None is not recommended if a preprocessor for the GPR training samples may have been refit after initialization). If a non-None value is passed, it is advisable to refit the hyperparameters of the kernel.

• fit_gpr (Bool or dict (default: True)) – Whether the GPR \(\theta\)-parameters should be optimised. It can be passed a dict, which is always interpreted as True, containing arguments for the _fit_hyperparameters method, namely n_restarts of the optimizer, start_from_current if the first optimizer run should start from the last optimum, or different hyperparameters_bounds. E.g. to perform a single GPR optimization run starting at the last hyperparameter optimum: fit_gpr={'start_from_current': True, 'n_restarts': 1}.

• fit_classifier (Bool (default: True)) – Whether the infinities classifier is refit. Overridden to True if fit_gpr is not False.

• i_iter (int or array-like, shape = (n_samples), optional) – An index to be assigned to the newly-appended points. If None, assigns the last entry +1.

• properies (dict {str: Sequence}, optional) – A dictionary of property names, with a list of the same length as X as values. These properties have no effect, beyond being kept track of.

Returns:

Returns an instance of self.

Return type:

self

predict(X, return_std=False, return_mean_grad=False, return_std_grad=False, validate=True, ignore_classifier=None)

Predict output for X.

In addition to the mean of the predictive distribution, also its standard deviation (return_std=True), the gradient of the mean and the standard-deviation with respect to X can be optionally provided.

Parameters:

• X (array-like, shape = (n_samples, n_features)) – Query points where the GP is evaluated.

• return_std (bool, default: False) – If True, the standard-deviation of the predictive distribution at the query points is returned along with the mean.

• return_mean_grad (bool, default: False) – Whether or not to return the gradient of the mean. Only valid when X is a single point.

• return_std_grad (bool, default: False) – Whether or not to return the gradient of the std. Only valid when X is a single point.

• validate (bool, default: True) – If False, X is assumed to be correctly formatted (2-d float array, with points as rows and dimensions/features as columns, C-contiguous), and no checks are performed on it. Reduces overhead. Use only for repeated calls when the input is programmatically generated to be correct at each stage.

• ignore_classifier (list, optional (default: None)) – If defined as a list, the classifiers with names on that list are not used to discard the given points whenever all inputs are predicted to be negative infinity.

• note:: (..) – Note that in contrast to the sklearn GP Regressor our implementation cannot return the full covariance matrix. This is to save on some complexity and since the full covariance cannot be calculated if either values are infinite or a y-preprocessor is used.

Returns:

• y_mean (array, shape = (n_samples)) – Mean of GPR’s predictive distribution a query points

• y_std (array, shape = (n_samples,), optional) – Standard deviation of GPR’s predictive distribution at query points. Only returned when return_std is True.

• y_mean_grad (shape = (n_samples, n_features), optional) – The gradient of GPR’s the predicted mean.

• y_std_grad (shape = (n_samples, n_features), optional) – The gradient of GPR’s the predicted std.

logp(X, validate=True, ignore_classifier=None)

Returns the surrogate log-posterior. Alias of predict() for the GP Regressor’s mean only.

Parameters:

• X (array-like, shape = (n_samples, n_features)) – Query points where the GP is evaluated.

• validate (bool, default: True) – If False, X is assumed to be correctly formatted (2-d float array, with points as rows and dimensions/features as columns, C-contiguous), and no checks are performed on it. Reduces overhead. Use only for repeated calls when the input is programmatically generated to be correct at each stage.

• ignore_classifier (list, optional (default: None)) – If defined as a list, the classifiers with names on that list are not used to discard the given points whenever all inputs are predicted to be negative infinity.

Returns:

log-posterior – Predicted log-posterior.

Return type:

array, shape = (n_samples)

predict_std(X, ignore_classifier=None, validate=True)

Predict output standart deviation for X.

Parameters:

• X (array-like, shape = (n_samples, n_features)) – Query points where the GP is evaluated.

• ignore_classifier (list, optional (default: None)) – If defined as a list, the classifiers with names on that list are not used to discard the given points whenever all inputs are predicted to be negative infinity.

• validate (bool, default: True) – If False, X is assumed to be correctly formatted (2-d float array, with points as rows and dimensions/features as columns, C-contiguous), and no checks are performed on it. Reduces overhead. Use only for repeated calls when the input is programmatically generated to be correct at each stage.

Returns:

y_std – Standard deviation of predictive distribution at query points. Only returned when return_std is True.

Return type:

array, shape = (n_samples,), optional

fraction_prior_finite(n_prior='100d')

Returns the faction of the prior that is classified as finite, assuming uniform density. Performs the calculation by drawing and classifying n uniform prior samples.

Parameters:

n_prior (int, str) – Number of samples to be drawn from the prior for getting the finite fraction.

Returns:

fraction_prior_finite – The fraction of the prior volume classified as finite.

Return type:

float

class gpry.surrogate.Clipper(clip_factor)

Bases: object

Handles the upper clipping of the y-output of the regressor, as a factor of the difference in its range.

property trivial