Preprocessing

This module provides different methods for preprocessing the training data in X- and Y-direction. These classes and methods are primarily used internally by the GP regressor and the optimizer of the acquisition function.

This module contains several methods of preprocessing the training and target values for both the GP Regressor and acquisition module. Instances of different preprocessors can be chained together thereby building a pipeline.

The preprocessors are implemented into the GP Acquisition and GP Regressor module in a way which performs the transformations behind the scenes meaning that the user can work in the non-transformed space and all transformations will be performed internally.

You can build your own preprocessor if you want. This requires you to build a custom class. How to do that for X- and y-preprocessors is explained in the Pipeline_X and Pipeline_y classes respectively.

NB: all transform-like methods should return a copy of the input, but avoid unnecessary copy statements.

class preprocessing.DummyPreprocessor

Bases: object

is_linear = True

classmethod fit(*args, **kwargs)

classmethod transform_bounds(bounds)

classmethod transform(_)

classmethod inverse_transform(_)

classmethod transform_scale(_)

classmethod inverse_transform_scale(_)

class preprocessing.Pipeline_X(preprocessors)

Bases: object

Used for building a pipeline for preprocessing X-values. This is provided with a list of preprocessors in the order they shall be applied. The transform_scale, fit, transform and inverse_transform methods can then be called as if the pipeline was a single preprocessor.

Parameters:

preprocessors (list of preprocessors for X.) –

The preprocessors in the order that the transformations shall be transformed. These need to either be inbuilt preprocessors or you can build your own preprocessor. For this you need to build a custom class with the signature:

from gpry.preprocessing import Preprocessor
class My_X_preprocessor:
    def __init__(self, ...):
        # Add here any objects that the preprocessor might need
        ...

    def fit(self, X, y):
        # This method should fit the transformation
        # (if neccessary).
        ...
        return self

    def transform_bounds(self, bounds):
        # This method should transform the bounds of the prior. If
        # the bounds remain unchanged after the transformation this
        # method should just return the (untransformed) bounds.
        ...
        return transformed_bounds

    def transform(self, X):
        # This method transforms the X-data. For this the fit
        # method has to have been called before at least once.
        ...
        return X_transformed

    def inverse_transform(self, X):
        # Applies the inverse transformation to ``transform``.
        ...
        return inverse_transformed_X

    def transform_scale(self, scale):
        # This method should transform a scale in X, e.g. the bounds
        # of the prior.
        ...
        return transformed_scale

    def transform_scale(self, scale):
        # Applies the inverse transform to ``transform_scale``.
        ...
        return transformed_scale

Note

All the preprocessor objects need to be initialized! Furthermore transform` and ``inverse_transform need to preserve the shape of X. All transformations must return a copy (but avoid unnecessary copy statements).

transform_bounds(bounds)

fit(X, y)

Consecutively fit several preprocessors by passing the transformed data through each one and fitting.

transform(X)

Transform the data through the pipeline

inverse_transform(X)

Inverse transform the data through the pipeline (by applying each inverse transformation in reverse order).

transform_scale(scale)

inverse_transform_scale(scale)

Inverse transform the data through the pipeline (by applying each inverse transformation in reverse order).

class preprocessing.Whitening(bounds, mean=None, cov=None, learn=False)

Bases: object

A class which can pre-transform the posterior in a way that it matches a multivariate normal distribution during the Regression step. This is done in the hope that by matching a normal distribution the GP will converge faster. The transformation used is the whitening transformation which is given by

\[X_k^i \to \frac{\mathbf{R}^{ij} (X_k^j - m^j)}{\sigma^i}\ .\]

\(\mathbf{R}\) is the matrix which solves \(\mathbf{C} = \mathbf{R}\mathbf{\Lambda}\mathbf{R}\) where \(\mathbf{C}\) is the empirical covariance matrix (along the dimensions of X) and \(\mathbf{\Lambda}\) a diagonal matrix. \(m^j\) the empirical mean and \(\sigma = \sqrt{\mathbf{C}^{ii}}\) the empirical standard deviation.

This can help with highly anisotropic distributions, especially if degeneracy directions are known a priori.

When adapting it with learn=True, this may not be very numerically robust, since the empirical mean and standard deviation are weighted by the posterior values which have a high dynamical range. An anisotropic kernel may be preferred.

static prepare_transform(cov)

Compute the relevant elements for the transform from the mean and covmat.

Raises ValueError if it fails at the eigen-decomposition.

static compute_mean_cov(X, logp)

Computes mean and cov using the given points weighted by the given log-probabilities.

Raises ValueError if failed to get a non-singular covmat.

fit(X, y)

Fits the whitening transformation, if initialised with learn=True.

If an error is encountered, keeps the previous transform.

transform(X)

inverse_transform(X)

transform_bounds(bounds)

class preprocessing.Normalize_bounds(bounds)

Bases: object

A class which transforms all bounds of the prior such that the prior hypervolume occupies the unit hypercube in the interval [0, 1]. This is done because of two reasons:

1. Confining the bounds while fitting the GP regressor ensures that the hyperparameters of the GP (particularly length-scales) are within the same order of magnitude if one assumes that the non-zero region of the posterior occupies roughly the same fraction of the prior in each direction. This is a reasonable assumption for most realistic cases.

2. If the with of the posterior distribution is similar along every dimension this makes it far easier for the optimizer of the acquisition function to navigate the acquisition function space (which has the same number of dimensions as the training data) especially if the optimizer uses a fixed jump-length.

Parameters:

bounds (array-like, shape = (n_dims, 2)) – Bounds [lower, upper] along each dimension.

Attributes:

• transformed_bounds (array-like, shape = (n_dims, 2)) – Array with [0, 1] along every dimension.

• bounds_min (array-like, shape = (n_dims,)) – Lower bounds along every dimension.

• bounds_max (array-like, shape = (n_dims,)) – Upper bounds along every dimension.

update_bounds(bounds)

transform_bounds(bounds)

fit(X, y)

Fits the transformer (which in reality does nothing)

transform(X)

Transforms X so that all values lie between 0 and 1.

Parameters:

X (array-like, shape = (n_samples, n_dims)) – X-values that one wants to transform. Must be between bounds.

Returns:

X_transformed – Transformed X-values

Return type:

array-like, shape = (n_samples, n_dims)

inverse_transform(X)

Applies the inverse transformation

Parameters:

X (array-like, shape = (n_samples, n_dims)) – Transformed X-values between 0 and 1.

Returns:

X – Inverse transformed (original) values.

Return type:

array-like, shape = (n_samples, n_dims)

inverse_transform_scale(X)

Applies the inverse transformation to an unbounded scale (e.g. the kernel length scale).

Parameters:

X (array-like, shape = (n_samples, n_dims)) – Transformed X-values between 0 and 1.

Returns:

X – Inverse transformed (original) values.

Return type:

array-like, shape = (n_samples, n_dims)

class preprocessing.Pipeline_y(preprocessors)

Bases: object

Used for building a pipeline for preprocessing y-values. This is provided with a list of preprocessors in the order they shall be applied. The fit, transform and inverse_transform methods can then be called as if the pipeline was a single preprocessor.

Parameters:

preprocessors (list of preprocessors for y.) –

The preprocessors in the order that the transformations shall be transformed. These need to either be inbuilt preprocessors or you can build your own preprocessor. For this you need to build a custom class with the signature:

class My_y_preprocessor:
    def __init__(self, ...):
        # Add here any objects that the preprocessor might need
        ...

    def fit(self, X, y):
        # This method should fit the transformation
        # (if neccessary).
        ...
        return self

    def transform(self, y):
        # This method transforms the y-data. For this the fit
        # method has to have been called before at least once.
        ...
        return transformed_y

    def inverse_transform(self, y):
        # Applies the inverse transformation to ``transform``.
        ...
        return inverse_transformed_y

    def transform_scale(self, scale):
        # This method should transform a scale-like quantity
        # (e.g. the noise level of the training data) such that
        # it represents the corresponding scale
        # of the transformed data.
        ...
        return transformed_scale

    def inverse_transform_scale(self, scale):
        # This method should invert the transformation applied by
        # ``transform_scale``.
        ...
        return inverse_transformed_scale

Note

All the preprocessor objects need to be initialized! Furthermore the transform and inverse_transform methods need to preserve the shape of y. In contrast to the preprocessors for X this does not need to contain a method to transform bounds, but it still needs one to transform scales such as the the noise level (alpha).

fit(X, y)

Consecutively fit several preprocessors by passing the transformed data through each one and fitting.

transform(y)

Transform the data through the pipeline

inverse_transform(y)

Inverse transform the data through the pipeline (by applying each inverse transformation in reverse order).

transform_scale(scale)

Transforms the scale through the pipeline

inverse_transform_scale(scale)

Inverse transforms the scale through the pipeline

class preprocessing.Normalize_y(use_median=False)

Bases: object

Transforms y-values (target values) such that they are centered around 0 with a standard deviation of 1. This is done so that the constant pre-factor in the kernel (constant kernel) stays within a numerically convenient range.

Attributes:

• mean_ (float) – Mean of the y-values

• std_ (float) – Standard deviation of the y-values

• **Methods (****)

• .. autosummary:: – :toctree: stubs

  transform inverse_transform

property is_linear

property fitted

fit(X, y)

Calculates the mean and standard deviation of y and saves them.

Parameters:

• X (array-like, shape = (n_samples, dimension)) – X-values (target values) that can be used to tune the transformation. determine the mean and std.

• y (array-like, shape = (n_samples,)) – y-values (target values) that are used to determine the mean and std.

transform(y)

Transforms y.

Parameters:

y (array-like, shape = (n_samples,)) – y-values that one wants to transform.

Returns:

y_transformed – Transformed y-values

Return type:

array-like, shape = (n_samples,)

inverse_transform(y)

Applies inverse transformation to y.

Parameters:

y_transformed (array-like, shape = (n_samples,)) – Transformed y-values.

Returns:

y – Original y-values.

Return type:

array-like, shape = (n_samples,)

transform_scale(scale)

inverse_transform_scale(scale)

class preprocessing.NormalizeChi2_y(nsigma=1)

Bases: Normalize_y

Transforms y-values (target values) such that they are centered around the Gaussian 1-sigma value with respect to the largest logp, so that the standard deviation is the distance between the maximum and the value that defines that 0.

Attributes:

• mean_ (float) – Mean of the y-values

• std_ (float) – Standard deviation of the y-values

• **Methods (****)

• .. autosummary:: – :toctree: stubs

  transform inverse_transform

fit(X, y)

Calculates the mean and standard deviation of y and saves them.

Parameters:

• X (array-like, shape = (n_samples, dimension)) – X-values (target values) that can be used to tune the transformation. determine the mean and std.

• y (array-like, shape = (n_samples,)) – y-values (target values) that are used to determine the mean and std.