run

`run`#

This module implements the Runner class, which handles the active learning loop (see How does GPry work), and gets MC samples from the surrogate model after convergence (see Drawing MC samples from the surrogate posterior). It also includes some methods for plotting and diagnostics. A callback=[callable] function can also be passed (see The callback function).

This class is initialized with information about the true model (the true likelihood, the prior bounds, optionally names and labels for the parameters), general options for the active learning loop (truth evaluation budget, number of candidates per iteration…), and the specification of the different components involved in the loop, which are then stored as attributes (see below). For an example call with some non-trivial parameter setting see the Advanced example.

The Runner also contains a table detailing progress information as Runner.progress (see Progress).

Furthermore the Runner object allows for checkpointing. The relevant arguments are checkpoint="/path/to/checkpoint", defining where to save the current progress, and load_checkpoint="resume"|"overwrite", which decides what to do if a previous run with the specified checkpoint is found (raises an error if path given but action not specified, in order to avoid losing previous work).

class gpry.run.Runner(loglike=None, bounds=None, ref_bounds=None, params=None, gpr=None, surrogate='RBF', gp_acquisition='LogExp', initial_proposer='reference', convergence_criterion=None, mc=None, callback=None, callback_is_MPI_aware=False, options=None, checkpoint=None, load_checkpoint=None, seed=None, plots=False, verbose=3)[source]#

Bases: object

Class that takes care of constructing the Bayesian quadrature/posterior characterization loop. After initialisation, the algorithm can be launched with Runner.run(), and, optionally after that, an MC process can be launched on the surrogate model with Runner.generate_mc_sample().

Parameters:

loglike (callable or Cobaya model object) – Log-likelihood function or Cobaya Model instance. If a Cobaya model is passed, arguments bounds, ref_bounds and params are ignored, since all that information is already contained in the Cobaya model. It must not be specified if ‘resuming’ from a checkpoint (see load_checkpoint below).
bounds (List of [min, max], or Dict {name: [min, max],...}) – List or dictionary of parameter bounds. If it is a dictionary, the keys need to correspond to the argument names of the likelihood function, and the values can be either bounds specified as [min, max], or bounds and labels, as {"prior": [min, max], "latex": [label]}. It does not need to be defined (will be ignored) if a Cobaya Model instance is passed as loglike.
ref_bounds (List of [min, max], or Dict {name: [min, max],...}) – List or dictionary of “reference” parameter bounds, i.e. bounds from within which to draw the initial set of training samples.
params (list of str, dict {str: str}, optional) – List of names for the parameters. Alternatively, a dictionary {name: label}, where label is a LaTeX-coded string, without $’s. By default, generic parameter names x_1, x_2,... will be used.
surrogate (SurrogateModel, str, dict, optional (default="RBF")) –
The surrogate model used for interpolating the posterior. A full specification consists of a dict detailing the regressor options (see gpr.GaussianProcessRegressor), and optionally, preprocessing_X|y transformations, an infinities_classifier dict (see infinities_classifier.InfinitiesClassifiers) and a clip factor for large regressor values. E.g.:
```
surrogate = {
    "regressor": {
        "kernel": "RBF",
        "noise_level" : 1e-2,
    },
    "clip_factor" : 1.1,
    "infinities_classifier" : {
        "svm": {"threshold": "20s"},
        "trust_region": {"threshold": "10s"},
    }
```
If None is passed, the regressor is created with an anisotropic RBF kernel, a noise level of 0.01, preprocessors normalizing X values within bounds and y values as a standard normal, a clipping factor of 1.1, and an SVM as an infinities classifier. If a string is passed, it is interpreted as a length correlation kernel specification, and the rest of defaults are kept. Alternatively, an already initialized instance can be passed.
gp_acquisition (GenericGPAcquisition, optional (default="LogExp")) – The acquisition object. If None is given the BatchOptimizer with a LogExp acquisition function is used (with the $\zeta$ value chosen automatically depending on the dimensionality of the prior). It can also be passed an initialized instance, or a dict with arguments with which to initialize one. The acquisition engine works in the space defined by the preprocessors of the surrogate model.
initial_proposer (InitialPointProposer, str, dict, optional (default="reference")) – Proposer used for drawing the initial training samples before running the Bayesian optimisation loop. By default the samples are drawn from the model reference (prior if no reference is specified). Alternative options which can be passed as strings are "prior", "uniform". The "reference" proposer defaults to the prior if no reference distribution is provided. If defined as a dict with the proposer name as single key, the values will be passed as kwargs to the proposer.
convergence_criterion (ConvergenceCriterion, str, dict, False, optional (default=None)) – The convergence criterion. If None is given the default criterion is used: CorrectCounter for BatchOptimizer with adaptive thresholds, and a combination of a less stringent CorrectCounter and a GaussianKL for NORA. Can be specified as a dict to initialize one or more ConvergenceCriterion classes with some arguments, or directly as an instance or class name of some ConvergenceCriterion. If False, no convergence criterion is used, and the process runs until the budget is exhausted.
mc (dict, str, optional (default=None)) – Sampler and it options for the diagnosis and final MC samples as {<sampler_name >: {'option1': value1, ...}}, or simply a struing specifying the sampler name. These settings can be overriden when calling run.Runner.generate_mc_sample() using its sampler argument. By default, the best available nested sampler is used, with standard precision settings.
options (dict, optional (default=None)) –
A dict containing all options regarding the bayesian optimization loop. The available options are:
- n_initial : Number of finite initial truth evaluations before starting the BO loop (default: 3 * number of dimensions)
- max_initial : Maximum number of truth evaluations at initialization. If it is reached before n_initial finite points have been found, the run will fail. To avoid that, try decreasing the volume of your prior (default: 30 * (number of dimensions)**1.5).
- max_total : Maximum number of attempted sampling points before the run stops. This is useful if you e.g. want to restrict the maximum computation resources (default: 70 * (number of dimensions)**1.5 or max_initial, whichever is largest).
- max_finite : Maximum number of sampling points accepted into the GP training set before the run stops. This might be useful if you use the DontConverge convergence criterion, specifying exactly how many points you want to have in your GP. If you set this limit by hand and find that it is easily saturated, try decreasing the volume of your prior (default: max_total).
- n_points_per_acq : Number of points which are aquired with Kriging believer for every acquisition step. It will be adjusted within a 20% tolerance to match a multiple of the number of parallel processes (default: number of dimensions).
- fit_full_every : Number of iterations between full GP hyperparameters fits, including several restarts of the optimiser. Pass ‘np.inf’ or a large number to never refit with restarts (default : 2 * sqrt(number of dimensions)).
- fit_simple_every : Similar to fit_full_every, but with a single optimiser run from the last optimum hyperparameters. Overridden by fit_full_every where it matches its periodicity. Pass np.inf or a large number to never refit from last optimum (default : 1, i.e. every iteration).
callback (callable, optional (default=None)) – Function run each iteration after adapting the recently acquired points and the computation of the convergence criterion. This function should take the runner as argument: callback(runner_instance). When running in parallel, the function is run by the main process only, unless callback_is_MPI_aware=True.
callback_is_MPI_aware (bool (default: False)) – If True, the callback function is called for every process simultaneously, and it is expected to handle parallelisation internally. If false, only the main process calls it.
checkpoint (str, optional (default=None)) – Path for storing checkpointing information from which to resume in case the algorithm crashes. If None is given no checkpoint is saved.
load_checkpoint ("resume" or "overwrite", must be specified if path is not None.) – Whether to resume from the checkpoint files if existing ones are found at the location specified by checkpoint.
seed (int or numpy.random.Generator, optional) – Seed for the random number generator, or an already existing generator, for reproducible runs. For MPI runs, if a seed is passed, numpy.random.SeedSequence will be used to generate seeds for every ranks.
plots (bool, dict (default: True)) – If True, produces some progress plots. One can also pass the arguments of Runner.plot_progress as a dict for finer control, e.g. {"timing": True, "convergence": True, "trace": False, "slices": False, "ext": "svg"}.
verbose (1, 2, 3, optional (default: 3)) – Level of verbosity. 3 prints Infos, Warnings and Errors, 2 Warnings and Errors, and 1 only Errors. Should be set to 2 or 3 if problems arise. Is passed to the GP, Acquisition and Convergence criterion if they are built automatically.

Attributes:

truth (truth.Truth) – The model whose log-posterior is learned.
surrogate (surrogate.SurrogateModel) – Surrogate model of truth. It can be used to call an MCMC sampler for getting marginalized properties.
gp_acquisition (GenericGPAcquisition) – The acquisition engine that was used for the active sampling procedure.
convergence_criterion (Convergence_criterion) – The convergence critera used for determining convergence.
options (dict) – The options dict used for the active sampling loop.
progress (Progress) – Object containing per-iteration progress information: number of finite training points, number of GP evaluations, timing of different parts of the algorithm, and value of the convergence criterion.

property d#: Dimensionality of the problem.

property prior_bounds#

property params#: Returns the list of parameter names.

property labels#: Returns the list of labels.

property gpr#: Deprecated. Use Runner.surrogate instead.

logp(X)[source]#

Wrapper for the surrogate posterior. Call with a point or a list of them.

This is the full posterior. If the prior is uniform the likelihood function can be recovered by summing self.log_prior_volume to this function.

Always returns an array.

logL(X)[source]#

Wrapper for the surrogate likelihood. Call with a point or a list of them.

Always returns an array.

logp_truth(X)[source]#

Wrapper for the true log-posterior. Call with a single point.

This is the full posterior. If the prior is uniform the likelihood function can be recovered by summing self.log_prior_volume to this function.

Always returns a scalar.

logL_truth(X)[source]#

Wrapper for the true log-likelihood. Call with a single point.

Always returns a scalar.

logpost_eval_and_report(X, level=None)[source]#: Simple wrapper to evaluate and return the true log-posterior at X, and log it with the given level.

logprior(X)[source]#: Returns the log-prior.

log(msg, level=None)[source]#: Print a message if its verbosity level is equal or lower than the given one (or always if level=None.

ensure_paths(plots=False)[source]#: Creates paths for checkpoint and plots.

property n_total_left#: Number of truth evaluations before stopping.

property n_finite_left#: Number of truth evaluations with finite return value before stopping.

banner(text, max_line_length=79, prefix='| ', suffix=' |', header='=', footer='=', level=3)[source]#: Creates an iteration banner.

read_checkpoint(truth=None)[source]#

Loads checkpoint files to be able to resume a run or save the results for further processing.

Parameters:: truth (gpry.truth.Truth, optional) – If passed, it will be used instead of the loaded one.

save_checkpoint(update_truth=False)[source]#: Saves checkpoint files to be able to resume a run or save the results for further processing.

run()[source]#: Runs the acquisition-training-convergence loop until either convergence or a stopping condition is reached.

do_initial_training()[source]#

Draws an initial sample for the surrogate GP model until it has a training set of size n_initial, counting only finite-target points (“finite” here meaning over the threshold of the SVM classifier, if present).

This function is MPI-aware and broadcasts the initialized surrogate model to all processes.

update_mean_cov(use_mc_sample=None)[source]#: Updates and shares mean and cov if available, preferring the MC-sample one if indicated, and otherwise checking GPAcquisition first, and Convergence second if not present in GPAcquisition.

set_fiducial_point(X, logpost=None, loglike=None)[source]#

Set a single fiducial point in parameter space, to be included in the plots and in some tests.

Recover with self.fiducial_[X|logpost|loglike|weight].

Parameters:

X (array-like, shape = (n_dim)) – Fiducial point coordinates.
logpost (float, optional) – Fiducial point log-posterior density
loglike (float, optional) – Fiducial point log-likelihood density (use if the prior density not known).

:raises TypeError : if a mismatch found between the different inputs.:

set_fiducial_mc(X, logpost=None, loglike=None, weights=None)[source]#

Set a reference Monte Carlo sample of the true posterior, to be included in the plots and in some tests.

Recover with self.fiducial_mc_[X|logpost|loglike|weight].

Parameters:

X (array-like, shape = (n_samples, n_dim)) – Fiducial 2d array of MC samples.
logpost (array-like, shape = (n_samples), optional) – Log-posterior density for the points in the MC sample.
loglike (array-like, shape = (n_samples), optional) – Log-likelihood density for the points in the MC sample (use if the prior density is not known).
weights (array-like, shape = (n_samples), optional) – Weights of points in the MC sample (assumed equal weights if not specified).

:raises TypeError : if a mismatch found between the different inputs.:

generate_mc_sample(sampler=None, add_options=None, output=None, resume=False)[source]#

Runs an MC process using a nested sampler, or Cobaya.

The result can be retrieved using the run.Runner.last_mc_samples() method.

Parameters:

sampler (string or dict, optional) – Sampler to be initialised. If a string, it must be nested or the name of a sampler recognised by Cobaya (e.g. mcmc). If nested, the best available nested sampler is used: PolyChord if available, otherwise UltraNest. If defined as a dict, the options specified as keys and values will be passed to the sampler, in the nested case nlive, num_repeats (length of slice chains for PolyChord), precision_criterion (fraction of the evidence contained in the set of live points), nprior (number of initial samples of the prior). If using a Cobaya sampler, the options mentioned in the Cobaya documentation of the particular sampler can be passed. If undefined, uses the sampler specified by the mc argument at initialzation (‘nested’, if unspecified).
add_options (dict, optional) – DEPRECATED: pass options by specifying the sampler argument as a dict.
output (path, optional (default: checkpoint/, if checkpoint != None)) – The path where the resulting Monte Carlo sample shall be stored. If passed explicitly False, produces no output.
resume (bool, optional (default=False)) – Whether to resume from existing output files (True) or force overwrite (False)

last_mc_samples(copy=True, as_pandas=False, as_getdist=False)[source]#

Returns the last MC sample from the surrogate model as a dict with keys w (weights), X, logpost, logprior and loglike.

If None is stored as weights, all samples should be assumed to have equal weight.

The MC samples can optionally be returned as a Pandas DataFrame or a GetDist MCSamples.

last_mc_logZ()[source]#

Returns the log-evidence of the last MC sample from the surrogate model, together with its standard deviation, as (logZ, std(logZ)).

The standard deviation corresponds to the uncertainty of the nested sampling calculation of the evidence of the mean GPR, not including the uncertainty associated with the GPR modelling and the infinities classifier.

Returns None if no MC samples from the surrogate model have been obtained, or if no evidence has been computed from them.

diagnose_last_mc_sample()[source]#

Diagnoses the last MC sample to check consistency.

This is an MPI-aware method that should be run simultaneously from all processes if running with MPI.

Return type:: True if the test succeeded, False otherwise.

plot_mc(samples_or_samples_folder=None, add_training=True, add_samples=None, output=None, output_dpi=200, ext='png', close=False)[source]#

Creates a triangle plot of an MC sample of the surrogate model, and optionally shows some evaluation locations.

Parameters:

samples_or_samples_folder (cobaya.SampleCollection, getdist.MCSamples, str) – MC samples returned by a call to the Runner.generate_mc_sample() method, the output path where they were written, or a getdist.MCSamples instance. If not specified (default) it will try to use the last set of samples generated.
add_training (bool, optional (default=True)) – Whether the training locations are plotted on top of the contours.
add_samples (dict(label, (cobaya.SampleCollection, getdist.MCSamples, str))) – Extra MC samples to be added to the plot, specified as dict with labels as keys, and the same type as samples_or_samples_folder as values. Default: None.
output (str or os.path, optional (default=None)) – The location to save the generated plot in. If None it will be saved in checkpoint_path/images/surrogate_corner.pdf or ./images/surrogate_corner.png if checkpoint_path is None
output_dpi (int (default: 200)) – The resolution of the generated plot in DPI.
ext (str (default: “png” if output not defined; else ignore)) – Format for the plot.
close (bool (default: False)) – Whether to close the figure (it cannot be shown with plt.show() after calling this method, but it will be saved). It should be True if called multiple times during a run, to avoid wasting memory.

plot_distance_distribution(samples_or_samples_folder=None, show_added=True, output=None, output_dpi=200, ext='png')[source]#

Plots the distance distribution of the training points with respect to the confidence ellipsoids (in a Gaussian approx) derived from an MC sample of the surrogate model.

Parameters:

samples_or_samples_folder (cobaya.SampleCollection, getdist.MCSamples, str) – MC samples returned by a call to the Runner.generate_mc_sample() method, the output path where they were written, or a getdist.MCSamples instance. If not specified (default) it will try to use the last set of samples generated.
show_added (bool (default True)) – Colours the stacks depending on how early or late the corresponding points were added (bluer stacks represent newer points).
output (str or os.path, optional (default=None)) – The location to save the generated plot in. If None it will be saved in .png format at checkpoint_path/images/, or ./images/ if checkpoint_path was None.
output_dpi (int (default: 200)) – The resolution of the generated plot in DPI.
ext (str (default: “png” if output not defined; else ignore)) – Format for the plot

plot_progress(ext='png', timing=True, convergence=False, trace=True, slices=False, corner=False, corner_final=None, close=False)[source]#

Creates some progress plots and saves them at path (assumes path exists).

Parameters:

ext (str (default "png")) – Format for the plots, among the available ones in matplotlib.
timing (bool (default: True)) – Plot histogram of timing per iteration (totals in legend).
convergence (bool (default: True)) – Plot the evolution of the convergence criterion (included in trace plot).
trace (bool (default: True)) – Plot the evolution of the run: convergence criterion, surrogate log(p) and parameters.
slices (bool (default: False)) – Plots slices per training samples. Slow – use for diagnosis only.
corner (bool (default: False)) – Creates a corner plot per iteration (contours for current GP shown only if using NORA). Slow – use for diagnosis only.
corner_final (bool, optional (default: None)) – Whether the final corner plot is created. Needs a surrogate mc sample. If undefined, it is created only if the run has converged.
close (bool (default: False)) – Whether to close the figures (they cannot be shown with plt.show() after calling this method, but they will be saved). It should be True if called multiple times during a run, to avoid wasting memory.

run

Contents

run#

`run`#