scnn.optimize

Optimize neural networks using convex reformulations.

scnn.optimize.optimize(formulation: typing_extensions.Literal[gated_relu, relu], max_neurons: int, X_train: ndarray, y_train: ndarray, X_test: ndarray | None = None, y_test: ndarray | None = None, regularizer: Regularizer | None = None, bias: bool = False, return_convex: bool = False, unitize_data: bool = True, verbose: bool = False, log_file: str | None = None, device: typing_extensions.Literal[cpu, cuda] = 'cpu', dtype: typing_extensions.Literal[float32, float64] = 'float32', seed: int = 778) → Tuple[Model, Metrics]

Convenience function for training neural networks by convex reformulation.

Parameters:

• formulation –

  the convex reformulation to solve. Must be one of

  • ”gated_relu”: train a network with Gated ReLU activations.

  • ”relu”: train a network with ReLU activations.

• max_neurons – the maximum number of neurons in the convex reformulation.

• X_train – an \(n \times d\) matrix of training examples.

• y_train – an \(n \times c\) or vector matrix of training targets.

• X_test – an \(m \times d\) matrix of test examples.

• y_test – an \(n \times c\) or vector matrix of test targets.

• regularizer – an optional regularizer for the convex reformulation. Defaults to no regularization.

• bias – whether or not to use a bias in the model.

• return_convex – whether or not to return the convex reformulation instead of the final non-convex model.

• unitize_data – whether or not to unitize the column norms of the training set. This can improve conditioning during optimization.

• verbose – whether or not the solver should print verbosely during optimization.

• log_file – a path to an optional log file.

• device – the device on which to run. Must be one of cpu (run on CPU) or cuda (run on cuda-enabled GPUs if available).

• dtype – the floating-point type to use. “float32” is faster than “float64” but can lead to excessive numerical errors on badly conditioned datasets.

• seed – an integer seed for reproducibility.

Returns:

the optimized model and metrics collected during optimization.

Return type:

(Model, Metrics)

scnn.optimize.optimize_model(model: Model, solver: Optimizer, metrics: Metrics, X_train: ndarray, y_train: ndarray, X_test: ndarray | None = None, y_test: ndarray | None = None, regularizer: Regularizer | None = None, return_convex: bool = False, unitize_data: bool = True, verbose: bool = False, log_file: str | None = None, device: typing_extensions.Literal[cpu, cuda] = 'cpu', dtype: typing_extensions.Literal[float32, float64] = 'float32', seed: int = 778) → Tuple[Model, Metrics]

Train a neural network by convex reformulation.

Parameters:

• model – a convex reformulation of a neural network model.

• solver – the optimizer to use when solving the reformulation.

• metrics – a object specifying which metrics to collect during optimization.

• X_train – an \(n \times d\) matrix of training examples.

• y_train – an \(n \times c\) or vector matrix of training targets.

• X_test – an \(m \times d\) matrix of test examples.

• y_test – an \(n \times c\) or vector matrix of test targets.

• regularizer – an optional regularizer for the convex reformulation.

• return_convex – whether or not to return the convex reformulation instead of the final non-convex model.

• unitize_data – whether or not to unitize the column norms of the training set. This can improve conditioning during optimization.

• verbose – whether or not the solver should print verbosely during optimization.

• log_file – a path to an optional log file.

• device – the device on which to run. Must be one of cpu (run on CPU) or cuda (run on cuda-enabled GPUs if available).

• dtype – the floating-point type to use. “float32” is faster than “float64” but can lead to excessive numerical errors on badly conditioned datasets.

• seed – an integer seed for reproducibility.

Returns:

The optimized model and metrics collected during optimization.

scnn.optimize.optimize_path(model: Model, solver: Optimizer, path: List[Regularizer], metrics: Metrics, X_train: ndarray, y_train: ndarray, X_test: ndarray | None = None, y_test: ndarray | None = None, warm_start: bool = True, save_path: str | None = None, return_convex: bool = False, unitize_data: bool = True, verbose: bool = False, log_file: str | None = None, device: typing_extensions.Literal[cpu, cuda] = 'cpu', dtype: typing_extensions.Literal[float32, float64] = 'float32', seed: int = 778) → Tuple[List[Model | str], List[Metrics]]

Train a neural network by convex reformulation.

Parameters:

• model – a convex reformulation of a neural network model.

• solver – the optimizer to use when solving the reformulation.

• path – a list of regularizer objects specifying the regularization path to explore.

• metrics – a object specifying which metrics to collect during optimization.

• X_train – an \(n \times d\) matrix of training examples.

• y_train – an \(n \times c\) or vector matrix of training targets.

• X_test – an \(m \times d\) matrix of test examples.

• y_test – an \(n \times c\) or vector matrix of test targets.

• warm_start – whether or not to warm-start each optimization problem in the path using the previous solution.

• save_path – string specifying a directory where models in the regularization path should be saved. All models will be retained in memory and returned if save_path = None.

• return_convex – whether or not to return the convex reformulation instead of the final non-convex model.

• unitize_data – whether or not to unitize the column norms of the training set. This can improve conditioning during optimization.

• verbose – whether or not the solver should print verbosely during optimization.

• log_file – a path to an optional log file.

• device – the device on which to run. Must be one of cpu (run on CPU) or cuda (run on cuda-enabled GPUs if available).

• dtype – the floating-point type to use. “float32” is faster than “float64” but can lead to excessive numerical errors on badly conditioned datasets.

• seed – an integer seed for reproducibility.