BQP#
purpose#
The bqp package uses a preconditioned, projected-gradient method to
solve a given bound-constrained convex quadratic program.
The aim is to minimize the quadratic objective function
See Section 4 of $GALAHAD/doc/bqp.pdf for a brief description of the method employed and other details.
terminology#
Any required solution \(x\) necessarily satisfies the primal optimality conditions
method#
Projected-gradient methods iterate towards a point that satisfies these conditions by ultimately aiming to satisfy \(H x + g = z\) and \(z = z_l + z_u\), while satifying the remaining optimality conditions at each stage. Appropriate norms of the amounts by which the optimality conditions fail to be satisfied are known as the primal and dual infeasibility, and the violation of complementary slackness, respectively.
The method is iterative. Each iteration proceeds in two stages. Firstly, the so-called generalized Cauchy point for the quadratic objective is found. (The purpose of this point is to ensure that the algorithm converges and that the set of bounds which are satisfied as equations at the solution is rapidly identified.) Thereafter an improvement to the objective is sought using either a direct-matrix or truncated conjugate-gradient algorithm.
reference#
This is a specialised version of the method presented in
A. R. Conn, N. I. M. Gould and Ph. L. Toint, Global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM Journal on Numerical Analysis 25 (1988) 433-460.
matrix storage#
The symmetric \(n\) by \(n\) matrix \(H\) may be presented and stored in a variety of formats. But crucially symmetry is exploited by only storing values from the lower triangular part (i.e, those entries that lie on or below the leading diagonal).
Dense storage format: The matrix \(H\) is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. Since \(H\) is symmetric, only the lower triangular part (that is the part \(H_{ij}\) for \(0 \leq j \leq i \leq n-1\)) need be held. In this case the lower triangle should be stored by rows, that is component \(i * i / 2 + j\) of the storage array H_val will hold the value \(H_{ij}\) (and, by symmetry, \(H_{ji}\)) for \(0 \leq j \leq i \leq n-1\). The string H_type = ‘dense’ should be specified.
Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the \(l\)-th entry, \(0 \leq l \leq ne-1\), of \(H\), its row index i, column index j and value \(H_{ij}\), \(0 \leq j \leq i \leq n-1\), are stored as the \(l\)-th components of the integer arrays H_row and H_col and real array H_val, respectively, while the number of nonzeros is recorded as H_ne = \(ne\). Note that only the entries in the lower triangle should be stored. The string H_type = ‘coordinate’ should be specified.
Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of \(H\) the i-th component of the integer array H_ptr holds the position of the first entry in this row, while H_ptr(n) holds the total number of entries. The column indices j, \(0 \leq j \leq i\), and values \(H_{ij}\) of the entries in the i-th row are stored in components l = H_ptr(i), …, H_ptr(i+1)-1 of the integer array H_col, and real array H_val, respectively. Note that as before only the entries in the lower triangle should be stored. For sparse matrices, this scheme almost always requires less storage than its predecessor. The string H_type = ‘sparse_by_rows’ should be specified.
Diagonal storage format: If \(H\) is diagonal (i.e., \(H_{ij} = 0\) for all \(0 \leq i \neq j \leq n-1\)) only the diagonals entries \(H_{ii}\), \(0 \leq i \leq n-1\) need be stored, and the first n components of the array H_val may be used for the purpose. The string H_type = ‘diagonal’ should be specified.
Multiples of the identity storage format: If \(H\) is a multiple of the identity matrix, (i.e., \(H = \alpha I\) where \(I\) is the n by n identity matrix and \(\alpha\) is a scalar), it suffices to store \(\alpha\) as the first component of H_val. The string H_type = ‘scaled_identity’ should be specified.
The identity matrix format: If \(H\) is the identity matrix, no values need be stored. The string H_type = ‘identity’ should be specified.
The zero matrix format: The same is true if \(H\) is the zero matrix, but now the string H_type = ‘zero’ or ‘none’ should be specified.
functions#
- bqp.initialize()#
Set default option values and initialize private data
Returns:
- optionsdict
- dictionary containing default control options:
- errorint
error and warning diagnostics occur on stream error.
- outint
general output occurs on stream out.
- print_levelint
the level of output required is specified by print_level. Possible values are
<=0
gives no output,
1
gives a one-line summary for every iteration.
2
gives a summary of the inner iteration for each iteration.
>=3
gives increasingly verbose (debugging) output.
- start_printint
on which iteration to start printing.
- stop_printint
on which iteration to stop printing.
- print_gapint
how many iterations between printing.
- maxitint
how many iterations to perform (-ve reverts to HUGE(1)-1).
- cold_startint
cold_start should be set to 0 if a warm start is required (with variable assigned according to B_stat, see below), and to any other value if the values given in prob.X suffice.
- ratio_cg_vs_sdint
the ratio of how many iterations use CG rather steepest descent.
- change_maxint
the maximum number of per-iteration changes in the working set permitted when allowing CG rather than steepest descent.
- cg_maxitint
how many CG iterations to perform per BQP iteration (-ve reverts to n+1).
- sif_file_deviceint
the unit number to write generated SIF file describing the current problem.
- infinityfloat
any bound larger than infinity in modulus will be regarded as infinite.
- stop_pfloat
the required accuracy for the primal infeasibility.
- stop_dfloat
the required accuracy for the dual infeasibility.
- stop_cfloat
the required accuracy for the complementary slackness.
- identical_bounds_tolfloat
any pair of constraint bounds (x_l,x_u) that are closer than i dentical_bounds_tol will be reset to the average of their values.
- stop_cg_relativefloat
the CG iteration will be stopped as soon as the current norm of the preconditioned gradient is smaller than max( stop_cg_relative * initial preconditioned gradient, stop_cg_absolute).
- stop_cg_absolutefloat
see stop_cg_relative.
- zero_curvaturefloat
threshold below which curvature is regarded as zero.
- cpu_time_limitfloat
the maximum CPU time allowed (-ve = no limit).
- exact_arcsearchbool
exact_arcsearch is True if an exact arcsearch is required, and False if approximation suffices.
- space_criticalbool
if space_critical is True, every effort will be made to use as little space as possible. This may result in longer computation times.
- deallocate_error_fatalbool
if deallocate_error_fatal is True, any array/pointer deallocation error will terminate execution. Otherwise, computation will continue.
- generate_sif_filebool
if generate_sif_file is True, a SIF file describing the current problem will be generated.
- sif_file_namestr
name (max 30 characters) of generated SIF file containing input problem.
- prefixstr
all output lines will be prefixed by the string contained in quotes within
prefix, e.g. ‘word’ (note the qutoes) will result in the prefix word.- sbls_optionsdict
default control options for SBLS (see
sbls.initialize).
- bqp.load(n, H_type, H_ne, H_row, H_col, H_ptr, options=None)#
Import problem data into internal storage prior to solution.
Parameters:
- nint
holds the number of variables.
- H_typestring
specifies the symmetric storage scheme used for the Hessian \(H\). It should be one of ‘coordinate’, ‘sparse_by_rows’, ‘dense’, ‘diagonal’, ‘scaled_identity’, ‘identity’, ‘zero’ or ‘none’; lower or upper case variants are allowed.
- H_neint
holds the number of entries in the lower triangular part of \(H\) in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
- H_rowndarray(H_ne)
holds the row indices of the lower triangular part of \(H\) in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes, and in this case can be None.
- H_colndarray(H_ne)
holds the column indices of the lower triangular part of \(H\) in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the other storage schemes are used, and in this case can be None.
- H_ptrndarray(n+1)
holds the starting position of each row of the lower triangular part of \(H\), as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be None.
- optionsdict, optional
dictionary of control options (see
bqp.initialize).
- bqp.solve_qp(n, f, g, H_ne, H_val, x_l, x_u, x, z)#
Find a solution to the bound-constrained convex quadratic program involving the quadratic objective function \(q(x)\).
Parameters:
- nint
holds the number of variables.
- ffloat
holds the constant term \(f\) in the objective function.
- gndarray(n)
holds the values of the linear term \(g\) in the objective function.
- H_neint
holds the number of entries in the lower triangular part of the Hessian \(H\).
- H_valndarray(H_ne)
holds the values of the nonzeros in the lower triangle of the Hessian \(H\) in the same order as specified in the sparsity pattern in
bqp.load.- x_lndarray(n)
holds the values of the lower bounds \(x_l\) on the variables. The lower bound on any component of \(x\) that is unbounded from below should be set no larger than minus
options.infinity.- x_undarray(n)
holds the values of the upper bounds \(x_l\) on the variables. The upper bound on any component of \(x\) that is unbounded from above should be set no smaller than
options.infinity.- xndarray(n)
holds the initial estimate of the minimizer \(x\), if known. This is not crucial, and if no suitable value is known, then any value, such as \(x=0\), suffices and will be adjusted accordingly.
- zndarray(n)
holds the initial estimate of the dual variables \(z\) associated with the simple bound constraints, if known. This is not crucial, and if no suitable value is known, then any value, such as \(z=0\), suffices and will be adjusted accordingly.
Returns:
- xndarray(n)
holds the values of the approximate minimizer \(x\) after a successful call.
- zndarray(n)
holds the values of the dual variables associated with the simple bound constraints.
- x_statndarray(n)
holds the return status for each variable. The i-th component will be negative if the \(i\)-th variable lies on its lower bound, positive if it lies on its upper bound, and zero if it lies between bounds.
- [optional] bqp.information()
Provide optional output information
Returns:
- informdict
- dictionary containing output information:
- statusint
return status. Possible values are:
0
The run was successful.
-1
An allocation error occurred. A message indicating the offending array is written on unit options[‘error’], and the returned allocation status and a string containing the name of the offending array are held in inform[‘alloc_status’] and inform[‘bad_alloc’] respectively.
-2
A deallocation error occurred. A message indicating the offending array is written on unit options[‘error’] and the returned allocation status and a string containing the name of the offending array are held in inform[‘alloc_status’] and inform[‘bad_alloc’] respectively.
-3
The restriction n > 0 or m > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’, ‘scaled_identity’, ‘identity’, ‘zero’ or ‘none’ has been violated.
-4
The bound constraints are inconsistent.
-7
The objective function appears to be unbounded from below on the feasible set.
-9
The analysis phase of the factorization failed; the return status from the factorization package is given by inform[‘factor_status’].
-10
The factorization failed; the return status from the factorization package is given by inform[‘factor_status’].
-11
The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given by inform[‘factor_status’].
-16
The problem is so ill-conditioned that further progress is impossible.
-18
Too many iterations have been performed. This may happen if options[‘maxit’] is too small, but may also be symptomatic of a badly scaled problem.
-19
The CPU time limit has been reached. This may happen if options[‘cpu_time_limit’] is too small, but may also be symptomatic of a badly scaled problem.
-20
The Hessian \(H\) appears to be indefinite.
-23
An entry from the strict upper triangle of \(H\) has been specified.
- alloc_statusint
the status of the last attempted allocation/deallocation.
- bad_allocstr
the name of the array for which an allocation/deallocation error occurred.
- factorization_statusint
status return from factorization.
- iterint
number of iterations required.
- cg_iterint
number of CG iterations required.
- objfloat
current value of the objective function.
- norm_pgfloat
current value of the projected gradient.
- timedict
- dictionary containing timing information:
- totalfloat
total time.
- analysefloat
time for the analysis phase.
- factorizefloat
time for the factorization phase.
- solvefloat
time for the linear solution phase.
- sbls_informdict
inform parameters for SBLS (see
sbls.information).
- bqp.terminate()#
Deallocate all internal private storage.
example code#
from galahad import bqp
import numpy as np
np.set_printoptions(precision=4,suppress=True,floatmode='fixed')
print("\n** python test: bqp")
# set parameters
n = 3
infinity = float("inf")
# describe objective function
f = 1.0
g = np.array([0.0,2.0,0.0])
H_type = 'coordinate'
H_ne = 4
H_row = np.array([0,1,2,2])
H_col = np.array([0,1,1,2])
H_ptr = None
H_val = np.array([1.0,2.0,1.0,3.0])
# describe constraints
x_l = np.array([-1.0,-infinity,-infinity])
#x_u = np.array([1.0,infinity,2.0])
x_u = np.array([1.0,infinity,0.0])
# allocate internal data and set default options
options = bqp.initialize()
# set some non-default options
options['print_level'] = 0
#print("options:", options)
# load data (and optionally non-default options)
bqp.load(n, H_type, H_ne, H_row, H_col, H_ptr, options)
# provide starting values (not crucial)
x = np.array([0.0,0.0,0.0])
z = np.array([0.0,0.0,0.0])
# find optimum of qp
#print("\nsolve bqp")
x, z, x_stat = bqp.solve_qp(n, f, g, H_ne, H_val, x_l, x_u, x, z)
print(" x:",x)
print(" z:",z)
print(" x_stat:",x_stat)
# get information
inform = bqp.information()
print(" f: %.4f" % inform['obj'])
print('** bqp exit status:', inform['status'])
# deallocate internal data
bqp.terminate()This example code is available in $GALAHAD/src/bqp/Python/test_bqp.py .