LPA

purpose

The lpa package uses the simplex method to solve a given linear program (LP). The aim is to minimize the linear objective function

\[q(x) = f + g^T x\]

subject to the general linear constraints and simple bounds

\[c_l \leq A x \leq c_u \;\;\mbox{and} \;\; x_l \leq x \leq x_u,\]

where \(A\) is a given \(m\) by \(n\) matrix, \(g\) is a vector, \(f\) is a scalar, and any of the components of the vectors \(c_l\), \(c_u\), \(x_l\) or \(x_u\) may be infinite. The method offers the choice of direct and iterative solution of the key regularization subproblems, and is most suitable for problems involving a large number of unknowns \(x\).

See Section 4 of $GALAHAD/doc/lpa.pdf for a brief description of the method employed and other details.

N.B. The package is simply a sophisticated interface to the HSL package LA04, and requires that a user has obtained the latter. LA04 is not included in GALAHAD but is available without charge to recognised academics, see http://www.hsl.rl.ac.uk/catalogue/la04.html. If LA04 is unavailable, the interior- point linear programming package lpb is an alternative.

terminology

Any required solution \(x\) necessarily satisfies the primal optimality conditions

\[A x = c\]

and

\[c_l \leq c \leq c_u, \;\; x_l \leq x \leq x_u,\]

the dual optimality conditions

\[g = A^{T} y + z,\;\; y = y_l + y_u \;\;\mbox{and}\;\; z = z_l + z_u,\]

and

\[y_l \geq 0, \;\; y_u \leq 0, \;\; z_l \geq 0 \;\;\mbox{and}\;\; z_u \leq 0,\]

and the complementary slackness conditions

\[( A x - c_l )^{T} y_l = 0,\;\; ( A x - c_u )^{T} y_u = 0,\;\; (x -x_l )^{T} z_l = 0 \;\;\mbox{and}\;\;(x -x_u )^{T} z_u = 0,\]

where the vectors \(y\) and \(z\) are known as the Lagrange multipliers for the general linear constraints, and the dual variables for the bounds, respectively, and where the vector inequalities hold component-wise.

The so-called dual to this problem is another linear program

\[- \mbox{minimize} \;\; c^{lT} y^l + c^{uT} y^u + x^{lT} z^l + x^{uT} z^u + f \;\; \mbox{subject to the dual optimality conditions}\]

that uses the same data. The solution to the two problems, it is exists, is the same, but if one is infeasible, the other is unbounded. It can be more efficient to solve the dual, particularly if \(m\) is much larger than \(n\).

method

The bulk of the work is peformed by the HSL package LA04. The main subbroutine from this package requires that the input problem be transformed into the ``standard form’’

\[\begin{split}\begin{array}{rl} \mbox{minimize} & g'^T x' \\ \mbox{subject to} & A' x' = b \\ & l_i \leq x'_i \leq u_i, \;\; (i\leq k) \\ \mbox{and} & x'_l \geq 0, \;\; (i \geq l) \end{array}\end{split}\]

by introducing slack an surpulus variables, reordering and removing fixed variables and free constraints. The resulting problem involves \(n'\) unknowns and \(m'\) general constraints. In order to deal with the possibility that the general constraints are inconsistent or not of full rank, LA04 introduces additional ``artifical’’ variables \(v\), replaces the constraints of the standard-form LP by the enlaarged set

\[A' x' + v = b,\]

and gradually encourages \(v\) to zero as a first solution phase.

Once a selection of \(m'\) independent (non-basic) variables is made, the enlarged constraints determine the remaining \(m'\) dependent ({basic) variables. The simplex method is a scheme for systematically adjusting the choice of basic and non-basic variables until a set which defines an optimal solution of the standard-form LP is obtained. Each iteration of the simplex method requires the solution of a number of sets of linear equations whose coefficient matrix is the basis matrix \(B\), made up of the columns of \([A' \;\; I]\) corresponding to the basic variables, or its transpose \(B^T\). As the basis matrices for consecutive iterations are closely related, it is normally advantageous to update (rather than recompute) their factorizations as the computation proceeds. If an initial basis is not provided by the user, a set of basic variables which provide a (permuted) triangular basis matrix is found by the simple crash algorithm of Gould and Reid (1989), and initial steepest-edge weights are calculated.

Phases one (finding a feasible solution) and two (solving the standard-form LP) of the simplex method are applied, as appropriate, with the choice of entering variable as described by Goldfarb and Reid (1977) and the choice of leaving variable as proposed by Harris (1973). Refactorizations of the basis matrix are performed whenever doing so will reduce the average iteration time or there is insufficient memory for its factors. The reduced cost for the entering variable is computed afresh. If it is found to be of a different sign from the recurred value or more than 10% different in magnitude, a fresh computation of all the reduced costs is performed. Details of the factorization and updating procedures are given by Reid (1982). Iterative refinement is encouraged for the basic solution and for the reduced costs after each factorization of the basis matrix and when they are recomputed at the end of phase 1.

references

D. Goldfarb and J. K. Reid, ``A practicable steepest-edge simplex algorithm’’. Mathematical Programming 12 (1977) 361-371.

N. I. M. Gould and J. K. Reid, ``New crash procedures for large systems of linear constraints’’. Mathematical Programming 45 (1989) 475-501.

P. M. J. Harris, ``Pivot selection methods of the Devex LP code’’. Mathematical Programming 5 (1973) 1-28.

J. K. Reid, ``A sparsity-exploiting variant of the Bartels-Golub decomposition for linear-programming bases’’. Mathematical Programming 24 (1982) 55-69.

matrix storage

The unsymmetric \(m\) by \(n\) matrix \(A\) may be presented and stored in a variety of convenient input formats.

Dense storage format: The matrix \(A\) is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. In this case, component \(n \ast i + j\) of the storage array A_val will hold the value \(A_{ij}\) for \(0 \leq i \leq m-1\), \(0 \leq j \leq n-1\). The string A_type = ‘dense’ should be specified.

Dense by columns storage format: The matrix \(A\) is stored as a compact dense matrix by columns, that is, the values of the entries of each column in turn are stored in order within an appropriate real one-dimensional array. In this case, component \(m \ast j + i\) of the storage array A_val will hold the value \(A_{ij}\) for \(0 \leq i \leq m-1\), \(0 \leq j \leq n-1\). The string A_type = ‘dense_by_columns’ should be specified.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the \(l\)-th entry, \(0 \leq l \leq ne-1\), of \(A\), its row index i, column index j and value \(A_{ij}\), \(0 \leq i \leq m-1\), \(0 \leq j \leq n-1\), are stored as the \(l\)-th components of the integer arrays A_row and A_col and real array A_val, respectively, while the number of nonzeros is recorded as A_ne = \(ne\). The string A_type = ‘coordinate’should be specified.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of \(A\) the i-th component of the integer array A_ptr holds the position of the first entry in this row, while A_ptr(m) holds the total number of entries. The column indices j, \(0 \leq j \leq n-1\), and values \(A_{ij}\) of the nonzero entries in the i-th row are stored in components l = A_ptr(i), \(\ldots\), A_ptr(i+1)-1, \(0 \leq i \leq m-1\), of the integer array A_col, and real array A_val, respectively. For sparse matrices, this scheme almost always requires less storage than its predecessor. The string A_type = ‘sparse_by_rows’ should be specified.

Sparse column-wise storage format: Once again only the nonzero entries are stored, but this time they are ordered so that those in column j appear directly before those in column j+1. For the j-th column of \(A\) the j-th component of the integer array A_ptr holds the position of the first entry in this column, while A_ptr(n) holds the total number of entries. The row indices i, \(0 \leq i \leq m-1\), and values \(A_{ij}\) of the nonzero entries in the j-th columns are stored in components l = A_ptr(j), \(\ldots\), A_ptr(j+1)-1, \(0 \leq j \leq n-1\), of the integer array A_row, and real array A_val, respectively. As before, for sparse matrices, this scheme almost always requires less storage than the co-ordinate format. The string A_type = ‘sparse_by_columns’ should be specified.

functions

lpa.initialize()

Set default option values and initialize private data

Returns:

optionsdict

dictionary containing default control options:

errorint

error and warning diagnostics occur on stream error.

outint

general output occurs on stream out.

print_levelint

the level of output required is specified by print_level. Possible values are

• <=0

  gives no output,

• 1

  gives a summary.

• 2

  gives a summary of the inner iteration for each iteration by enabling output from LA04.

• >=3

  gives increasingly verbose (debugging) output.

print_levelint

the level of output required is specified by print_level (>= 2 turns on LA04 output).

start_printint

any printing will start on this iteration.

stop_printint

any printing will stop on this iteration.

maxitint

at most maxit inner iterations are allowed.

max_iterative_refinementsint

maximum number of iterative refinements allowed.

min_real_factor_sizeint

initial size for real array for the factors and other data.

min_integer_factor_sizeint

initial size for integer array for the factors and other data.

random_number_seedint

the initial seed used when generating random numbers.

sif_file_deviceint

specifies the unit number to write generated SIF file describing the current problem.

qplib_file_deviceint

specifies the unit number to write generated QPLIB file describing the current problem.

infinityfloat

any bound larger than infinity in modulus will be regarded as infinite.

tol_datafloat

the tolerable relative perturbation of the data (\(A\),\(g,\ldots\)) defining the problem.

feas_tolfloat

any constraint violated by less than feas_tol will be considered to be satisfied.

relative_pivot_tolerancefloat

pivot threshold used to control the selection of pivot elements in the matrix factorization. Any potential pivot which is less than the largest entry in its row times the threshold is excluded as a candidate.

growth_limitfloat

limit to control growth in the upated basis factors. A refactorization occurs if the growth exceeds this limit.

zero_tolerancefloat

any entry in the basis smaller than this is considered zero.

change_tolerancefloat

any solution component whose change is smaller than a tolerence times the largest change may be considered to be zero.

identical_bounds_tolfloat

any pair of constraint bounds (c_l,c_u) or (x_l,x_u) that are closer than identical_bounds_tol will be reset to the average of their values.

cpu_time_limitfloat

the maximum CPU time allowed (-ve means infinite).

clock_time_limitfloat

the maximum elapsed clock time allowed (-ve means infinite).

scalebool

if scale is True, the problem will be automatically scaled prior to solution. This may improve computation time and accuracy.

dualbool

should the dual problem be solved rather than the primal?.

warm_startbool

should a warm start using the data in C_stat and X_stat be attempted?.

steepest_edgebool

should steepest-edge weights be used to detetrmine the variable leaving the basis?.

space_criticalbool

if space_critical is True, every effort will be made to use as little space as possible. This may result in longer computation time.

deallocate_error_fatalbool

if deallocate_error_fatal is True, any array/pointer deallocation error will terminate execution. Otherwise, computation will continue.

generate_sif_filebool

if generate_sif_file is True, a SIF file describing the current problem is to be generated.

generate_qplib_filebool

if generate_qplib_file is True, a QPLIB file describing the current problem is to be generated.

sif_file_namestr

name of generated SIF file containing input problem.

qplib_file_namestr

name of generated QPLIB file containing input problem.

prefixstr

all output lines will be prefixed by the string contained in quotes within prefix, e.g. ‘word’ (note the qutoes) will result in the prefix word.

lpa.load(n, m, A_type, A_ne, A_row, A_col, A_ptr, options=None)

Import problem data into internal storage prior to solution.

Parameters:

nint

holds the number of variables.

mint

holds the number of constraints.

A_typestring

specifies the unsymmetric storage scheme used for the constraints Jacobian \(A\). It should be one of ‘coordinate’, ‘sparse_by_rows’ or ‘dense’; lower or upper case variants are allowed.

A_neint

holds the number of entries in \(A\) in the sparse co-ordinate storage scheme. It need not be set for any of the other two schemes.

A_rowndarray(A_ne)

holds the row indices of \(A\) in the sparse co-ordinate storage scheme. It need not be set for any of the other two schemes, and in this case can be None.

A_colndarray(A_ne)

holds the column indices of \(A\) in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense storage scheme is used, and in this case can be None.

A_ptrndarray(m+1)

holds the starting position of each row of \(A\), as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be None.

optionsdict, optional

dictionary of control options (see lpa.initialize).

lpa.solve_lp(n, m, f, g, A_ne, A_val, c_l, c_u, x_l, x_u)

Find a solution to the convex quadratic program involving the quadratic objective function \(q(x)\).

Parameters:

nint

holds the number of variables.

mint

holds the number of residuals.

ffloat

holds the constant term \(f\) in the objective function.

gndarray(n)

holds the values of the linear term \(g\) in the objective function.

A_neint

holds the number of entries in the constraint Jacobian \(A\).

A_valndarray(A_ne)

holds the values of the nonzeros in the constraint Jacobian \(A\) in the same order as specified in the sparsity pattern in lpa.load.

c_lndarray(m)

holds the values of the lower bounds \(c_l\) on the constraints The lower bound on any component of \(A x\) that is unbounded from below should be set no larger than minus options.infinity.

c_undarray(m)

holds the values of the upper bounds \(c_l\) on the constraints The upper bound on any component of \(A x\) that is unbounded from above should be set no smaller than options.infinity.

x_lndarray(n)

holds the values of the lower bounds \(x_l\) on the variables. The lower bound on any component of \(x\) that is unbounded from below should be set no larger than minus options.infinity.

x_undarray(n)

holds the values of the upper bounds \(x_l\) on the variables. The upper bound on any component of \(x\) that is unbounded from above should be set no smaller than options.infinity.

xndarray(n)

holds the initial estimate of the minimizer \(x\), if known. This is not crucial, and if no suitable value is known, then any value, such as \(x=0\), suffices and will be adjusted accordingly.

yndarray(m)

holds the initial estimate of the Lagrange multipliers \(y\) associated with the general constraints, if known. This is not crucial, and if no suitable value is known, then any value, such as \(y=0\), suffices and will be adjusted accordingly.

zndarray(n)

holds the initial estimate of the dual variables \(z\) associated with the simple bound constraints, if known. This is not crucial, and if no suitable value is known, then any value, such as \(z=0\), suffices and will be adjusted accordingly.

Returns:

xndarray(n)

holds the values of the approximate minimizer \(x\) after a successful call.

cndarray(m)

holds the values of the residuals \(c(x) = Ax\).

yndarray(m)

holds the values of the Lagrange multipliers associated with the general linear constraints.

zndarray(n)

holds the values of the dual variables associated with the simple bound constraints.

c_statndarray(m)

holds the return status for each constraint. The i-th component will be negative if the value of the \(i\)-th constraint \((Ax)_i\)) lies on its lower bound, positive if it lies on its upper bound, and zero if it lies between bounds.

x_statndarray(n)

holds the return status for each variable. The i-th component will be negative if the \(i\)-th variable lies on its lower bound, positive if it lies on its upper bound, and zero if it lies between bounds.

[optional] lpa.information()

Provide optional output information

Returns:

informdict

dictionary containing output information:

statusint

return status. Possible values are:

• 0

  The run was successful.

• -1

  An allocation error occurred. A message indicating the offending array is written on unit options[‘error’], and the returned allocation status and a string containing the name of the offending array are held in inform[‘alloc_status’] and inform[‘bad_alloc’] respectively.

• -2

  A deallocation error occurred. A message indicating the offending array is written on unit options[‘error’] and the returned allocation status and a string containing the name of the offending array are held in inform[‘alloc_status’] and inform[‘bad_alloc’] respectively.

• -3

  The restriction n > 0 or m > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’ or ‘sparse_by_rows’ has been violated.

• -4

  The bound constraints are inconsistent.

• -5

  The constraints appear to have no feasible point.

• -7

  The objective function appears to be unbounded from below on the feasible set.

• -9

  The analysis phase of the factorization failed; the return status from the factorization package is given by inform[‘factor_status’].

• -10

  The factorization failed; the return status from the factorization package is given by inform[‘factor_status’].

• -11

  The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given by inform[‘factor_status’].

• -16

  The problem is so ill-conditioned that further progress is impossible.

• -18

  Too many iterations have been performed. This may happen if options[‘maxit’] is too small, but may also be symptomatic of a badly scaled problem.

• -19

  The CPU time limit has been reached. This may happen if options[‘cpu_time_limit’] is too small, but may also be symptomatic of a badly scaled problem.

alloc_statusint

the status of the last attempted allocation/deallocation.

bad_allocstr

the name of the array for which an allocation/deallocation error occurred.

iterint

the total number of iterations required.

la04_jobint

the final value of LA04’s job argument.

la04_job_infoint

any extra information from an unsuccessfull call to LA04 (LA04’s RINFO(35).

objfloat

the value of the objective function at the best estimate of the solution.

primal_infeasibilityfloat

the value of the primal infeasibility.

feasiblebool

is the returned “solution” feasible?.

RINFOndarray(40)

the information array from LA04.

timedict

dictionary containing timing information:

totalfloat

the total CPU time spent in the package.

preprocessfloat

the CPU time spent preprocessing the problem.

clock_totalfloat

the total clock time spent in the package.

clock_preprocessfloat

the clock time spent preprocessing the problem.

rpd_informdict

inform parameters for RPD (see rpd.information).

lpa.terminate()

Deallocate all internal private storage.

example code

from galahad import lpa
import numpy as np
np.set_printoptions(precision=4,suppress=True,floatmode='fixed')
print("\n** python test: lpa")

# set parameters
n = 3
m = 2
infinity = float("inf")

#  describe objective function

f = 1.0
g = np.array([0.0,2.0,0.0])

#  describe constraints

A_type = 'coordinate'
A_ne = 4
A_row = np.array([0,0,1,1])
A_col = np.array([0,1,1,2])
A_ptr = None
A_val = np.array([2.0,1.0,1.0,1.0])
c_l = np.array([1.0,2.0])
c_u = np.array([2.0,2.0])
x_l = np.array([-1.0,-infinity,-infinity])
x_u = np.array([1.0,infinity,2.0])

# allocate internal data and set default options
options = lpa.initialize()

# set some non-default options
options['print_level'] = 0
#print("options:", options)

# load data (and optionally non-default options)
lpa.load(n, m, A_type, A_ne, A_row, A_col, A_ptr, options)

#  provide starting values (not crucial)

x = np.array([0.0,0.0,0.0])
y = np.array([0.0,0.0])
z = np.array([0.0,0.0,0.0])

# find optimum of lp
#print("\n solve lp")
x, c, y, z, x_stat, c_stat \
  = lpa.solve_lp(n, m, f, g, A_ne, A_val,
                 c_l, c_u, x_l, x_u, x, y, z)
print(" x:",x)
print(" c:",c)
print(" y:",y)
print(" z:",z)
print(" x_stat:",x_stat)
print(" c_stat:",c_stat)

# get information
inform = lpa.information()
print(" f: %.4f" % inform['obj'])
print('** lpa exit status:', inform['status'])

# deallocate internal data

lpa.terminate()

This example code is available in $GALAHAD/src/lpa/Python/test_lpa.py .