GALAHAD LLSR package

purpose

Given a real \(m\) by \(n\) model matrix \(A\), a real \(n\) by \(n\) symmetric diagonally-dominant matrix \(S\), a real \(m\) vector of observations \(b\) and scalars \(\sigma>0\) and \(p \geq 2\), the llsr package finds a minimizer of the regularized linear least-squares objective function

\[\frac{1}{2} \| A x - b \|^2_2 + \frac{\sigma}{p}\|x\|_S^p,\]

where the \(S\)-norm of \(x\) is \(\|x\|_S = \sqrt{x^T S x}\). This problem commonly occurs as a subproblem in nonlinear least-squares calculations. The matrix \(S\) need not be provided in the commonly-occurring \(\ell_2\)-regularization case for which \(S = I\), the \(n\) by \(n\) identity matrix.

Factorization of matrices of the form

\[\begin{split}\begin{pmatrix}\lambda S & A^T \\ A & - I\end{pmatrix}\mspace{5em}\mbox{(1)}\end{split}\]

for a succession of scalars \(\lambda\) will be required, so this package is most suited for the case where such a factorization may be found efficiently. If this is not the case, the package lsrt may be preferred.

See Section 4 of $GALAHAD/doc/llsr.pdf for additional details.

method

The required solution \(x_*\) necessarily satisfies the optimality condition \(A^T A x_* + \lambda_* S x_* = A^T b\), where \(\lambda_* = \sigma \|x_*\|^{p-2}\).

The method is iterative, and proceeds in two phases. Firstly, lower and upper bounds, \(\lambda_L\) and \(\lambda_U\), on \(\lambda_*\) are computed using Gershgorin’s theorems and other eigenvalue bounds, including those that may involve the Cholesky factorization of \(S\) The first phase of the computation proceeds by progressively shrinking the bound interval \([\lambda_L,\lambda_U]\) until a value \(\lambda\) for which \(\|x(\lambda)\|_{M} \geq \sigma \|x(\lambda)\|_S^{p-2}\) is found. Here \(x(\lambda)\) and its companion \(y(\lambda)\) are defined to be a solution of

\[(A^T A + \lambda S)x(\lambda) = A^T b. \mspace{5em}\mbox{(2)}\]

Once the terminating \(\lambda\) from the first phase has been discovered, the second phase consists of applying Newton or higher-order iterations to the nonlinear secular equation \(\lambda = \sigma \|x(\lambda)\|_S^{p-2}\) with the knowledge that such iterations are both globally and ultimately rapidly convergent.

The dominant cost is the requirement that we solve a sequence of linear systems (2). This may be rewritten as

\[\begin{split}\begin{pmatrix}\lambda S & A^T \\ A & - I\end{pmatrix} \begin{pmatrix}x(\lambda) \\ y(\lambda)\end{pmatrix} = \begin{pmatrix}A^T b \\ 0\end{pmatrix} \mspace{5em} \mbox{(3)}\end{split}\]

for some auxiliary vector \(y(\lambda)\). In general a sparse symmetric, indefinite factorization of the coefficient matrix of (3) is often preferred to a Cholesky factorization of that of (2).

reference

The method is the obvious adaptation to the linear least-squares problem of that described in detail in

H. S. Dollar, N. I. M. Gould and D. P. Robinson. ``On solving trust-region and other regularised subproblems in optimization’’. Mathematical Programming Computation 2(1) (2010) 21–57.

matrix storage

The unsymmetric \(m\) by \(n\) model matrix \(A\) may be presented and stored in a variety of convenient input formats.

Dense storage format: The matrix \(A\) is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. In this case, component \(n \ast i + j\) of the storage array A_val will hold the value \(A_{ij}\) for \(0 \leq i \leq m-1\), \(0 \leq j \leq n-1\). The string A_type = ‘dense’ should be specified.

Dense by columns storage format: The matrix \(A\) is stored as a compact dense matrix by columns, that is, the values of the entries of each column in turn are stored in order within an appropriate real one-dimensional array. In this case, component \(m \ast j + i\) of the storage array A_val will hold the value \(A_{ij}\) for \(0 \leq i \leq m-1\), $0 \leq j \leq n-1 The string A_type = ‘dense_by_columns’ should be specified.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the \(l\)-th entry, \(0 \leq l \leq ne-1\), of \(A\), its row index i, column index j and value \(A_{ij}\), \(0 \leq i \leq m-1\), \(0 \leq j \leq n-1\), are stored as the \(l\)-th components of the integer arrays A_row and A_col and real array A_val, respectively, while the number of nonzeros is recorded as A_ne = \(ne\). The string A_type = ‘coordinate’should be specified.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of \(A\) the i-th component of the integer array A_ptr holds the position of the first entry in this row, while A_ptr(m) holds the total number of entries. The column indices j, \(0 \leq j \leq n-1\), and values \(A_{ij}\) of the nonzero entries in the i-th row are stored in components l = A_ptr(i), \(\ldots\), A_ptr(i+1)-1, \(0 \leq i \leq m-1\), of the integer array A_col, and real array A_val, respectively. For sparse matrices, this scheme almost always requires less storage than its predecessor. The string A_type = ‘sparse_by_rows’ should be specified.

Sparse column-wise storage format: Once again only the nonzero entries are stored, but this time they are ordered so that those in column j appear directly before those in column j+1. For the j-th column of \(A\) the j-th component of the integer array A_ptr holds the position of the first entry in this column, while A_ptr(n) holds the total number of entries. The row indices i, \(0 \leq i \leq m-1\), and values \(A_{ij}\) of the nonzero entries in the j-th columnsare stored in components l = A_ptr(j), \(\ldots\), A_ptr(j+1)-1, \(0 \leq j \leq n-1\), of the integer array A_row, and real array A_val, respectively. As before, for sparse matrices, this scheme almost always requires less storage than the co-ordinate format. The string A_type = ‘sparse_by_columns’ should be specified.

The symmetric \(n\) by \(n\) scaing matrix \(S\) may also be presented and stored in a variety of formats. But crucially symmetry is exploited by only storing values from the lower triangular part (i.e, those entries that lie on or below the leading diagonal).

Dense storage format: The matrix \(S\) is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. Since \(S\) is symmetric, only the lower triangular part (that is the part \(S_{ij}\) for \(0 \leq j \leq i \leq n-1\)) need be held. In this case the lower triangle should be stored by rows, that is component \(i * i / 2 + j\) of the storage array S_val will hold the value \(S_{ij}\) (and, by symmetry, \(S_{ji}\)) for \(0 \leq j \leq i \leq n-1\). The string S_type = ‘dense’ should be specified.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the \(l\)-th entry, \(0 \leq l \leq ne-1\), of \(S\), its row index i, column index j and value \(S_{ij}\), \(0 \leq j \leq i \leq n-1\), are stored as the \(l\)-th components of the integer arrays S_row and S_col and real array S_val, respectively, while the number of nonzeros is recorded as S_ne = \(ne\). Note that only the entries in the lower triangle should be stored. The string S_type = ‘coordinate’ should be specified.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of \(S\) the i-th component of the integer array S_ptr holds the position of the first entry in this row, while S_ptr(n) holds the total number of entries. The column indices j, \(0 \leq j \leq i\), and values \(S_{ij}\) of the entries in the i-th row are stored in components l = S_ptr(i), …, S_ptr(i+1)-1 of the integer array S_col, and real array S_val, respectively. Note that as before only the entries in the lower triangle should be stored. For sparse matrices, this scheme almost always requires less storage than its predecessor. The string S_type = ‘sparse_by_rows’ should be specified.

Diagonal storage format: If \(S\) is diagonal (i.e., \(S_{ij} = 0\) for all \(0 \leq i \neq j \leq n-1\)) only the diagonals entries \(S_{ii}\), \(0 \leq i \leq n-1\) need be stored, and the first n components of the array S_val may be used for the purpose. The string S_type = ‘diagonal’ should be specified.

Multiples of the identity storage format: If \(S\) is a multiple of the identity matrix, (i.e., \(H = \alpha I\) where \(I\) is the n by n identity matrix and \(\alpha\) is a scalar), it suffices to store \(\alpha\) as the first component of S_val. The string S_type = ‘scaled_identity’ should be specified.

The identity matrix format: If \(S\) is the identity matrix, no values need be stored. The string S_type = ‘identity’ should be specified. Strictly this is not required as \(S\) will be assumed to be \(I\) if it is not explicitly provided.

The zero matrix format: The same is true if \(S\) is the zero matrix, but now the string S_type = ‘zero’ or ‘none’ should be specified.

introduction to function calls

To solve a given problem, functions from the llsr package must be called in the following order:

• llsr_initialize - provide default control parameters and set up initial data structures

• llsr_read_specfile (optional) - override control values by reading replacement values from a file

• llsr_import - set up problem data structures and fixed values

• llsr_import_scaling (optional) - set up problem data structures for \(S\) if required

• llsr_reset_control (optional) - possibly change control parameters if a sequence of problems are being solved

• llsr_solve_problem - solve the regularization problem

• llsr_information (optional) - recover information about the solution and solution process

• llsr_terminate - deallocate data structures

See the examples section for illustrations of use.

callable functions

overview of functions provided

// typedefs

typedef float spc_;
typedef double rpc_;
typedef int ipc_;

// structs

struct llsr_control_type;
struct llsr_history_type;
struct llsr_inform_type;
struct llsr_time_type;

// global functions

void llsr_initialize(
    void **data,
    struct llsr_control_type* control,
    ipc_ *status
);

void llsr_read_specfile(
    struct llsr_control_type* control,
    const char specfile[]
);

void llsr_import(
    struct llsr_control_type* control,
    void **data,
    ipc_ *status,
    ipc_ m,
    ipc_ n,
    const char A_type[],
    ipc_ A_ne,
    const ipc_ A_row[],
    const ipc_ A_col[],
    const ipc_ A_ptr[]
);

void llsr_import_scaling(
    struct llsr_control_type* control,
    void **data,
    ipc_ *status,
    ipc_ n,
    const char S_type[],
    ipc_ S_ne,
    const ipc_ S_row[],
    const ipc_ S_col[],
    const ipc_ S_ptr[]
);

void llsr_reset_control(
    struct llsr_control_type* control,
    void **data,
    ipc_ *status
);

void llsr_solve_problem(
    void **data,
    ipc_ *status,
    ipc_ m,
    ipc_ n,
    const rpc_ power,
    const rpc_ weight,
    ipc_ A_ne,
    const rpc_ A_val[],
    const rpc_ b[],
    rpc_ x[],
    ipc_ S_ne,
    const rpc_ S_val[]
);

void llsr_information(void **data, struct llsr_inform_type* inform, ipc_ *status);

void llsr_terminate(
    void **data,
    struct llsr_control_type* control,
    struct llsr_inform_type* inform
);

typedefs

typedef float spc_

spc_ is real single precision

typedef double rpc_

rpc_ is the real working precision used, but may be changed to float by defining the preprocessor variable SINGLE.

typedef int ipc_

ipc_ is the default integer word length used, but may be changed to int64_t by defining the preprocessor variable INTEGER_64.

function calls

void llsr_initialize(
    void **data,
    struct llsr_control_type* control,
    ipc_ *status
)

Set default control values and initialize private data

Parameters:

data	holds private internal data
control	is a struct containing control information (see llsr_control_type)
status	is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are (currently): 0 The initialization was successful.

void llsr_read_specfile(
    struct llsr_control_type* control,
    const char specfile[]
)

Read the content of a specification file, and assign values associated with given keywords to the corresponding control parameters. An in-depth discussion of specification files is available, and a detailed list of keywords with associated default values is provided in $GALAHAD/src/llsr/LLSR.template. See also Table 2.1 in the Fortran documentation provided in $GALAHAD/doc/llsr.pdf for a list of how these keywords relate to the components of the control structure.

Parameters:

control	is a struct containing control information (see llsr_control_type)
specfile	is a character string containing the name of the specification file

void llsr_import(
    struct llsr_control_type* control,
    void **data,
    ipc_ *status,
    ipc_ m,
    ipc_ n,
    const char A_type[],
    ipc_ A_ne,
    const ipc_ A_row[],
    const ipc_ A_col[],
    const ipc_ A_ptr[]
)

Import problem data into internal storage prior to solution.

Parameters:

control	is a struct whose members provide control paramters for the remaining prcedures (see llsr_control_type)
data	holds private internal data
status	is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are: 1 The import was successful, and the package is ready for the solve phase -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’ or ‘absent’ has been violated.
m	is a scalar variable of type ipc_, that holds the number of residuals, i.e., the number of rows of \(A\). m must be positive.
n	is a scalar variable of type ipc_, that holds the number of variables, i.e., the number of columns of \(A\). n must be positive.
A_type	is a one-dimensional array of type char that specifies the unsymmetric storage scheme used for the constraint Jacobian, \(A\) if any. It should be one of ‘coordinate’, ‘sparse_by_rows’ or ‘dense’; lower or upper case variants are allowed.
A_ne	is a scalar variable of type ipc_, that holds the number of entries in \(A\), if used, in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
A_row	is a one-dimensional array of size A_ne and type ipc_, that holds the row indices of \(A\) in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes, and in this case can be NULL.
A_col	is a one-dimensional array of size A_ne and type ipc_, that holds the column indices of \(A\) in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense or diagonal storage schemes are used, and in this case can be NULL.
A_ptr	is a one-dimensional array of size n+1 and type ipc_, that holds the starting position of each row of \(A\), as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be NULL.

void llsr_import_scaling(
    struct llsr_control_type* control,
    void **data,
    ipc_ *status,
    ipc_ n,
    const char S_type[],
    ipc_ S_ne,
    const ipc_ S_row[],
    const ipc_ S_col[],
    const ipc_ S_ptr[]
)

Import the scaling matrix \(S\) into internal storage prior to solution. Thus must have been preceeded by a call to llsr_import.

Parameters:

control	is a struct whose members provide control paramters for the remaining prcedures (see llsr_control_type)
data	holds private internal data
status	is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are: 1 The import was successful, and the package is ready for the solve phase -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’ or ‘diagonal’ has been violated.
n	is a scalar variable of type ipc_, that holds the number of variables, i.e., the number of rows and columns of \(S\). n must be positive.
S_type	is a one-dimensional array of type char that specifies the symmetric storage scheme used for the matrix \(S\). It should be one of ‘coordinate’, ‘sparse_by_rows’, ‘dense’ or ‘diagonal’; lower or upper case variants are allowed.
S_ne	is a scalar variable of type ipc_, that holds the number of entries in the lower triangular part of \(S\) in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
S_row	is a one-dimensional array of size S_ne and type ipc_, that holds the row indices of the lower triangular part of \(S\) in the sparse co-ordinate storage scheme. It need not be set for any of the other three schemes, and in this case can be NULL.
S_col	is a one-dimensional array of size S_ne and type ipc_, that holds the column indices of the lower triangular part of \(S\) in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense, diagonal or (scaled) identity storage schemes are used, and in this case can be NULL.
S_ptr	is a one-dimensional array of size n+1 and type ipc_, that holds the starting position of each row of the lower triangular part of \(S\), as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be NULL.

void llsr_reset_control(
    struct llsr_control_type* control,
    void **data,
    ipc_ *status
)

Reset control parameters after import if required.

Parameters:

control	is a struct whose members provide control paramters for the remaining prcedures (see llsr_control_type)
data	holds private internal data
status	is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are: The import was successful, and the package is ready for the solve phase

void llsr_solve_problem(
    void **data,
    ipc_ *status,
    ipc_ m,
    ipc_ n,
    const rpc_ power,
    const rpc_ weight,
    ipc_ A_ne,
    const rpc_ A_val[],
    const rpc_ b[],
    rpc_ x[],
    ipc_ S_ne,
    const rpc_ S_val[]
)

Solve the regularization-region problem.

Parameters:

data	holds private internal data
status	is a scalar variable of type ipc_, that gives the entry and exit status from the package. Possible exit values are: 0 The run was successful. -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restrictions n > 0 and m > 0 or requirement that A_type or A_type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’ or ‘diagonal’ has been violated. -9 The analysis phase of the factorization failed; the return status from the factorization package is given in the component inform.factor_status -10 The factorization failed; the return status from the factorization package is given in the component inform.factor_status. -11 The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given in the component inform.factor_status. -15 The matrix \(S\) does not appear to be strictly diagonally dominant. -16 The problem is so ill-conditioned that further progress is impossible. -17 The step is too small to make further impact.
m	is a scalar variable of type ipc_, that holds the number of residuals
n	is a scalar variable of type ipc_, that holds the number of variables
power	is a scalar of type rpc_, that holds the regularization power, \(p\), used. power must be greater than or equal to two.
weight	is a scalar of type rpc_, that holds the regularization power, \(\sigma\), used. weight must be strictly positive
A_ne	is a scalar variable of type ipc_, that holds the number of entries in the observation matrix \(A\).
A_val	is a one-dimensional array of size A_ne and type rpc_, that holds the values of the entries of the observation matrix \(A\) in any of the available storage schemes.
b	is a one-dimensional array of size m and type rpc_, that holds the values \(b\) of observations. The i-th component of b, i = 0, … , m-1, contains \(b_i\).
x	is a one-dimensional array of size n and type rpc_, that holds the values \(x\) of the optimization variables. The j-th component of x, j = 0, … , n-1, contains \(x_j\).
S_ne	is a scalar variable of type ipc_, that holds the number of entries in the scaling matrix \(S\) if it not the identity matrix.
S_val	is a one-dimensional array of size S_ne and type rpc_, that holds the values of the entries of the scaling matrix \(S\) in any of the available storage schemes. If S_val is NULL, \(S\) will be taken to be the identity matrix.

void llsr_information(void **data, struct llsr_inform_type* inform, ipc_ *status)

Provides output information

Parameters:

data	holds private internal data
inform	is a struct containing output information (see llsr_inform_type)
status	is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are (currently): 0 The values were recorded successfully

void llsr_terminate(
    void **data,
    struct llsr_control_type* control,
    struct llsr_inform_type* inform
)

Deallocate all internal private storage

Parameters:

data	holds private internal data
control	is a struct containing control information (see llsr_control_type)
inform	is a struct containing output information (see llsr_inform_type)

available structures

llsr_control_type structure

#include <galahad_llsr.h>

struct llsr_control_type {
    // fields

    bool f_indexing;
    ipc_ error;
    ipc_ out;
    ipc_ print_level;
    ipc_ new_a;
    ipc_ new_s;
    ipc_ max_factorizations;
    ipc_ taylor_max_degree;
    rpc_ initial_multiplier;
    rpc_ lower;
    rpc_ upper;
    rpc_ stop_normal;
    bool use_initial_multiplier;
    bool space_critical;
    bool deallocate_error_fatal;
    char definite_linear_solver[31];
    char prefix[31];
    struct sbls_control_type sbls_control;
    struct sls_control_type sls_control;
    struct ir_control_type ir_control;
};

detailed documentation

control derived type as a C struct

components

bool f_indexing

use C or Fortran sparse matrix indexing

ipc_ error

unit for error messages

ipc_ out

unit for monitor output

ipc_ print_level

controls level of diagnostic output

ipc_ new_a

how much of \(A\) has changed since the previous call. Possible values are

• 0 unchanged

• 1 values but not indices have changed

• 2 values and indices have changed

ipc_ new_s

how much of \(S\) has changed since the previous call. Possible values are

• 0 unchanged

• 1 values but not indices have changed

• 2 values and indices have changed

ipc_ max_factorizations

the maximum number of factorizations (=iterations) allowed. -ve implies no limit

ipc_ taylor_max_degree

maximum degree of Taylor approximant allowed (<= 3)

rpc_ initial_multiplier

initial estimate of the Lagrange multipler

rpc_ lower

lower and upper bounds on the multiplier, if known

rpc_ upper

see lower

rpc_ stop_normal

stop when \(| \|x\| - (\lambda/\sigma)^{1/(p-2)| \leq\) stop_normal \* max( 1, \(\|x\|\))

bool use_initial_multiplier

ignore initial_multiplier?

bool space_critical

if space is critical, ensure allocated arrays are no bigger than needed

bool deallocate_error_fatal

exit if any deallocation fails

char definite_linear_solver[31]

the name of the definite linear equation solver used. Possible choices are currently: ‘sils’, ‘ma27’, ‘ma57’, ‘ma77’, ‘ma86’, ‘ma87’, ‘ma97’, ‘ssids’, ‘mumps’, ‘pardiso’, ‘mkl_pardiso’, ‘pastix’, ‘wsmp’, ‘potr’, ‘sytr’ and ‘pbtr’, although only ‘potr’, ‘sytr’, ‘pbtr’ and, for OMP 4.0-compliant compilers, ‘ssids’ are installed by default; others are easily installed (see README.external). More details of the capabilities of each solver are provided in the documentation for galahad_sls.

char prefix[31]

all output lines will be prefixed by prefix(2:LEN(TRIM(.prefix))-1) where prefix contains the required string enclosed in quotes, e.g. “string” or ‘string’

struct sbls_control_type sbls_control

control parameters for the symmetric factorization and related linear solves (see sbls_c documentation)

struct sls_control_type sls_control

control parameters for the factorization of S and related linear solves (see sls_c documentation)

struct ir_control_type ir_control

control parameters for iterative refinement for definite system solves (see ir_c documentation)

llsr_time_type structure

#include <galahad_llsr.h>

struct llsr_time_type {
    // fields

    rpc_ total;
    rpc_ assemble;
    rpc_ analyse;
    rpc_ factorize;
    rpc_ solve;
    rpc_ clock_total;
    rpc_ clock_assemble;
    rpc_ clock_analyse;
    rpc_ clock_factorize;
    rpc_ clock_solve;
};

detailed documentation

time derived type as a C struct

components

rpc_ total

total CPU time spent in the package

rpc_ assemble

CPU time assembling \(K(\lambda)\) in (1)

rpc_ analyse

CPU time spent analysing \(K(\lambda)\).

rpc_ factorize

CPU time spent factorizing \(K(\lambda)\).

rpc_ solve

CPU time spent solving linear systems inolving \(K(\lambda)\).

rpc_ clock_total

total clock time spent in the package

rpc_ clock_assemble

clock time assembling \(K(\lambda)\)

rpc_ clock_analyse

clock time spent analysing \(K(\lambda)\)

rpc_ clock_factorize

clock time spent factorizing \(K(\lambda)\)

rpc_ clock_solve

clock time spent solving linear systems inolving \(K(\lambda)\)

llsr_history_type structure

#include <galahad_llsr.h>

struct llsr_history_type {
    // fields

    rpc_ lambda;
    rpc_ x_norm;
    rpc_ r_norm;
};

detailed documentation

history derived type as a C struct

components

rpc_ lambda

the value of \(\lambda\)

rpc_ x_norm

the corresponding value of \(\|x(\lambda)\|_M\)

rpc_ r_norm

the corresponding value of \(\|A x(\lambda) - b\|_2\)

llsr_inform_type structure

#include <galahad_llsr.h>

struct llsr_inform_type {
    // fields

    ipc_ status;
    ipc_ alloc_status;
    ipc_ factorizations;
    ipc_ len_history;
    rpc_ r_norm;
    rpc_ x_norm;
    rpc_ multiplier;
    char bad_alloc[81];
    struct llsr_time_type time;
    struct llsr_history_type history[100];
    struct sbls_inform_type sbls_inform;
    struct sls_inform_type sls_inform;
    struct ir_inform_type ir_inform;
};

detailed documentation

inform derived type as a C struct

components

ipc_ status

reported return status:

• 0

  the solution has been found

• -1

  an array allocation has failed

• -2

  an array deallocation has failed

• -3

  n and/or Delta is not positive

• -10

  the factorization of \(K(\lambda)\) failed

• -15

  \(S\) does not appear to be strictly diagonally dominant

• -16

  ill-conditioning has prevented furthr progress

ipc_ alloc_status

STAT value after allocate failure.

ipc_ factorizations

the number of factorizations performed

ipc_ len_history

the number of (\(\|x\|_S\), \(\lambda\)) pairs in the history

rpc_ r_norm

corresponding value of the two-norm of the residual, \(\|A x(\lambda) - b\|\)

rpc_ x_norm

the S-norm of x, \(\|x\|_S\)

rpc_ multiplier

the multiplier corresponding to the regularization term

char bad_alloc[81]

name of array which provoked an allocate failure

struct llsr_time_type time

time information

struct llsr_history_type history[100]

history information

struct sbls_inform_type sbls_inform

information from the symmetric factorization and related linear solves (see sbls_c documentation)

struct sls_inform_type sls_inform

information from the factorization of S and related linear solves (see sls_c documentation)

struct ir_inform_type ir_inform

information from the iterative refinement for definite system solves (see ir_c documentation)

Index

example calls

This is an example of how to use the package to solve a regularized linear least-squares subproblem; the code is available in $GALAHAD/src/llsr/C/llsrt.c . A variety of supported Hessian and constraint matrix storage formats are shown.

Notice that C-style indexing is used, and that this is flagged by setting control.f_indexing to false. The floating-point type rpc_ is set in galahad_precision.h to double by default, but to float if the preprocessor variable SINGLE is defined. Similarly, the integer type ipc_ from galahad_precision.h is set to int by default, but to int64_t if the preprocessor variable INTEGER_64 is defined.

/* llsrtf.c */
/* Full test for the LLSR C interface using C sparse matrix indexing */

#include <stdio.h>
#include <math.h>
#include <string.h>
#include "galahad_precision.h"
#include "galahad_cfunctions.h"
#include "galahad_llsr.h"

int main(void) {

    // Derived types
    void *data;
    struct llsr_control_type control;
    struct llsr_inform_type inform;
    ipc_ i, l;

    // Set problem data
    // set dimensions
    ipc_ m = 100;
    ipc_ n = 2*m+1;
    // A = ( I : Diag(1:n) : e )
    ipc_ A_ne = 3*m;
    ipc_ A_row[A_ne];
    ipc_ A_col[A_ne];
    ipc_ A_ptr[m+1];
    rpc_ A_val[A_ne];

    // store A in sparse formats
    l=0;
    for( i=0; i < m; i++){
     A_ptr[i] = l;
     A_row[l] = i;
     A_col[l] = i;
     A_val[l] = 1.0;
     l++;
     A_row[l] = i;
     A_col[l] = m+i;
     A_val[l] = i+1;
     l++;
     A_row[l] = i;
     A_col[l] = n-1;
     A_val[l] = 1.0;
     l++;
    }
    A_ptr[m] = l;

    // store A in dense format
    ipc_ A_dense_ne = m * n;
    rpc_ A_dense_val[A_dense_ne];
    for( i=0; i < A_dense_ne; i++) A_dense_val[i] = 0.0;
    l=-1;
    for( i=1; i <= m; i++){
     A_dense_val[l+i] = 1.0;
     A_dense_val[l+m+i] = i;
     A_dense_val[l+n] = 1.0;
     l=l+n;
    }

    // S = diag(1:n)**2
    ipc_ S_ne = n;
    ipc_ S_row[S_ne];
    ipc_ S_col[S_ne];
    ipc_ S_ptr[n+1];
    rpc_ S_val[S_ne];

    // store S in sparse formats
    for( i=0; i < n; i++){
     S_row[i] = i;
     S_col[i] = i;
     S_ptr[i] = i;
     S_val[i] = (i+1)*(i+1);
    }
    S_ptr[n] = n;

    // store S in dense format
    ipc_ S_dense_ne = n*(n+1)/2;
    rpc_ S_dense_val[S_dense_ne];
    for( i=0; i < S_dense_ne; i++) S_dense_val[i] = 0.0;
    l=-1;
    for( i=1; i <= n; i++){
      S_dense_val[l+i] = i*i;
      l=l+i;
    }

   // b is a vector of ones
    rpc_ b[m]; // observations
    for( i=0; i < m; i++){
      b[i] = 1.0;
    }

   // cubic regularization, weight is one
    rpc_ power = 3.0;
    rpc_ weight = 1.0;

   // Set output storage
    rpc_ x[n]; // solution
    char st = ' ';
    ipc_ status;

    printf(" C sparse matrix indexing\n\n");

    printf(" basic tests of problem storage formats\n\n");

    // loop over storage formats
    for( ipc_ d=1; d<=4; d++){

        // Initialize LLSR
        llsr_initialize( &data, &control, &status );
        strcpy(control.definite_linear_solver, "potr ") ;
        strcpy(control.sbls_control.symmetric_linear_solver, "sytr ") ;
        strcpy(control.sbls_control.definite_linear_solver, "potr ") ;
        // control.print_level = 1;

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing

        // use s or not (1 or 0)
        for( ipc_ use_s=0; use_s<=1; use_s++){
           switch(d){
               case 1: // sparse co-ordinate storage
                   st = 'C';
                   llsr_import( &control, &data, &status, m, n,
                               "coordinate", A_ne, A_row, A_col, NULL );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, 0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "coordinate", S_ne, S_row,
                                           S_col, NULL );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, S_ne, S_val );
                   }
                   break;
               case 2: // sparse by rows
                   st = 'R';
                   llsr_import( &control, &data, &status, m, n,
                                "sparse_by_rows", A_ne, NULL, A_col, A_ptr );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, 0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "sparse_by_rows", S_ne, NULL,
                                           S_col, S_ptr );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, S_ne, S_val );
                   }
                   break;
               case 3: // dense
                   st = 'D';
                   llsr_import( &control, &data, &status, m, n,
                                "dense", A_dense_ne, NULL, NULL, NULL );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_dense_ne, A_dense_val, b, x,
                                          0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "dense", S_dense_ne,
                                           NULL, NULL, NULL );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_dense_ne, A_dense_val, b, x,
                                          S_dense_ne, S_dense_val );
                   }
                   break;
               case 4: // diagonal
                   st = 'I';
                   llsr_import( &control, &data, &status, m, n,
                                "coordinate", A_ne, A_row, A_col, NULL );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, 0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "diagonal", S_ne, NULL, NULL, NULL );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, S_ne, S_val );
                   }
                   break;
               }
           llsr_information( &data, &inform, &status );

           if(inform.status == 0){
               printf("storage type %c%1" i_ipc_ ":  status = %1" i_ipc_ ", ||r|| = %5.2f\n",
                      st, use_s, inform.status, inform.r_norm );
           }else{
               printf("storage type %c%1" i_ipc_ ": LLSR_solve exit status = %1" i_ipc_ "\n",
                      st, use_s, inform.status);
           }
        }
        //printf("x: ");
        //for( ipc_ i = 0; i < n; i++) printf("%f ", x[i]);
        //printf("\n");

        // Delete internal workspace
        llsr_terminate( &data, &control, &inform );
    }
}

This is the same example, but now fortran-style indexing is used; the code is available in $GALAHAD/src/llsr/C/llsrtf.c .

/* llsrtf.c */
/* Full test for the LLSR C interface using Fortran sparse matrix indexing */

#include <stdio.h>
#include <math.h>
#include <string.h>
#include "galahad_precision.h"
#include "galahad_cfunctions.h"
#include "galahad_llsr.h"

int main(void) {

    // Derived types
    void *data;
    struct llsr_control_type control;
    struct llsr_inform_type inform;
    ipc_ i, l;

    // Set problem data
    // set dimensions
    ipc_ m = 100;
    ipc_ n = 2*m+1;
    // A = ( I : Diag(1:n) : e )
    ipc_ A_ne = 3*m;
    ipc_ A_row[A_ne];
    ipc_ A_col[A_ne];
    ipc_ A_ptr[m+1];
    rpc_ A_val[A_ne];

    // store A in sparse formats
    l=0;
    for( i=1; i <= m; i++){
     A_ptr[i-1] = l+1;
     A_row[l] = i;
     A_col[l] = i;
     A_val[l] = 1.0;
     l++;
     A_row[l] = i;
     A_col[l] = m+i;
     A_val[l] = i;
     l++;
     A_row[l] = i;
     A_col[l] = n;
     A_val[l] = 1.0;
     l++;
    }
    A_ptr[m] = l+1;

    // store A in dense format
    ipc_ A_dense_ne = m * n;
    rpc_ A_dense_val[A_dense_ne];
    for( i=0; i < A_dense_ne; i++) A_dense_val[i] = 0.0;
    l=-1;
    for( i=1; i <= m; i++){
     A_dense_val[l+i] = 1.0;
     A_dense_val[l+m+i] = i;
     A_dense_val[l+n] = 1.0;
     l=l+n;
    }

    // S = diag(1:n)**2
    ipc_ S_ne = n;
    ipc_ S_row[S_ne];
    ipc_ S_col[S_ne];
    ipc_ S_ptr[n+1];
    rpc_ S_val[S_ne];

    // store S in sparse formats
    for( i=0; i < n; i++){
     S_row[i] = i+1;
     S_col[i] = i+1;
     S_ptr[i] = i+1;
     S_val[i] = (i+1)*(i+1);
    }
    S_ptr[n] = n+1;

    // store S in dense format
    ipc_ S_dense_ne = n*(n+1)/2;
    rpc_ S_dense_val[S_dense_ne];
    for( i=0; i < S_dense_ne; i++) S_dense_val[i] = 0.0;
    l=-1;
    for( i=1; i <= n; i++){
      S_dense_val[l+i] = i*i;
      l=l+i;
    }

   // b is a vector of ones
    rpc_ b[m]; // observations
    for( i=0; i < m; i++){
      b[i] = 1.0;
    }

   // cubic regularization, weight is one
    rpc_ power = 3.0;
    rpc_ weight = 1.0;

    // Set output storage
    rpc_ x[n]; // solution
    char st = ' ';
    ipc_ status;

    printf(" Fortran sparse matrix indexing\n\n");

    printf(" basic tests of problem storage formats\n\n");

    // loop over storage formats
    for( ipc_ d=1; d<=4; d++){

        // Initialize LLSR
        llsr_initialize( &data, &control, &status );
        strcpy(control.definite_linear_solver, "potr ") ;
        strcpy(control.sbls_control.symmetric_linear_solver, "sytr ") ;
        strcpy(control.sbls_control.definite_linear_solver, "potr ") ;
        // control.print_level = 1;

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing

        // use s or not (1 or 0)
        for( ipc_ use_s=0; use_s<=1; use_s++){
           switch(d){
               case 1: // sparse co-ordinate storage
                   st = 'C';
                   llsr_import( &control, &data, &status, m, n,
                               "coordinate", A_ne, A_row, A_col, NULL );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, 0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "coordinate", S_ne, S_row,
                                           S_col, NULL );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, S_ne, S_val );
                   }
                   break;
               case 2: // sparse by rows
                   st = 'R';
                   llsr_import( &control, &data, &status, m, n,
                                "sparse_by_rows", A_ne, NULL, A_col, A_ptr );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, 0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "sparse_by_rows", S_ne, NULL,
                                           S_col, S_ptr );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, S_ne, S_val );
                   }
                   break;
               case 3: // dense
                   st = 'D';
                   llsr_import( &control, &data, &status, m, n,
                                "dense", A_dense_ne, NULL, NULL, NULL );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_dense_ne, A_dense_val, b, x,
                                          0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "dense", S_dense_ne,
                                           NULL, NULL, NULL );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_dense_ne, A_dense_val, b, x,
                                          S_dense_ne, S_dense_val );
                   }
                   break;
               case 4: // diagonal
                   st = 'I';
                   llsr_import( &control, &data, &status, m, n,
                                "coordinate", A_ne, A_row, A_col, NULL );
                   if(use_s == 0){
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, 0, NULL );
                   }else{
                      llsr_import_scaling( &control, &data, &status, n,
                                           "diagonal", S_ne, NULL, NULL, NULL );
                      llsr_solve_problem( &data, &status, m, n, power, weight,
                                          A_ne, A_val, b, x, S_ne, S_val );
                   }
                   break;
               }
           llsr_information( &data, &inform, &status );

           if(inform.status == 0){
               printf("storage type %c%1" i_ipc_ ":  status = %1" i_ipc_ ", ||r|| = %5.2f\n",
                      st, use_s, inform.status, inform.r_norm );
           }else{
               printf("storage type %c%1" i_ipc_ ": LLSR_solve exit status = %1" i_ipc_ "\n",
                      st, use_s, inform.status);
           }
        }
        //printf("x: ");
        //for( ipc_ i = 0; i < n; i++) printf("%f ", x[i]);
        //printf("\n");

        // Delete internal workspace
        llsr_terminate( &data, &control, &inform );
    }
}