GALAHAD LLSR package#

purpose#

Given a real $m$ by $n$ model matrix $A$, a real $n$ by $n$ symmetric diagonally-dominant matrix $S$, a real $m$ vector of observations $b$ and scalars $\sigma>0$ and $p \geq 2$, the llsr package finds a minimizer of the regularized linear least-squares objective function

\[\frac{1}{2} \| A x - b \|^2_2 + \frac{\sigma}{p}\|x\|_S^p,\]

where the $S$-norm of $x$ is $\|x\|_S = \sqrt{x^T S x}$. This problem commonly occurs as a subproblem in nonlinear least-squares calculations. The matrix $S$ need not be provided in the commonly-occurring $\ell_2$-regularization case for which $S = I$, the $n$ by $n$ identity matrix.

Factorization of matrices of the form

\[\begin{split}\begin{pmatrix}\lambda S & A^T \\ A & - I\end{pmatrix}\mspace{5em}\mbox{(1)}\end{split}\]

for a succession of scalars $\lambda$ will be required, so this package is most suited for the case where such a factorization may be found efficiently. If this is not the case, the package lsrt may be preferred.

See Section 4 of $GALAHAD/doc/llsr.pdf for additional details.

method#

The required solution $x_*$ necessarily satisfies the optimality condition $A^T A x_* + \lambda_* S x_* = A^T b$, where $\lambda_* = \sigma \|x_*\|^{p-2}$.

The method is iterative, and proceeds in two phases. Firstly, lower and upper bounds, $\lambda_L$ and $\lambda_U$, on $\lambda_*$ are computed using Gershgorin’s theorems and other eigenvalue bounds, including those that may involve the Cholesky factorization of $S$ The first phase of the computation proceeds by progressively shrinking the bound interval $[\lambda_L,\lambda_U]$ until a value $\lambda$ for which $\|x(\lambda)\|_{M} \geq \sigma \|x(\lambda)\|_S^{p-2}$ is found. Here $x(\lambda)$ and its companion $y(\lambda)$ are defined to be a solution of

\[(A^T A + \lambda S)x(\lambda) = A^T b. \mspace{5em}\mbox{(2)}\]

Once the terminating $\lambda$ from the first phase has been discovered, the second phase consists of applying Newton or higher-order iterations to the nonlinear secular equation $\lambda = \sigma \|x(\lambda)\|_S^{p-2}$ with the knowledge that such iterations are both globally and ultimately rapidly convergent.

The dominant cost is the requirement that we solve a sequence of linear systems (2). This may be rewritten as

\[\begin{split}\begin{pmatrix}\lambda S & A^T \\ A & - I\end{pmatrix} \begin{pmatrix}x(\lambda) \\ y(\lambda)\end{pmatrix} = \begin{pmatrix}A^T b \\ 0\end{pmatrix} \mspace{5em} \mbox{(3)}\end{split}\]

for some auxiliary vector $y(\lambda)$. In general a sparse symmetric, indefinite factorization of the coefficient matrix of (3) is often preferred to a Cholesky factorization of that of (2).

reference#

The method is the obvious adaptation to the linear least-squares problem of that described in detail in

H. S. Dollar, N. I. M. Gould and D. P. Robinson. ``On solving trust-region and other regularised subproblems in optimization’’. Mathematical Programming Computation 2(1) (2010) 21–57.

matrix storage#

unsymmetric storage#

The unsymmetric $m$ by $n$ model matrix $A$ may be presented and stored in a variety of convenient input formats.

Dense storage format: The matrix $A$ is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. In this case, component $n \ast i + j$ of the storage array A_val will hold the value $A_{ij}$ for $1 \leq i \leq m$, $1 \leq j \leq n$. The string A_type = ‘dense’ should be specified.

Dense by columns storage format: The matrix $A$ is stored as a compact dense matrix by columns, that is, the values of the entries of each column in turn are stored in order within an appropriate real one-dimensional array. In this case, component $m \ast j + i$ of the storage array A_val will hold the value $A_{ij}$ for $1 \leq i \leq m$, $1 \leq j \leq n$. The string A_type = ‘dense_by_columns’ should be specified.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the $l$-th entry, $1 \leq l \leq ne$, of $A$, its row index i, column index j and value $A_{ij}$, $1 \leq i \leq m$, $1 \leq j \leq n$, are stored as the $l$-th components of the integer arrays A_row and A_col and real array A_val, respectively, while the number of nonzeros is recorded as A_ne = $ne$. The string A_type = ‘coordinate’should be specified.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of $A$ the i-th component of the integer array A_ptr holds the position of the first entry in this row, while A_ptr(m+1) holds the total number of entries plus one. The column indices j, $1 \leq j \leq n$, and values $A_{ij}$ of the nonzero entries in the i-th row are stored in components l = A_ptr(i), $\ldots$, A_ptr(i+1)-1, $1 \leq i \leq m$, of the integer array A_col, and real array A_val, respectively. For sparse matrices, this scheme almost always requires less storage than its predecessor. The string A_type = ‘sparse_by_rows’ should be specified.

Sparse column-wise storage format: Once again only the nonzero entries are stored, but this time they are ordered so that those in column j appear directly before those in column j+1. For the j-th column of $A$ the j-th component of the integer array A_ptr holds the position of the first entry in this column, while A_ptr(n+1) holds the total number of entries plus one. The row indices i, $1 \leq i \leq m$, and values $A_{ij}$ of the nonzero entries in the j-th columnsare stored in components l = A_ptr(j), $\ldots$, A_ptr(j+1)-1, $1 \leq j \leq n$, of the integer array A_row, and real array A_val, respectively. As before, for sparse matrices, this scheme almost always requires less storage than the co-ordinate format. The string A_type = ‘sparse_by_columns’ should be specified.

symmetric storage#

The symmetric $n$ by $n$ scaing matrix $S$ may also be presented and stored in a variety of formats. But crucially symmetry is exploited by only storing values from the lower triangular part (i.e, those entries that lie on or below the leading diagonal).

Dense storage format: The matrix $S$ is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. Since $S$ is symmetric, only the lower triangular part (that is the part $S_{ij}$ for $1 \leq j \leq i \leq n$) need be held. In this case the lower triangle should be stored by rows, that is component $(i-1) * i / 2 + j$ of the storage array S_val will hold the value $S_{ij}$ (and, by symmetry, $S_{ji}$) for $1 \leq j \leq i \leq n$. The string S_type = ‘dense’ should be specified.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the $l$-th entry, $1 \leq l \leq ne$, of $S$, its row index i, column index j and value $S_{ij}$, $1 \leq j \leq i \leq n$, are stored as the $l$-th components of the integer arrays S_row and S_col and real array S_val, respectively, while the number of nonzeros is recorded as S_ne = $ne$. Note that only the entries in the lower triangle should be stored. The string S_type = ‘coordinate’ should be specified.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of $S$ the i-th component of the integer array S_ptr holds the position of the first entry in this row, while S_ptr(n+1) holds the total number of entries plus one. The column indices j, $1 \leq j \leq i$, and values $S_{ij}$ of the entries in the i-th row are stored in components l = S_ptr(i), …, S_ptr(i+1)-1 of the integer array S_col, and real array S_val, respectively. Note that as before only the entries in the lower triangle should be stored. For sparse matrices, this scheme almost always requires less storage than its predecessor. The string S_type = ‘sparse_by_rows’ should be specified.

Diagonal storage format: If $S$ is diagonal (i.e., $S_{ij} = 0$ for all $1 \leq i \neq j \leq n$) only the diagonals entries $S_{ii}$, $1 \leq i \leq n$ need be stored, and the first n components of the array S_val may be used for the purpose. The string S_type = ‘diagonal’ should be specified.

Multiples of the identity storage format: If $S$ is a multiple of the identity matrix, (i.e., $H = \alpha I$ where $I$ is the n by n identity matrix and $\alpha$ is a scalar), it suffices to store $\alpha$ as the first component of S_val. The string S_type = ‘scaled_identity’ should be specified.

The identity matrix format: If $S$ is the identity matrix, no values need be stored. The string S_type = ‘identity’ should be specified. Strictly this is not required as $S$ will be assumed to be $I$ if it is not explicitly provided.

The zero matrix format: The same is true if $S$ is the zero matrix, but now the string S_type = ‘zero’ or ‘none’ should be specified.

introduction to function calls#

To solve a given problem, functions from the llsr package must be called in the following order:

llsr_initialize - provide default control parameters and set up initial data structures
llsr_read_specfile (optional) - override control values by reading replacement values from a file
llsr_import - set up problem data structures and fixed values
llsr_import_scaling (optional) - set up problem data structures for $S$ if required
llsr_reset_control (optional) - possibly change control parameters if a sequence of problems are being solved
llsr_solve_problem - solve the regularization problem
llsr_information (optional) - recover information about the solution and solution process
llsr_terminate - deallocate data structures

See the examples section for illustrations of use.

parametric real type T#

Below, the symbol T refers to a parametric real type that may be Float32 (single precision) or Float64 (double precision).

callable functions#

    function llsr_initialize(T, data, control, status)

Set default control values and initialize private data

Parameters:

data

holds private internal data

control

is a structure containing control information (see llsr_control_type)

status

is a scalar variable of type Int32 that gives the exit status from the package. Possible values are (currently):

0
The initialization was successful.

    function llsr_read_specfile(T, control, specfile)

Read the content of a specification file, and assign values associated with given keywords to the corresponding control parameters. An in-depth discussion of specification files is available, and a detailed list of keywords with associated default values is provided in $GALAHAD/src/llsr/LLSR.template. See also Table 2.1 in the Fortran documentation provided in $GALAHAD/doc/llsr.pdf for a list of how these keywords relate to the components of the control structure.

Parameters:

control	is a structure containing control information (see llsr_control_type)
specfile	is a one-dimensional array of type Vararg{Cchar} that must give the name of the specification file

    function llsr_import(T, control, data, status, m, n,
                         A_type, A_ne, A_row, A_col, A_ptr)

Import problem data into internal storage prior to solution.

Parameters:

control	is a structure whose members provide control parameters for the remaining procedures (see llsr_control_type)
data	holds private internal data
status	is a scalar variable of type Int32 that gives the exit status from the package. Possible values are: 1 The import was successful, and the package is ready for the solve phase -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’ or ‘absent’ has been violated.
m	is a scalar variable of type Int32 that holds the number of residuals, i.e., the number of rows of $A$. m must be positive.
n	is a scalar variable of type Int32 that holds the number of variables, i.e., the number of columns of $A$. n must be positive.
A_type	is a one-dimensional array of type Vararg{Cchar} that specifies the unsymmetric storage scheme used for the constraint Jacobian, $A$ if any. It should be one of ‘coordinate’, ‘sparse_by_rows’ or ‘dense’; lower or upper case variants are allowed.
A_ne	is a scalar variable of type Int32 that holds the number of entries in $A$, if used, in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
A_row	is a one-dimensional array of size A_ne and type Int32 that holds the row indices of $A$ in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes, and in this case can be C_NULL.
A_col	is a one-dimensional array of size A_ne and type Int32 that holds the column indices of $A$ in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense or diagonal storage schemes are used, and in this case can be C_NULL.
A_ptr	is a one-dimensional array of size n+1 and type Int32 that holds the starting position of each row of $A$, as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be C_NULL.

    function llsr_import_scaling(T, control, data, status, n,
                                 S_type, S_ne, S_row, S_col, S_ptr)

Import the scaling matrix $S$ into internal storage prior to solution. Thus must have been preceeded by a call to llsr_import.

Parameters:

control	is a structure whose members provide control parameters for the remaining procedures (see llsr_control_type)
data	holds private internal data
status	is a scalar variable of type Int32 that gives the exit status from the package. Possible values are: 1 The import was successful, and the package is ready for the solve phase -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’ or ‘diagonal’ has been violated.
n	is a scalar variable of type Int32 that holds the number of variables, i.e., the number of rows and columns of $S$. n must be positive.
S_type	is a one-dimensional array of type Vararg{Cchar} that specifies the symmetric storage scheme used for the matrix $S$. It should be one of ‘coordinate’, ‘sparse_by_rows’, ‘dense’ or ‘diagonal’; lower or upper case variants are allowed.
S_ne	is a scalar variable of type Int32 that holds the number of entries in the lower triangular part of $S$ in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
S_row	is a one-dimensional array of size S_ne and type Int32 that holds the row indices of the lower triangular part of $S$ in the sparse co-ordinate storage scheme. It need not be set for any of the other three schemes, and in this case can be C_NULL.
S_col	is a one-dimensional array of size S_ne and type Int32 that holds the column indices of the lower triangular part of $S$ in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense, diagonal or (scaled) identity storage schemes are used, and in this case can be C_NULL.
S_ptr	is a one-dimensional array of size n+1 and type Int32 that holds the starting position of each row of the lower triangular part of $S$, as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be C_NULL.

    function llsr_reset_control(T, control, data, status)

Reset control parameters after import if required.

Parameters:

control

is a structure whose members provide control parameters for the remaining procedures (see llsr_control_type)

data

holds private internal data

status

is a scalar variable of type Int32 that gives the exit status from the package. Possible values are:

1. The import was successful, and the package is ready for the solve phase

    function llsr_solve_problem(T, data, status, m, n, power, weight,
                                A_ne, A_val, b, x, S_ne, S_val)

Solve the regularization-region problem.

Parameters:

data	holds private internal data
status	is a scalar variable of type Int32 that gives the entry and exit status from the package. Possible exit values are: 0 The run was successful. -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restrictions n > 0 and m > 0 or requirement that A_type or A_type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’ or ‘diagonal’ has been violated. -9 The analysis phase of the factorization failed; the return status from the factorization package is given in the component inform.factor_status -10 The factorization failed; the return status from the factorization package is given in the component inform.factor_status. -11 The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given in the component inform.factor_status. -15 The matrix $S$ does not appear to be strictly diagonally dominant. -16 The problem is so ill-conditioned that further progress is impossible. -17 The step is too small to make further impact.
m	is a scalar variable of type Int32 that holds the number of residuals
n	is a scalar variable of type Int32 that holds the number of variables
power	is a scalar of type T that holds the regularization power, $p$, used. power must be greater than or equal to two.
weight	is a scalar of type T that holds the regularization power, $\sigma$, used. weight must be strictly positive
A_ne	is a scalar variable of type Int32 that holds the number of entries in the observation matrix $A$.
A_val	is a one-dimensional array of size A_ne and type T that holds the values of the entries of the observation matrix $A$ in any of the available storage schemes.
b	is a one-dimensional array of size m and type T that holds the values $b$ of observations. The i-th component of `b`, i = 1, … , m, contains $b_i$.
x	is a one-dimensional array of size n and type T that holds the values $x$ of the optimization variables. The j-th component of `x`, j = 1, … , n, contains $x_j$.
S_ne	is a scalar variable of type Int32 that holds the number of entries in the scaling matrix $S$ if it not the identity matrix.
S_val	is a one-dimensional array of size S_ne and type T that holds the values of the entries of the scaling matrix $S$ in any of the available storage schemes. If S_val is C_NULL, $S$ will be taken to be the identity matrix.

    function llsr_information(T, data, inform, status)

Provides output information

Parameters:

data

holds private internal data

inform

is a structure containing output information (see llsr_inform_type)

status

is a scalar variable of type Int32 that gives the exit status from the package. Possible values are (currently):

0
The values were recorded successfully

    function llsr_terminate(T, data, control, inform)

Deallocate all internal private storage

Parameters:

data	holds private internal data
control	is a structure containing control information (see llsr_control_type)
inform	is a structure containing output information (see llsr_inform_type)

available structures#

llsr_control_type structure#

    struct llsr_control_type{T}
      f_indexing::Bool
      error::Int32
      out::Int32
      print_level::Int32
      new_a::Int32
      new_s::Int32
      max_factorizations::Int32
      taylor_max_degree::Int32
      initial_multiplier::T
      lower::T
      upper::T
      stop_normal::T
      use_initial_multiplier::Bool
      space_critical::Bool
      deallocate_error_fatal::Bool
      definite_linear_solver::NTuple{31,Cchar}
      prefix::NTuple{31,Cchar}
      sbls_control::sbls_control_type{T}
      sls_control::sls_control_type{T}
      ir_control::ir_control_type{T}

detailed documentation#

control derived type as a Julia structure

components#

Bool f_indexing

use C or Fortran sparse matrix indexing

Int32 error

unit for error messages

Int32 out

unit for monitor output

Int32 print_level

controls level of diagnostic output

Int32 new_a

how much of $A$ has changed since the previous call. Possible values are

0 unchanged
1 values but not indices have changed
2 values and indices have changed

Int32 new_s

how much of $S$ has changed since the previous call. Possible values are

0 unchanged
1 values but not indices have changed
2 values and indices have changed

Int32 max_factorizations

the maximum number of factorizations (=iterations) allowed. -ve implies no limit

Int32 taylor_max_degree

maximum degree of Taylor approximant allowed (<= 3)

T initial_multiplier

initial estimate of the Lagrange multipler

T lower

lower and upper bounds on the multiplier, if known

T upper

see lower

T stop_normal

stop when $| \|x\| - (\lambda/\sigma)^{1/(p-2)| \leq$ stop_normal \* max( 1, $\|x\|$)

Bool use_initial_multiplier

ignore initial_multiplier?

Bool space_critical

if space is critical, ensure allocated arrays are no bigger than needed

Bool deallocate_error_fatal

exit if any deallocation fails

char definite_linear_solver[31]

definite linear equation solver

NTuple{31,Cchar} prefix

all output lines will be prefixed by prefix(2:LEN(TRIM(.prefix))-1) where prefix contains the required string enclosed in quotes, e.g. “string” or ‘string’

struct sbls_control_type sbls_control

control parameters for the symmetric factorization and related linear solves (see sbls_c documentation)

struct sls_control_type sls_control

control parameters for the factorization of S and related linear solves (see sls_c documentation)

struct ir_control_type ir_control

control parameters for iterative refinement for definite system solves (see ir_c documentation)

llsr_time_type structure#

    struct llsr_time_type{T}
      total::T
      assemble::T
      analyse::T
      factorize::T
      solve::T
      clock_total::T
      clock_assemble::T
      clock_analyse::T
      clock_factorize::T
      clock_solve::T

detailed documentation#

time derived type as a Julia structure

components#

T total

total CPU time spent in the package

T assemble

CPU time assembling $K(\lambda)$ in (1)

T analyse

CPU time spent analysing $K(\lambda)$.

T factorize

CPU time spent factorizing $K(\lambda)$.

T solve

CPU time spent solving linear systems inolving $K(\lambda)$.

T clock_total

total clock time spent in the package

T clock_assemble

clock time assembling $K(\lambda)$

T clock_analyse

clock time spent analysing $K(\lambda)$

T clock_factorize

clock time spent factorizing $K(\lambda)$

T clock_solve

clock time spent solving linear systems inolving $K(\lambda)$

llsr_history_type structure#

    struct llsr_history_type{T}
      lambda::T
      x_norm::T
      r_norm::T

detailed documentation#

history derived type as a Julia structure

components#

T lambda

the value of $\lambda$

T x_norm

the corresponding value of $\|x(\lambda)\|_M$

T r_norm

the corresponding value of $\|A x(\lambda) - b\|_2$

llsr_inform_type structure#

    struct llsr_inform_type{T}
      status::Int32
      alloc_status::Int32
      factorizations::Int32
      len_history::Int32
      r_norm::T
      x_norm::T
      multiplier::T
      bad_alloc::NTuple{81,Cchar}
      time::llsr_time_type{T}
      history::NTuple{100,llsr_history_type{T}}
      sbls_inform::sbls_inform_type{T}
      sls_inform::sls_inform_type{T}
      ir_inform::ir_inform_type{T}

detailed documentation#

inform derived type as a Julia structure

components#

Int32 status

reported return status:

0

the solution has been found
-1

an array allocation has failed
-2

an array deallocation has failed
-3

n and/or Delta is not positive
-10

the factorization of $K(\lambda)$ failed
-15

$S$ does not appear to be strictly diagonally dominant
-16

ill-conditioning has prevented furthr progress

Int32 alloc_status

STAT value after allocate failure.

Int32 factorizations

the number of factorizations performed

Int32 len_history

the number of ($\|x\|_S$, $\lambda$) pairs in the history

T r_norm

corresponding value of the two-norm of the residual, $\|A x(\lambda) - b\|$

T x_norm

the S-norm of x, $\|x\|_S$

T multiplier

the multiplier corresponding to the regularization term

NTuple{81,Cchar} bad_alloc

name of array which provoked an allocate failure

struct llsr_time_type time

time information

struct llsr_history_type history[100]

history information

struct sbls_inform_type sbls_inform

information from the symmetric factorization and related linear solves (see sbls_c documentation)

struct sls_inform_type sls_inform

information from the factorization of S and related linear solves (see sls_c documentation)

struct ir_inform_type ir_inform

information from the iterative refinement for definite system solves (see ir_c documentation)

Index

example calls#

This is an example of how to use the package to solve a regularized linear least-squares subproblem; the code is available in $GALAHAD/src/llsr/Julia/test_llsr.jl . A variety of supported Hessian and constraint matrix storage formats are shown.

# test_llsr.jl
# Simple code to test the Julia interface to LLSR

using GALAHAD
using Test
using Printf
using Accessors

function test_llsr(::Type{T}) where T
  # Derived types
  data = Ref{Ptr{Cvoid}}()
  control = Ref{llsr_control_type{T}}()
  inform = Ref{llsr_inform_type{T}}()

  # Set problem data
  # set dimensions
  m = 100
  n = 2 * m + 1

  # # A = (I : Diag(1:n) : e)
  A_ne = 3 * m
  A_row = zeros(Cint, A_ne)
  A_col = zeros(Cint, A_ne)
  A_ptr = zeros(Cint, m + 1)
  A_val = zeros(T, A_ne)

  # store A in sparse formats
  l = 1
  for i in 1:m
    A_ptr[i] = l
    A_row[l] = i
    A_col[l] = i
    A_val[l] = 1.0
    l = l + 1
    A_row[l] = i
    A_col[l] = m + i
    A_val[l] = i
    l = l + 1
    A_row[l] = i
    A_col[l] = n
    A_val[l] = 1.0
    l = l + 1
  end
  A_ptr[m + 1] = l

  # store A in dense format
  A_dense_ne = m * n
  A_dense_val = zeros(T, A_dense_ne)
  l = 0
  for i in 1:m
    A_dense_val[l + i] = 1.0
    A_dense_val[l + m + i] = i
    A_dense_val[l + n] = 1.0
    l = l + n
  end

  # S = diag(1:n)**2
  S_ne = n
  S_row = zeros(Cint, S_ne)
  S_col = zeros(Cint, S_ne)
  S_ptr = zeros(Cint, n + 1)
  S_val = zeros(T, S_ne)

  # store S in sparse formats
  for i in 1:n
    S_row[i] = i
    S_col[i] = i
    S_ptr[i] = i
    S_val[i] = i * i
  end
  S_ptr[n + 1] = n + 1

  # store S in dense format
  S_dense_ne = div(n * (n + 1), 2)
  S_dense_val = zeros(T, S_dense_ne)
  l = 0
  for i in 1:n
    S_dense_val[l + i] = i * i
    l = l + i
  end

  # b is a vector of ones
  b = ones(T, m) # observations

  # cubic regularization, weight is one
  power = 3.0
  weight = 1.0

  # Set output storage
  x = zeros(T, n) # solution
  st = ' '
  status = Ref{Cint}()

  @printf(" Fortran sparse matrix indexing\n\n")
  @printf(" basic tests of problem storage formats\n\n")

  # loop over storage formats
  for d in 1:4

    # Initialize LLSR
    llsr_initialize(T, data, control, status)
    @reset control[].definite_linear_solver = galahad_linear_solver("potr")
    @reset control[].sbls_control.symmetric_linear_solver = galahad_linear_solver("sytr")
    @reset control[].sbls_control.definite_linear_solver = galahad_linear_solver("potr")
    # @reset control[].print_level = 1

    # Set user-defined control options
    @reset control[].f_indexing = true # Fortran sparse matrix indexing

    # use s or not (1 or 0)
    for use_s in 0:1

      # sparse co-ordinate storage
      if d == 1
        st = 'C'
        llsr_import(T, control, data, status, m, n,
                    "coordinate", A_ne, A_row, A_col, C_NULL)

        if use_s == 0
          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_ne, A_val, b, x, 0, C_NULL)
        else
          llsr_import_scaling(T, control, data, status, n,
                              "coordinate", S_ne, S_row,
                              S_col, C_NULL)

          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_ne, A_val, b, x, S_ne, S_val)
        end
      end

      # sparse by rows
      if d == 2
        st = 'R'
        llsr_import(T, control, data, status, m, n,
                    "sparse_by_rows", A_ne, C_NULL, A_col, A_ptr)

        if use_s == 0
          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_ne, A_val, b, x, 0, C_NULL)
        else
          llsr_import_scaling(T, control, data, status, n,
                              "sparse_by_rows", S_ne, C_NULL,
                              S_col, S_ptr)

          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_ne, A_val, b, x, S_ne, S_val)
        end
      end

      # dense
      if d == 3
        st = 'D'
        llsr_import(T, control, data, status, m, n,
                    "dense", A_dense_ne, C_NULL, C_NULL, C_NULL)
        if use_s == 0
          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_dense_ne, A_dense_val, b, x,
                             0, C_NULL)
        else
          llsr_import_scaling(T, control, data, status, n,
                              "dense", S_dense_ne,
                              C_NULL, C_NULL, C_NULL)

          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_dense_ne, A_dense_val, b, x,
                             S_dense_ne, S_dense_val)
        end
      end

      # diagonal
      if d == 4
        st = 'I'
        llsr_import(T, control, data, status, m, n,
                    "coordinate", A_ne, A_row, A_col, C_NULL)
        if use_s == 0
          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_ne, A_val, b, x, 0, C_NULL)
        else
          llsr_import_scaling(T, control, data, status, n,
                              "diagonal", S_ne, C_NULL, C_NULL, C_NULL)

          llsr_solve_problem(T, data, status, m, n, power, weight,
                             A_ne, A_val, b, x, S_ne, S_val)
        end
      end

      llsr_information(T, data, inform, status)

      if inform[].status == 0
        @printf("storage type %c%1i:  status = %1i, ||r|| = %5.2f\n", st, use_s,
                inform[].status, inform[].r_norm)
      else
        @printf("storage type %c%1i: LLSR_solve exit status = %1i\n", st, use_s,
                inform[].status)
      end
    end

    # @printf("x: ")
    # for i = 1:n
    #   @printf("%f ", x[i])
    # end
    # @printf("\n")

    # Delete internal workspace
    llsr_terminate(T, data, control, inform)
  end
  return 0
end

@testset "LLSR" begin
  @test test_llsr(Float32) == 0
  @test test_llsr(Float64) == 0
end