GALAHAD NLS package#

purpose#

The nls package uses a regularization method to find a (local) unconstrained minimizer of a differentiable weighted sum-of-squares objective function

\[f(x) := \frac{1}{2} \sum_{i=1}^m w_i c_i^2(x) \equiv \frac{1}{2} \|c(x)\|^2_W\]

of many variables $x$ involving positive weights $w_i$, $i=1,\ldots,m$. The method offers the choice of direct and iterative solution of the key regularization subproblems, and is most suitable for large problems. First derivatives of the residual function $c(x)$ are required, and if second derivatives of the $c_i(x)$ can be calculated, they may be exploited.

See Section 4 of $GALAHAD/doc/nls.pdf for additional details.

terminology#

The gradient $\nabla_x f(x)$ of a function $f(x)$ is the vector whose $i$-th component is $\partial f(x)/\partial x_i$. The Hessian $\nabla_{xx} f(x)$ of $f(x)$ is the symmetric matrix whose $i,j$-th entry is $\partial^2 f(x)/\partial x_i \partial x_j$. The Hessian is sparse if a significant and useful proportion of the entries are universally zero.

The algorithm used by the package is iterative. From the current best estimate of the minimizer $x_k$, a trial improved point $x_k + s_k$ is sought. The correction $s_k$ is chosen to improve a model $m_k(s)$ of the objective function $f(x_k+s)$ built around $x_k$. The model is the sum of two basic components, a suitable approximation $t_k(s)$ of $f(x_k+s)$, %another approximation of $(\rho/r) \|x_k+s\|_r^r$ (if $\rho > 0$), and a regularization term $(\sigma_k/p) \|s\|_{S_k}^p$ involving a weight $\sigma_k$, power $p$ and a norm $\|s\|_{S_k} := \sqrt{s^T S_k s}$ for a given positive definite scaling matrix $S_k$ that is included to prevent large corrections. The weight $\sigma_k$ is adjusted as the algorithm progresses to ensure convergence.

The model $t_k(s)$ is a truncated Taylor-series approximation, and this relies on being able to compute or estimate derivatives of $c(x)$. Various models are provided, and each has different derivative requirements. We denote the $m$ by $n$ residual Jacobian $J(x) \equiv \nabla_x c(x)$ as the matrix whose $i,j$-th component

\[J(x)_{i,j} := \partial c_i(x) / \partial x_j \;\; \mbox{for $i=1,\ldots,m$ and $j=1,\ldots,n$.}\]

For a given $m$-vector $y$, the weighted-residual Hessian is the sum

\[H(x,y) := \sum_{\ell=1}^m y_{\ell} H_{\ell}(x), \;\; \mbox{where}\;\; H_{\ell}(x)_{i,j} := \partial^2 c_{\ell}(x) / \partial x_i \partial x_j \;\; \mbox{for $i,j=1,\ldots,n$}\]

is the Hessian of $c_\ell(x)$. Finally, for a given vector $v$, we define the residual-Hessians-vector product matrix

\[P(x,v) := (H_1(x) v, \ldots, H_m(x) v).\]

The models $t_k(s)$ provided are,

the first-order Taylor approximation $f(x_k) + g(x_k)^T s$, where $g(x) = J^T(x) W c(x)$,
a barely second-order approximation $f(x_k) + g(x_k)^T s + \frac{1}{2} s^T W s$,
the Gauss-Newton approximation $\frac{1}{2} \| c(x_k) + J(x_k) s\|^2_W$,
the Newton (second-order Taylor) approximation

$f(x_k) + g(x_k)^T s + \frac{1}{2} s^T [ J^T(x_k) W J(x_k) + H(x_k,W c(x_k))] s$, and
the tensor Gauss-Newton approximation $\frac{1}{2} \| c(x_k) + J(x_k) s + \frac{1}{2} s^T \cdot P(x_k,s) \|^2_W$, where the $i$-th component of $s^T \cdot P(x_k,s)$ is shorthand for the scalar $s^T H_i(x_k) s$, where $W$ is the diagonal matrix of weights $w_i$, $i = 1, \ldots m$0.

method#

An adaptive regularization method is used. In this, an improvement to a current estimate of the required minimizer, $x_k$ is sought by computing a step $s_k$. The step is chosen to approximately minimize a model $t_k(s)$ of $f_{\rho,r}(x_k+s)$ that includes a weighted regularization term $\frac{\sigma_k}{p} \|s\|_{S_k}^p$ for some specified positive weight $\sigma_k$. The quality of the resulting step $s_k$ is assessed by computing the “ratio” $(f(x_k) - f(x_k + s_k))/(t_k(0) - t_k(s_k))$. The step is deemed to have succeeded if the ratio exceeds a given $\eta_s > 0$, and in this case $x_{k+1} = x_k + s_k$. Otherwise $x_{k+1} = x_k$, and the weight is increased by powers of a given increase factor up to a given limit. If the ratio is larger than $\eta_v \geq \eta_d$, the weight will be decreased by powers of a given decrease factor again up to a given limit. The method will terminate as soon as $f(x_k)$ or $\|\nabla_x f(x_k)\|$ is smaller than a specified value.

A choice of linear, quadratic or quartic models $t_k(s)$ is available (see the previous section), and normally a two-norm regularization will be used, but this may change if preconditioning is employed.

If linear or quadratic models are employed, an appropriate, approximate model minimizer is found using either a direct approach involving factorization of a shift of the model Hessian $B_k$ or an iterative (conjugate-gradient/Lanczos) approach based on approximations to the required solution from a so-called Krlov subspace. The direct approach is based on the knowledge that the required solution satisfies the linear system of equations $(B_k + \lambda_k I) s_k = - \nabla_x f(x_k)$ involving a scalar Lagrange multiplier $\lambda_k$. This multiplier is found by uni-variate root finding, using a safeguarded Newton-like process, by RQS. The iterative approach uses GLRT, and is best accelerated by preconditioning with good approximations to the Hessian of the model using PSLS. The iterative approach has the advantage that only Hessian matrix-vector products are required, and thus the Hessian $B_k$ is not required explicitly. However when factorizations of the Hessian are possible, the direct approach is often more efficient.

When a quartic model is used, the model is itself of least-squares form, and the package calls itself recursively to approximately minimize its model. The quartic model often gives a better approximation, but at the cost of more involved derivative requirements.

references#

The generic adaptive cubic regularization method is described in detail in

C. Cartis, N. I. M. Gould and Ph. L. Toint, ``Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results’’ Mathematical Programming 127(2) (2011) 245–295,

and uses ``tricks’’ as suggested in

N. I. M. Gould, M. Porcelli and Ph. L. Toint, ``Updating the regularization parameter in the adaptive cubic regularization algorithm’’. Computational Optimization and Applications 53(1) (2012) 1–22.

The specific methods employed here are discussed in

N. I. M. Gould, J. A. Scott and T. Rees, ``Convergence and evaluation-complexity analysis of a regularized tensor-Newton method for solving nonlinear least-squares problems’’. Computational Optimization and Applications 73(1) (2019) 1–35.

matrix storage#

The unsymmetric $m$ by $n$ Jacobian matrix $J = J(x)$ and the residual-Hessians-vector product matrix $P(x,v)$ may be presented and stored in a variety of convenient input formats. Let $A$ be $J$ or $P$ as appropriate.

Dense storage format: The matrix $A$ is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. In this case, component $n \ast i + j$ of the storage array A_val will hold the value $A_{ij}$ for $0 \leq i \leq m-1$, $0 \leq j \leq n-1$.

Dense by columns storage format: The matrix $A$ is stored as a compact dense matrix by columns, that is, the values of the entries of each column in turn are stored in order within an appropriate real one-dimensional array. In this case, component $m \ast j + i$ of the storage array A_val will hold the value $A_{ij}$ for $0 \leq i \leq m-1$, $0 \leq j \leq n-1$.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the $l$-th entry, $0 \leq l \leq ne-1$, of $A$, its row index i, column index j and value $A_{ij}$, $0 \leq i \leq m-1$, $0 \leq j \leq n-1$, are stored as the $l$-th components of the integer arrays A_row and A_col and real array A_val, respectively, while the number of nonzeros is recorded as A_ne = $ne$.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of $A$ the i-th component of the integer array A_ptr holds the position of the first entry in this row, while A_ptr(m) holds the total number of entries. The column indices j, $0 \leq j \leq n-1$, and values $A_{ij}$ of the nonzero entries in the i-th row are stored in components l = A_ptr(i), $\ldots$, A_ptr(i+1)-1, $0 \leq i \leq m-1$, of the integer array A_col, and real array A_val, respectively. For sparse matrices, this scheme almost always requires less storage than its predecessor.

Sparse column-wise storage format: Once again only the nonzero entries are stored, but this time they are ordered so that those in column j appear directly before those in column j+1. For the j-th column of $A$ the j-th component of the integer array A_ptr holds the position of the first entry in this column, while A_ptr(n) holds the total number of entries. The row indices i, $0 \leq i \leq m-1$, and values $A_{ij}$ of the nonzero entries in the j-th columnsare stored in components l = A_ptr(j), $\ldots$, A_ptr(j+1)-1, $0 \leq j \leq n-1$, of the integer array A_row, and real array A_val, respectively. As before, for sparse matrices, this scheme almost always requires less storage than the co-ordinate format.

The symmetric $n$ by $n$ matrix $H = H(x,y)$ may be presented and stored in a variety of formats. But crucially symmetry is exploited by only storing values from the lower triangular part (i.e, those entries that lie on or below the leading diagonal).

Dense storage format: The matrix $H$ is stored as a compact dense matrix by rows, that is, the values of the entries of each row in turn are stored in order within an appropriate real one-dimensional array. Since $H$ is symmetric, only the lower triangular part (that is the part $H_{ij}$ for $0 \leq j \leq i \leq n-1$) need be held. In this case the lower triangle should be stored by rows, that is component $i * i / 2 + j$ of the storage array H_val will hold the value $H_{ij}$ (and, by symmetry, $H_{ji}$) for $0 \leq j \leq i \leq n-1$.

Sparse co-ordinate storage format: Only the nonzero entries of the matrices are stored. For the $l$-th entry, $0 \leq l \leq ne-1$, of $H$, its row index i, column index j and value $H_{ij}$, $0 \leq j \leq i \leq n-1$, are stored as the $l$-th components of the integer arrays H_row and H_col and real array H_val, respectively, while the number of nonzeros is recorded as H_ne = $ne$. Note that only the entries in the lower triangle should be stored.

Sparse row-wise storage format: Again only the nonzero entries are stored, but this time they are ordered so that those in row i appear directly before those in row i+1. For the i-th row of $H$ the i-th component of the integer array H_ptr holds the position of the first entry in this row, while H_ptr(n) holds the total number of entries. The column indices j, $0 \leq j \leq i$, and values $H_{ij}$ of the entries in the i-th row are stored in components l = H_ptr(i), …, H_ptr(i+1)-1 of the integer array H_col, and real array H_val, respectively. Note that as before only the entries in the lower triangle should be stored. For sparse matrices, this scheme almost always requires less storage than its predecessor.

Diagonal storage format: If $H$ is diagonal (i.e., $H_{ij} = 0$ for all $0 \leq i \neq j \leq n-1$) only the diagonals entries $H_{ii}$, $0 \leq i \leq n-1$ need be stored, and the first n components of the array H_val may be used for the purpose.

Multiples of the identity storage format: If $H$ is a multiple of the identity matrix, (i.e., $H = \alpha I$ where $I$ is the n by n identity matrix and $\alpha$ is a scalar), it suffices to store $\alpha$ as the first component of H_val.

The identity matrix format: If $H$ is the identity matrix, no values need be stored.

The zero matrix format: The same is true if $H$7 is the zero matrix.

introduction to function calls#

To solve a given problem, functions from the nls package must be called in the following order:

nls_initialize - provide default control parameters and set up initial data structures
nls_read_specfile (optional) - override control values by reading replacement values from a file
nls_import - set up problem data structures and fixed values
nls_reset_control (optional) - possibly change control parameters if a sequence of problems are being solved
solve the problem by calling one of
- nls_solve_with_mat - solve using function calls to evaluate function, gradient and Hessian values
- nls_solve_without_mat - solve using function calls to evaluate function and gradient values and Hessian-vector products
- nls_solve_reverse_with_mat - solve returning to the calling program to obtain function, gradient and Hessian values, or
- nls_solve_reverse_without_mat - solve returning to the calling prorgram to obtain function and gradient values and Hessian-vector products
nls_information (optional) - recover information about the solution and solution process
nls_terminate - deallocate data structures

See the examples section for illustrations of use.

callable functions#

overview of functions provided#

// typedefs

typedef float spc_;
typedef double rpc_;
typedef int ipc_;

// structs

struct nls_subproblem_control_type;
struct nls_control_type;
struct nls_subproblem_inform_type;
struct nls_inform_type;
struct nls_time_type;

// function calls

void nls_initialize(
    void **data,
    struct nls_control_type* control,
    struct nls_inform_type* inform
);

void nls_read_specfile(struct nls_control_type* control, const char specfile[]);

void nls_import(
    struct nls_control_type* control,
    void **data,
    ipc_ *status,
    ipc_ n,
    ipc_ m,
    const char J_type[],
    ipc_ J_ne,
    const ipc_ J_row[],
    const ipc_ J_col[],
    const ipc_ J_ptr[],
    const char H_type[],
    ipc_ H_ne,
    const ipc_ H_row[],
    const ipc_ H_col[],
    const ipc_ H_ptr[],
    const char P_type[],
    ipc_ P_ne,
    const ipc_ P_row[],
    const ipc_ P_col[],
    const ipc_ P_ptr[],
    const rpc_ w[]
);

void nls_reset_control(
    struct nls_control_type* control,
    void **data,
    ipc_ *status
);

void nls_solve_with_mat(
    void **data,
    void *userdata,
    ipc_ *status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    ipc_(*)(ipc_, ipc_, const rpc_[], rpc_[], const void*) eval_c,
    ipc_ j_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], rpc_[], const void*) eval_j,
    ipc_ h_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], const void*) eval_h,
    ipc_ p_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], bool, const void*) eval_hprods
);

void nls_solve_without_mat(
    void **data,
    void *userdata,
    ipc_ *status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    ipc_(*)(ipc_, ipc_, const rpc_[], rpc_[], const void*) eval_c,
    ipc_(*)(ipc_, ipc_, const rpc_[], const bool, rpc_[], const rpc_[], bool, const void*) eval_jprod,
    ipc_(*)(ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], const rpc_[], bool, const void*) eval_hprod,
    ipc_ p_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], bool, const void*) eval_hprods
);

void nls_solve_reverse_with_mat(
    void **data,
    ipc_ *status,
    ipc_ *eval_status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    ipc_ j_ne,
    rpc_ J_val[],
    const rpc_ y[],
    ipc_ h_ne,
    rpc_ H_val[],
    rpc_ v[],
    ipc_ p_ne,
    rpc_ P_val[]
);

void nls_solve_reverse_without_mat(
    void **data,
    ipc_ *status,
    ipc_ *eval_status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    bool* transpose,
    rpc_ u[],
    rpc_ v[],
    rpc_ y[],
    ipc_ p_ne,
    rpc_ P_val[]
);

void nls_information(void **data, struct nls_inform_type* inform, ipc_ *status);

void nls_terminate(
    void **data,
    struct nls_control_type* control,
    struct nls_inform_type* inform
);

typedefs#

typedef float spc_

spc_ is real single precision

typedef double rpc_

rpc_ is the real working precision used, but may be changed to float by defining the preprocessor variable SINGLE.

typedef int ipc_

ipc_ is the default integer word length used, but may be changed to int64_t by defining the preprocessor variable INTEGER_64.

function calls#

void nls_initialize(
    void **data,
    struct nls_control_type* control,
    struct nls_inform_type* inform
)

Set default control values and initialize private data

Parameters:

data	holds private internal data
control	is a struct containing control information (see nls_control_type)
inform	is a struct containing output information (see nls_inform_type)

void nls_read_specfile(struct nls_control_type* control, const char specfile[])

Read the content of a specification file, and assign values associated with given keywords to the corresponding control parameters. An in-depth discussion of specification files is available, and a detailed list of keywords with associated default values is provided in $GALAHAD/src/nls/NLS.template. See also Table 2.1 in the Fortran documentation provided in $GALAHAD/doc/nls.pdf for a list of how these keywords relate to the components of the control structure.

Parameters:

control	is a struct containing control information (see nls_control_type)
specfile	is a character string containing the name of the specification file

void nls_import(
    struct nls_control_type* control,
    void **data,
    ipc_ *status,
    ipc_ n,
    ipc_ m,
    const char J_type[],
    ipc_ J_ne,
    const ipc_ J_row[],
    const ipc_ J_col[],
    const ipc_ J_ptr[],
    const char H_type[],
    ipc_ H_ne,
    const ipc_ H_row[],
    const ipc_ H_col[],
    const ipc_ H_ptr[],
    const char P_type[],
    ipc_ P_ne,
    const ipc_ P_row[],
    const ipc_ P_col[],
    const ipc_ P_ptr[],
    const rpc_ w[]
)

Import problem data into internal storage prior to solution.

Parameters:

control	is a struct whose members provide control paramters for the remaining prcedures (see nls_control_type)
data	holds private internal data
status	is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are: 1 The import was successful, and the package is ready for the solve phase -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restrictions n > 0, m > 0 or requirement that J/H/P_type contains its relevant string ‘dense’, ‘dense_by_columns’, ‘coordinate’, ‘sparse_by_rows’, ‘sparse_by_columns’, ‘diagonal’ or ‘absent’ has been violated.
n	is a scalar variable of type ipc_, that holds the number of variables.
m	is a scalar variable of type ipc_, that holds the number of residuals.
J_type	is a one-dimensional array of type char that specifies the unsymmetric storage scheme used for the Jacobian, $J$. It should be one of ‘coordinate’, ‘sparse_by_rows’, ‘dense’ or ‘absent’, the latter if access to the Jacobian is via matrix-vector products; lower or upper case variants are allowed.
J_ne	is a scalar variable of type ipc_, that holds the number of entries in $J$ in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
J_row	is a one-dimensional array of size J_ne and type ipc_, that holds the row indices of $J$ in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes, and in this case can be NULL.
J_col	is a one-dimensional array of size J_ne and type ipc_, that holds the column indices of $J$ in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense or diagonal storage schemes are used, and in this case can be NULL.
J_ptr	is a one-dimensional array of size m+1 and type ipc_, that holds the starting position of each row of $J$, as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be NULL.
H_type	is a one-dimensional array of type char that specifies the symmetric storage scheme used for the Hessian, $H$. It should be one of ‘coordinate’, ‘sparse_by_rows’, ‘dense’, ‘diagonal’ or ‘absent’, the latter if access to $H$ is via matrix-vector products; lower or upper case variants are allowed.
H_ne	is a scalar variable of type ipc_, that holds the number of entries in the lower triangular part of $H$ in the sparse co-ordinate storage scheme. It need not be set for any of the other three schemes.
H_row	is a one-dimensional array of size H_ne and type ipc_, that holds the row indices of the lower triangular part of $H$ in the sparse co-ordinate storage scheme. It need not be set for any of the other three schemes, and in this case can be NULL.
H_col	is a one-dimensional array of size H_ne and type ipc_, that holds the column indices of the lower triangular part of $H$ in either the sparse co-ordinate, or the sparse row-wise storage scheme. It need not be set when the dense or diagonal storage schemes are used, and in this case can be NULL.
H_ptr	is a one-dimensional array of size n+1 and type ipc_, that holds the starting position of each row of the lower triangular part of $H$, as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be NULL.
P_type	is a one-dimensional array of type char that specifies the unsymmetric storage scheme used for the residual-Hessians-vector product matrix, $P$. It should be one of ‘coordinate’, ‘sparse_by_columns’, ‘dense_by_columns’ or ‘absent’, the latter if access to $P$ is via matrix-vector products; lower or upper case variants are allowed.
P_ne	is a scalar variable of type ipc_, that holds the number of entries in $P$ in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes.
P_row	is a one-dimensional array of size P_ne and type ipc_, that holds the row indices of $P$ in either the sparse co-ordinate, or the sparse column-wise storage scheme. It need not be set when the dense storage scheme is used, and in this case can be NULL.
P_col	is a one-dimensional array of size P_ne and type ipc_, that holds the row indices of $P$ in the sparse co-ordinate storage scheme. It need not be set for any of the other schemes, and in this case can be NULL.
P_ptr	is a one-dimensional array of size n+1 and type ipc_, that holds the starting position of each row of $P$, as well as the total number of entries, in the sparse row-wise storage scheme. It need not be set when the other schemes are used, and in this case can be NULL.
w	is a one-dimensional array of size m and type rpc_, that holds the values $w$ of the weights on the residuals in the least-squares objective function. It need not be set if the weights are all ones, and in this case can be NULL.

void nls_reset_control(
    struct nls_control_type* control,
    void **data,
    ipc_ *status
)

Reset control parameters after import if required.

Parameters:

control

is a struct whose members provide control paramters for the remaining prcedures (see nls_control_type)

data

holds private internal data

status

is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are:

1. The import was successful, and the package is ready for the solve phase

void nls_solve_with_mat(
    void **data,
    void *userdata,
    ipc_ *status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    ipc_(*)(ipc_, ipc_, const rpc_[], rpc_[], const void*) eval_c,
    ipc_ j_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], rpc_[], const void*) eval_j,
    ipc_ h_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], const void*) eval_h,
    ipc_ p_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], bool, const void*) eval_hprods
)

Find a local minimizer of a given function using a trust-region method.

This call is for the case where $H = \nabla_{xx}f(x)$ is provided specifically, and all function/derivative information is available by function calls.

Parameters:

data	holds private internal data
userdata	is a structure that allows data to be passed into the function and derivative evaluation programs.
status	is a scalar variable of type ipc_, that gives the entry and exit status from the package. On initial entry, status must be set to 1. Possible exit values are: 0 The run was successful -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’ or ‘absent’ has been violated. -9 The analysis phase of the factorization failed; the return status from the factorization package is given in the component inform.factor_status -10 The factorization failed; the return status from the factorization package is given in the component inform.factor_status. -11 The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given in the component inform.factor_status. -16 The problem is so ill-conditioned that further progress is impossible. -17 The step is too small to make further impact. -18 Too many iterations have been performed. This may happen if control.maxit is too small, but may also be symptomatic of a badly scaled problem. -19 The CPU time limit has been reached. This may happen if control.cpu_time_limit is too small, but may also be symptomatic of a badly scaled problem. -82 The user has forced termination of solver by removing the file named control.alive_file from unit unit control.alive_unit.
n	is a scalar variable of type ipc_, that holds the number of variables.
m	is a scalar variable of type ipc_, that holds the number of residuals.
x	is a one-dimensional array of size n and type rpc_, that holds the values $x$ of the optimization variables. The j-th component of x, j = 0, … , n-1, contains $x_j$.
c	is a one-dimensional array of size m and type rpc_, that holds the residual $c(x)$. The i-th component of c, j = 0, … , n-1, contains $c_j(x)$.
g	is a one-dimensional array of size n and type rpc_, that holds the gradient $g = \nabla_xf(x)$ of the objective function. The j-th component of g, j = 0, … , n-1, contains $g_j$.
eval_c	is a user-supplied function that must have the following signature: ipc_ eval_c( ipc_ n, const rpc_ x[], rpc_ c[], const void *userdata ) The componnts of the residual function $c(x)$ evaluated at x= $x$ must be assigned to c, and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_c` via the structure `userdata`.
j_ne	is a scalar variable of type ipc_, that holds the number of entries in the Jacobian matrix $J$.
eval_j	is a user-supplied function that must have the following signature: ipc_ eval_j( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ j[], const void *userdata ) The components of the Jacobian $J = \nabla_x c(x$) of the residuals must be assigned to j in the same order as presented to nls_import, and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_j` via the structure `userdata`.
h_ne	is a scalar variable of type ipc_, that holds the number of entries in the lower triangular part of the Hessian matrix $H$ if it is used.
eval_h	is a user-supplied function that must have the following signature: ipc_ eval_h( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[], rpc_ h[], const void *userdata ) The nonzeros of the matrix $H = \sum_{i=1}^m y_i \nabla_{xx}c_i(x)$ of the weighted residual Hessian evaluated at x= $x$ and y= $y$ must be assigned to h in the same order as presented to nls_import, and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_h` via the structure `userdata`.
p_ne	is a scalar variable of type ipc_, that holds the number of entries in the residual-Hessians-vector product matrix $P$ if it is used.
eval_hprods	is an optional user-supplied function that may be NULL. If non-NULL, it must have the following signature: ipc_ eval_hprods( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[], const rpc_ v[], rpc_ p[], bool got_h, const void *userdata ) ); The entries of the matrix $P$, whose i-th column is the product $\nabla_{xx}c_i(x) v$ between $\nabla_{xx}c_i(x)$, the Hessian of the i-th component of the residual $c(x)$ at x= $x$, and v= $v$ must be returned in p and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_hprods` via the structure `userdata`.

void nls_solve_without_mat(
    void **data,
    void *userdata,
    ipc_ *status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    ipc_(*)(ipc_, ipc_, const rpc_[], rpc_[], const void*) eval_c,
    ipc_(*)(ipc_, ipc_, const rpc_[], const bool, rpc_[], const rpc_[], bool, const void*) eval_jprod,
    ipc_(*)(ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], const rpc_[], bool, const void*) eval_hprod,
    ipc_ p_ne,
    ipc_(*)(ipc_, ipc_, ipc_, const rpc_[], const rpc_[], rpc_[], bool, const void*) eval_hprods
)

Find a local minimizer of a given function using a trust-region method.

This call is for the case where access to $H = \nabla_{xx}f(x)$ is provided by Hessian-vector products, and all function/derivative information is available by function calls.

Parameters:

data	holds private internal data
userdata	is a structure that allows data to be passed into the function and derivative evaluation programs.
status	is a scalar variable of type ipc_, that gives the entry and exit status from the package. On initial entry, status must be set to 1. Possible exit values are: 0 The run was successful -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’ or ‘absent’ has been violated. -9 The analysis phase of the factorization failed; the return status from the factorization package is given in the component inform.factor_status -10 The factorization failed; the return status from the factorization package is given in the component inform.factor_status. -11 The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given in the component inform.factor_status. -16 The problem is so ill-conditioned that further progress is impossible. -17 The step is too small to make further impact. -18 Too many iterations have been performed. This may happen if control.maxit is too small, but may also be symptomatic of a badly scaled problem. -19 The CPU time limit has been reached. This may happen if control.cpu_time_limit is too small, but may also be symptomatic of a badly scaled problem. -82 The user has forced termination of solver by removing the file named control.alive_file from unit unit control.alive_unit.
n	is a scalar variable of type ipc_, that holds the number of variables
m	is a scalar variable of type ipc_, that holds the number of residuals.
x	is a one-dimensional array of size n and type rpc_, that holds the values $x$ of the optimization variables. The j-th component of x, j = 0, … , n-1, contains $x_j$.
c	is a one-dimensional array of size m and type rpc_, that holds the residual $c(x)$. The i-th component of c, j = 0, … , n-1, contains $c_j(x)$.
g	is a one-dimensional array of size n and type rpc_, that holds the gradient $g = \nabla_xf(x)$ of the objective function. The j-th component of g, j = 0, … , n-1, contains $g_j$.
eval_c	is a user-supplied function that must have the following signature: ipc_ eval_c( ipc_ n, const rpc_ x[], rpc_ c[], const void *userdata ) The componnts of the residual function $c(x)$ evaluated at x= $x$ must be assigned to c, and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_c` via the structure `userdata`.
eval_jprod	is a user-supplied function that must have the following signature: ipc_ eval_jprod( ipc_ n, ipc_ m, const rpc_ x[], bool transpose, rpc_ u[], const rpc_ v[], bool got_j, const void *userdata ) The sum $u + \nabla_{x}c_(x) v$ (if tranpose is false) or The sum $u + (\nabla_{x}c_(x))^T v$ (if tranpose is true) bewteen the product of the Jacobian $\nabla_{x}c_(x)$ or its tranpose with the vector v= $v$ and the vector $ $u$ must be returned in u, and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_jprod` via the structure `userdata`.
eval_hprod	is a user-supplied function that must have the following signature: ipc_ eval_hprod( ipc_ n, ipc_ m, const rpc_ x[], const rpc_ y[], rpc_ u[], const rpc_ v[], bool got_h, const void *userdata ) The sum $u + \sum_{i=1}^m y_i \nabla_{xx}c_i(x) v$ of the product of the weighted residual Hessian $H = \sum_{i=1}^m y_i \nabla_{xx}c_i(x)$ evaluated at x= $x$ and y= $y$ with the vector v= $v$ and the vector $ $u$ must be returned in u, and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. The Hessians have already been evaluated or used at x if got_h is true. Data may be passed into `eval_hprod` via the structure `userdata`.
p_ne	is a scalar variable of type ipc_, that holds the number of entries in the residual-Hessians-vector product matrix $P$ if it is used.
eval_hprods	is an optional user-supplied function that may be NULL. If non-NULL, it must have the following signature: ipc_ eval_hprods( ipc_ n, ipc_ m, ipc_ p_ne, const rpc_ x[], const rpc_ v[], rpc_ pval[], bool got_h, const void *userdata ) The entries of the matrix $P$, whose i-th column is the product $\nabla_{xx}c_i(x) v$ between $\nabla_{xx}c_i(x)$, the Hessian of the i-th component of the residual $c(x)$ at x= $x$, and v= $v$ must be returned in pval and the function return value set to 0. If the evaluation is impossible at x, return should be set to a nonzero value. Data may be passed into `eval_hprods` via the structure `userdata`.

void nls_solve_reverse_with_mat(
    void **data,
    ipc_ *status,
    ipc_ *eval_status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    ipc_ j_ne,
    rpc_ J_val[],
    const rpc_ y[],
    ipc_ h_ne,
    rpc_ H_val[],
    rpc_ v[],
    ipc_ p_ne,
    rpc_ P_val[]
)

Find a local minimizer of a given function using a trust-region method.

This call is for the case where $H = \nabla_{xx}f(x)$ is provided specifically, but function/derivative information is only available by returning to the calling procedure

Parameters:

data	holds private internal data
status	is a scalar variable of type ipc_, that gives the entry and exit status from the package. On initial entry, status must be set to 1. Possible exit values are: 0 The run was successful -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’ or ‘absent’ has been violated. -9 The analysis phase of the factorization failed; the return status from the factorization package is given in the component inform.factor_status -10 The factorization failed; the return status from the factorization package is given in the component inform.factor_status. -11 The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given in the component inform.factor_status. -16 The problem is so ill-conditioned that further progress is impossible. -17 The step is too small to make further impact. -18 Too many iterations have been performed. This may happen if control.maxit is too small, but may also be symptomatic of a badly scaled problem. -19 The CPU time limit has been reached. This may happen if control.cpu_time_limit is too small, but may also be symptomatic of a badly scaled problem. -82 The user has forced termination of solver by removing the file named control.alive_file from unit unit control.alive_unit. 2 The user should compute the vector of residuals $c(x)$ at the point $x$ indicated in x and then re-enter the function. The required value should be set in c, and eval_status should be set to 0. If the user is unable to evaluate $c(x)$ for instance, if the function is undefined at $x$ the user need not set c, but should then set eval_status to a non-zero value. 3 The user should compute the Jacobian of the vector of residual functions, $\nabla_x c(x)$, at the point $x$ indicated in x and then re-enter the function. The l-th component of the Jacobian stored according to the scheme specified for the remainder of $J$ in the earlier call to nls_import should be set in J_val[l], for l = 0, …, J_ne-1 and eval_status should be set to 0. If the user is unable to evaluate a component of $J$ for instance, if a component of the matrix is undefined at $x$ the user need not set J_val, but should then set eval_status to a non-zero value. 4 The user should compute the matrix $H = \sum_{i=1}^m v_i \nabla_{xx}c_i(x)$ of weighted residual Hessian evaluated at x= $x$ and v= $v$ and then re-enter the function. The l-th component of the matrix stored according to the scheme specified for the remainder of $H$ in the earlier call to nls_import should be set in H_val[l], for l = 0, …, H_ne-1 and eval_status should be set to 0. If the user is unable to evaluate a component of $H$ for instance, if a component of the matrix is undefined at $x$ the user need not set H_val, but should then set eval_status to a non-zero value. Note that this return will not happen if the Gauss-Newton model is selected. 7 The user should compute the entries of the matrix $P$, whose i-th column is the product $\nabla_{xx}c_i(x) v$ between $\nabla_{xx}c_i(x)$, the Hessian of the i-th component of the residual $c(x)$ at x= $x$, and v= $v$ and then re-enter the function. The l-th component of the matrix stored according to the scheme specified for the remainder of $P$ in the earlier call to nls_import should be set in P_val[l], for l = 0, …, P_ne-1 and eval_status should be set to 0. If the user is unable to evaluate a component of $P$ for instance, if a component of the matrix is undefined at $x$ the user need not set P_val, but should then set eval_status to a non-zero value. Note that this return will not happen if either the Gauss-Newton or Newton models is selected.
eval_status	is a scalar variable of type ipc_, that is used to indicate if objective function/gradient/Hessian values can be provided (see above)
n	is a scalar variable of type ipc_, that holds the number of variables
m	is a scalar variable of type ipc_, that holds the number of residuals.
x	is a one-dimensional array of size n and type rpc_, that holds the values $x$ of the optimization variables. The j-th component of x, j = 0, … , n-1, contains $x_j$.
c	is a one-dimensional array of size m and type rpc_, that holds the residual $c(x)$. The i-th component of c, j = 0, … , n-1, contains $c_j(x)$. See status = 2, above, for more details.
g	is a one-dimensional array of size n and type rpc_, that holds the gradient $g = \nabla_xf(x)$ of the objective function. The j-th component of g, j = 0, … , n-1, contains $g_j$.
j_ne	is a scalar variable of type ipc_, that holds the number of entries in the Jacobian matrix $J$.
J_val	is a one-dimensional array of size j_ne and type rpc_, that holds the values of the entries of the Jacobian matrix $J$ in any of the available storage schemes. See status = 3, above, for more details.
y	is a one-dimensional array of size m and type rpc_, that is used for reverse communication. See status = 4 above for more details.
h_ne	is a scalar variable of type ipc_, that holds the number of entries in the lower triangular part of the Hessian matrix $H$.
H_val	is a one-dimensional array of size h_ne and type rpc_, that holds the values of the entries of the lower triangular part of the Hessian matrix $H$ in any of the available storage schemes. See status = 4, above, for more details.
v	is a one-dimensional array of size n and type rpc_, that is used for reverse communication. See status = 7, above, for more details.
p_ne	is a scalar variable of type ipc_, that holds the number of entries in the residual-Hessians-vector product matrix, $P$.
P_val	is a one-dimensional array of size p_ne and type rpc_, that holds the values of the entries of the residual-Hessians-vector product matrix, $P$. See status = 7, above, for more details.

void nls_solve_reverse_without_mat(
    void **data,
    ipc_ *status,
    ipc_ *eval_status,
    ipc_ n,
    ipc_ m,
    rpc_ x[],
    rpc_ c[],
    rpc_ g[],
    bool* transpose,
    rpc_ u[],
    rpc_ v[],
    rpc_ y[],
    ipc_ p_ne,
    rpc_ P_val[]
)

Find a local minimizer of a given function using a trust-region method.

This call is for the case where access to $H = \nabla_{xx}f(x)$ is provided by Hessian-vector products, but function/derivative information is only available by returning to the calling procedure.

Parameters:

data	holds private internal data
status	is a scalar variable of type ipc_, that gives the entry and exit status from the package. On initial entry, status must be set to 1. Possible exit values are: 0 The run was successful -1 An allocation error occurred. A message indicating the offending array is written on unit control.error, and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -2 A deallocation error occurred. A message indicating the offending array is written on unit control.error and the returned allocation status and a string containing the name of the offending array are held in inform.alloc_status and inform.bad_alloc respectively. -3 The restriction n > 0 or requirement that type contains its relevant string ‘dense’, ‘coordinate’, ‘sparse_by_rows’, ‘diagonal’ or ‘absent’ has been violated. -9 The analysis phase of the factorization failed; the return status from the factorization package is given in the component inform.factor_status -10 The factorization failed; the return status from the factorization package is given in the component inform.factor_status. -11 The solution of a set of linear equations using factors from the factorization package failed; the return status from the factorization package is given in the component inform.factor_status. -16 The problem is so ill-conditioned that further progress is impossible. -17 The step is too small to make further impact. -18 Too many iterations have been performed. This may happen if control.maxit is too small, but may also be symptomatic of a badly scaled problem. -19 The CPU time limit has been reached. This may happen if control.cpu_time_limit is too small, but may also be symptomatic of a badly scaled problem. -82 The user has forced termination of solver by removing the file named control.alive_file from unit unit control.alive_unit. 2 The user should compute the vector of residuals $c(x)$ at the point $x$ indicated in x and then re-enter the function. The required value should be set in c, and eval_status should be set to 0. If the user is unable to evaluate $c(x)$ for instance, if the function is undefined at $x$ the user need not set c, but should then set eval_status to a non-zero value. 5 The user should compute the sum $u + \nabla_{x}c_(x) v$ (if tranpose is false) or $u + (\nabla_{x}c_(x))^T v$ (if tranpose is true) between the product of the Jacobian $\nabla_{x}c_(x)$ or its tranpose with the vector v= $v$ and the vector u = $ $u$, and then re-enter the function. The result should be set in u, and eval_status should be set to 0. If the user is unable to evaluate the sum for instance, if the Jacobian is undefined at $x$ the user need not set u, but should then set eval_status to a non-zero value. 6 The user should compute the sum $u + \sum_{i=1}^m y_i \nabla_{xx}c_i(x) v$ between the product of the weighted residual Hessian $H = \sum_{i=1}^m y_i \nabla_{xx}c_i(x)$ evaluated at x= $x$ and y= $y$ with the vector v= $v$ and the the vector u = $ $u$, and then re-enter the function. The result should be set in u, and eval_status should be set to 0. If the user is unable to evaluate the sum for instance, if the weifghted residual Hessian is undefined at $x$ the user need not set u, but should then set eval_status to a non-zero value. 7 The user should compute the entries of the matrix $P$, whose i-th column is the product $\nabla_{xx}c_i(x) v$ between $\nabla_{xx}c_i(x)$, the Hessian of the i-th component of the residual $c(x)$ at x= $x$, and v= $v$ and then re-enter the function. The l-th component of the matrix stored according to the scheme specified for the remainder of $P$ in the earlier call to nls_import should be set in P_val[l], for l = 0, …, P_ne-1 and eval_status should be set to 0. If the user is unable to evaluate a component of $P$ for instance, if a component of the matrix is undefined at $x$ the user need not set P_val, but should then set eval_status to a non-zero value. Note that this return will not happen if either the Gauss-Newton or Newton models is selected.
eval_status	is a scalar variable of type ipc_, that is used to indicate if objective function/gradient/Hessian values can be provided (see above)
n	is a scalar variable of type ipc_, that holds the number of variables
m	is a scalar variable of type ipc_, that holds the number of residuals.
x	is a one-dimensional array of size n and type rpc_, that holds the values $x$ of the optimization variables. The j-th component of x, j = 0, … , n-1, contains $x_j$.
c	is a one-dimensional array of size m and type rpc_, that holds the residual $c(x)$. The i-th component of c, j = 0, … , n-1, contains $c_j(x)$. See status = 2, above, for more details.
g	is a one-dimensional array of size n and type rpc_, that holds the gradient $g = \nabla_xf(x)$ of the objective function. The j-th component of g, j = 0, … , n-1, contains $g_j$.
transpose	is a scalar variable of type bool, that indicates whether the product with Jacobian or its transpose should be obtained when status=5.
u	is a one-dimensional array of size max(n,m) and type rpc_, that is used for reverse communication. See status = 5,6 above for more details.
v	is a one-dimensional array of size max(n,m) and type rpc_, that is used for reverse communication. See status = 5,6,7 above for more details.
y	is a one-dimensional array of size m and type rpc_, that is used for reverse communication. See status = 6 above for more details.
p_ne	is a scalar variable of type ipc_, that holds the number of entries in the residual-Hessians-vector product matrix, $P$.
P_val	is a one-dimensional array of size P_ne and type rpc_, that holds the values of the entries of the residual-Hessians-vector product matrix, $P$. See status = 7, above, for more details.

void nls_information(void **data, struct nls_inform_type* inform, ipc_ *status)

Provides output information

Parameters:

data

holds private internal data

inform

is a struct containing output information (see nls_inform_type)

status

is a scalar variable of type ipc_, that gives the exit status from the package. Possible values are (currently):

0
The values were recorded successfully

void nls_terminate(
    void **data,
    struct nls_control_type* control,
    struct nls_inform_type* inform
)

Deallocate all internal private storage

Parameters:

data	holds private internal data
control	is a struct containing control information (see nls_control_type)
inform	is a struct containing output information (see nls_inform_type)

available structures#

nls_subproblem_control_type structure#

#include <galahad_nls.h>

struct nls_subproblem_control_type {
    // components

    ipc_ error;
    ipc_ out;
    ipc_ print_level;
    ipc_ start_print;
    ipc_ stop_print;
    ipc_ print_gap;
    ipc_ maxit;
    ipc_ alive_unit;
    char alive_file[31];
    ipc_ jacobian_available;
    ipc_ hessian_available;
    ipc_ model;
    ipc_ norm;
    ipc_ non_monotone;
    ipc_ weight_update_strategy;
    rpc_ stop_c_absolute;
    rpc_ stop_c_relative;
    rpc_ stop_g_absolute;
    rpc_ stop_g_relative;
    rpc_ stop_s;
    rpc_ power;
    rpc_ initial_weight;
    rpc_ minimum_weight;
    rpc_ initial_inner_weight;
    rpc_ eta_successful;
    rpc_ eta_very_successful;
    rpc_ eta_too_successful;
    rpc_ weight_decrease_min;
    rpc_ weight_decrease;
    rpc_ weight_increase;
    rpc_ weight_increase_max;
    rpc_ reduce_gap;
    rpc_ tiny_gap;
    rpc_ large_root;
    rpc_ switch_to_newton;
    rpc_ cpu_time_limit;
    rpc_ clock_time_limit;
    bool subproblem_direct;
    bool renormalize_weight;
    bool magic_step;
    bool print_obj;
    bool space_critical;
    bool deallocate_error_fatal;
    char prefix[31];
    struct rqs_control_type rqs_control;
    struct glrt_control_type glrt_control;
    struct psls_control_type psls_control;
    struct bsc_control_type bsc_control;
    struct roots_control_type roots_control;
};

detailed documentation#

subproblem_control derived type as a C struct

components#

ipc_ error

error and warning diagnostics occur on stream error

ipc_ out

general output occurs on stream out

ipc_ print_level

the level of output required.

$\leq$ 0 gives no output,
= 1 gives a one-line summary for every iteration,
= 2 gives a summary of the inner iteration for each iteration,
$\geq$ 3 gives increasingly verbose (debugging) output

ipc_ start_print

any printing will start on this iteration

ipc_ stop_print

any printing will stop on this iteration

ipc_ print_gap

the number of iterations between printing

ipc_ maxit

the maximum number of iterations performed

ipc_ alive_unit

removal of the file alive_file from unit alive_unit terminates execution

char alive_file[31]

see alive_unit

ipc_ jacobian_available

is the Jacobian matrix of first derivatives available ($\geq$ 2), is access only via matrix-vector products (=1) or is it not available ($\leq$ 0) ?

ipc_ hessian_available

is the Hessian matrix of second derivatives available ($\geq$ 2), is access only via matrix-vector products (=1) or is it not available ($\leq$ 0) ?

ipc_ model

the model used.

Possible values are

0 dynamic (not yet implemented)
1 first-order (no Hessian)
2 barely second-order (identity Hessian)
3 Gauss-Newton ($J^T J$ Hessian)
4 second-order (exact Hessian)
5 Gauss-Newton to Newton transition
6 tensor Gauss-Newton treated as a least-squares model
7 tensor Gauss-Newton treated as a general model
8 tensor Gauss-Newton transition from a least-squares to a general mode

ipc_ norm

the regularization norm used.

The norm is defined via $\|v\|^2 = v^T S v$, and will define the preconditioner used for iterative methods. Possible values for $S$ are

-3 user’s own regularization norm
-2 $S$ = limited-memory BFGS matrix (with .PSLS_control.lbfgs_vectors history) (not yet implemented)
-1 identity (= Euclidan two-norm)
0 automatic (not yet implemented)
1 diagonal, $S$ = diag( max($J^TJ$ Hessian, .PSLS_control.min_diagonal ) )
2 diagonal, $S$ = diag( max( Hessian, .PSLS_control.min_diagonal ) )
3 banded, $S$ = band( Hessian ) with semi-bandwidth .PSLS_control.semi_bandwidth
4 re-ordered band, P=band(order(A)) with semi-bandwidth .PSLS_control.semi_bandwidth
5 full factorization, $S$ = Hessian, Schnabel-Eskow modification
6 full factorization, $S$ = Hessian, GMPS modification (not yet implemented)
7 incomplete factorization of Hessian, Lin-More’
8 incomplete factorization of Hessian, HSL_MI28
9 incomplete factorization of Hessian, Munskgaard (not yet implemented)
10 expanding band of Hessian (not yet implemented)

ipc_ non_monotone

non-monotone $\leq$ 0 monotone strategy used, anything else non-monotone strategy with this history length used

ipc_ weight_update_strategy

define the weight-update strategy: 1 (basic), 2 (reset to zero when very successful), 3 (imitate TR), 4 (increase lower bound), 5 (GPT)

rpc_ stop_c_absolute

overall convergence tolerances. The iteration will terminate when $||c(x)||_2 \leq$ MAX( .stop_c_absolute, .stop_c_relative $* \|c(x_{\mbox{initial}})\|_2$, or when the norm of the gradient, $g = J^T(x) c(x) / \|c(x)\|_2$, of \|\|c\|\|_2, satisfies $\|g\|_2 \leq$ MAX( .stop_g_absolute, .stop_g_relative $* \|g_{\mbox{initial}}\|_2$, or if the step is less than .stop_s

rpc_ stop_c_relative

see stop_c_absolute

rpc_ stop_g_absolute

see stop_c_absolute

rpc_ stop_g_relative

see stop_c_absolute

rpc_ stop_s

see stop_c_absolute

rpc_ power

the regularization power (<2 => chosen according to the model)

rpc_ initial_weight

initial value for the regularization weight (-ve => $1/\|g_0\|)$)

rpc_ minimum_weight

minimum permitted regularization weight

rpc_ initial_inner_weight

initial value for the inner regularization weight for tensor GN (-ve => 0)

rpc_ eta_successful

a potential iterate will only be accepted if the actual decrease f - f(x_new) is larger than .eta_successful times that predicted by a quadratic model of the decrease. The regularization weight will be decreaed if this relative decrease is greater than .eta_very_successful but smaller than .eta_too_successful

rpc_ eta_very_successful

see eta_successful

rpc_ eta_too_successful

see eta_successful

rpc_ weight_decrease_min

on very successful iterations, the regularization weight will be reduced by the factor .weight_decrease but no more than .weight_decrease_min while if the iteration is unsucceful, the weight will be increased by a factor .weight_increase but no more than .weight_increase_max (these are delta_1, delta_2, delta3 and delta_max in Gould, Porcelli and Toint, 2011)

rpc_ weight_decrease

see weight_decrease_min

rpc_ weight_increase

see weight_decrease_min

rpc_ weight_increase_max

see weight_decrease_min

rpc_ reduce_gap

expert parameters as suggested in Gould, Porcelli and Toint, “Updating t regularization parameter in the adaptive cubic regularization algorithm” RAL-TR-2011-007, Rutherford Appleton Laboratory, England (2011), http://epubs.stfc.ac.uk/bitstream/6181/RAL-TR-2011-007.pdf (these are denoted beta, epsilon_chi and alpha_max in the paper)

rpc_ tiny_gap

see reduce_gap

rpc_ large_root

see reduce_gap

rpc_ switch_to_newton

if the Gauss-Newto to Newton model is specified, switch to Newton as soon as the norm of the gradient g is smaller than switch_to_newton

rpc_ cpu_time_limit

the maximum CPU time allowed (-ve means infinite)

rpc_ clock_time_limit

the maximum elapsed clock time allowed (-ve means infinite)

bool subproblem_direct

use a direct (factorization) or (preconditioned) iterative method to find the search direction

bool renormalize_weight

should the weight be renormalized to account for a change in scaling?

bool magic_step

allow the user to perform a “magic” step to improve the objective

bool print_obj

print values of the objective/gradient rather than ||c|| and its gradien

bool space_critical

if .space_critical true, every effort will be made to use as little space as possible. This may result in longer computation time

bool deallocate_error_fatal

if .deallocate_error_fatal is true, any array/pointer deallocation error will terminate execution. Otherwise, computation will continue

char prefix[31]

all output lines will be prefixed by .prefix(2:LEN(TRIM(.prefix))-1) where .prefix contains the required string enclosed in quotes, e.g. “string” or ‘string’

struct rqs_control_type rqs_control

control parameters for RQS

struct glrt_control_type glrt_control

control parameters for GLRT

struct psls_control_type psls_control

control parameters for PSLS

struct bsc_control_type bsc_control

control parameters for BSC

struct roots_control_type roots_control

control parameters for ROOTS

nls_control_type structure#

#include <galahad_nls.h>

struct nls_control_type {
    // components

    bool f_indexing;
    ipc_ error;
    ipc_ out;
    ipc_ print_level;
    ipc_ start_print;
    ipc_ stop_print;
    ipc_ print_gap;
    ipc_ maxit;
    ipc_ alive_unit;
    char alive_file[31];
    ipc_ jacobian_available;
    ipc_ hessian_available;
    ipc_ model;
    ipc_ norm;
    ipc_ non_monotone;
    ipc_ weight_update_strategy;
    rpc_ stop_c_absolute;
    rpc_ stop_c_relative;
    rpc_ stop_g_absolute;
    rpc_ stop_g_relative;
    rpc_ stop_s;
    rpc_ power;
    rpc_ initial_weight;
    rpc_ minimum_weight;
    rpc_ initial_inner_weight;
    rpc_ eta_successful;
    rpc_ eta_very_successful;
    rpc_ eta_too_successful;
    rpc_ weight_decrease_min;
    rpc_ weight_decrease;
    rpc_ weight_increase;
    rpc_ weight_increase_max;
    rpc_ reduce_gap;
    rpc_ tiny_gap;
    rpc_ large_root;
    rpc_ switch_to_newton;
    rpc_ cpu_time_limit;
    rpc_ clock_time_limit;
    bool subproblem_direct;
    bool renormalize_weight;
    bool magic_step;
    bool print_obj;
    bool space_critical;
    bool deallocate_error_fatal;
    char prefix[31];
    struct rqs_control_type rqs_control;
    struct glrt_control_type glrt_control;
    struct psls_control_type psls_control;
    struct bsc_control_type bsc_control;
    struct roots_control_type roots_control;
    struct nls_subproblem_control_type subproblem_control;
};

detailed documentation#

control derived type as a C struct

components#

bool f_indexing

use C or Fortran sparse matrix indexing

ipc_ error

error and warning diagnostics occur on stream error

ipc_ out

general output occurs on stream out

ipc_ print_level

the level of output required.

$\leq$ 0 gives no output,
= 1 gives a one-line summary for every iteration,
= 2 gives a summary of the inner iteration for each iteration,
$\geq$ 3 gives increasingly verbose (debugging) output

ipc_ start_print

any printing will start on this iteration

ipc_ stop_print

any printing will stop on this iteration

ipc_ print_gap

the number of iterations between printing

ipc_ maxit

the maximum number of iterations performed

ipc_ alive_unit

removal of the file alive_file from unit alive_unit terminates execution

char alive_file[31]

see alive_unit

ipc_ jacobian_available

is the Jacobian matrix of first derivatives available ($\geq$ 2), is access only via matrix-vector products (=1) or is it not available ($\leq$ 0) ?

ipc_ hessian_available

is the Hessian matrix of second derivatives available ($\geq$ 2), is access only via matrix-vector products (=1) or is it not available ($\leq$ 0) ?

ipc_ model

the model used.

Possible values are

0 dynamic (not yet implemented)
1 first-order (no Hessian)
2 barely second-order (identity Hessian)
3 Gauss-Newton ($J^T J$ Hessian)
4 second-order (exact Hessian)
5 Gauss-Newton to Newton transition
6 tensor Gauss-Newton treated as a least-squares model
7 tensor Gauss-Newton treated as a general model
8 tensor Gauss-Newton transition from a least-squares to a general mode

ipc_ norm

the regularization norm used.

The norm is defined via $\|v\|^2 = v^T S v$, and will define the preconditioner used for iterative methods. Possible values for $S$ are

-3 user’s own regularization norm
-2 $S$ = limited-memory BFGS matrix (with .PSLS_control.lbfgs_vectors history) (not yet implemented)
-1 identity (= Euclidan two-norm)
0 automatic (not yet implemented)
1 diagonal, $S$ = diag( max($J^TJ$ Hessian, .PSLS_control.min_diagonal ) )
2 diagonal, $S$ = diag( max( Hessian, .PSLS_control.min_diagonal ) )
3 banded, $S$ = band( Hessian ) with semi-bandwidth .PSLS_control.semi_bandwidth
4 re-ordered band, P=band(order(A)) with semi-bandwidth .PSLS_control.semi_bandwidth
5 full factorization, $S$ = Hessian, Schnabel-Eskow modification
6 full factorization, $S$ = Hessian, GMPS modification (not yet implemented)
7 incomplete factorization of Hessian, Lin-More’
8 incomplete factorization of Hessian, HSL_MI28
9 incomplete factorization of Hessian, Munskgaard (not yet implemented)
10 expanding band of Hessian (not yet implemented)

ipc_ non_monotone

non-monotone $\leq$ 0 monotone strategy used, anything else non-monotone strategy with this history length used

ipc_ weight_update_strategy

define the weight-update strategy: 1 (basic), 2 (reset to zero when very successful), 3 (imitate TR), 4 (increase lower bound), 5 (GPT)

rpc_ stop_c_absolute

overall convergence tolerances. The iteration will terminate when $||c(x)||_2 \leq$ MAX( .stop_c_absolute, .stop_c_relative $* \|c(x_{\mbox{initial}})\|_2$ or when the norm of the gradient, $g = J^T(x) c(x) / \|c(x)\|_2$, of \|\|c(x)\|\|_2 satisfies $\|g\|_2 \leq$ MAX( .stop_g_absolute, .stop_g_relative $* \|g_{\mbox{initial}}\|_2$, or if the step is less than .stop_s

rpc_ stop_c_relative

see stop_c_absolute

rpc_ stop_g_absolute

see stop_c_absolute

rpc_ stop_g_relative

see stop_c_absolute

rpc_ stop_s

see stop_c_absolute

rpc_ power

the regularization power (<2 => chosen according to the model)

rpc_ initial_weight

initial value for the regularization weight (-ve => $1/\|g_0\|)$)

rpc_ minimum_weight

minimum permitted regularization weight

rpc_ initial_inner_weight

initial value for the inner regularization weight for tensor GN (-ve => 0)

rpc_ eta_successful

rpc_ eta_very_successful

see eta_successful

rpc_ eta_too_successful

see eta_successful

rpc_ weight_decrease_min

rpc_ weight_decrease

see weight_decrease_min

rpc_ weight_increase

see weight_decrease_min

rpc_ weight_increase_max

see weight_decrease_min

rpc_ reduce_gap

expert parameters as suggested in Gould, Porcelli and Toint, “Updating the regularization parameter in the adaptive cubic: regularization algorithm”, RAL-TR-2011-007, Rutherford Appleton Laboratory, England (2011), http://epubs.stfc.ac.uk/bitstream/6181/RAL-TR-2011-007.pdf (these are denoted beta, epsilon_chi and alpha_max in the paper)

rpc_ tiny_gap

see reduce_gap

rpc_ large_root

see reduce_gap

rpc_ switch_to_newton

if the Gauss-Newto to Newton model is specified, switch to Newton as soon as the norm of the gradient g is smaller than switch_to_newton

rpc_ cpu_time_limit

the maximum CPU time allowed (-ve means infinite)

rpc_ clock_time_limit

the maximum elapsed clock time allowed (-ve means infinite)

bool subproblem_direct

use a direct (factorization) or (preconditioned) iterative method to find the search direction

bool renormalize_weight

should the weight be renormalized to account for a change in scaling?

bool magic_step

allow the user to perform a “magic” step to improve the objective

bool print_obj

print values of the objective/gradient rather than ||c|| and its gradient

bool space_critical

if .space_critical true, every effort will be made to use as little space as possible. This may result in longer computation time

bool deallocate_error_fatal

if .deallocate_error_fatal is true, any array/pointer deallocation error will terminate execution. Otherwise, computation will continue

char prefix[31]

all output lines will be prefixed by .prefix(2:LEN(TRIM(.prefix))-1) where .prefix contains the required string enclosed in quotes, e.g. “string” or ‘string’

struct rqs_control_type rqs_control

control parameters for RQS

struct glrt_control_type glrt_control

control parameters for GLRT

struct psls_control_type psls_control

control parameters for PSLS

struct bsc_control_type bsc_control

control parameters for BSC

struct roots_control_type roots_control

control parameters for ROOTS

struct nls_subproblem_control_type subproblem_control

control parameters for the step-finding subproblem

nls_time_type structure#

#include <galahad_nls.h>

struct nls_time_type {
    // components

    spc_ total;
    spc_ preprocess;
    spc_ analyse;
    spc_ factorize;
    spc_ solve;
    rpc_ clock_total;
    rpc_ clock_preprocess;
    rpc_ clock_analyse;
    rpc_ clock_factorize;
    rpc_ clock_solve;
};

detailed documentation#

time derived type as a C struct

components#

spc_ total

the total CPU time spent in the package

spc_ preprocess

the CPU time spent preprocessing the problem

spc_ analyse

the CPU time spent analysing the required matrices prior to factorization

spc_ factorize

the CPU time spent factorizing the required matrices

spc_ solve

the CPU time spent computing the search direction

rpc_ clock_total

the total clock time spent in the package

rpc_ clock_preprocess

the clock time spent preprocessing the problem

rpc_ clock_analyse

the clock time spent analysing the required matrices prior to factorization

rpc_ clock_factorize

the clock time spent factorizing the required matrices

rpc_ clock_solve

the clock time spent computing the search direction

nls_subproblem_inform_type structure#

#include <galahad_nls.h>

struct nls_subproblem_inform_type {
    // components

    ipc_ status;
    ipc_ alloc_status;
    char bad_alloc[81];
    char bad_eval[13];
    ipc_ iter;
    ipc_ cg_iter;
    ipc_ c_eval;
    ipc_ j_eval;
    ipc_ h_eval;
    ipc_ factorization_max;
    ipc_ factorization_status;
    int64_t max_entries_factors;
    int64_t factorization_integer;
    int64_t factorization_real;
    rpc_ factorization_average;
    rpc_ obj;
    rpc_ norm_c;
    rpc_ norm_g;
    rpc_ weight;
    struct nls_time_type time;
    struct rqs_inform_type rqs_inform;
    struct glrt_inform_type glrt_inform;
    struct psls_inform_type psls_inform;
    struct bsc_inform_type bsc_inform;
    struct roots_inform_type roots_inform;
};

detailed documentation#

subproblem_inform derived type as a C struct

components#

ipc_ status

return status. See NLS_solve for details

ipc_ alloc_status

the status of the last attempted allocation/deallocation

char bad_alloc[81]

the name of the array for which an allocation/deallocation error occurred

char bad_eval[13]

the name of the user-supplied evaluation routine for which an error occurred

ipc_ iter

the total number of iterations performed

ipc_ cg_iter

the total number of CG iterations performed

ipc_ c_eval

the total number of evaluations of the residual function c(x)

ipc_ j_eval

the total number of evaluations of the Jacobian J(x) of c(x)

ipc_ h_eval

the total number of evaluations of the scaled Hessian H(x,y) of c(x)

ipc_ factorization_max

the maximum number of factorizations in a sub-problem solve

ipc_ factorization_status

the return status from the factorization

int64_t max_entries_factors

the maximum number of entries in the factors

int64_t factorization_integer

the total integer workspace required for the factorization

int64_t factorization_real

the total real workspace required for the factorization

rpc_ factorization_average

the average number of factorizations per sub-problem solve

rpc_ obj

the value of the objective function $\frac{1}{2}\|c(x)\|^2_W$ at the best estimate the solution, x, determined by NLS_solve

rpc_ norm_c

the norm of the residual $\|c(x)\|_W$ at the best estimate of the solution x, determined by NLS_solve

rpc_ norm_g

the norm of the gradient of $\|c(x)\|_W$ of the objective function at the best estimate, x, of the solution determined by NLS_solve

rpc_ weight

the final regularization weight used

struct nls_time_type time

timings (see above)

struct rqs_inform_type rqs_inform

inform parameters for RQS

struct glrt_inform_type glrt_inform

inform parameters for GLRT

struct psls_inform_type psls_inform

inform parameters for PSLS

struct bsc_inform_type bsc_inform

inform parameters for BSC

struct roots_inform_type roots_inform

inform parameters for ROOTS

nls_inform_type structure#

#include <galahad_nls.h>

struct nls_inform_type {
    // components

    ipc_ status;
    ipc_ alloc_status;
    char bad_alloc[81];
    char bad_eval[13];
    ipc_ iter;
    ipc_ cg_iter;
    ipc_ c_eval;
    ipc_ j_eval;
    ipc_ h_eval;
    ipc_ factorization_max;
    ipc_ factorization_status;
    int64_t max_entries_factors;
    int64_t factorization_integer;
    int64_t factorization_real;
    rpc_ factorization_average;
    rpc_ obj;
    rpc_ norm_c;
    rpc_ norm_g;
    rpc_ weight;
    struct nls_time_type time;
    struct rqs_inform_type rqs_inform;
    struct glrt_inform_type glrt_inform;
    struct psls_inform_type psls_inform;
    struct bsc_inform_type bsc_inform;
    struct roots_inform_type roots_inform;
    struct nls_subproblem_inform_type subproblem_inform;
};

detailed documentation#

inform derived type as a C struct

components#

ipc_ status

return status. See NLS_solve for details

ipc_ alloc_status

the status of the last attempted allocation/deallocation

char bad_alloc[81]

the name of the array for which an allocation/deallocation error occurred

char bad_eval[13]

the name of the user-supplied evaluation routine for which an error occurred

ipc_ iter

the total number of iterations performed

ipc_ cg_iter

the total number of CG iterations performed

ipc_ c_eval

the total number of evaluations of the residual function c(x)

ipc_ j_eval

the total number of evaluations of the Jacobian J(x) of c(x)

ipc_ h_eval

the total number of evaluations of the scaled Hessian H(x,y) of c(x)

ipc_ factorization_max

the maximum number of factorizations in a sub-problem solve

ipc_ factorization_status

the return status from the factorization

int64_t max_entries_factors

the maximum number of entries in the factors

int64_t factorization_integer

the total integer workspace required for the factorization

int64_t factorization_real

the total real workspace required for the factorization

rpc_ factorization_average

the average number of factorizations per sub-problem solve

rpc_ obj

the value of the objective function $\frac{1}{2}\|c(x)\|^2_W$ at the best estimate the solution, x, determined by NLS_solve

rpc_ norm_c

the norm of the residual $\|c(x)\|_W$ at the best estimate of the solution x, determined by NLS_solve

rpc_ norm_g

the norm of the gradient of $\|c(x)\|_W$ of the objective function at the best estimate, x, of the solution determined by NLS_solve

rpc_ weight

the final regularization weight used

struct nls_time_type time

timings (see above)

struct rqs_inform_type rqs_inform

inform parameters for RQS

struct glrt_inform_type glrt_inform

inform parameters for GLRT

struct psls_inform_type psls_inform

inform parameters for PSLS

struct bsc_inform_type bsc_inform

inform parameters for BSC

struct roots_inform_type roots_inform

inform parameters for ROOTS

struct nls_subproblem_inform_type subproblem_inform

inform parameters for subproblem

Index

example calls#

This is an example of how to use the package to solve a nonlinear least-squares problem; the code is available in $GALAHAD/src/nls/C/nlst.c . A variety of supported Hessian and constraint matrix storage formats are shown.

Notice that C-style indexing is used, and that this is flagged by setting control.f_indexing to false. The floating-point type rpc_ is set in galahad_precision.h to double by default, but to float if the preprocessor variable SINGLE is defined. Similarly, the integer type ipc_ from galahad_precision.h is set to int by default, but to int64_t if the preprocessor variable INTEGER_64 is defined.

/* nlst.c */
/* Full test for the NLS C interface using C sparse matrix indexing */
/* Jari Fowkes & Nick Gould, STFC-Rutherford Appleton Laboratory, 2021 */

#include <stdio.h>
#include <math.h>
#include "galahad_precision.h"
#include "galahad_cfunctions.h"
#include "galahad_nls.h"

// Define imax
ipc_ imax(ipc_ a, ipc_ b) {
    return (a > b) ? a : b;
};

// Custom userdata struct
struct userdata_type {
   rpc_ p;
};

// Function prototypes

ipc_ res( ipc_ n, ipc_ m, const rpc_ x[], rpc_ c[], const void * );
ipc_ jac( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[], const void * );
ipc_ hess( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
          rpc_ hval[], const void * );
ipc_ jacprod( ipc_ n, ipc_ m, const rpc_ x[], const bool transpose, rpc_ u[],
             const rpc_ v[], bool got_j, const void * );
ipc_ hessprod( ipc_ n, ipc_ m, const rpc_ x[], const rpc_ y[], rpc_ u[],
              const rpc_ v[], bool got_h, const void * );
ipc_ rhessprods( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[], const rpc_ v[],
                rpc_ pval[], bool got_h, const void * );
ipc_ scale( ipc_ n, ipc_ m, const rpc_ x[], rpc_ u[],
           const rpc_ v[], const void * );
ipc_ jac_dense( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
               const void * );
ipc_ hess_dense( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
                rpc_ hval[], const void * );
ipc_ rhessprods_dense( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[],
                      const rpc_ v[], rpc_ pval[], bool got_h,
                      const void * );

int main(void) {

    // Derived types
    void *data;
    struct nls_control_type control;
    struct nls_inform_type inform;

    // Set user data
    struct userdata_type userdata;
    userdata.p = 1.0;

    // Set problem data
    ipc_ n = 2; // # variables
    ipc_ m = 3; // # residuals
    ipc_ j_ne = 5; // Jacobian elements
    ipc_ h_ne = 2; // Hesssian elements
    ipc_ p_ne = 2; // residual-Hessians-vector products elements
    ipc_ J_row[] = {0, 1, 1, 2, 2}; // Jacobian J
    ipc_ J_col[] = {0, 0, 1, 0, 1}; //
    ipc_ J_ptr[] = {0, 1, 3, 5};    // row pointers
    ipc_ H_row[] = {0, 1};          // Hessian H
    ipc_ H_col[] = {0, 1};          // NB lower triangle
    ipc_ H_ptr[] = {0, 1, 2};       // row pointers
    ipc_ P_row[] = {0, 1};          // residual-Hessians-vector product matrix
    ipc_ P_ptr[] = {0, 1, 2, 2};    // column pointers

    // Set storage
    rpc_ g[n]; // gradient
    rpc_ c[m]; // residual
    rpc_ y[m]; // multipliers
    char st = ' ';
    ipc_ status;

    printf(" C sparse matrix indexing\n\n");

    printf(" tests options for all-in-one storage format\n\n");

    for( ipc_ d=1; d <= 5; d++){
//  for( ipc_ d=5; d <= 5; d++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = 6;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        switch(d){
            case 1: // sparse co-ordinate storage
                st = 'C';
                nls_import( &control, &data, &status, n, m,
                            "coordinate", j_ne, J_row, J_col, NULL,
                            "coordinate", h_ne, H_row, H_col, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac,
                                    h_ne, hess, p_ne, rhessprods );
                break;
            case 2: // sparse by rows
                st = 'R';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac,
                                    h_ne, hess, p_ne, rhessprods );
                break;
            case 3: // dense
                st = 'D';
                nls_import( &control, &data, &status, n, m,
                            "dense", j_ne, NULL, NULL, NULL,
                            "dense", h_ne, NULL, NULL, NULL,
                            "dense", p_ne, NULL, NULL, NULL, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac_dense,
                                    h_ne, hess_dense, p_ne, rhessprods_dense );
                break;
            case 4: // diagonal
                st = 'I';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "diagonal", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac,
                                    h_ne, hess, p_ne, rhessprods );
                break;
            case 5: // access by products
                st = 'P';
                nls_import( &control, &data, &status, n, m,
                            "absent", j_ne, NULL, NULL, NULL,
                            "absent", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_without_mat( &data, &userdata, &status,
                                       n, m, x, c, g, res, jacprod,
                                       hessprod, p_ne, rhessprods );
                break;
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("%c:%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   st, inform.iter, inform.obj, inform.status);
        }else{
            printf("%c: NLS_solve exit status = %1" i_ipc_ "\n", st, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n tests reverse-communication options\n\n");

    // reverse-communication input/output
    ipc_ eval_status;
    rpc_ u[imax(m,n)], v[imax(m,n)];
    rpc_ J_val[j_ne], J_dense[m*n];
    rpc_ H_val[h_ne], H_dense[n*(n+1)/2], H_diag[n];
    rpc_ P_val[p_ne], P_dense[m*n];
    bool transpose;
    bool got_j = false;
    bool got_h = false;

    for( ipc_ d=1; d <= 5; d++){
//  for( ipc_ d=1; d <= 4; d++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = 6;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        switch(d){
            case 1: // sparse co-ordinate storage
                st = 'C';
                nls_import( &control, &data, &status, n, m,
                            "coordinate", j_ne, J_row, J_col, NULL,
                            "coordinate", h_ne, H_row, H_col, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, j_ne, J_val, y,
                                              h_ne, H_val, v, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac( n, m, j_ne, x, J_val, &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess( n, m, h_ne, x, y, H_val, &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 2: // sparse by rows
                st = 'R';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, j_ne, J_val, y,
                                              h_ne, H_val, v, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac( n, m, j_ne, x, J_val, &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess( n, m, h_ne, x, y, H_val, &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 3: // dense
                st = 'D';
                nls_import( &control, &data, &status, n, m,
                            "dense", j_ne, NULL, NULL, NULL,
                            "dense", h_ne, NULL, NULL, NULL,
                            "dense", p_ne, NULL, NULL, NULL, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, m*n, J_dense, y,
                                              n*(n+1)/2, H_dense, v, m*n,
                                              P_dense );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac_dense( n, m, j_ne, x, J_dense,
                                               &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess_dense( n, m, h_ne, x, y, H_dense,
                                                &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods_dense( n, m, p_ne, x, v, P_dense,
                                                      got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 4: // diagonal
                st = 'I';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "diagonal", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, j_ne, J_val, y,
                                              n, H_diag, v, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac( n, m, j_ne, x, J_val, &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess( n, m, h_ne, x, y, H_diag, &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 5: // access by products
                st = 'P';
//              control.print_level = 1;
                nls_import( &control, &data, &status, n, m,
                            "absent", j_ne, NULL, NULL, NULL,
                            "absent", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_without_mat( &data, &status, &eval_status,
                                                 n, m, x, c, g, &transpose,
                                                 u, v, y, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 5){ // evaluate u + J v or u + J'v
                      eval_status = jacprod( n, m, x, transpose, u, v, got_j,
                                             &userdata );
                  }else if(status == 6){ // evaluate u + H v
                      eval_status = hessprod( n, m, x, y, u, v, got_h,
                                              &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("%c:%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   st, inform.iter, inform.obj, inform.status);
        }else{
            printf("%c: NLS_solve exit status = %1" i_ipc_ "\n", st, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, direct access\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                    "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        nls_solve_with_mat( &data, &userdata, &status,
                            n, m, x, c, g, res, j_ne, jac,
                            h_ne, hess, p_ne, rhessprods );

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf(" %1" i_ipc_ ":%6" i_ipc_
                   " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf(" %" i_ipc_ ": NLS_solve exit status = %1" i_ipc_ "\n",
                   model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, access by products\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "absent", j_ne, NULL, NULL, NULL,
                    "absent", h_ne, NULL, NULL, NULL,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        nls_solve_without_mat( &data, &userdata, &status,
                               n, m, x, c, g, res, jacprod,
                               hessprod, p_ne, rhessprods );
        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("P%1" i_ipc_ ":%6" i_ipc_
                   " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf("P%" i_ipc_ ": NLS_solve exit status = %1" i_ipc_
                   "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, reverse access\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                    "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        while(true){ // reverse-communication loop
          nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                      n, m, x, c, g, j_ne, J_val, y,
                                      h_ne, H_val, v, p_ne, P_val );
          if(status == 0){ // successful termination
                break;
          }else if(status < 0){ // error exit
              break;
          }else if(status == 2){ // evaluate c
              eval_status = res( n, m, x, c, &userdata );
          }else if(status == 3){ // evaluate J
              eval_status = jac( n, m, j_ne, x, J_val, &userdata );
          }else if(status == 4){ // evaluate H
              eval_status = hess( n, m, h_ne, x, y, H_val, &userdata );
          }else if(status == 7){ // evaluate P
              eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                        got_h, &userdata );
          }else{
              printf(" the value %1" i_ipc_ " of status should not occur\n",
                status);
              break;
          }
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("P%1" i_ipc_ ":%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf(" %" i_ipc_ ": NLS_solve exit status = %1" i_ipc_ "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, reverse access by products\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = false; // C sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "absent", j_ne, NULL, NULL, NULL,
                    "absent", h_ne, NULL, NULL, NULL,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        while(true){ // reverse-communication loop
          nls_solve_reverse_without_mat( &data, &status, &eval_status,
                                         n, m, x, c, g, &transpose,
                                         u, v, y, p_ne, P_val );
          if(status == 0){ // successful termination
                break;
          }else if(status < 0){ // error exit
              break;
          }else if(status == 2){ // evaluate c
              eval_status = res( n, m, x, c, &userdata );
          }else if(status == 5){ // evaluate u + J v or u + J'v
              eval_status = jacprod( n, m, x, transpose, u, v, got_j,
                                     &userdata );
          }else if(status == 6){ // evaluate u + H v
              eval_status = hessprod( n, m, x, y, u, v, got_h,
                                      &userdata );
          }else if(status == 7){ // evaluate P
              eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                        got_h, &userdata );
          }else{
              printf(" the value %1" i_ipc_ " of status should not occur\n",
                status);
              break;
          }
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("P%1" i_ipc_ ":%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf("P%" i_ipc_ ": NLS_solve exit status = %1" i_ipc_ "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }
}

// compute the residuals
ipc_ res( ipc_ n, ipc_ m, const rpc_ x[], rpc_ c[], const void *userdata ){
    struct userdata_type *myuserdata = ( struct userdata_type * ) userdata;
    rpc_ p = myuserdata->p;
    c[0] = pow(x[0],2.0) + p;
    c[1] = x[0] + pow(x[1],2.0);
    c[2] = x[0] - x[1];
    return 0;
}

// compute the Jacobian
ipc_ jac( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
         const void *userdata ){
    jval[0] = 2.0 * x[0];
    jval[1] = 1.0;
    jval[2] = 2.0 * x[1];
    jval[3] = 1.0;
    jval[4] = - 1.0;
    return 0;
}

// compute the Hessian
ipc_ hess( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
           rpc_ hval[], const void *userdata ){
    hval[0] = 2.0 * y[0];
    hval[1] = 2.0 * y[1];
    return 0;
}

// compute Jacobian-vector products
ipc_ jacprod( ipc_ n, ipc_ m, const rpc_ x[], const bool transpose, rpc_ u[],
             const rpc_ v[], bool got_j, const void *userdata ){
    if (transpose) {
      u[0] = u[0] + 2.0 * x[0] * v[0] + v[1] + v[2];
      u[1] = u[1] + 2.0 * x[1] * v[1] - v[2];
    }else{
      u[0] = u[0] + 2.0 * x[0] * v[0];
      u[1] = u[1] + v[0]  + 2.0 * x[1] * v[1];
      u[2] = u[2] + v[0] - v[1];
    }
    return 0;
}

// compute Hessian-vector products
ipc_ hessprod( ipc_ n, ipc_ m, const rpc_ x[], const rpc_ y[], rpc_ u[],
              const rpc_ v[], bool got_h, const void *userdata ){
    u[0] = u[0] + 2.0 * y[0] * v[0];
    u[1] = u[1] + 2.0 * y[1] * v[1];
    return 0;
}

// compute residual-Hessians-vector products
ipc_ rhessprods( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[], const rpc_ v[],
                rpc_ pval[], bool got_h, const void *userdata ){
    pval[0] = 2.0 * v[0];
    pval[1] = 2.0 * v[1];
    return 0;
}

// scale v
ipc_ scale( ipc_ n, ipc_ m, const rpc_ x[], rpc_ u[],
           const rpc_ v[], const void *userdata ){
    u[0] = v[0];
    u[1] = v[1];
    return 0;
}

// compute the dense Jacobian
ipc_ jac_dense( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
               const void *userdata ){
    jval[0] = 2.0 * x[0];
    jval[1] = 0.0;
    jval[2] = 1.0;
    jval[3] = 2.0 * x[1];
    jval[4] = 1.0;
    jval[5] = - 1.0;
    return 0;
}

// compute the dense Hessian
ipc_ hess_dense( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
                rpc_ hval[], const void *userdata ){
    hval[0] = 2.0 * y[0];
    hval[1] = 0.0;
    hval[2] = 2.0 * y[1];
    return 0;
}

// compute dense residual-Hessians-vector products
ipc_ rhessprods_dense( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[],
                      const rpc_ v[], rpc_ pval[], bool got_h,
                      const void *userdata ){
    pval[0] = 2.0 * v[0];
    pval[1] = 0.0;
    pval[2] = 0.0;
    pval[3] = 2.0 * v[1];
    pval[4] = 0.0;
    pval[5] = 0.0;
    return 0;
}

This is the same example, but now fortran-style indexing is used; the code is available in $GALAHAD/src/nls/C/nlstf.c .

/* nlstf.c */
/* Full test for the NLS interface using Fortran sparse matrix indexing */
/* Jari Fowkes & Nick Gould, STFC-Rutherford Appleton Laboratory, 2021 */

#include <stdio.h>
#include <math.h>
#include "galahad_precision.h"
#include "galahad_cfunctions.h"
#include "galahad_nls.h"

// Define imax
ipc_ imax(ipc_ a, ipc_ b) {
    return (a > b) ? a : b;
};

// Custom userdata struct
struct userdata_type {
   rpc_ p;
};

// Function prototypes

ipc_ res( ipc_ n, ipc_ m, const rpc_ x[], rpc_ c[], const void * );
ipc_ jac( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
         const void * );
ipc_ hess( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
          rpc_ hval[], const void * );
ipc_ jacprod( ipc_ n, ipc_ m, const rpc_ x[], const bool transpose,
             rpc_ u[], const rpc_ v[], bool got_j, const void * );
ipc_ hessprod( ipc_ n, ipc_ m, const rpc_ x[], const rpc_ y[],
              rpc_ u[], const rpc_ v[], bool got_h, const void * );
ipc_ rhessprods( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[], const rpc_ v[],
                rpc_ pval[], bool got_h, const void * );
ipc_ scale( ipc_ n, ipc_ m, const rpc_ x[], rpc_ u[],
           const rpc_ v[], const void * );
ipc_ jac_dense( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
               const void * );
ipc_ hess_dense( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
                rpc_ hval[], const void * );
ipc_ rhessprods_dense( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[],
                      const rpc_ v[], rpc_ pval[], bool got_h,
                      const void * );

int main(void) {

    // Derived types
    void *data;
    struct nls_control_type control;
    struct nls_inform_type inform;

    // Set user data
    struct userdata_type userdata;
    userdata.p = 1.0;

    // Set problem data
    ipc_ n = 2; // # variables
    ipc_ m = 3; // # residuals
    ipc_ j_ne = 5; // Jacobian elements
    ipc_ h_ne = 2; // Hesssian elements
    ipc_ p_ne = 2; // residual-Hessians-vector products elements
    ipc_ J_row[] = {1, 2, 2, 3, 3}; // Jacobian J
    ipc_ J_col[] = {1, 1, 2, 1, 2}; //
    ipc_ J_ptr[] = {1, 2, 4, 6};    // row pointers
    ipc_ H_row[] = {1, 2};          // Hessian H
    ipc_ H_col[] = {1, 2};          // NB lower triangle
    ipc_ H_ptr[] = {1, 2, 3};       // row pointers
    ipc_ P_row[] = {1, 2};          // residual-Hessians-vector product matrix
    ipc_ P_ptr[] = {1, 2, 3, 3};    // column pointers

    // Set storage
    rpc_ g[n]; // gradient
    rpc_ c[m]; // residual
    rpc_ y[m]; // multipliers
    char st = ' ';
    ipc_ status;

    printf(" Fortran sparse matrix indexing\n\n");

    printf(" tests options for all-in-one storage format\n\n");

    for( ipc_ d=1; d <= 5; d++){
//  for( ipc_ d=5; d <= 5; d++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing
        // control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = 6;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        switch(d){
            case 1: // sparse co-ordinate storage
                st = 'C';
                nls_import( &control, &data, &status, n, m,
                            "coordinate", j_ne, J_row, J_col, NULL,
                            "coordinate", h_ne, H_row, H_col, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac,
                                    h_ne, hess, p_ne, rhessprods );
                break;
            case 2: // sparse by rows
                st = 'R';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac,
                                    h_ne, hess, p_ne, rhessprods );
                break;
            case 3: // dense
                st = 'D';
                nls_import( &control, &data, &status, n, m,
                            "dense", j_ne, NULL, NULL, NULL,
                            "dense", h_ne, NULL, NULL, NULL,
                            "dense", p_ne, NULL, NULL, NULL, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac_dense,
                                    h_ne, hess_dense, p_ne, rhessprods_dense );
                break;
            case 4: // diagonal
                st = 'I';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "diagonal", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_with_mat( &data, &userdata, &status,
                                    n, m, x, c, g, res, j_ne, jac,
                                    h_ne, hess, p_ne, rhessprods );
                break;
            case 5: // access by products
                st = 'P';
                nls_import( &control, &data, &status, n, m,
                            "absent", j_ne, NULL, NULL, NULL,
                            "absent", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                nls_solve_without_mat( &data, &userdata, &status,
                                       n, m, x, c, g, res, jacprod,
                                       hessprod, p_ne, rhessprods );
                break;
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("%c:%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   st, inform.iter, inform.obj, inform.status);
        }else{
            printf("%c: NLS_solve exit status = %1" i_ipc_ "\n", st, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n tests reverse-communication options\n\n");

    // reverse-communication input/output
    ipc_ eval_status;
    rpc_ u[imax(m,n)], v[imax(m,n)];
    rpc_ J_val[j_ne], J_dense[m*n];
    rpc_ H_val[h_ne], H_dense[n*(n+1)/2], H_diag[n];
    rpc_ P_val[p_ne], P_dense[m*n];
    bool transpose;
    bool got_j = false;
    bool got_h = false;

    for( ipc_ d=1; d <= 5; d++){
//  for( ipc_ d=1; d <= 4; d++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = 6;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        switch(d){
            case 1: // sparse co-ordinate storage
                st = 'C';
                nls_import( &control, &data, &status, n, m,
                            "coordinate", j_ne, J_row, J_col, NULL,
                            "coordinate", h_ne, H_row, H_col, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, j_ne, J_val, y,
                                              h_ne, H_val, v, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac( n, m, j_ne, x, J_val, &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess( n, m, h_ne, x, y, H_val, &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 2: // sparse by rows
                st = 'R';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, j_ne, J_val, y,
                                              h_ne, H_val, v, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac( n, m, j_ne, x, J_val, &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess( n, m, h_ne, x, y, H_val, &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 3: // dense
                st = 'D';
                nls_import( &control, &data, &status, n, m,
                            "dense", j_ne, NULL, NULL, NULL,
                            "dense", h_ne, NULL, NULL, NULL,
                            "dense", p_ne, NULL, NULL, NULL, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, m*n, J_dense, y,
                                              n*(n+1)/2, H_dense, v, m*n,
                                              P_dense );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac_dense( n, m, j_ne, x, J_dense,
                                               &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess_dense( n, m, h_ne, x, y, H_dense,
                                                &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods_dense( n, m, p_ne, x, v, P_dense,
                                                      got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 4: // diagonal
                st = 'I';
                nls_import( &control, &data, &status, n, m,
                            "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                            "diagonal", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                              n, m, x, c, g, j_ne, J_val, y,
                                              n, H_diag, v, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 3){ // evaluate J
                      eval_status = jac( n, m, j_ne, x, J_val, &userdata );
                  }else if(status == 4){ // evaluate H
                      eval_status = hess( n, m, h_ne, x, y, H_diag, &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
            case 5: // access by products
                st = 'P';
//              control.print_level = 1;
                nls_import( &control, &data, &status, n, m,
                            "absent", j_ne, NULL, NULL, NULL,
                            "absent", h_ne, NULL, NULL, NULL,
                            "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
                while(true){ // reverse-communication loop
                  nls_solve_reverse_without_mat( &data, &status, &eval_status,
                                                 n, m, x, c, g, &transpose,
                                                 u, v, y, p_ne, P_val );
                  if(status == 0){ // successful termination
                        break;
                  }else if(status < 0){ // error exit
                      break;
                  }else if(status == 2){ // evaluate c
                      eval_status = res( n, m, x, c, &userdata );
                  }else if(status == 5){ // evaluate u + J v or u + J'v
                      eval_status = jacprod( n, m, x, transpose, u, v, got_j,
                                             &userdata );
                  }else if(status == 6){ // evaluate u + H v
                      eval_status = hessprod( n, m, x, y, u, v, got_h,
                                              &userdata );
                  }else if(status == 7){ // evaluate P
                      eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                                got_h, &userdata );
                  }else{
                      printf(" the value %1" i_ipc_ " of status should not occur\n",
                        status);
                      break;
                  }
                }
                break;
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("%c:%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   st, inform.iter, inform.obj, inform.status);
        }else{
            printf("%c: NLS_solve exit status = %1" i_ipc_ "\n", st, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, direct access\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                    "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        nls_solve_with_mat( &data, &userdata, &status,
                            n, m, x, c, g, res, j_ne, jac,
                            h_ne, hess, p_ne, rhessprods );

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf(" %1" i_ipc_ ":%6" i_ipc_
                   " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf(" %" i_ipc_ ": NLS_solve exit status = %1" i_ipc_
                   "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, access by products\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "absent", j_ne, NULL, NULL, NULL,
                    "absent", h_ne, NULL, NULL, NULL,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        nls_solve_without_mat( &data, &userdata, &status,
                               n, m, x, c, g, res, jacprod,
                               hessprod, p_ne, rhessprods );
        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("P%1" i_ipc_ ":%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf("P%" i_ipc_ ": NLS_solve exit status = %1" i_ipc_ "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, reverse access\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "sparse_by_rows", j_ne, NULL, J_col, J_ptr,
                    "sparse_by_rows", h_ne, NULL, H_col, H_ptr,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        while(true){ // reverse-communication loop
          nls_solve_reverse_with_mat( &data, &status, &eval_status,
                                      n, m, x, c, g, j_ne, J_val, y,
                                      h_ne, H_val, v, p_ne, P_val );
          if(status == 0){ // successful termination
                break;
          }else if(status < 0){ // error exit
              break;
          }else if(status == 2){ // evaluate c
              eval_status = res( n, m, x, c, &userdata );
          }else if(status == 3){ // evaluate J
              eval_status = jac( n, m, j_ne, x, J_val, &userdata );
          }else if(status == 4){ // evaluate H
              eval_status = hess( n, m, h_ne, x, y, H_val, &userdata );
          }else if(status == 7){ // evaluate P
              eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                        got_h, &userdata );
          }else{
              printf(" the value %1" i_ipc_ " of status should not occur\n",
                status);
              break;
          }
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("P%1" i_ipc_ ":%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf(" %" i_ipc_ ": NLS_solve exit status = %1" i_ipc_ "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }

    printf("\n basic tests of models used, reverse access by products\n\n");

    for( ipc_ model=3; model <= 8; model++){

        // Initialize NLS
        nls_initialize( &data, &control, &inform );

        // Set user-defined control options
        control.f_indexing = true; // Fortran sparse matrix indexing
        //control.print_level = 1;
        control.jacobian_available = 2;
        control.hessian_available = 2;
        control.model = model;
        rpc_ x[] = {1.5,1.5}; // starting point
        rpc_ W[] = {1.0, 1.0, 1.0}; // weights

        nls_import( &control, &data, &status, n, m,
                    "absent", j_ne, NULL, NULL, NULL,
                    "absent", h_ne, NULL, NULL, NULL,
                    "sparse_by_columns", p_ne, P_row, NULL, P_ptr, W );
        while(true){ // reverse-communication loop
          nls_solve_reverse_without_mat( &data, &status, &eval_status,
                                         n, m, x, c, g, &transpose,
                                         u, v, y, p_ne, P_val );
          if(status == 0){ // successful termination
                break;
          }else if(status < 0){ // error exit
              break;
          }else if(status == 2){ // evaluate c
              eval_status = res( n, m, x, c, &userdata );
          }else if(status == 5){ // evaluate u + J v or u + J'v
              eval_status = jacprod( n, m, x, transpose, u, v, got_j,
                                     &userdata );
          }else if(status == 6){ // evaluate u + H v
              eval_status = hessprod( n, m, x, y, u, v, got_h,
                                      &userdata );
          }else if(status == 7){ // evaluate P
              eval_status = rhessprods( n, m, p_ne, x, v, P_val,
                                        got_h, &userdata );
          }else{
              printf(" the value %1" i_ipc_ " of status should not occur\n",
                status);
              break;
          }
        }

        nls_information( &data, &inform, &status );

        if(inform.status == 0){
            printf("P%1" i_ipc_ ":%6" i_ipc_ " iterations. Optimal objective value = %5.2f"
                   " status = %1" i_ipc_ "\n",
                   model, inform.iter, inform.obj, inform.status);
        }else{
            printf("P%" i_ipc_ ": NLS_solve exit status = %1" i_ipc_ "\n", model, inform.status);
        }
        // Delete internal workspace
        nls_terminate( &data, &control, &inform );
    }
}

// compute the residuals
ipc_ res( ipc_ n, ipc_ m, const rpc_ x[], rpc_ c[], const void *userdata ){
    struct userdata_type *myuserdata = ( struct userdata_type * ) userdata;
    rpc_ p = myuserdata->p;
    c[0] = pow(x[0],2.0) + p;
    c[1] = x[0] + pow(x[1],2.0);
    c[2] = x[0] - x[1];
    return 0;
}

// compute the Jacobian
ipc_ jac( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
         const void *userdata ){
    jval[0] = 2.0 * x[0];
    jval[1] = 1.0;
    jval[2] = 2.0 * x[1];
    jval[3] = 1.0;
    jval[4] = - 1.0;
    return 0;
}

// compute the Hessian
ipc_ hess( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
           rpc_ hval[], const void *userdata ){
    hval[0] = 2.0 * y[0];
    hval[1] = 2.0 * y[1];
    return 0;
}

// compute Jacobian-vector products
ipc_ jacprod( ipc_ n, ipc_ m, const rpc_ x[], const bool transpose, rpc_ u[],
             const rpc_ v[], bool got_j, const void *userdata ){
    if (transpose) {
      u[0] = u[0] + 2.0 * x[0] * v[0] + v[1] + v[2];
      u[1] = u[1] + 2.0 * x[1] * v[1] - v[2];
    }else{
      u[0] = u[0] + 2.0 * x[0] * v[0];
      u[1] = u[1] + v[0]  + 2.0 * x[1] * v[1];
      u[2] = u[2] + v[0] - v[1];
    }
    return 0;
}

// compute Hessian-vector products
ipc_ hessprod( ipc_ n, ipc_ m, const rpc_ x[], const rpc_ y[], rpc_ u[],
              const rpc_ v[], bool got_h, const void *userdata ){
    u[0] = u[0] + 2.0 * y[0] * v[0];
    u[1] = u[1] + 2.0 * y[1] * v[1];
    return 0;
}

// compute residual-Hessians-vector products
ipc_ rhessprods( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[], const rpc_ v[],
                rpc_ pval[], bool got_h, const void *userdata ){
    pval[0] = 2.0 * v[0];
    pval[1] = 2.0 * v[1];
    return 0;
}

// scale v
ipc_ scale( ipc_ n, ipc_ m, const rpc_ x[], rpc_ u[],
           const rpc_ v[], const void *userdata ){
    u[0] = v[0];
    u[1] = v[1];
    return 0;
}

// compute the dense Jacobian
ipc_ jac_dense( ipc_ n, ipc_ m, ipc_ jne, const rpc_ x[], rpc_ jval[],
               const void *userdata ){
    jval[0] = 2.0 * x[0];
    jval[1] = 0.0;
    jval[2] = 1.0;
    jval[3] = 2.0 * x[1];
    jval[4] = 1.0;
    jval[5] = - 1.0;
    return 0;
}

// compute the dense Hessian
ipc_ hess_dense( ipc_ n, ipc_ m, ipc_ hne, const rpc_ x[], const rpc_ y[],
                rpc_ hval[], const void *userdata ){
    hval[0] = 2.0 * y[0];
    hval[1] = 0.0;
    hval[2] = 2.0 * y[1];
    return 0;
}

// compute dense residual-Hessians-vector products
ipc_ rhessprods_dense( ipc_ n, ipc_ m, ipc_ pne, const rpc_ x[],
                      const rpc_ v[], rpc_ pval[], bool got_h,
                      const void *userdata ){
    pval[0] = 2.0 * v[0];
    pval[1] = 0.0;
    pval[2] = 0.0;
    pval[3] = 2.0 * v[1];
    pval[4] = 0.0;
    pval[5] = 0.0;
    return 0;
}