Anymatrix: an extensible MATLAB matrix collection

Anymatrix is a MATLAB toolbox that provides an extensible collection of matrices with the ability to search the collection by matrix properties. Each matrix is implemented as a MATLAB function and the matrices are arranged in groups. Compared with previous collections, Anymatrix offers three novel features. First, it allows a user to share a collection of matrices by putting them in a group, annotating them with properties, and placing the group on a public repository, for example on GitHub; the group can then be incorporated into another user’s local Anymatrix installation. Second, it provides a tool to search for matrices by their properties, with Boolean expressions supported. Third, it provides organization into sets, which are subsets of matrices from the whole collection appended with notes, which facilitate reproducible experiments. Anymatrix v1.0 comes with 146 built-in matrices organized into 7 groups with 49 recognized properties. The authors continue to extend the collection and welcome contributions from the community.


Introduction
two years later. Clement's motivation was that "For testing and experimentation it is important to have matrices for which exact inverses and eigenvalues are known". Many authors have subsequently studied particular matrices and published papers documenting their properties. Some have published collections of matrices, early examples being Rutishauser's article [43], the book by Gregory and Karney [20], and an appendix in the book by Westlake [53].
In addition to their use in computation, matrices with known properties can also be employed in theoretical investigations to test conjectures, explore the sharpness of bounds, and formulate possible new results.
Several collections of test matrices are available in software. These include • matrices built into MATLAB through the gallery function and other MAT-LAB functions that generate individual matrices; • the Test Matrix Toolbox [26,27], much of which was subsequently incorporated into the MATLAB gallery function; • Matrix Market 1 [4]; • the SuiteSparse Matrix Collection (previously known as the University of Florida Sparse Matrix Collection) 2 [10], which contains large, sparse matrices of fixed size, coming from applications; • CONTEST [51], a MATLAB toolbox providing adjacency matrices from random network models; • the Regularization Tools toolbox 3 [21,22], the IR Tools toolbox 4 [17], and the AIR Tools II toolbox 5 [23], all of which contain several test problems, each defining a matrix and a right-hand side; • the Julia Matrix Depot package [55], which has similar functionality to the MATLAB gallery function and also gives access to matrices from Regularization Tools, the SuiteSparse Matrix Collection, and Matrix Market; • the NLEVP collection 6 of nonlinear eigenvalue problems, each of which is defined by two or more matrices [2,35]. Matrix Depot and SuiteSparse allow users to contribute to the collection (via https:// sparse.tamu.edu/submit in the latter case).
Anymatrix is a new MATLAB toolbox, available at https://github.com/mmikaitis/ anymatrix, that provides an extensible matrix collection and has several key features: • it is large and easily extensible; • it includes parametrized matrices and both fixed size and variable size matrices; • it offers the ability to search for matrices with specific properties; • it organizes matrices into groups and the user can easily create a new group that behaves like the built-in groups and share it on a public repository; and • it enables subsets of matrices to be easily created, stored, and reused to help make experiments reproducible. The collections listed above do not have all these features; see Table 1. In particular, the MATLAB gallery function does not provide a means to search matrices by properties and is not extensible by the user; Matrix Market has not been updated for many years and is not tightly integrated with MATLAB; and SuiteSparse focuses on sparse matrices and provides only matrices of fixed size.
Unlike the NLEVP collection [2], where all matrix integration was done by hand (hard-coded into a function called nlevp_query), in Anymatrix properties are defined within the program files defining the matrices or groups and are read in when Anymatrix initializes itself on the first call in a MATLAB session, and it is this feature than makes the extensibility possible.
Our aim in designing Anymatrix was that anyone who comes across interesting matrices in their research should be able to document them and easily make them available for inclusion in Anymatrix.

The Anymatrix toolbox
In this section we outline the design of Anymatrix and how it is used. Full details are given in the users' guide [34].

Groups of matrices
The matrices in Anymatrix are organized in groups and there are seven built-in groups: • contest: the matrices from CONTEST [51], listed in Table 2.
• core: a miscellaneous selection of matrices, listed in Table 3. Several of these matrices are adapted from the Matrix Computation Toolbox [25]. • gallery: the 61 MATLAB gallery matrices [24, [46], together with some complex Hadamard matrices from [48]; see Table 4. A Hadamard matrix is a real n × n matrix with elements ±1 satisfying H T H = nI . A complex Hadamard matrix has entries of modulus 1 and satisfies H * H = nI . • matlab: matrices that are in MATLAB but are not part of gallery, namely the 12 matrices in Table 5. • nessie: matrices from the NESSIE collection coming from real-life networks [52], listed in Table 6. • regtools: matrices from the Regularization Tools toolbox [21,22], listed in Table 7. The whole toolbox, including the regularization routines, is included in Anymatrix, but only the matrices are used. The regtools matrices have optional output arguments that generate a right-hand side b and, in some cases, a solution x (which are discretizations of the right-hand side and exact solution of the underlying Fredholm integral equation). We note that the problems in Regularization Tools are discretizations of mainly one-dimensional problems that are regarded as easy problems today. Discretizations of more challenging two-dimensional problems can be found in the IR Tools toolbox [17] and the AIR Tools II toolbox [23].  Every matrix is accessed through an identifier that combines the group name and the matrix name. For example, the 5 × 5 beta matrix from the core group is constructed by  Importantly, as long as the group names are distinct (which is enforced by the directory structure), matrices in different groups have different identifiers even if they have the same matrix name. Each matrix is a MATLAB function, but these functions are not on the MATLAB path (but rather in a private directory), so there is no danger of namespace clashes with other MATLAB functions.  Help for the core/beta matrix is obtained with Most of the functionality of Anymatrix is implemented with name-value pairs of input parameters, and the name and the value can be given in any order. Anymatrix supports both the comma-separated "name, value" syntax and the "name = value" syntax introduced in MATLAB R2021a. Taking account of the command/function duality of MATLAB, the following lines are equivalent: Moreover, "name" can be abbreviated as long as there is a unique completion, so are further equivalent forms. The contents of a group, and a cell array of matrix identifiers for the group, can be obtained as follows, where we denote omitted output by [...].

Matrix properties
Every matrix has a set of properties associated with it and it is possible to extract the identifiers of all matrices having a specified set of properties.
For the most part a matrix is assigned a property if it has that property for the default values of any input parameters. An exception is "rectangular", which is assigned if the matrix is rectangular for some choice of input parameters, and if the default shape is square then the matrix is given both the "square" and "rectangular" properties. Examples of matrices having both properties are gallery/krylov and gallery/randsvd. Likewise a matrix can be "real", "complex", or both.
Anymatrix recognizes the properties listed in Table 8. The properties are not casesensitive. For definitions of the properties not defined in the table see linear algebra or numerical linear algebra textbooks.
We do not regard the properties defined for real matrices as being included in the corresponding ones for complex matrices, thus "symmetric" is not included in "Hermitian" and "orthogonal symmetric" is not included in "unitary".
The properties of the core/beta matrix are obtained with We can find which other matrices are nonnegative and either stochastic or totally positive (abbreviating 'properties' to 'p'): In Anymatrix properties are read in when Anymatrix initializes itself on the first call in a MATLAB session. Anymatrix provides a warning if a matrix is detected to have a property not contained in Table 8. Properties can be defined in two ways. The first way is within the functions defining the matrices. A template for a function is This function returns properties in its final output argument and the properties can be obtained without generating the matrix by calling the function with no input  This second option makes it easier to incorporate an existing group of matrices, as just one file needs to be edited. For example, for the built-in gallery and matlab groups we did not modify the corresponding M-files (which are part of MATLAB) but instead wrote an am properties.m function for each group.

Remote groups
We have incorporated Git integration to allow remote groups to be downloaded from Git repositories into Anymatrix storage space and treated in the same way as the builtin groups. For example, we can incorporate the following four matrix collections.
• https://github.com/higham/matrices-correlation-invalid: invalid correlation matrices collected by Higham and Strabić [36,37]. These are matrices that are intended to be correlation matrices but for various reasons relating to their construction have a negative eigenvalue and so are not positive semidefinite. • https://github.com/higham/hpl-ai-matrix: a parametrized n × n matrix designed for use in the HPL-AI Mixed Precision Benchmark 7 [14].
Anymatrix automatically rescans the file storage to pick up any new groups and matrices that were downloaded. If the repository is updated we can run the command that was used to create the group again, which runs git pull to update the group.

Sets of matrices
A matrix set is a user-created subset of all the matrices in an Anymatrix collection, from one or multiple groups, gathered into one named entity. It provides a convenient way to record and later reuse a set of matrices that have something in common, such as being used in an experiment. It thereby aids reproducibility of experiments. A set is defined by creating a text file in anymatrix/sets/.
Here is an example usage of Anymatrix sets: In this case my set is the name of the set. The file that describes this set is anymatrix/sets/my set.txt and it has the following contents (it can be viewed with type my_set.txt).
A set entry comprises a matrix ID followed by a colon and a set of input arguments, and it is terminated by a new line character. Adding a new set into Anymatrix involves creating such a file and rescanning the filestore with anymatrix('scan').
Note that comments can only be made by starting a line with a percentage sign.

Testing
Anymatrix contains two types of tests. The first type uses the MATLAB unit testing framework to check that the matrices have the claimed properties, which is done mainly for properties that are structural and do not require numerical tolerances. We do not test for positive definiteness or diagonal dominance, for example.
The tests are carried out on samples of the matrices for several small dimensions, with default values of any parameters. These tests are run by calling the function test_anymatrix_properties located in the anymatrix/testing directory. A second set of tests is specific to the groups. Tests for all the groups that have them can be run by anymatrix('test'). Specific group tests can be run by anymatrix('test',group_name). Currently, the hadamard group and the hpl-ai-matrix and randsvdfast downloadable groups have such specific tests.

Examples of use of Anymatrix
Here we provide some examples to illustrate how Anymatrix can be useful in research.

Growth factors for LU factorization
The growth factor for LU factorization on A ∈ C n×n is defined by where A (1) = A and the a (k) ij are the elements at the start of the kth stage of LU factorization [28, sect. 9.3; 54]. For complete pivoting, which selects as the pivot element at each stage an element of largest modulus in the active submatrix, it is known that ρ n ≤ n usually holds in practice, but that ρ n > n is possible [11,18]. Cryer [9, Conjecture 5.1] conjectured that ρ n = n for any Hadamard matrix. This conjecture is still open [40,44] and substantial effort was required even to show that ρ 16 = 16 for any 16 × 16 Hadamard matrix [39]. We will check the value of ρ n for the real and complex Hadamard matrices in Anymatrix using the following code. The code uses the function gep from the Matrix Computation Toolbox [25], which implements LU factorization with several pivoting strategies and returns the growth factor. The code tries all 659 Hadamard matrices of dimension up to 428 in the group as well as 9 complex Hadamard matrices of dimension up to 13 (even though Cryer's conjecture does not apply to the latter matrices). The code finds that the growth factor is always equal to n to within the tolerance (which accounts for rounding errors), lending support to the conjecture. The same is true if we repeat the computation for partial pivoting and rook pivoting.
We note that because of rounding errors the pivot sequences may be different from those that would arise in exact arithmetic, so care is needed in interpreting these results; see the discussion in [11].

Specific matrices
Anymatrix provides a convenient way to record and make available particular matrices of interest. We mention three examples.
Consider this computation with the matrix core/edelman27, a matrix of order 27 whose elements are all ±1: The determinant is obviously an integer and d2, computed in 32-digit variableprecision arithmetic (VPA) using the Symbolic Math Toolbox is the exact value. Yet in double precision arithmetic, det computes an inexact value. An explanation for this surprising behavior is given in [31].
In numerical linear algebra we often deal with matrices of integers, but the factorizations we compute typically have some noninteger entries. It is not well known that for n ≤ 7 every symmetric positive definite n × n matrix A of integers with determinant 1 has a factorization A = Z T Z with Z an n × n matrix of integers, though in general such a factorization does not exist for n ≥ 8. This result is mentioned by Taussky [49, p. 336], [50, pp. 812-813] and goes back to Hermite, Minkowski, and Mordell [42]. Finding an integer factor Z is a nontrivial task. Higham, Lettington, and Schmidt [33] have recently derived conditions for integer factorizations to exist and have developed an approach to computing them. A case in point is the Wilson matrix, a moderately ill-conditioned symmetric positive definite matrix that has a long history as a test matrix. An integer factor was discovered in [32] and two ratio-nal factors were discovered in [33]. The matrix and its factors are available through core/wilson: Here, W = Z T Z = Y T Y = X T X, and Z can be thought of as a block Cholesky factor.
A group could be constructed of matrices that provide examples and counterexamples for a particular problem, such as the embeddability problem. This problem, which arises in Markov models, concerns whether a stochastic matrix can be written as the exponential of a matrix with zero row sums and nonnegative diagonal entries. Necessary and sufficient conditions for the existence of a generator are not known. Specific matrices have been identified that demonstrate different possibilities as regards existence of generators and the nature of a generator as a logarithm of the stochastic matrix; see, for example, [5], [30, sec. 2.3], [45]. Collecting such matrices into a group would facilitate numerical experiments in this area of research.

Matrix generators
Often one wants to generate many matrices of a particular type, either randomly or by varying parameters. Examples of MATLAB functions with these capabilities are gallery/randsvd, for generating random nonsymmetric matrices with a specified singular value distribution or symmetric positive definite matrices with a specified eigenvalue distribution, and gallery/randcorr, for generating random correlation matrices with specified eigenvalues. Anymatrix includes two other routines of this type.
The function core/totally_nonneg, generates totally nonnegative matrices, which are square matrices for which every submatrix has nonnegative determinant. It uses a representation of such matrices as a product of nonnegative bidiagonal matrices [16] and allows the elements of the factors to be specified or to be chosen randomly.
The function core/symmstoch generates symmetric stochastic matrices (symmetric nonnegative matrices with unit row and column sums) with eigenvalues determined by an n-vector of input parameters (which must satisfy a certain nonnegativity condition). The construction makes use of a Soules matrix, an orthogonal matrix of a special type generated by core/soules [47].

Optimal matrices
Much research has focused on finding matrices in a given class with largest determinant, largest condition number, smallest singular value, and so on. Anymatrix includes a number of optimal matrices that have been identified, as summarized in Table 9.

Condition estimator
Our final example shows how to run a test across all the matrices in Anymatrix, which can be useful in various circumstances. MATLAB does not provide any easy way to access sequentially all the matrices in gallery.
For the square matrices in Anymatrix, the following code computes the underestimation ratio for the MATLAB condition number estimator condest, which uses the algorithm of Higham and Tisseur [38] to compute a lower bound for κ 1 (A) = ||A|| 1 ||A −1 || 1 . The code filters out matrices defined in the cell array omit, which  [12,15].
would require a specific dimension or more than one input argument.

The output is
The results of this experiment are consistent with the experience that the estimator is usually within a factor 3 of the true condition number [38].

How to extend or contribute to the collection
Users can extend Anymatrix in several ways. New groups can easily be added, as explained in the users' guide [34,Sec. 10.1]. Such groups are on the same footing as the built-in groups.
One can make a group available to others by putting it in an online GitHub repository, as explained in Section 2.3.
We welcome suggestions for matrices to add to existing groups as well as suggestions for new groups to be added, and will be happy to link to remote groups on the Anymatrix GitHub page.

Conclusion
The MATLAB gallery matrix collection is a valuable tool, but it is so large that it is difficult to find within it matrices with specific properties. We designed Anymatrix to provide a unified, searchable interface to a wide variety of matrices, in such a way that remote groups can be included, allowing the basic Anymatrix collection to be expanded by users. We have been using Anymatrix in our research throughout its development and find that it matches our needs for finding suitable matrices for testing and experimentation, as well as being a convenient way to make available specific matrices that have been found in the research of us or our colleagues (such as core/edelman27 and the integer factors of core/wilson).
Our code makes use of many recent features of MATLAB, including certain string handling functions and the unit testing framework. Much of it is not specific to organizing matrices and so it can be reused to organize, into groups and sets, any kind of objects appended with properties. licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.