In 2021, after a one-year postponement due to the pandemic emergency, the biennial congress of the Italian Society of Industrial and Applied Mathematics (SIMAI) was held at Parma (Italy). Also this edition, like the previous ones, was aimed to bring together researchers and professionals from academia and industry who are active in the study of mathematical and numerical models as well as their application to industrial and in general real life problems, to stimulate interdisciplinary research in applied mathematics and to foster interactions of the scientific community with industry. The congress was one of the first in Italy, after the isolation due to the pandemic, giving the opportunity to meet again in the presence. A considerable number of minisymposia was dedicated to the computational aspects of numerical optimization, testifying to the fact that it plays a crucial and increasingly relevant role in most of the real-world applications in which a growing amount of data must be processed. The aim of this special issue is to present some high-quality research papers that provide a snapshot, of course not exhaustive, of how advances in numerical optimization are stimulated and motivated by real-life applications. There is a common thread that connects these papers. They demonstrate how well-established optimization techniques can be combined with new approaches to broaden the field of addressable problems.

An example is the paper of Stankewitz et al. [13], where the well-known properties of standard gradient descent method can provide numerous advances in the field of optimization for machine learning. Combining the techniques of the inexact optimization with probabilistic results, namely gradient concentration, the authors derive bounds for the error prediction on future data, highlighting the implicit regularization properties of optimization for learning.

On the same line is the paper by Crisci et al. [7] that aims to make effective gradient projection methods for constrained optimization problems. Moving from an approach originally proposed by Fletcher [8] for the unconstrained case, in [7] the authors propose a limited memory strategy based on the Ritz-like values of the Hessian matrix restricted to the inactive constraints, when the final active set is reached. Computational experiments show the nice behaviuor of their approach on large scale problems for which classical gradient descent methods usually yield very slow convergence.

An alternative to projected gradient approaches for the constrained optimization of a smooth objective function is given by the Frank-Wolfe (FW) type methods, which are used in many applications. The main drawback of FW methods is the slow convergence that may occur, mainly close to the solution, due to the so called "bad steps", i.e. long sequence of very short steps. This problems is addressed in [12], by proposing a theoretically sound algorithmic framework aimed to rule out slow convergence rates due to a large number of bad steps. The approach could have a meaningful impact on the effective use of FW type methods in the general framework of practical applications.

Following the novel and original approach of Ulyanov et al. [11] to image processing, in [5] two Deep Image Prior-inspired models for image restoration tasks are proposed. The strength of the proposed models is their independence from the identification of a suitable regularization parameter. Indeed, in the first model, which is unconstrained, the authors propose to adapt the pixel-wise regularization parameters at each iteration by exploiting a scheme based on the Uniform PENalty principle. In the second model the need of a regularization parameter is avoided by exploiting the Morozov’s discrepancy principle and imposing the data fidelity as a convex constraint for which the noise level is estimated a priori. The authors compare the two models with state-of-the-art Deep Image Prior models on a variety of test problems.

In [1], the authors presents a strategy to segment images in two parts, by considering an image as the sum of a geometric component, the cartoon one, and of a oscillatory component, this latter containing texture and/or noise. Since the oscillatory content of these regions may badly affect the segmentation result, a joint cartoon-texture decomposition model is merged with the binary segmentation approach to improve the quality of a rough given initialization. Thus the segmentation is achieved by solving a non-smooth constrained optimization problem which involves the discrete version of the well-known functional of Chan et al. [6] and the divergence of Kullback–Leibler to control the oscillatory term. Interesting results are obtained by a suitable implementation of the ADMM method.

In [4], the focus is on the very popular FISTA method and its numerous inertial forward-backward variants, widely applied in the composite optimization problems arising in data processing. The paper addresses the drawback of inexact computation of the proximal operator in the backward step and generalizes several nested primal-dual algorithms already available in the literature; in particular the authors propose to use a prefixed number of inner primal-dual iterations for the proximal evaluation and a warm-start strategy for starting the inner loop. Convergence results and numerical evidence on image restoration problems enable to affirm that the combination of warm-start strategy with an appropriate choice of the inertial parameters is strictly required in order to guarantee the convergence to the real minimum point of the objective function.

In [9] the authors deal with the numerical solution of mixed integer PDE constrained optimization (MIPDECO) problem with a linear time-dependent constraint and integer constraints on the control. An improved penalty algorithm (IPA), recently developed by the authors in [10], is suitably adapted to this linear time-dependent setting, having partial observations in the objective function. The core of the IPA is a local optimization solver paired with both a probabilistic basin hopping strategy and an updating tool for the penalty parameter. A combination of an interior point method (IPM), model order reduction (MOR), and preconditioning resulting in the MOR-IPM is introduced to address the large-scale context of the time-dependent PDE constraint. Numerical investigation for the heat equation as well as a convection-diffusion problem shows that the approach enables to overcome the challenges due to the mixture of combinatorial aspects, induced by integer variables, and large scale linear algebra issues, arising from the PDE discretization.

Modern finance is getting more and more sophisticated, requiring complicated mathematical models, based on stochastic analysis, dynamical system theory, game theory, optimal control and dynamic programming. In this framework, the paper by Barbagallo and Guarino Lo Bianco [2] deals with the analysis of an effective oligopolistic market equilibrium model using the theory of stochastic and time-dependent variational inequalities combined with the Nash equilibrium theory, focusing on the policymaker’s point of view. Equilibrium conditions and well-posedness are stated, and some numerical experiences are reported.

The finite-sum minimization problem is addressed in [3] with a stochastic first-order trust-region method, based on inexact function and gradient evaluations. Namely, this work combines the inexact restoration approach for constrained optimization with the trust-region procedure and random models. Randomness helps relieve the computational burden of the minimization, with feasibility and optimality which are gradually improved in a modular way. A theoretical analysis with complexity results is carried out, whereas computational results on problems arising in binary classification and regression show the effectiveness of the proposed approach.

We sincerely thank all the authors who submitted their papers, and the referees for their effort and very valuable contribution to this Special Issue. We also want to thank Prof. William W. Hager, Editor-in-Chief of Computational Optimization and Application for his constant support.