Teaching Students Nonlinear Programming with Computer Algebra System

This paper presents several didactic examples of the nonlinear programming (NLP) problems solved with Mathematica. We solved examples of Karush–Kuhn–Tucker necessary conditions, Lagrange multipliers method, convex optimization, and graphical method. We compared the hand calculation in Karush–Kuhn–Tucker method with Lagrange multipliers method. The paper contains Mathematica symbolic codes used for Karush–Kuhn–Tucker necessary conditions and the Hessian analysis in convex optimization. We present also some didactic graphs for various aspects of NLP problems using plots and dynamic plots. The use of Mathematica during teaching students about NLP by Computer Algebra System (CAS) seems to be very useful both as the calculations support (checking hand calculation) and when creating didactic graphical visualizations using dynamic plots. We did not find in available literature any similar example of NLP problems solved with CAS or the use of dynamic plots.

of some aspects of used methods. This paper presents two examples of NLP problems for Karush-Kuhn-Tucker necessary conditions, one example of NLP problem for Lagrange multipliers method, three examples of NLP problems for graphical method only, and one example of NLP problem for convex optimization. We compare the hand calculation in Karush-Kuhn-Tucker necessary conditions with calculation in Lagrange multipliers method.
Introduction to NLP theory is presented in [1][2][3][4][5]7]. We will consider NLP problems in the following form: f (x 1 , x 2 , . . . , x n ) subject to: where n and m are positive integers, X is a subset of R n and f, g i are real-valued functions on X with at least one function of f, g i (i = 1, 2, . . . , m) being nonlinear. A feasible region of the NLP problem is defined as a set of all possible points that satisfy all the problem's constrains.
If (x = 0 and y = −1/2) then then The set D is compact and f is continuous on it so finally we get: the greatest value 1/2 and smallest value −1 of f on D are attained corresponding at points (0, 1/2) and (1, 0) respectively.
In Fig. 1 we present graphical interpretation of Karush-Kuhn-Tucker necessary conditions for the above problem.
In the Fig. 1 we see that: We solve this problem also using Mathematica procedures. We present the solution below: Example 2 Let us solve the following NLP problem: We solve it only using Mathematica because a number of cases to consider is 16 and with many subcases.

Listing 3 Mathematica code:
From the above listing we get that the greatest value 20 of f on feasible set is attained at points (4, −2, 0) and (4, 2, 0).

Listing 5 Mathematica code Lagrange multipliers method:
The set D is compact and f is continuous on it so the greatest value 1/2 and the smallest value −1 of f on D are attained respectively at points (0, 1/2) and (1, 0).

Graphical Method
In the case of two or three decision variables, we can try to solve the NLP problem using graphical method. In this method we first plot the feasible region. Next, we consider the family of level sets of the objective function. We try to select level sets from this family that have at least one common point with the feasible region and on which the minimum value of the objective function is attained-when the task is for minimum or maximum when the task is for maximum.

Example 4
We solve the problem from Example 1 using graphical method. In the Fig. 3 we present Mathematica dynamic plot for the Example 1. The level sets is a family of parabolas y − x 2 = c.
The dynamic version of the Fig. 3 can be found in Electronic supplementary material. Global maximum of the function f (x, y) = y − x 2 at point (0, 1/2) we can determine graphically from the level set: y − x 2 = 1/2. Similarly global minimum of the function f at point (1, 0) we can determine graphically from the level set: y − x 2 = −1.

Example 5
We solve the problem from Example 2 using graphical method. In the Fig. 4 we present Mathematica dynamic plot for the Example 2 in two versions: for global minimum and global maximum. The level sets is a family of paraboloids x 2 + y 2 − z = c.   The dynamic version of the Fig. 4 can be found in Electronic supplementary material. We can determine graphically a global minimum of the function f (x, y, z) = x 2 + y 2 − z at point (0, 0, 3) from the level set: The dynamic version of the Fig. 5 can be found in Electronic supplementary material. We can determine graphically a global maximum of the function f (x, y, z) at points (4, −2, 0), (4, 2, 0) from the level set: x 2 + y 2 − z = 20.
Example 6 Consider the following nonlinear integer problem: The dynamic version of the Fig. 6 can be found in Electronic supplementary material. From the dynamic plot in Fig. 6 we can find that the global minimum of the integer problem of function f (x, y) = x + y is attained at point (0, −2) (on the level set x + y = −2), f (0, −2) = −2. Similarly we can find that the global maximum of the function f is attained at point (4, 2) (on the level set x + y = 6), f (4, 2) = 6.
We can check these results using Mathematica build-in function Maximize and Minimize as presented below:

Convex Optimization
This approach based on the fact that for convex function defined on convex set, local minimum is a global minimum (Figs. 7 and 8).
Using Mathematica we can check that the function f is convex on the convex set D and find the stationary points of f .   (−1, 0).
The dynamic version of the Fig. 8 can be found in Electronic supplementary material.

Summary and Conclusions
This paper presents some didactic examples of NLP problems solved with Mathematica. In general, teaching students about NLP, for the first time, is not an easy task without computer support. Examining formal conditions by hand (Karush-Kuhn-Tucker necessary conditions or Lagrange multiple constrains) in case of the more difficult problems can be quite laborious. Also, graphical interpretations are not easy to present without the support of computer programs such as CAS. In Example 1 (Karush-Kuhn-Tucker necessary conditions) with two constrains we get four main cases to consider (with two subcases) what is possible to do by hand with students. But, in Example 2 with four constrains we get sixteen main cases with some subcases, what is rather not appropriate for hand calculations during classes, but it's possible to do with support of computer programs such as CAS. In Example 3 we use Lagrange multiple constrains method for Example 1 to compare Lagrange method with its generalization, the Karush-Kuhn-Tucker method. In Example 3 we finally get only four cases. Using Lagrange method for this problem seems to be significantly easier than Karush-Kuhn-Tucker method. In general, CAS programs give us a possibility to support hand calculations for NLP problems with more constrains. We can also check the convexity condition of the function in convex optimization problem and determine local extremes using CAS functions (Example 7). The possibility to visualize some aspects of NLP problems with CAS is also important. In Examples 1 and 2 we present graphical interpretation of Karush-Kuhn-Tucker necessary conditions. Using CAS dynamic plot we can see normal vectors to the surfaces in Example 2 from different sides. In Examples 4, 5, and 6 we solve the problems using graphical method with dynamic plots. The possibility of graphical visualization and the dynamic plots use are very important in this approach. In case of the graphical method, we analyze the family of level sets of the optimized function in order to determine optimal level set during NLP problems solving. We try to translate the initial level set in the right direction to the optimal position. The above translation is to increase the function values if the problem is for maximum or decrease the function values if the problem is for minimum. The method itself has a dynamic character. Therefore, the CAS dynamic plots seems to be an appropriate tool for visual presentation. This allows students to follow the method steps and provides deeper understanding consequently. In case of two variables and not so many constrains we can draw feasible region and level-sets by hand of course, but in case of three variables (Example 5) it is much more difficult, so computer graphics support seems to be very useful. In case of determining the global minimum of the convex function (Example 7), the use of CAS dynamic plots allows students to dynamically view the surface being a graph of the optimized function and observe the global minimum point on the surface. Generally, the use of Mathematica during teaching students about NLP by CAS programs seems to be very useful both as the hand calculations support and when creating graphical visualizations of various aspects of NLP problems. We did not find any analogical approach to the NLP problems solving using CAS in available literature.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.