Parallel computing technologies 2018

Looking at the current growing trends in parallel programming (PP), it would be interesting now to predict future features of PP systems for parallel implementation of applications; to understand what new parallel computing technologies should/can be developed in the near future, and what education would be the best in the near term.

Wide use of supercomputers in modeling involves many mathematicians in the process of numerical/mathematical models development, and involves a number of highly qualified parallel programmers to assist to mathematicians in implementing simulation programs. However, every mathematician should be able to actually create any necessary simulating parallel programs without the assistance of a programmer. For now, this is not the case.

Of the field’s growing trends, which do I find of primary interest? The main current trend in PP is a gradual but consistent increase over recent years in the awareness of the necessity, partially or fully, to replace procedural PP (like C + MPI) with automatic parallel program construction for application implementation. (Here, “applications” are actually different problems for a supercomputer’s numerical simulation). Additional motivation for the creation of a practical system of automatic generation of parallel program (PSAG) for supercomputer large-scale numerical simulation is the appearance of actual problems which can be solved only with the use of PSAG, for example, creation of active knowledge technology.

Implementation of PSAG involves many technological problems. Theoretical algorithms for the automatic implementation/generation of the schemes below are well known in theories of program synthesis, at least from 1970s. However, because of low technological quality of the theoretical algorithms, they are not acceptable for the development of PSAG systems. Thus, the problem of automatic construction of parallel programs lies in the development of technological algorithms¹ for implementation of the schemes below.

Currently, in many object domains, a variety of practically comprehensive, object oriented libraries of modules have accumulated. On this basis the necessary algorithms can be derived from the axioms of AT and the necessary target programs can be constructed out of library modules.² Now with the use of current hardware and software there is a possibility to develop and implement effective technological algorithms for automatic parallel program construction/generation.

The following technological algorithms, programs and components should be developed:

• 1. AT—axiomatic theory describing an object domain. This AT contains the description all of the algorithms of an object domain that can be constructed out of library modules. Therefore, with AT a desirable algorithm of a problem solution could be derived first, and then the proper implementing program could be constructed.

• 2. Derivation of an algorithm of application problem solution (simulating algorithm) from the AT axioms and problem formulation. Problem formulation is a pair (V, W), where V is the set of input variables of the deriving algorithm and W is the set of output variables. Algorithm of derivation is described in mathematical logic.

• 3. Algorithms of automatic construction/generation of a program, implementing derived simulating algorithm. In the course of automatic generation of the target simulating parallel program, the properties all of a model’s data should be taken into consideration, because the data properties often define the numerical model’s behavior.

• 4. For automatic generation of better simulating programs the peculiarities of the computer system should be taken into account.

• 5. Various algorithms of dynamic distributed distribution of distributed resources (4DR), including dynamic load balancing.

Creation of PSAG will open the gates to the development of active knowledge technology. What will be the main work of parallel programmers over the next 20 or 50 (or maybe more) years? The reachable goal of the parallel programming community should be elimination, fully or partially, of procedural PP from the practice of supercomputer modeling. Soon, we should be able to resort primariely to automatic generation of application programs, instead of their programming.

What should the current computer science and computer engineering students be taught in universities? Many colleagues consider that basic mathematical education should rely equally on courses in discrete mathematics, mathematical logic first, and calculus. The coder that does not know logic and mathematics of the object domain will not be able to program satisfactory problem solutions.

Main projects for the future

• 1. For the next 20 or maybe 50 years, the efforts of parallel programmers should be concentrated on full or partial elimination of procedural parallel programming from the practice of supercomputer modeling. This means that, for different application domains, knowledge bases should be developed and implemented.

• 2. Development of a system of automatic construction of parallel application programs, focusing first on all different programs of numerical supercomputer simulation.

• 3. Development of intelligent algorithms for generation of parallel programs of numerical supercomputer simulation that take into account the behavior of models.

• 4. Finally, it is necessary to point out that education of computer specialists definitely should be based less on calculus and mostly on discrete mathematics courses such as set theory, algorithm theory and, first of all, on mathematical logic.