# Basics of Differential Evolution

• Anyong Qing
Chapter
Part of the Evolutionary Learning and Optimization book series (ALO, volume 4)

## A Short History

Inception

Differential evolution was proposed by K.V. Price and R. Storn in 1995 [1]. At that time, Price was asked to solve the Chebyshev polynomial fitting problem [1]-[5] by Storn [2], [5]. Initially, he tried to solve it by using genetic annealing algorithm [6]. However, although he eventually found the solution to the 5-dimensional Chebyshev polynomial fitting problem by using genetic annealing algorithm, he was frustrated to notice that genetic annealing algorithm fails to fulfill the three requirements for a practical optimization technique: strong global search capability, fast convergence, and user friendliness.

A breakthrough happened when Price came up with an innovative scheme for generating trial parameter vectors. In this scheme, a new parameter vector is generated by adding the weighted difference vector between two population members to a third member. Such a scheme was named as differential mutation and has been well known to be the crucial idea behind the success of differential evolution. The cornerstone for differential evolution was therefore laid.

Price wrapped up his invention with other critical ideas: natural real code, arithmetic operations, mother-child competition and selection, and execution of evolutionary operations in the order of mutation-crossover-selection. Consequently, differential evolution, a very reliable, efficient, robust, and simple evolutionary algorithm was developed.

## Keywords

Differential Evolution Differential Evolution Algorithm General Regression Neural Network Arithmetic Crossover Hybrid Differential Evolution
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

1. 1.
Storn, R., Price, K.V.: Differential Evolution - A Simple and Efficient Adaptive Scheme for Global Optimization over Continuous Spaces, Technical Report TR-95-012, International Computer Science Insitute (March 1995)Google Scholar
2. 2.
Price, K.V.: Differential evolution vs. the functions of the 2nd ICEO. In: 1997 IEEE Int. Conf. Evolutionary Computation, Indianapolis, IN, April 13-16, pp. 153–157 (1997)Google Scholar
3. 3.
Price, K.V., Storn, R.M., Lampinen, J.A.: Differential Evolution: a Practical Approach to Global Optimization. Springer, Berlin (2005)
4. 4.
Qing, A.: Differential Evolution: Fundamentals and Applications in Electrical Engineering. John Wiley, New York (2009)Google Scholar
5. 5.
Storn, R.: Differential evolution (DE) for continuous function optimization (an algorithm by Kenneth Price and Rainer Storn) (2009), http://www.icsi.berkeley.edu/~storn/code.html (last accessed on October 23, 2009)
6. 6.
Price, K.V.: Genetic annealing. Dr. Bobb’s J. 19(10), 127–132 (1994)Google Scholar
7. 7.
Price, K.V.: Differential evolution: a fast and simple numerical optimizer. In: 1996 Biennial Conf. North American Fuzzy Information Processing Society, Berkeley, CA, June 19-22, pp. 524–527 (1996)Google Scholar
8. 8.
Storn, R., Price, K.V.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optimization 11(4), 341–359 (1997)
9. 9.
Brutovský, B., Ulicný, J., Miškovský, P.: Application of genetic algorithms based techniques in the theoretical analysis of molecular vibrations. In: 1st Int. Conf. Genetic Algorithms Occasion 130th Anniversary Mendel’s Laws in Brno, Brno, Czech Republic, September 26-28, pp. 29–33 (1995)Google Scholar
10. 10.
Storn, R., Price, K.V.: Minimizing the real functions of the ICEC 1996 contest by differential evolution. In: 1996 IEEE Int. Conf. Evolutionary Computation, Nagoya, May 20-22, pp. 842–844 (1996)Google Scholar
11. 11.
Storn, R.: Modeling and Optimization of PET-Redundancy Assignment for MPEG-Sequences, Technical Report TR-95-018, International Computer Science Institute (May 1995)Google Scholar
12. 12.
Storn, R.: Differential Evolution Design of an IIR-Filter with Requirements for Magnitude and Group Delay, Technical Report TR-95-026, International Computer Science Institute (June 1995)Google Scholar
13. 13.
Storn, R.: Differential evolution design of an IIR-filter. In: 1996 IEEE Int. Conf. Evolutionary Computation, Nagoya, May 20-22, pp. 268–273 (1996)Google Scholar
14. 14.
Storn, R.: On the usage of differential evolution for function optimization. In: 1996 Biennial Conf. North American Fuzzy Information Processing Society, Berkeley, CA, June 19-22, pp. 519–523 (1996)Google Scholar
15. 15.
Storn, R.: System Design by Constraint Adaptation and Differential Evolution, Technical Report TR-96-039, International Computer Science Institute (November 1996)Google Scholar
16. 16.
Joshi, R., Sanderson, A.C.: Multisensor fusion and model selection us-ing a minimal representation size framework. In: 1996 IEEE/SICE/RSJ Int. Conf. Multisensor Fusion Integration Intelligent Systems, Washington, DC, Deemeber 8-11, pp. 25–32 (1996)Google Scholar
17. 17.
Chiou, J.P., Wang, F.S.: Hybrid differential evolution for parameter estimation of a batch bioprocess. In: IEEE Int. Symp. Control Theory Applications, Singapore, July 29-30, pp. 171–174 (1997)Google Scholar
18. 18.
Fleiner, C.: Parallel Optimizations: Advanced Constructs and Compiler Optimizations for a Parallel, Object Oriented, Shared Memory Language Running on a Distributed System, Ph. D. Thesis, University of Fribourg (April 11, 1997)Google Scholar
19. 19.
Joshi, R., Sanderson, A.C.: Experimental studies on minimal representation multisensor fusion. In: 8th Int. Conf. Advanced Robotics, Monterey, CA, July 7-9, pp. 603–610 (1997a)Google Scholar
20. 20.
Joshi, R., Sanderson, A.C.: Minimal representation multisensor fusion using differential evolution. In: 1997 IEEE Int. Symp. Computational Intelligence Robotics Automation, Monterey, CA, July 10-11, pp. 266–273 (1997b)Google Scholar
21. 21.
Joshi, R., Sanderson, A.C.: Multisensor fusion of touch and vision using minimal representation size. In: 1997 IEEE/RSJ Int. Conf. Intelligent Robots Systems, Grenoble, September 7-11, vol. 3, pp. v4–v5 (1997c)Google Scholar
22. 22.
Masters, T., Land, W.: A new training algorithm for the general regression neural network. In: 1997 IEEE Int. Conf. Systems Man Cybernetics, Orlando, FL, October 12-15, vol. 3, pp. 1990–1994 (1997)Google Scholar
23. 23.
Michael, C., McGraw, G.: Opportunism and Diversity in Automated Software Test Data Generation, Technical Report RSTR-003-97-13, ver-sion 1.3, RST Corporation, Sterling, VA, USA (December 8, 1997)Google Scholar
24. 24.
Rüttgers, M.: Differential Evolution: A Method for Optimization of Real Scheduling Problems, Technical Report TR-97-013, International Computer Science Institute (March 1997)Google Scholar
25. 25.
Rüttgers, M.: Design of a method for machine scheduling for core blowers in foundries. In: Reusch, B. (ed.) Fuzzy Days 1997. LNCS, vol. 1226, p. 572. Springer, Heidelberg (1997)Google Scholar
26. 26.
Rüttgers, M.: Design of a new algorithm for scheduling in parallel machine shops. In: 1997 5th European Congress Intelligent Techniques Soft Computing, Aachen, Germany, September 8-11, vol. 3, pp. 2182–2187 (1997)Google Scholar
27. 27.
Thomas, P., Vernon, D.: Image registration by differential evolution. In: 1st Irish Machine Vision Image Processing Conf., Magee College, University of Ulster, pp. 221–225 (1997)Google Scholar
28. 28.
Wang, F.S., Chiou, J.P.: Differential evolution for dynamic optimization of differential algebraic systems. In: 1997 IEEE Int. Conf. Evolutionary Computation, Indianapolis, IN, April 13-16, pp. 531–536 (1997)Google Scholar
29. 29.
Wang, F.S., Chiou, J.P.: Optimal control and optimal time location problems of differential-algebraic systems by differential evolution. Industrial Engineering Chemistry Research 36(12), 5348–5357 (1997)
30. 30.
Price, K., Storn, R.: Differential evolution: a simple evolution strategy for fast optimization. Dr. Dobb’s J. 22(4), 18–24, 78 (1997)
31. 31.
Chang, T.T., Chang, H.C.: Application of differential evolution to passive shunt harmonic filter planning. In: 8th Int. Conf. Harmonics Quality Power, Athens, Greece, October 14-16, vol. 1, pp. 149–153 (1998)Google Scholar
32. 32.
Meyer, M.: Construction of a multi-purpose X-ray CCD detector and its implementation on a 4-circle kappa goniometer, Ph. D. Thesis, l’Université de Lausanne (1998)Google Scholar
33. 33.
Mastorakis, N.E. (ed.): Recent Advances in Circuits and Systems. World Scientific, Singapore (1998)Google Scholar
34. 34.
Corn, D., Dorigo, M., Glover, F. (eds.): New Ideas in Optimization. McGraw-Hill, London (1999)Google Scholar
35. 35.
Qing, A.: Dynamic differential evolution strategy and applications in electromagnetic inverse scattering problems. IEEE Trans. Geosci. Remote Sens. 44(1), 116–125 (2006)
36. 36.
Bergey, P.K.: An agent enhanced intelligent spreadsheet solver for multi-criteria decision making. In: 1999 Americas Conf. Information Systems, Milwaukee, August 13-15, pp. 966–968 (1999)Google Scholar
37. 37.
Chang, C.S., Xu, D.Y., Quek, H.B.: Pareto-optimal set based multiobjective tuning of fuzzy automatic train operation for mass transit system. IEE Proc. B-Electric Power Applications 146(5), 577–583 (1999)
38. 38.
Rigling, B.D., Moore, F.W.: Exploitation of sub-populations in evolution strategies for improved numerical optimization. In: 10th Midwest Artificial Intelligence Cognitive Science Conf., Bloomington, Indiana, April 23-25, pp. 80–88 (1999)Google Scholar
39. 39.
Lee, M.H., Han, C., Chang, K.S.: Dynamic optimization of a continuous polymer reactor using a modified differential evolution algorithm. Industrial Engineering Chemistry Research 38(12), 4825–4831 (1999)
40. 40.
Michalski, K.A.: Electromagnetic imaging of circular-cylindrical conductors and tunnels using a differential evolution algorithm. Microwave Optical Technology Letters 27(5), 330–334 (2000)
41. 41.
Babu, B.V., Chaturvedi, G.: Evolutionary computation strategy for optimization of an alkylation reaction. In: Int. Symp. 53rd Annual Session IIChE, Science City, Calcutta, December 18-21 (2000)Google Scholar
42. 42.
Babu, B.V., Munawar, S.A.: Differential evolution for the optimal design of heat exchangers. In: All India Seminar Chemical Engineering Progress Resource Development: A Vision 2010 Beyond, Orissa State Center, Bhuvaneshwar (March 13, 2000)Google Scholar
43. 43.
Pahner, U., Hameyer, K.: Adaptive coupling of differential evolution and multiquadrics approximation for the tuning of the optimization process. IEEE Trans. Magnetics 36(4), 1047–1051 (2000)
44. 44.
Lampinen, J.: A bibliography on differential evolution algorithm, Technical Report, Lappeenranta University of Technology, Department of Information Technology, Laboratory of Information Processing (2001) (last updated on October 14, 2002), available via internet, http://www2.lut.fi/~jlampine/debiblio.htm (accessed on October 12, 2009)
45. 45.
Lampinen, J.: Solving problems subject to multiple nonlinear constraints by the differential evolution. In: 7th Int. Conf. Soft Computing, Brno, Czech Republic, June 6-8, pp. 50–57 (2001)Google Scholar
46. 46.
Angira, R., Babu, B.V.: Non-dominated sorting differential evolution (NSDE): an extension of differential evolution for multi-objective optimization. In: 2nd Indian Int. Conf. Artificial Intelligence, Pune, India, December 20-22, pp. 1428–1443 (2005)Google Scholar
47. 47.
Qin, A.K., Suganthan, P.N.: Self-adaptive differential evolution algorithm for numerical optimization. In: 2005 IEEE Congress Evolutionary Computation, Edinburgh, UK, September 2-5, vol. 2, pp. 1785–1791 (2005)Google Scholar
48. 48.
Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.A.: Opposition-based differential evolution for optimization of noisy problems. In: 2006 IEEE Congress Evolutionary Computation, Vancouver, Canada, July 16-21, pp. 1865–1872 (2006)Google Scholar
49. 49.
Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.A.: Opposition-based differential evolution algorithms. In: 2006 IEEE Congress Evolutionary Computation, Vancouver, Canada, July 16-21, pp. 2010–2017 (2006)Google Scholar
50. 50.
Zaharie, D.: A comparative analysis of crossover variants in differential evolution. In: Int. Multiconference Computer Science Information Technology, pp. 171–181 (2007)Google Scholar
51. 51.
Lawson, K.: Darwin and Evolution for Kids: His Life and Ideas with 21 Activities. Chicago Review Press, Chicago (2003)Google Scholar
52. 52.
Chen, C.W., Chen, D.Z., Cao, G.Z.: An improved differential evolution algorithm in training and encoding prior knowledge into feedforward networks with application in chemistry. Chemometrics Intelligent Laboratory Systems 64(1), 27–43 (2002)
53. 53.
Chakraborty, U.K. (ed.): Advances in Differential Evolution. Springer, Berlin (March 2008)
54. 54.
Fan, H.Y., Lampinen, J.: A trigonometric mutation operation to differential evolution. J. Global Optimization 27, 105–129 (2003)
55. 55.
Fan, H.Y., Lampinen, J.: A directed mutation operation for the differential evolution algorithm. Int. J. Industrial Engineering-Theory Applications Practice 10(1), 6–15 (2003)
56. 56.
Fischer, M.M., Hlavackova-Schindler, K., Reismann, M.: An evolutionary mutation-based algorithm for weight training in neural networks for telecommunication flow modelling, Computational Intelligence Modelling, Control Automation. In: Evolutionary Computation and Fuzzy Logic for Intelligent Control, Knowledge Acquisition and Information Retrieval, Vienna, Austria, Febuary 17-19. Concurrent Systems Engineering Series, vol. 55, pp. 54–59 (1999)Google Scholar
57. 57.
Qing, A.: A parametric study on differential evolution based on benchmark electromagnetic inverse scattering problem. In: 2007 IEEE Congress Evolutionary Computation, Singapore, September 25-28, pp. 1904–1909 (2007)Google Scholar
58. 58.
Qing, A.: A study on base vector for differential evolution. In: 2008 IEEE World Congress Computational Intelligence/2008 IEEE Congress Evolutionary Computation, Hong Kong, June 1-6, pp. 550–556 (2008)Google Scholar
59. 59.
Lampinen, J., Zelinka, I.: Mixed variable non-linear optimization by differential evolution. In: 2nd Int. Prediction Conf., Zlin, Czech Republic, October 7-8, pp. 45–55 (1999)Google Scholar
60. 60.
Krink, T., Filipič, B., Fogel, G.B., Thomsen, R.: Noisy optimization problems - a particular challenge for differential evolution? In: 2004 IEEE Congress Evolutionary Computation, Portland, OR, June 19-23, vol. 1, pp. 332–339 (2004)Google Scholar
61. 61.
Bindal, A., Ierapetritou, M.G., Balakrishnan, S., Armaou, A., Makeev, A.G., Kevrekidis, I.G.: Equation-free, coarse-grained computational optimization using timesteppers. Chemical Engineering Science 61(2), 779–793 (2006)