Other Applications of ADP

  • Huaguang Zhang
  • Derong Liu
  • Yanhong Luo
  • Ding Wang
Part of the Communications and Control Engineering book series (CCE)


In this chapter, the optimal control problems of modern wireless networks and automotive engines are studied by using ADP methods. In the first part, a novel learning control architecture is proposed based on adaptive critic designs/ADP, with only a single module instead of two or three modules. The choice of utility function for the present self-learning control scheme makes the present learning process much more efficient than existing learning control methods. The call admission controller can perform learning in real time as well as in off-line environments and the controller improves its performance as it gains more experience. In the second part, an ADP-based learning algorithm is designed according to certain criteria and calibrated for vehicle operation over the entire operating regime. The algorithm is optimized for the engine in terms of performance, fuel economy, and tailpipe emissions through a significant effort in the research and development and calibration process. After the controller has learned to provide optimal control signals under various operating conditions off-line or on-line, it is applied to perform the task of engine control in real time. The performance of the controller can be further refined and improved through continuous learning in real-time vehicle operations.


Utility Function Code Division Multiple Access Service Class Call Admission Control Handoff Call 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    A guide to DECT features that influence the traffic capacity and the maintenance of high radio link transmission quality, including the results of simulations. ETSI technical report: ETR 042, July 1992. Available on-line at
  2. 2.
    Ariyavisitakul S (1994) Signal and interference statistics of a CDMA system with feedback power control—Part II. IEEE Trans Commun 42:597–605 CrossRefGoogle Scholar
  3. 3.
    Bambos N, Chen SC, Pottie GJ (2000) Channel access algorithms with active link protection for wireless communication networks with power control. IEEE/ACM Trans Netw 8:583–597 CrossRefGoogle Scholar
  4. 4.
    Demuth H, Beale M (1998) Neural network toolbox user’s guide. MathWorks, Natick Google Scholar
  5. 5.
    Dobner DJ (1980) A mathematical engine model for development of dynamic engine control. SAE paper no 800054 Google Scholar
  6. 6.
    Dobner DJ (1983) Dynamic engine models for control development—Part I: non-linear and linear model formation. Int J Veh Des Spec Publ SP4:54–74 Google Scholar
  7. 7.
    Dziong Z, Jia M, Mermelstein P (1996) Adaptive traffic admission for integrated services in CDMA wireless-access networks. IEEE J Sel Areas Commun 14:1737–1747 CrossRefGoogle Scholar
  8. 8.
    Freeman RL (1996) Telecommunication system engineering. Wiley, New York Google Scholar
  9. 9.
    Gilhousen KS, Jacobs IM, Padovani R, Viterbi AJ, Weaver LA, Wheatley CE III (1991) On the capacity of a cellular CDMA system. IEEE Trans Veh Technol 40:303–312 CrossRefGoogle Scholar
  10. 10.
    Guerin RA (1987) Channel occupancy time distribution in a cellular radio system. IEEE Trans Veh Technol 35:89–99 CrossRefGoogle Scholar
  11. 11.
    Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5:989–993 CrossRefGoogle Scholar
  12. 12.
    Hong D, Rappaport SS (1986) Traffic model and performance analysis for cellular mobile radio telephone systems with prioritized and nonprioritized handoff procedures. IEEE Trans Veh Technol 35:77–92 CrossRefGoogle Scholar
  13. 13.
    Kim DK, Sung DK (2000) Capacity estimation for an SIR-based power-controlled CDMA system supporting ON–OFF traffic. IEEE Trans Veh Technol 49:1094–1100 CrossRefGoogle Scholar
  14. 14.
    Kim YW, Kim DK, Kim JH, Shin SM, Sung DK (2001) Radio resource management in multiple-chip-rate DS/CDMA systems supporting multiclass services. IEEE Trans Veh Technol 50:723–736 CrossRefGoogle Scholar
  15. 15.
    Kovalenko O, Liu D, Javaherian H (2001) Neural network modeling and adaptive critic control of automotive fuel-injection systems. In: Proceedings of IEEE international symposium on intelligent control, Taipei, Taiwan, pp 368–373 Google Scholar
  16. 16.
    Lendaris GG, Paintz C (1997) Training strategies for critic and action neural networks in dual heuristic programming method. In: Proceedings of international conference on neural networks, Houston, TX, pp 712–717 Google Scholar
  17. 17.
    Liu Z, Zarki ME (1994) SIR-based call admission control for DS-CDMA cellular systems. IEEE J Sel Areas Commun 12:638–644 CrossRefGoogle Scholar
  18. 18.
    Liu D, Zhang Y (2002) A new learning control approach suitable for problems with finite action space. In: Proceedings of international conference on control and automation, Xiamen, China, pp 1669–1673 Google Scholar
  19. 19.
    Liu D, Xiong X, Zhang Y (2001) Action-dependent adaptive critic designs. In: Proceedings of INNS-IEEE international joint conference on neural networks, Washington, DC, pp 990–995 Google Scholar
  20. 20.
    Liu D, Zhang Y, Hu S (2004) Call admission policies based on calculated power control setpoints in SIR-based power-controlled DS-CDMA cellular networks. Wirel Netw 10:473–483 CrossRefGoogle Scholar
  21. 21.
    Liu D, Zhang Y, Zhang H (2005) A self-learning call admission control scheme for CDMA cellular networks. IEEE Transactions on Neural Networks 16:1219–1228 CrossRefGoogle Scholar
  22. 22.
    Liu D, Hu S, Zhang HG (2006) Simultaneous blind separation of instantaneous mixtures with arbitrary rank. IEEE Trans Circuits Syst I, Regul Pap 53:2287–2298 MathSciNetCrossRefGoogle Scholar
  23. 23.
    Liu D, Xiong X, DasGupta B, Zhang HG (2006) Motif discoveries in unaligned molecular sequences using self-organizing neural networks. IEEE Trans Neural Netw 17:919–928 CrossRefGoogle Scholar
  24. 24.
    Liu D, Javaherian H, Kovalenko O (2008) Adaptive critic learning techniques for engine torque and air–fuel ratio control. IEEE Trans Syst Man Cybern, Part B, Cybern 38:988–993 CrossRefGoogle Scholar
  25. 25.
    Puskorius GV, Feldkamp LA, Davis LL (1996) Dynamic neural network methods applied to on-vehicle idle speed control. Proc IEEE 84:1407–1420 CrossRefGoogle Scholar
  26. 26.
    Ramjee R, Towsley D, Nagarajan R (1997) On optimal call admission control in cellular networks. Wirel Netw 3:29–41 CrossRefGoogle Scholar
  27. 27.
    Rappaport SS, Purzynski C (1996) Prioritized resource assignment for mobile cellular communication systems with mixed services and platform types. IEEE Trans Veh Technol 45:443–458 CrossRefGoogle Scholar
  28. 28.
    Sampath A, Holtzman JM (1997) Access control of data in integrated voice/data CDMA systems: benefits and tradeoffs. IEEE J Sel Areas Commun 15:1511–1526 CrossRefGoogle Scholar
  29. 29.
    Shin SM, Cho CH, Sung DK (1999) Interference-based channel assignment for DS-CDMA cellular systems. IEEE Trans Veh Technol 48:233–239 CrossRefGoogle Scholar
  30. 30.
    Veeravalli VV, Sendonaris A (1999) The coverage-capacity tradeoff in cellular CDMA systems. IEEE Trans Veh Technol 48:1443–1450 CrossRefGoogle Scholar
  31. 31.
    Visnevski NA, Prokhorov DV (1996) Control of a nonlinear multivariable system with adaptive critic designs. In: Proceedings of conference on artificial neural networks in engineering, St Louis, MO, pp 559–565 Google Scholar
  32. 32.
    Viterbi AJ, Viterbi AM, Zehavi E (1994) Other-cell interference in cellular power-controlled CDMA. IEEE Trans Commun 42:1501–1504 CrossRefGoogle Scholar
  33. 33.
    Werbos PJ, McAvoy T, Su T (1992) Neural networks, system identification, and control in the chemical process industries. In: White DA, Sofge DA (eds) Handbook of intelligent control: neural, fuzzy, and adaptive approaches, Van Nostrand Reinhold, New York, NY Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Huaguang Zhang
    • 1
  • Derong Liu
    • 2
  • Yanhong Luo
    • 1
  • Ding Wang
    • 2
  1. 1.College of Information Science Engin.Northeastern UniversityShenyangPeople’s Republic of China
  2. 2.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations