ACSAC 2005: Advances in Computer Systems Architecture pp 818-830 | Cite as
Implementation and Analysis of TCP/IP Offload Engine and RDMA Transfer Mechanisms on an Embedded System
Abstract
The speed of present-day network technology exceeds a gigabit and is developing rapidly. When using TCP/IP in these high-speed networks, a high load is incurred in processing TCP/IP protocol in a host CPU. To solve this problem, research has been carried out into TCP/IP Offload Engine (TOE) and Remote Direct Memory Access (RDMA). The TOE processes TCP/IP on a network adapter instead of using a host CPU; this reduces the processing burden on the host CPU, and RDMA eliminates any copy overhead of incoming data streams by allowing incoming data packets to be placed directly into the correct destination memory location. We have implemented the TOE and RDMA transfer mechanisms on an embedded system. The experimental results show that TOE and RDMA on an embedded system have considerable latencies despite of their advantages in reducing CPU utilization and data copy on the receiver side. An analysis of the experimental results and a method to overcome the high latencies of TOE and RDMA transfer mechanisms are presented.
Keywords
Embed System User Memory Network Adapter Device Driver Packet ProcessingPreview
Unable to display preview. Download preview PDF.
References
- 1.Bierbaum, N.: MPI and Embedded TCP/IP Gigabit Ethernet Cluster Computing. In: Proc. of the 27th Annual IEEE Conference on Local Computer Networks, pp. 733–734 (November 2002)Google Scholar
- 2.
- 3.Kay, J., Pasquale, J.: Profiling and reducing processing overheads in TCP/IP. IEEE/ACM Transactions on Networking 4(6), 817–828 (1996)CrossRefGoogle Scholar
- 4.Tak, S.-W., Son, J.-M., Kim, T.-K.: Experience with TCP/IP networking protocol S/W over embedded OS for network appliance. In: Proc. of 1999 International Workshops on Parallel Processing, September 1999, pp. 556–561 (1999)Google Scholar
- 5.Camarda, P., Pipio, F., Piscitelli, G.: Performance evaluation of TCP/IP protocol implementations in end systems. IEEE Proc. of Computers and Digital Techniques 146(1), 32–40 (1999)CrossRefGoogle Scholar
- 6.Kung, H.T., Wang, S.Y.: TCP trunking: design, implementation and performance. In: Proc. of 7th International Conference on Network Protocols, vol. 1999, pp. 222–231 (October 1999)Google Scholar
- 7.Bilic, H., Birk, Y., Chirashnya, I., Machulsky, Z.: Deferred segmentation for wire-speed transmission of large TCP frames over standard GbE networks. Hot Interconnects 9, 81–85 (2001)CrossRefGoogle Scholar
- 8.Liu, J., Panda, D.K., Banikazemi, M.: Evaluating the Impact of RDMA on Storage I/O over InfiniBand. In: SAN-03 Workshop (in conjunction with HPCA), (February 2004)Google Scholar
- 9.Hellwagner, H., Ohlenroth, M.: VI architecture communication features and performance on the Gigabit cluster LAN. Future Generation Computer Systems 18(3) (January 2002)Google Scholar
- 10.
- 11.Aiken, S., Grunwald, D., Pleszkun, A.R., Willeke, J.: A Performance Analysis of the iSCSI Protocol. In: Proc. of the 20th IEEE Symposium on Mass Storage Systems 2003 (April 2003)Google Scholar
- 12.
- 13.
- 14.
- 15.Clark, D.D., Jacobson, V., Romkey, J., Salwen, H.: An Analysis of TCP Processing Overhead. IEEE Communications 27(6), 23–29 (1989)CrossRefGoogle Scholar
- 16.Dunkels, A.: Full TCP/IP for 8-Bit Architectures. In: Proc. of the First International Conference on Mobile Applications, Systems and Services, San Francisco, CA, USA (May 2003)Google Scholar
- 17.
- 18.Ghadia, H.: Benefits of full TCP/IP offload in the NFS Services. In: 2003 NFS Industry Conference (September 2003)Google Scholar