Thread Transparency in Information Flow Middleware
Applications that process continuous information flows are challenging to write because the application programmer must deal with flow-specific concurrency and timing requirements, necessitating the explicit management of threads, synchronization, scheduling and timing. We believe that middleware can ease this burden, but middleware that supports control-flow centric interaction models such as remote method invocation does not match the structure of these applications. Indeed, it abstracts away from the very things that the information-flow centric programmer must control
We are defining Infopipes as a high-level abstraction for information flow, and we are developing a middleware framework that supports this abstraction directly. Infopipes handle the complexities associated with control flow and multi-threading, relieving the programmer of this task. Starting from a high-level description of an information flow configuration, the framework determines which parts of a pipeline require separate threads or coroutines, and handles synchronization transparently to the application programmer. The framework also gives the programmer the freedom to write or reuse components in a passive style, even though the configuration will actually require the use of a thread or coroutine. Conversely, it is possible to write a component using a thread and know that the thread will be eliminated if it is not needed in a pipeline. This allows the most appropriate programming model to be chosen for a given task, and existing code to be reused irrespective of its activity model
KeywordsControl Event Pipeline Stage Remote Method Invocation Thread Creation Push Mode
- 1.A. P. Black. An asymmetric stream communication system. In Proceedings of the Ninth ACM Symposium on Operating Systems Principles, pages 4–10, October 1983.Google Scholar
- 2.A. P. Black, J. Huang, and J. Walpole. Reifying communication at the application level. In Proceedings of the International Workshop on Multimedia Middleware. ACM, October 2001. Also available as OGI technical report CSE-01-006.Google Scholar
- 3.G. S. Blair, G. Coulson, P. Robin, and M. Papathomas. An architecture for next-generation middleware. In International Conference on Distributed Systems Platforms and Open Distributed Processing (Middleware), pages 191–206. IFIP, September 1998.Google Scholar
- 4.J.-P. Briot, R. Guearraoui, and K.-P. Löhr. Concurrency and distribution in object-oriented programming. ACM Computing Surveys, 30(3), September 1998.Google Scholar
- 6.S. Cen, C. Pu, R. Staehli, C. Cowan, and J. Walpole. A distributed real-time MPEG video audio player. In Proceedings of the Fifth International Workshop on Network and Operating Systems Support for Digital Audio and Video, volume 1018 of Lecture Notes in Computer Science, pages 142–153. Springer Verlag, April 1995.Google Scholar
- 7.G. Eisenhauer, F. Bustamante, and K. Schwan. Event services for high performance computing. In International Conference on High Performance Distributed Computing (HPDC), August 2000.Google Scholar
- 8.A. Goel, D. Steere, C. Pu, and J. Walpole. Adaptive resource management via modular feedback control. Technical Report CSE-99-003, Oregon Graduate Institute, January 1999.Google Scholar
- 9.J. Huang, A. P. Black, J. Walpole, and C. Pu. Infopipes — an abstraction for information flow. In ECOOP 2001 Workshop on The Next 700 Distributed Object Systems, June 2001. Also available as OGI technical report CSE-01-007.Google Scholar
- 11.C. Isert and K. Schwan. ACDS: Adapting computational data streams for high performance computing. In International Parallel and Distributed Processing Symposium (IPDPS), May 2000.Google Scholar
- 12.R. Koster and T. Kramp. A multithreading platform for multimedia applications. In Proceedings of Multimedia Computing and Networking 2001. SPIE, January 2001.Google Scholar
- 13.R. Koster and T. Kramp. Using message-based threading for multimedia applications. In Proceedings of the International Conference on Multimedia and Expo (ICME). IEEE, August 2001.Google Scholar
- 14.T. Kramp and G. Coulson. The design of a flexible communications framework for next-generation middleware. In Proceedings of the Second International Symposium on Distributed Objects and Applications (DOA). IEEE, September 2000.Google Scholar
- 15.T. Kramp and R. Koster. Flexible event-based threading for QoS-supporting middleware. In Proceedings of the Second International Working Conference on Distributed Applications and Interoperable Systems (DAIS). IFIP, July 1999.Google Scholar
- 16.C. Krasic and J. Walpole. QoS scalability for streamed media delivery. Technical Report CSE-99-011, Oregon Graduate Institute, September 1999.Google Scholar
- 17.F. Kuhns, C. O’Ryan, D. C. Schmidt, O. Othman, and J. Parsons. The design and performance of a pluggable protocols framework for object request broker middleware. In Proceedings of the sixth IFIP International Workshop on Protocols for High-Speed Networks (PfHSN), August 1999.Google Scholar
- 18.D. Lea. Concurrent Programming in Java. Addison-Wesley, 1997.Google Scholar
- 20.J. Magee, N. Dulay, and J. Kramer. Regis: A constructive development environment for distributed programs. Distributed Systems Engineering Journal, 1(5), September 1994.Google Scholar
- 21.Mircosoft. DirectX 8.0: DirectShow overview. http://msdn.microsoft.com/library/psdk/directx/dx8_c/ds/Oview/about_dshow.htm, January 2001.
- 22.J. P. Morrison. Flow-Based Programming: A New Approach to Application Development. Van Nostrand Reinhold, July 1994.Google Scholar
- 23.D. Mosberger and L. L. Peterson. Making paths explicit in the Scout operating system. In Proceedings of the second USENIX symposium on Operating systems design and implementation (OSDI). USENIX, October 1996.Google Scholar
- 24.S. Mungee, N. Surendran, and D. C. Schmidt. The design and performance of a CORBA audio/video streaming service. In HICSS-32 International Conference on System Sciences, minitrack on Multimedia DBMS and WWW, January 1999.Google Scholar
- 25.OMG. CORBA telecoms specification. http://www.omg.org/corba/ctfull.html, June 1998. formal/98-07-12.
- 26.J. Ousterhout. Why threads are a bad idea (for most purposes), 1996. Invited talk given at USENIX Technical Conference, available at http://www.scriptics.com/people/john.ousterhout/threads.ps.
- 27.T. Plagemann and B. Plattner. CoRA: A heuristic for protocol configuration and resource allocation. In Proceedings of the Workshop on Protocols for High-Speed Networks. IFIP, August 1994.Google Scholar
- 28.C. Pu, K. Schwan, and J. Walpole. Infosphere project: System support for information flow applications. ACM SIGMOD Record, 30(1), March 2001.Google Scholar
- 29.H. O. Rafaelsen and F. Eliassen. Trading and negotiating stream bindings. In Proceedings of the Second International Conference on Distributed Systems Platforms and Open Distributed Processing (Middleware), LNCS 1795, pages 273–288. IFIP/ACM, Springer, April 2000.Google Scholar
- 30.RealNetworks. Documentation of RealSystem G2 SDK, gold r4 release. http://www.realnetworks.com/devzone/tools/index.html, May 2000.
- 31.D. Steere, A. Goel, J. Gruenberg, D. McNamee, C. Pu, and J. Walpole. A feedbackdriven proportion allocator for real-rate scheduling. In Proceedings of the Third Symposium on Operating Systems Design and Implementation, pages 145–158, February 1999.Google Scholar
- 32.W. Taymans. GStreamer application development manual. http://www.gstreamer.net/documentation.shtml, mtJanuary 2001.
- 33.R. van Renesse. Goal-oriented programming, or composition using events, or threads considered harmful. In Proceeding of the 8th ACM SIGOPS European Workshop, September 1998.Google Scholar
- 34.R. van Renesse, K. Birman, M. Hayden, A. Vaysburd, and D. Karr. Building adaptive systems using Ensemble. Technical Report TR97-1638, Computer Science Department, Cornell University, 1997.Google Scholar
- 35.R. Vanegas, J. A. Zinky, J. P. Loyall, D. A. Karr, R. E. Schantz, and D. E. Bakken. QuO’s runtime support for quality of service in distributed objects. In Proceedings of the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing (Middleware’98). Springer Verlag, September 1998.Google Scholar
- 36.J. Walpole, R. Koster, S. Cen, C. Cowan, D. Maier, D. McNamee, C. Pu, D. Steere, and L. Yu. A player for adaptive mpeg video streaming over the internet. In Proceedings of the 26th Applied Imagery Pattern Recognition Workshop (AIPR-97). SPIE, October 1997.Google Scholar