Scaling-Up and Speeding-Up Video Analytics Inside Database Engine
- Qiming ChenAffiliated withHP Labs
- , Meichun HsuAffiliated withHP Labs
- , Rui LiuAffiliated withHP Labs, Hewlett Packard Co.
- , Weihong WangAffiliated withHP Labs, Hewlett Packard Co.
Most conventional video processing platforms treat database merely as a storage engine rather than a computation engine, which causes inefficient data access and massive amount of data movement. Motivated by providing a convergent platform, we push down video processing to the database engine using User Defined Functions (UDFs).
However, the existing UDF technology suffers from two major limitations. First, a UDF cannot take a set of tuples as input or as output, which restricts the modeling capability for complex applications, and the tuple-wise pipelined UDF execution often leads to inefficiency and rules out the potential for enabling data-parallel computation inside the function. Next, the UDFs coded in non-SQL language such as C, either involve hard-to-follow DBMS internal system calls for interacting with the query executor, or sacrifice performance by converting input objects to strings.
To solve the above problems, we realized the notion of Relation Valued Function (RVF) in an industry-scale database engine. With tuple-set input and output, an RVF can have enhanced modeling power, efficiency and in-function data-parallel computation potential. To have RVF execution interact with the query engine efficiently, we introduced the notion of RVF invocation patterns and based on that developed RVF containers for focused system support.
We have prototyped these mechanisms on the Postgres database engine, and tested their power with Support Vector Machine (SVM) classification and learning, the most widely used analytics model for video understanding. Our experience reveals the value of the proposed approach in multiple dimensions: modeling capability, efficiency, in-function data-parallelism with multi-core CPUs, as well as usability; all these are fundamental to converging data-intensive analytics and data management.
- Scaling-Up and Speeding-Up Video Analytics Inside Database Engine
- Book Title
- Database and Expert Systems Applications
- Book Subtitle
- 20th International Conference, DEXA 2009, Linz, Austria, August 31 – September 4, 2009. Proceedings
- pp 244-254
- Print ISBN
- Online ISBN
- Series Title
- Lecture Notes in Computer Science
- Series Volume
- Series ISSN
- Springer Berlin Heidelberg
- Copyright Holder
- Springer-Verlag Berlin Heidelberg
- Additional Links
- Industry Sectors
- eBook Packages
- Editor Affiliations
- 16. Nanyang Technological University
- 17. University of Linz
- Author Affiliations
- 18. HP Labs, Palo Alto, California, USA
- 19. HP Labs, Hewlett Packard Co., Beijing, China
To view the rest of this content please follow the download PDF link above.