2016-06-01
VectorH: taking SQL-on-Hadoop to the next level
Publication
Publication
Presented at the
ACM SIGMOD International Conference on Management of Data, San Francisco
In this paper we describe VectorH: a new SQL-on-Hadoop system built on top of the fast Vectorwise analytical database system. VectorH achieves fault tolerance and scalable data storage by relying on HDFS, extending the state-of-the-art in SQL-on-Hadoop systems by instrumenting the HDFS block replication policy to ensure local reads under most circumstances. VectorH integrates with YARN for workload management, achieving a high degree of elasticity .
Even though HDFS is an append-only filesystem, and it supports ordered table storage, VectorH can accommodate trickle updates through Positional Delta Trees (PDTs), a differential update structure that can be queried efficiently.
We describe the main technical extensions to single-server Vectorwise that turned it into a Hadoop-based
MPP system, in terms of workload management, parallel query optimization and execution, HDFS storage, transaction processing and Spark integration. In the evaluation section we compare VectorH with HAWQ, Impala, SparkSQL and Hive, showing orders of magnitude better performance than these competitors.
Additional Metadata | |
---|---|
, , , , | |
Actian Corp., Amsterdam, Netherlands | |
Actian CWI Research Grant | |
ACM SIGMOD International Conference on Management of Data | |
Organisation | Database Architectures |
Switakowski, M., Costea, A., Ionescu, A., Raducanu, B., Bârca, C., Sompolski, J., … Boncz, P. (2016). VectorH: taking SQL-on-Hadoop to the next level. |