In this paper we describe VectorH: a new SQL-on-Hadoop system built on top of the fast Vectorwise analytical database system. VectorH achieves fault tolerance and scalable data storage by relying on HDFS, extending the state-of-the-art in SQL-on-Hadoop systems by instrumenting the HDFS block replication policy to ensure local reads under most circumstances. VectorH integrates with YARN for workload management, achieving a high degree of elasticity . Even though HDFS is an append-only filesystem, and it supports ordered table storage, VectorH can accommodate trickle updates through Positional Delta Trees (PDTs), a differential update structure that can be queried efficiently. We describe the main technical extensions to single-server Vectorwise that turned it into a Hadoop-based MPP system, in terms of workload management, parallel query optimization and execution, HDFS storage, transaction processing and Spark integration. In the evaluation section we compare VectorH with HAWQ, Impala, SparkSQL and Hive, showing orders of magnitude better performance than these competitors.
data storage, query optimization, parallel query execution, cluster computing, hadoop
Information (theme 2)
Actian Corp., Amsterdam, Netherlands
Actian CWI Research Grant
ACM SIGMOD International Conference on Management of Data
This work was funded by the CWI PPS samenwerking; grant id pps/05050504 - Actian CWI Research Grant
Database Architectures

Switakowski, M, Costea, A, Ionescu, A, Raducanu, B, Bârca, C, Sompolski, J, … Boncz, P.A. (2016). VectorH: taking SQL-on-Hadoop to the next level.

Additional Files
24383B.pdf Author Manuscript , 844kb