Breaking the memory wall in MonetDB
In the past decades, advances in speed of commodity CPUs have far outpaced advances in RAM latency. Main-memory access has therefore become a performance bottleneck for many computer applications; a phenomenon that is widely known as the "memory wall." In this paper, we report how research around the MonetDB database system has led to a redesign of database architecture in order to take advantage of modern hardware, and in particular to avoid hitting the memory wall. This encompasses (i) a redesign of the query execution model to better exploit pipelined CPU architectures and CPU instruction caches; (ii) the use of columnar rather than row-wise data storage to better exploit CPU data caches; (iii) the design of new cache-conscious query processing algorithms; and (iv) the design and automatic calibration of memory cost models to choose and tune these cache-conscious algorithms in the query optimizer.
|Keywords||MonetDB, database architecture, column-store, memory wall, hierarchical memory|
|ACM||Systems (acm H.2.4)|
|THEME||Information (theme 2)|
|Journal||Communications of the ACM|
Boncz, P.A, Kersten, M.L, & Manegold, S. (2008). Breaking the memory wall in MonetDB. Communications of the ACM, 51(12), 77–85.