The internals of data management systems have largely made a number of incremental (but significant) changes over the last three decades, gradually improving their performance and functionality. But a fundamental shift is now under way because of two major trends. First, driven by power consumption characteristics, processor architecture has gone through a dramatic transformation towards multicore and richer memory organizations.  Furthermore, in the near future even more dramatic changes are likely that will requires software to explicitly deal with processor power consumption.  Second, the role of hard disks is being dramatically diminished as data has started to move nearly permanently to higher levels of the memory hierarchy.  A natural question that then follows is: “Should we design and build the internals of a modern data processing engine in dramatically different ways?” This  talk presents what we have discovered so far in answering this question, with an emphasis on how two core traditional DBMS internal components -- namely  ad hoc join algorithms and query optimization -- must be re-architected for the new hardware reality. The talk concludes that future data processing engines  must be energy-conscious and optimized for data that will largely be resident in main-memory.

Jignesh Patel is a Professor in Computer Sciences at the University of Wisconsin-Madison, which is where he also got his PhD. He has been working on what is now called “big data” for the last two decades. He is the recipient of an NSF Career Award, and multiple IBM and Microsoft faculty awards.