Data-centric computing with Netezza Architecture

DISC reading group September 24, 2007

High Level Points

Supercomputer use model today:
Compile, submit, wait Does a poor job of taking advantage of human insight available in interactive models

Large datasets can be interactively processed using Netezza

What is Netezza?

Essentially: A big, fast SQL database

What is Netezza?

Frontend provides SQL interface Backend is a large rack of specialized blades

Custom Backend Blades

Commodity CPU, NIC, disk Custom FPGA replaces disk interface

Can do basic filtering in hardware, i.e., stream processing before data hits main memory

Division of Data
Database distributed across multiple (100+) SPUs Each SPU controls, manages its slice of DB

No info on data management, replciation, etc.

Division of Labor
SPU FPGA handles basic filtering tasks SPU CPU handles record level processing: filtering, parsing, projecting, logging, etc. SPU CPU handles most operations on intermediate results: sorts, joins, aggregates Frontend CPU handles remaining operation

>>> Processing close to disk

What can this be used for?

Paper gives 3 examples:
Citation graph processing Search for particular structure in electrical netlist Word meaning disambiguation through search of ontology

Citation graph example

Look through large, sparse graph (16 million nodes, 388 million edges) Find both strong (direct edge) and weak couplings (e.g., two papers cite the same work) Essentially same code for workstation and Netezza no need to expose parallel architecture Workstation DNF; 80-100x speedup on smaller tests

IC netlist example
Flattened netlist of 3.5 million transistors, 10 million wires Search for AND structure

IC example results
Combinatorial explosion makes directly joining all possibilities for each element impossible Can constrain better using fanouts of signals internal to the circuit Individual SQL queries for finding possible matches for the individual transistors took under 10 seconds Found all uses of the AND macro, as well as many other (1300+) identical structures generated through other means

Ontology example
Expand out all possible interpretations of a phrase Ontology specifies lexical elements, IS-A relations, concepts, and constraints on concepts Goal is to search the space, expand concepts to find all matches to given phrase

Ontology results
Partially unfolded ontology
Greatly expands database size, but reduces iterations / recursions

Recoded ontology triples as integers 5.58 sec. vs. 262 sec. can pipeline multiple queries

Issues
Works if you can reduce your problem to SQL queries All of the problems were based on graph expansion / exploration how about other domains? Issues of database partitioning? How does arbitrary slicing across 108 blades affect performance / scalability, esp. for non-sparse problems? Strawman comparison to workstation class machine: how does a traditional DB server / storage cluster compare?