Databases: The Next Generation

Presented by Larry Garfield

Databases: The Next Generation

Larry Garfield Senior Architect, Palantir.net Database maintainer General gadfly

Drupal 6 database layer

ext/mysql, ext/mysqli, ext/pgsql
PHP 3 / MySQL 3 MySQL-centric

Limitations
No DB-specific optimizations
No automated type handling No master/slave replication No transaction support No dynamic queries

DB Fail!

PDO
Formal prepared statements

Type-agnostic queries

Object-oriented API Unified access API (not abstraction API)

Databases: Drupal 7
Built on PDO
Fully object-oriented Procedural wrappers New driver model More DB-agnostic Oodles of goodies

Did someone say goodies?

Automatic DB-specific optimizations
Automated type handling master/slave replication transaction support dynamic query builders Core: MySQL, PostgreSQL, SQLite Contrib: Oracle

New concepts
Database targets
Placeholders Fluent API

All modifier queries Never write INSERT again

Targets
Alternate database connection
May be used or not Fallback to default Master/slave replication Multi-slave configuration

Configuration
$db_url; $databases['default']['default'] = array( 'driver' => 'mysql', 'database' => 'drupaldb', 'username' => 'username', 'password' => 'secret', 'host' => 'localhost', );

Master/slave Configuration
$databases['default']['default'] = array( 'driver' => 'mysql', 'database' => 'drupaldb1', // ... 'host' => 'dbserver1', ); $databases['default']['slave'][] = array( 'driver' => 'mysql', 'database' => 'drupaldb2', // ... 'host' => 'dbserver2', ); $databases['default']['slave'][] = array( 'driver' => 'mysql', 'database' => 'drupaldb3', // ... 'host' => 'dbserver3', );

Types of queries
Static
Dynamic Insert Update Delete Merge Truncate

Static queries
Doesn't change based on environment
Takes named placeholders

Self-documenting Arbitrary order Always an array

DB_Query()

// Drupal 6 $result = db_query("SELECT nid, title FROM {node} WHERE uid = %d AND type = '%s'", 5, 'page'); // Drupal 7 $result = db_query("SELECT nid, title FROM {node} WHERE uid = :uid AND type = :type", array( ':uid' => 5, ':type' => 'page', ));

Auto-expanding placeholders
// This code: db_query("SELECT * FROM {node} WHERE nid IN (:nids)", array( ':nids' => array(13, 42, 144), ); // Auto-expands into the equivalent of: db_query("SELECT * FROM {node} WHERE nid IN (13, 42, 144)");

A look inside...
function db_query($query, $args = array(), $options = array()) { if (empty($options['target'])) { $options['target'] = 'default'; } $db = Database::getConnection($options['target']); $db->query($query, $args, $options); }

Result Sets
$result = db_query("SELECT, nid, title FROM {node}"); foreach ($result as $record) { // Do stuff with $record, which is an object } $result = db_query("SELECT ...", array(), array( 'fetch' => PDO::FETCH_ASSOC, )); foreach ($result as $record) { // Do stuff with $record, which is an array }

Query Options

db_query("Select, nid, title FROM {node}", array(), array( 'target' => 'slave', 'fetch' => PDO::FETCH_ASSOC, ));

Fetch Options
// Fetch into a stdClass object. $options['fetch'] = PDO::FETCH_OBJ; // Fetch into an associative array. $options['fetch'] = PDO::FETCH_ASSOC; // Fetch into a numeric array. $options['fetch'] = PDO::FETCH_NUM; // Fetch into a dual-keyed array. $options['fetch'] = PDO::FETCH_BOTH; // Fetch into a custom class. $options['fetch'] = 'MyClass';

Fetch Data, Druplicon!

$record = $result->fetchObject(); $record = $result->fetchAssoc(); $field = $result->fetchField($column_index); $records = $result->fetchAll(); $records = $result->fetchAllAssoc('nid'); $records = $result->fetchAllKeyed(); $ids = $result->fetchCol(); $nodes = db_query("SELECT nid, title FROM {node}") ->fetchAllKeyed();

Dynamic queries
SELECT queries using query builder
db_select() Only use if you really need dynamic behavior

Extra building overhead

Extenders for Pager and Tablesort

hook_query_alter()
All dynamic SELECT statements
Query tagging Structured data is easy to alter Access internal query structure hook_db_rewrite_sql()

Dynamic queries
$query = db_select('users', 'u', $options) ->condition('u.uid', 0, '<>') ->fields('u', array('uid', 'name', 'created')) ->orderBy('u.created') ->range(0, 50) ->addTag('foo'); $result = $query->execute(); function mymod_query_foo_alter($query) { $query->range(0, 10); } // SELECT u.uid AS uid, u.name AS name, u.created AS created // FROM {users} u // WHERE u.uid <> 0 // ORDER BY u.created // LIMIT 0, 10

More dynamic queries

$query = db_select('node', 'n'); $query->join('users', 'u', 'n.uid = u.uid'); $query->addField('u', 'name', 'user_name'); $result = $query ->fields('n', 'nid', 'title') ->where('SUBSTRING(n.title,0,5) = :prefix', array( ':prefix' => $prefix, )) ->orderBy('n.title') ->addTag('node_access') ->execute();

Extenders
$query = db_select('node', 'n') ->extend('PagerDefault'); $query ->fields('n', array('nid', 'title') // This method is part of PagerDefault! ->limit(5) ->orderBy('title'); $result = $query->execute();

Insert, Update, Delete

db_insert(), db_update(), db_delete()
Builder is required

Cross-database BLOB and LOB

Fully chainable "Fluent API"

Insert
// Drupal 6 db_query("INSERT INTO {mytable} (intvar, stringvar, floatvar) VALUES (%d, '%s', %f)", 5, 'hello world', 3.14); $id = db_last_insert_id(); // Drupal 7 $id = db_insert('mytable') ->fields(array( 'intvar' => 5, 'stringvar' => 'hello world', 'floatvar' => 3.14, )) ->execute();

Multi-Insert

$query = db_insert('mytable') ->fields(array('field1', 'field2', 'field3'));

foreach ($imported_data as $record) { $query->values($record); } $query->execute();

Insert FROM
// Build a SELECT query. $query = db_select('node', 'n'); // Join to the users table. $query->join('users', 'u', 'n.uid = u.uid'); // Add the fields we want. $query->addField('n','nid'); $query->addField('u','name'); // Add a condition to only get page nodes. $query->condition('type', 'page'); // Perform the insert. db_insert('mytable') ->from($query) ->execute();

Update

// Drupal 6 db_query("UPDATE {node} SET title='%s', status=%d WHERE uid=%d", 'hello world', 1, 5); // Drupal 7 db_update('node') ->fields(array( 'title' => 'hello world', 'status' => 1, )) ->condition('uid', 5) ->execute();

Delete

// Drupal 6 db_query("DELETE FROM {node} WHERE uid=%d AND created < %d", 5, time() - 3600); // Drupal 7 db_delete('node') ->condition('uid', 5) ->condition('created', REQUEST_TIME - 3600, '<') ->execute();

Merge
db_merge()
"INSERT, UPDATE, whichever" Update to same value or new Database-specific implementation

MySQL: INSERT ... ON DUPLICATE KEY UPDATE ... PostgreSQL: Needs function...

Merge (Just set it)

db_merge('example') ->key(array('name' => $name)) ->fields(array( 'field1' => $value1, 'field2' => $value2, )) ->execute();

Merge (Counter increment)

db_merge('example') ->key(array('name' => $name)) ->fields(array( 'field1' => $value1, 'field2' => $value2, )) ->expression('field1', 'field1 + :inc', array(':inc' => 1)) ->execute();

Transactions
Force multiple queries to run atomically
Only supported on some databases

(Silly MyISAM)

Use exceptions We have our own wrapper

Transactions
function my_transaction_function() { try { $txn = db_transaction(); $id = db_insert('example') ->fields(array( 'field1' => 'mystring', 'field2' => 5, )) ->execute(); my_other_function($id); return $id; } catch (Exception $e) { $transaction->rollback('node', $e->getMessage(), array(), WATCHDOG_ERROR); } }

Technical MaNual

http://drupal.org/node/310069

Your command crew

Database maintainer

PostgreSQL

Larry Garfield (Crell)

Damien Tournoud (DamZ)

MySQL

Josh Waihi (fiasco)

Damien Tournoud (DamZ) Kroly Ngyesi (chx)

Larry Garfield (Crell) David Strauss (David Strauss)

SQLite

Questions?

What did you think?

Step 1) Locate this session on the DCSF site http://sf2010.drupal.org/conference/schedule Step 2) Click the Take Survey link