SQL Server Internals &

Architecture
Naomi Williams, SQL DBA
Twitter @naomithesqldba
LinkedIn www.linkedin.com/in/nfwilliams/
 Thank You Sponsors!

Free Training at SQL Saturday is not

possible without our Sponsors.

2
 #SQLHELP
Tips

Get a copy, study it and become a master!

http://www.sqlsaturday.com/652/Sessions/Schedule.aspx

3
 Why do I need to know SQL Internals?

 Do you work in SQL

 DBA
 Developer
 BI
 Architect

4
5
Agenda
 ACID
 SQLOS
 Components of database engine
 Components of storage engine
 Cache aging
 Differences of Hekaton

6
 Why is there ACID in my SQL Server?

 ACID properties of Transactions

 Atomic
 Consistent
 Isolated
 Durable

7
Welcome to SQL Server! Where does the
path begin?

8
SELECT SNI (SQL Server Server Network
SQL Server Interface Interface)
aka API  Protocols
 TCP/IP SQLOS
 Shared Memory
 Named Pipes
 Virtual Interface Adapter (VIA)
TDS packets

SNI (SQL Server Client

Network Interface)

9
 SQLOS, Workers and Schedulers

 SQLOS creates a set of schedulers upon start equal to the number of logical CPU’s.
 Each scheduler manages workers, for every task to execute it’s assigned a worker that is in an idle state.
 Workers do not move between schedulers, tasks are never moved between workers. However SQLOS can
create child tasks and assign them different workers.
 Tasks can have one of six states
 Pending; task is waiting for an available worker
 Done; task is completed
 Running; task is currently executing
 Runnable; task is waiting for the scheduler
 Suspended; task is waiting for external event or resource
 Spinloop; task is processing a spinlock 10
 SQLOS, Workers and Schedulers, cont

 Running state
 Suspended queue and suspended state
 Runnable queue and runnable state

11
SELECT SNI (SQL Server Server Network
SQL Server Interface Interface)
aka API  Protocols
 TCP/IP SQLOS
 Shared Memory
 Named Pipes
 Virtual Interface Adapter (VIA)
TDS packets

SNI (SQL Server Client

Network Interface) Buffer Pool
 Language
Language Event  Hash of execution Data Cache
plan

If query hash does Database Engine

not exist in Plan Cmd Parser Optimizer
Cache Plan Cache
Query Tree Query Plan
Does the hash exist
Query Executor in the Plan Cache?
12

YES! Query has plan exists!

Logical Query Tree
Select * from Customers c inner join Orders o on
c.cid=o.cid where o.date=‘2014-12-12’;

Project
(*)

Filter (o.date=‘2014-12-12’)

Inner Join
c.cid=o.cid

Get (orders) as
Get (customers) as C O

13
SELECT SNI (SQL Server Server Network
SQL Server Interface
aka API Interface)
 Protocols
SQLOS
 TCP/IP
 Shared Memory
 Named Pipes
TDS packets  Virtual Interface Adapter (VIA)

SNI (SQL Server Client

Network Interface) Language Event Buffer Pool
Data Cache

Database Engine
Cmd Parser Optimizer
Plan Cache
Query Tree Query Plan

Query Executor
14
Query Optimizer

Parse and logical Binding, do they Generate physical

query tree exist? query plan
Does it make sense and how can I Do the objects exist? Create path in Most efficient plan (path) to the
get there? binary from algebrizer for data.
optimizer.

15
Physical Query Plan

16
SELECT
SQL Server Interface
aka API SNI (SQL Server Server Network Database Engine
Interface) Cmd Parser Optimizer
 Protocols
 TCP/IP SQLOS Query Tree Query Plan

 Shared Memory
 Named Pipes Query Executor
TDS packets
 Virtual Interface Adapter (VIA)
SNI (SQL Server Client
Network Interface) Language Event
Does Data exist in
OLE DB
Cache?
Transaction
Log File Storage Engine Buffer Pool
Access Methods
Transaction Data Cache
Manager
Buffer Manager
Data File Retrieves data from
disk to cache
17 Plan Cache
Insert, Update, Delete
SQL Server Interface
aka API SNI (SQL Server Server Network Database Engine
Interface) Cmd Parser Optimizer
 Protocols
 TCP/IP SQLOS Query Tree Query Plan

 Shared Memory
 Named Pipes Query Executor
TDS packets
 Virtual Interface Adapter (VIA)
SNI (SQL Server Client
Network Interface) Language Event
CheckPoint OLE DB
Transaction
Log File Storage Engine Buffer Pool
Transaction Access Methods Data Cache
Manager

Data File Lock and Log

manager Buffer Manager
18 Plan Cache
WAIT, WAIT WAIT
PageIOLatch_x,
Async_IO_Completion
SOS_Scheduler_Yield , IO_Completion
Async_Network_IO

Pagelatch_x, Latch_x,
Locks Resource_Semaphore
LCK_x, LCK_M_x
Writelog,
LogBuffer
Latches

BackupIO CXPacket

OLEDB IO_Completion

MSQL_DQ
MSQL_XP PreEmptive_OS_

ThreadPool 19
WAIT, WAIT WAIT
SQL Server Interface SOS_Scheduler_Yield
Async_Network_IO
aka API SNI (SQL Server Server Network Database Engine
, CXPacket
Interface) Cmd Parser Optimizer
 Protocols
 TCP/IP Query Tree Query Plan
SQLOS
 Shared Memory
 Named Pipes Query Executor
TDS packets
 Virtual Interface Adapter (VIA)
SNI (SQL Server Client
Network Interface) Language Event
Writelog, Pagelatch_x, Latch_x,
PageIOLatch_x,
LogBuffer Resource_Semaphore
Async_IO_Completion OLE DB
Transaction , IO_Completion
Log File Storage Engine Buffer Pool
Latches
Locks Data Cache
Transaction Access Methods
Manager

Data File Lock and Log

manager Buffer Manager
LCK_x, 20 Plan Cache
LCK_M_x
Data Cache and LRU-K Free Buffer List
Buffer Pool ABC Data File  Every buffer has a SQLOS
header
Data Cache  Last two times the Lazy
page was Writer
0
referenced
 Status information
0
LAZY WRITER

ABC AB

21
Plan Cache and LRU SQLOS
Buffer Pool
Plan Cache Resource
0 Monitor

9 52 730 92
ABC ABC AB

22
Hekaton
SQL Server Interface
aka API SNI (SQL Server Server Network Database Engine
Natively compiled stored
Interface) procedures
Cmd Parser Optimizerand schema
 Protocols
 TCP/IP SQLOS Query Tree Query Plan

 Shared Memory
 Named Pipes Query Executor
TDS packets In-Memory OLTP
 Virtual Interface Adapter (VIA)
SNI (SQL Server Client Compiler
Network Interface) Language Event
OLE DB
Transaction Memory Optimized
Log File Tables and Indexes
Storage Engine Buffer Pool
Transaction Access Methods Data Cache
Manager

Data File Lock and Log

manager Buffer Manager
23 Plan Cache
Summary
 ACID
 SQLOS
 Components of database engine
 Components of storage engine
 Cache aging
 Differences of Hekaton

24