SQL Server MVP Deep Dives, Volume 2

Copyright

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact

Special Sales Department

Manning Publications Co.

20 Baldwin Road

PO Box 261

Shelter Island, NY 11964

Email: 

orders@manning.com

©2012 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

Printed in the United States of America

1 2 3 4 5 6 7 8 9 10 – MAL – 16 15 14 13 12 11

Dedication

To all the children of Operation Smile

Brief Table of Contents

Copyright

Brief Table of Contents

Table of Contents

MVP authors and their chapters

Preface

Acknowledgments

About Operation Smile

About this book

About the Editors

About SQL Server MVPs

1. Architecture

Chapter 1. Where are my keys?

Chapter 2. Yes, we are all individuals A look at uniqueness in the world of SQL

Chapter 3. Architectural growth pains

Chapter 4. Characteristics of a great relational database

Chapter 5. Storage design considerations

Chapter 6. Generalization: the key to a well-designed schema

2. Database administration

Chapter 7. Increasing availability through testing

Chapter 8. Page restores

Chapter 9. Capacity planning

Chapter 10. Discovering your servers with PowerShell and SMO

Chapter 11. Will the real Mr. Smith please stand up?

Chapter 12. Build your own SQL Server 2008 performance dashboard

Chapter 13. SQL Server cost recovery

Chapter 14. Best practice compliance with Policy-Based Management

Chapter 15. Using SQL Server Management Studio to the fullest

Chapter 16. Multiserver management and Utility Explorer—best tools for the DBA

Chapter 17. Top 10 SQL Server admin student misconceptions

Chapter 18. High availability of SQL Server in the context of Service Level Agreements

3. Database development

Chapter 19. T-SQL: bad habits to kick

Chapter 20. Death by UDF

Chapter 21. Using regular expressions in SSMS

Chapter 22. SQL Server Denali: what’s coming next in T-SQL

Chapter 23. Creating your own data type

Chapter 24. Extracting data with regular expressions

Chapter 25. Relational division

Chapter 26. SQL FILESTREAM: to BLOB or not to BLOB

Chapter 27. Writing unit tests for Transact-SQL

Chapter 28. Getting asynchronous with Service Broker

Chapter 29. Effective use of HierarchyId

Chapter 30. Let Service Broker help you scale your application

4. Performance tuning and optimization

Chapter 31. Hardware 201: selecting and sizing database server hardware

Chapter 32. Parameter sniffing: your best friend... except when it isn’t

Chapter 33. Investigating the plan cache

Chapter 34. What are you waiting for? An introduction to waits and queues

Chapter 35. You see sets, and I see loops

Chapter 36. Performance-tuning the transaction log for OLTP workloads

Chapter 37. Strategies for unraveling tangled code

Chapter 38. Using PAL to analyze SQL Server performance

Chapter 39. Tuning JDBC for SQL Server

5. Business intelligence

Chapter 40. Creating a formal Reporting Services report part library

Chapter 41. Improving report layout and visualization

Chapter 42. Developing sharable managed code expressions in SSRS

Chapter 43. Designing reports with custom MDX queries

Chapter 44. Building a scale-out Reporting Services farm

Chapter 45. Creating SSRS reports from SSAS

Chapter 46. Optimizing SSIS for dimensional data loads

Chapter 47. SSIS configurations management

Chapter 48. Exploring different types of enumerators in the SSIS Foreach Loop container

Chapter 49. Late-arriving dimensions in SSIS

Chapter 50. Why automate tasks with SSIS?

Chapter 51. Extending SSIS using the Script component

Chapter 52. ETL design checklist

Chapter 53. Autogenerating SSAS cubes

Chapter 54. Scripting SSAS databases – AMO and PowerShell, Better Together

Chapter 55. Managing context in MDX

Chapter 56. Using time intelligence functions in PowerPivot

Chapter 57. Easy BI with Silverlight PivotViewer

Chapter 58. Excel as a BI frontend tool

Chapter 59. Real-time BI with StreamInsight

Chapter 60. BI solution development design considerations

Index

List of Figures

List of Tables

List of Listings

Table of Contents

Copyright

Brief Table of Contents

Table of Contents

MVP authors and their chapters

Preface

Acknowledgments

About Operation Smile

About this book

About the Editors

About SQL Server MVPs

1. Architecture

Chapter 1. Where are my keys?

Keys in the relational model

The debate

The arguments

Pro artificial keys

Pro natural keys

Additional considerations

Natural keys assist the optimizer

Artificial keys are the de facto standard

Modularity, portability, and foreseeing the future

IDENTITY columns may result in value gaps and run out of values

Recommendations

Simplicity and aesthetics

Summary

About the author

Chapter 2. Yes, we are all individuals A look at uniqueness in the world of SQL

Introducing uniqueness

Constrained to uniqueness

Primary keys

Unique constraints

Unique indexes

Unique constraint or unique index?

Advantages of the unique index

Advantages of the unique constraint

Uniqueness in results

The good and the bad of DISTINCT

DISTINCT or GROUP BY

Are they needed at all?

Unnecessary grouping

Being guided by that error

Summary

About the author

Chapter 3. Architectural growth pains

Manage your data types

IDENTITY case in point

Database design and scalability

Naming conventions

Inconsistent design

Normalization

Overnormalized

Undernormalized

Primary keys and foreign keys

GUIDs as primary keys

System-generated integers as primary keys

Generating your own ID values

Indexes

Underindexing

Overindexing

Maintenance

Fill factor

Summary

About the author

Chapter 4. Characteristics of a great relational database

Coherent

Standards based

Reasonable names and data types

Cohesive

Needs little documentation

Normal

Fundamentally sound

Documented

Secure

Encapsulated

Well performing

Summary

About the author

Chapter 5. Storage design considerations

Selecting the correct RAID type

RAID 0

RAID 1

RAID 5

RAID 6

RAID 10

RAID 50

When to use RAID 5

When to use RAID 6

When to use RAID 10

File placement

Index files

Transaction log files

tempdb database

Disk alignment

Correcting disk alignment on Windows 2003 and earlier

Correcting disk alignment in Windows 2008 and later

Correcting after the partition has been created

Aligning on the array

Snapshots

Snapshots with a VSS-enabled storage array

Snapshots with a non-VSS-enabled storage array

Snapshots as a backup process

Using snapshots to present storage to downstream environments

Clones

Summary

About the author

Chapter 6. Generalization: the key to a well-designed schema

A place for normalization

Lessons from the UIX discipline

Generalization defined

Benefits of generalization

Summary

About the author

2. Database administration

Chapter 7. Increasing availability through testing

Testing—it’s not just for application functionality

The missing link

Knowledge is power

Test early, test often

Automated versus manual testing

What needs to be tested?

First things first

Remember the big picture, too

Summary

About the author

Chapter 8. Page restores

Restore granularities

Requirements and limitations

Recovery model and availability of log backups

SQL Server Edition

Page type of the damaged page

Performing a page restore

What’s coming?

Summary

About the author

Chapter 9. Capacity planning

What is capacity planning?

Gathering current database disk space usage

Performance metrics

Summary

About the author

Chapter 10. Discovering your servers with PowerShell and SMO

Using PowerShell and Excel

Using SMO with PowerShell

Collecting instance and machine information

Collecting SQL Agent job information

Collecting database information

Summary

About the author

Chapter 11. Will the real Mr. Smith please stand up?

Personally identifiable data

Today’s superhero: the DBA

Our superpowers

Tools of the trade

Summary

About the author

Chapter 12. Build your own SQL Server 2008 performance dashboard

DMVs as the source of performance-related information

Using SQLCLR to get the performance counter values

Sample solution for performance monitoring

Use Reporting Services for performance monitoring

Some ideas to improve the solution

Summary

About the author

Chapter 13. SQL Server cost recovery

The context for SQL Server as a Service

What’s SQL Server as a Service?

An introduction to chargebacks

Implementing a chargeback model

Summary

About the author

Chapter 14. Best practice compliance with Policy-Based Management

The context for contemporary database administration

The importance of best practice compliance

Central Management Servers

Policy-Based Management

Surface area configuration

Sysadmin membership

Policy-Based Management with Central Management Servers

Summary

About the author

Chapter 15. Using SQL Server Management Studio to the fullest

Querying many servers at once

Creating and using a scripting solution with templates

Scripting multiple objects and now data, too

Summary

About the author

Chapter 16. Multiserver management and Utility Explorer—best tools for the DBA

SQL Server 2008 R2 tools for the DBA

Tools of the trade

Managing multiple instances using Utility Control Point

Multiserver management and administration

Best practices

Summary

About the author

Chapter 17. Top 10 SQL Server admin student misconceptions

Simple recovery model

Default collation

Table-level backups

Using replication for high availability

Timing query performance

Shrinking databases

Auditing login access

Tail-log backups

Database defaults

Difficulty

Summary

About the author

Chapter 18. High availability of SQL Server in the context of Service Level Agreements

High availability—a definition

Types of unavailability

Unavailability indicators

High availability options in SQL Server

Service Level Agreement

Measurement indicators

The structure of a Service Level Agreement

Service Level Agreements: the context for high availability

Summary

Useful links

About the author

3. Database development

Chapter 19. T-SQL: bad habits to kick

SELECT *

Declaring VARCHAR without length

Not choosing the right data type

Mishandling date range queries

Making assumptions about ORDER BY

Summary

About the author

Chapter 20. Death by UDF

Poor estimates

Row-by-row processing

What can you do about it?

Inline table valued function solution

Set-based solution

What about code reuse?

One last example of how bad scalar UDFs can be

Summary

About the author

Chapter 21. Using regular expressions in SSMS

Eliminating blank lines

Removing extra newline characters

Collapsing multiple lines into a single line

Using the beginning-of-line metacharacter

Using the end-of-line metacharacter

Summary

About the author

Chapter 22. SQL Server Denali: what’s coming next in T-SQL

OFFSET and FETCH

SQL Server 2005 and 2008 solution

SQL Server Denali solution

Comparing execution plan results

SEQUENCE

Tips for using SEQUENCE

Restrictions

EXECUTE...WITH RESULT SETS

THROW

Summary

About the author

Chapter 23. Creating your own data type

Anatomy of a CLR user-defined type

...But do you need it at all?

Representations and conversions

How about NULL?

Building the data type: the bare basics

Starting the project

Adding the fields for the native representation

Editing the signature

Converting between .NET and text

Converting between .NET and serialized

Handling NULLs

Using the data type

Summary

About the author

Chapter 24. Extracting data with regular expressions

Understanding before coding

Background

An incredibly brief introduction to regular expressions, matches, and groups

Regular expressions and SQL Server

Regular expressions and the .NET Framework

The solution

The core

The SQL CLR user-defined function

The SSIS script component

Homework

Summary

About the author

Chapter 25. Relational division

Why use relational division?

Defining relational division

Background

Sample data for two simple cases

Comparison charts

Let’s go on with the real stuff

Set-based solution to common relational division

Does one query exist for all types of relational division?

Summary

About the author

Chapter 26. SQL FILESTREAM: to BLOB or not to BLOB

To FILESTREAM or not to FILESTREAM

Configuring FILESTREAM in SQL Server

Operating system configuration

SQL Server configuration

Database configuration

Creating a table that uses FILESTREAM

Things to consider

How do I use FILESTREAM?

Summary

About the author

Chapter 27. Writing unit tests for Transact-SQL

Unit test basics

Unit test for databases

T-SQL unit test walkthrough

Automating unit test execution

Summary

About the author

Chapter 28. Getting asynchronous with Service Broker

The Service Broker usage template

Creating Service Broker objects

Summary

About the author

Chapter 29. Effective use of HierarchyId

Hierarchies in a database

Introduction to the HierarchyId data type

Using the HierarchyId data type

Physical HierarchyId data organization

Effective indexing

Depth-first indexes

Breadth-first indexes

More information about HierarchyId

Summary

About the author

Chapter 30. Let Service Broker help you scale your application

Scalable solutions

Service Broker objects

Security

Message types

Contracts

Queues

Services

Conversations

Endpoints

Routes

Remote service binding

ETL trigger demonstration

Summary

About the author

4. Performance tuning and optimization

Chapter 31. Hardware 201: selecting and sizing database server hardware

Why database server hardware is important

Scaling up or scaling out

SQL Server and hardware selection

Database server–specific hardware factors

Intel vs. AMD processors

Memory recommendations

Traditional storage subsystems

New developments in storage subsystems

Benchmarking and sizing tools

Summary

About the author

Chapter 32. Parameter sniffing: your best friend... except when it isn’t

Understanding parameter sniffing

Parameter sniffing gone wrong

Dealing with bad parameter sniffing

OPTIMIZE FOR

WITH RECOMPILE

Local variables

Plan guides

Turn off parameter sniffing

Summary

About the author

Chapter 33. Investigating the plan cache

Plan cache dynamic management objects

sys.dm_exec_cached_plans

sys.dm_exec_query_plan

Investigating missing indexes

Investigating index usage

Investigating operations

Investigating index scans

Investigating parameters

Plan cache considerations

Summary

About the author

Chapter 34. What are you waiting for? An introduction to waits and queues

Introduction to total response time

What are wait stats?

Why use wait stats?

Wait type categories

The execution model

Viewing and reporting on wait statistics

Calculating wait time: signal waits vs. resource waits

Correlating performance data: putting it together

General I/O issues

Buffer I/O latch issues

Blocking and locking

CPU pressure

Parallelism

Memory pressure

Summary

About the author

Chapter 35. You see sets, and I see loops

What loops?

The loop perspective

Loops in a query execution plan

Loops in complex queries

User-defined scalar functions in implicit loops

Merging multiple loops into one

Parallelizing loops

Linked server calls in a loop

Squeezing the fat out of loops with a slim table

Summary

About the author

Chapter 36. Performance-tuning the transaction log for OLTP workloads

How can the transaction log be a bottleneck?

Factors that contribute to transaction log bottlenecks

Determining whether the transaction log is a bottleneck

Strategies for dealing with transaction log I/O bottlenecks

Start with standard performance-tuning techniques

Take advantage of minimally logged operations if appropriate

Select a fast I/O subsystem

Align disk partitions

Remove physical file fragmentation

Preallocate transaction log file size

Separating data and log files

Managing virtual log files

Perform transaction log backups often

Schedule database maintenance during slow times

Summary

About this author

Chapter 37. Strategies for unraveling tangled code

Organize: make it readable

Formatting

Comments

Break down: what does it do?

SELECT columns

Data

Sketch

Pseudocode

Streamline: resolve obvious issues

Too much data

Functions

Non-SARGable WHERE Clauses

Streamline: optimize joins

Streamline: similar subqueries and queries

Streamline: dynamic SQL

Summary

About the author

Chapter 38. Using PAL to analyze SQL Server performance

Performance Analysis of Logs (PAL)

Using PAL with SQL Server

Capturing performance counter data

Data collector sets using PAL template files

Logman

SQLDiag

Performing PAL analysis

Counter Log

Threshold File

Questions

Output Options

File Output

Queue

Execute

The PAL report

Summary

About the author

Chapter 39. Tuning JDBC for SQL Server

JDBC performance tuning can be effective

Recommendations for tuning jTDS configuration

Unicode character issues

API cursor issues

ResultSet Type issue

Prepared statement mechanism issue

Controlling global configuration

Summary

About the author

5. Business intelligence

Chapter 40. Creating a formal Reporting Services report part library

Report parts defined

Deciding to create a report part library

Selecting report parts

Level 1

Level 2

Level 3

Level 4

Using a report part library

Creating the initial report part

Using report parts and change requests

Existing report part changes and publication

Summary

About the author

Chapter 41. Improving report layout and visualization

Target-based rendering

Control over pagination

Joining across datasets

Aggregates of aggregates

Writing mode

Domain scope

Databars

Sparklines

Summary

About the author

Chapter 42. Developing sharable managed code expressions in SSRS

Coding report expressions

Adding a report element property expression

Coding a report Code property expression

Creating sharable managed code expressions

Referencing DLLs from your report

Summary

About the author

Chapter 43. Designing reports with custom MDX queries

Using the Adventure Works sample data

MDX query builder

Building a report

Adding a calculated member

Handling parameters

Passing parameters

Parameterizing measures

Summary

About the author

Chapter 44. Building a scale-out Reporting Services farm

What is network load balancing?

Preparing your network

Create a DNS entry for the NLB cluster application

Configure the server network cards

Adding the network load balancing feature

Creating the NLB cluster

Adding hosts to the NLB cluster

Installing Reporting Services on the NLB cluster

Configuring the first Reporting Services instance

Configuring the second Reporting Services instance

Joining the second Reporting Services instance

Configuring view state validation

Configuring the hostname and UrlRoot

Workarounds for the HTTP 401 error message

Summary

About the author

Chapter 45. Creating SSRS reports from SSAS

Creating the report project

Adding a shared data source

Creating shared datasets

Creating the main dataset

Creating the parameter list shared dataset

The report

Datasets

The matrix

Adding hierarchy drill-down capabilities

Parameters

Charts

Summary

About the author

Chapter 46. Optimizing SSIS for dimensional data loads

Optimization quick wins

Type 0 dimensions

Type 1 SCDs

Type 2 SCDs

Summary

About the author

Chapter 47. SSIS configurations management

Building the demo database

Starting in the middle

Changing the connection

Externalizing the connection

Taking a step back

Abstracting a bit

Let’s take a peek

Runtime overrides

Summary

About the author

Chapter 48. Exploring different types of enumerators in the SSIS Foreach Loop container

Make it dynamic

Foreach Loop enumerators

Summary

About the authors

Chapter 49. Late-arriving dimensions in SSIS

A late-arriving dimension scenario

Natural keys and surrogate keys

The example data structure

Working around late-arriving dimensions

File it

Ignore it

Update the fact later

Handling late-arriving dimension members in fact processing

Processing the dimension update

Summary

About the author

Chapter 50. Why automate tasks with SSIS?

Automation by example

SSIS key tasks and components

Creating reusable and mobile SSIS packages

Precedence and manipulating control flow

Monitoring the results

Summary

About the author

Chapter 51. Extending SSIS using the Script component

The Swiss Army knife of SSIS

Before we get started...a word of caution

Sources, destinations, and transformations...oh my!

Synchronous and asynchronous behavior

Script component inputs and outputs

Into the code

Summary

About the author

Chapter 52. ETL design checklist

Discovering data realities

Extract phase

Detecting changed data

Data staging

Transform phase

Load phase

Surrogate key generation

Data update strategy

ETL system instrumentation and management

Alerting

Recovery and restart ability

Audit, balance, and control support

Runtime event logging

ETL metadata and reporting

Summary

About the author

Chapter 53. Autogenerating SSAS cubes

Background

Sample usage scenarios

Technology overview

Developing InstantCube, a simple cube generator

Creating the Visual Studio project

Creating the relational database emitter

Creating the Analysis Services emitter

Creating the SSAS database

Creating the data sources

Creating the Data Source View

Creating the dimensions

Creating the cubes

Summary

References

About the author

Chapter 54. Scripting SSAS databases – AMO and PowerShell, Better Together

Advantages of PowerShell

Advantages of compiled code

Automating processing

Repetitive design changes

Scripting databases

Modules and snap-ins

Summary

About the author

Chapter 55. Managing context in MDX

Named sets

Calculated members

Scope assignments

Summary

About the author

Chapter 56. Using time intelligence functions in PowerPivot

Introducing Data Analysis Expressions

DAX data types

Sample database

Time intelligence functions

Golden rules

Functions that return a single date

Functions that return a table of dates

Functions that evaluate expressions over a time period

Samples

Summary

About the author

Chapter 57. Easy BI with Silverlight PivotViewer

Presenting Silverlight PivotViewer

What makes up your pivot

Cards (or images)

Slicing and dicing options

Filtering and sorting options

The way to go

Which seat should I take?

The source

Preparation

Implementation

Summary

About the author

Chapter 58. Excel as a BI frontend tool

Key points to consider when choosing a BI frontend tool

Why Excel?

Assumptions

Why use OLAP as a source?

Dashboard implementation

The business perspective

The technology perspective

Summary

About the author

Chapter 59. Real-time BI with StreamInsight

What is StreamInsight?

What are events and event streams?

Event shapes

Deployment

Architecture

How does querying streams differ from querying an RDBMS?

Where is StreamInsight useful?

Time

Querying the streams

Summary

About the author

Chapter 60. BI solution development design considerations

Architecture design

Other aspects influencing architecture design

Solution development

Conceptual design

ETL layer

Dimensional modeling and cube design

Reporting platform

Dashboard development

Summary

About the author

Index

List of Figures

List of Tables

List of Listings

MVP authors and their chapters

Johan Åhlén 53

Gogula Aryalingam 57

Glenn Berry 31

Aaron Bertrand 19

Kevin G. Boles 20

Robert Cain 45

Tim Chapman 38

Denny Cherry 5

Michael Coles 46

Rod Colledge 14

John Paul Cook 21

Louis Davidson 4

Rob Farley 2

Grant Fritchey 32

Darren Gosbell 54

Sergio Govoni 22

Allan Hirt 7

Satya Jayanty 16

Tibor Karaszi 17

Jungsun Kim 39

Tobiasz Koprowski 18

Hugo Kornelis 23

Ted Krueger 50

Matija Lah 24

Rodney Landrum 15

Greg Larsen 9

Peter Larsson 25

Andy Leonard 47

Ami Levin 1

Greg Low 41

John Magnabosco 11

Jennifer McCown 37

Brad McGehee 36

Siddharth Mehta 60

Ben Miller 26

Allan Mitchell 59

Tim Mitchell 51

Luciano Moreira 27

Jessica M. Moss 40

Paul Nielsen 6

Shahriar Nikkhah 48

Robert Pearl 34

Boyan Penev 55

Pedro Perfeito 58

Pawel Potasinski 12

Mladen Prajdić 28

Abolfazl Radgoudarzi 48

Denis Reznik 29

Rafael Salas 52

Edwin Sarmiento 44

Chris Shaw 3

Gail Shaw 8

Linchi Shea 35

Jason Strate 33

Paul Turley 43

William Vaughn 42

Peter Ward 13

Joe Webb 10

John Welch 49

Allen White 30

Thiago Zavaschi 56

Preface

This is the second volume of a book that many people thought would never see the light of day. In early 2007, the editor of the first volume, Paul Nielsen, had an extraordinary idea. I’ll let him tell you about how this idea came into being, by including a section from the preface to Volume 1:

Each year Microsoft invites all the MVPs from every technology and country to Redmond for an MVP Summit—all top secret—don’t tweet what you see! During the MVP Summit, each product team holds a series of presentations where they explain their technologies, share their vision, and listen to some honest feedback. At the 2007 MVP Summit in Seattle, Bill Gates presented his vision of the future of computing to the MVPs and then took questions for about an hour. I really enjoy these dialogues. I get the sense that if BillG wasn’t the founder of Microsoft, he’d make a great MVP. You can tell he likes us as fellow Geeks, and he’s rather candid in the MVP Q&A time. It’s one of my favorite parts of the MVP Summit.

During the Q&A, the lines at the microphones are far too long to bother to join, so I daydream a few questions I’d ask BillG:

As the world’s wealthiest Geek, what can you tell me about your PC?

Even with all your accomplishments, do you still find the most happiness and satisfaction with your family?

Do you play Age of Empires 2, and if so, want to join me in a game?

Kirk or Picard?