SQL Performance Tuning

Table of Contents Overview..............................................................................................................................2 Introduction..........................................................................................................................3 Creating Query Lists with a SQL Profiler Trace ................................................................4 Long Running and New Queries......................................................................................5 Recompiling Queries.......................................................................................................8 Compilation Process..........................................................................................................10 Conclusion.........................................................................................................................12 Reference...........................................................................................................................12

SQL Performance Tuning Overview

Most companies will usually only work on post-deployment queries when they create a problem and are placed on a "bug" list of some kind. Yes, you can find problem queries that way, but if you wait until they are on a "bug" list, you tend to only have the minimum time needed to "fix" the problem and get the application back up and running. If you are lucky, you will have time to incorporate any new performance tuning tips you learned since you last worked on the query as you fix the new problem This document provides information related to SQL performance tuning. Also, provides steps that can be considered in query tuning and explains the compilation process for queries in SQL server.

SQL Performance Tuning Introduction

The first step you need to do is to identify what queries are in need of tuning. When you boil it down, there are actually two types of queries in a SQL Server installation: pre-deployment queries and post-deployment queries. It goes without saying that all new (pre-deployment) queries need to be optimized fully before you even schedule them for deployment. However, those queries that have already been deployed, even those you have already optimize, should be looked at again as the database environment changes over time to determine if they have developed new problems. This ad-hoc way of tuning queries may or may not allow you to identify and correct major performance issues with your stored procedures because it deals with only those that are bad enough to gain major attention. A better plan would allow you to not only work on problem queries as they arise, but also to obtain lists of queries that show performance problems before they make your "bug" list. These queries can then be optimized as you have time or as you enhance them for new application releases. Using the power of the SQL Profiler and running a simple trace to capture the performance of your stored procedures can easily obtain these new query lists. This trace is simple to create, and depending on your installation, should be run during your production time frame off and on for a few days to obtain a good sampling of data.

SQL Performance Tuning Creating Query Lists with a SQL Profiler Trace
To create a lists of queries, you will need to create and run a trace capturing all the Stored Procedure events, as well as the following data columns: EventClass EventSubClass TextData CPU Reads Writes Duration SPID StartTime EndTime NestLevel ObjectID ObjectName ObjectType

This trace may create a large amount of data, so it might be easier to create the trace so it will place the information to a table so you can use a query to view the data. If can find a way to filter the trace in order to limit the data output, I would suggest that you do so. I usually start my stored procedure names with 'spXXXX' so I can filter by object name. But as you look into your particular situation, you can usually come up with something to use to filter some of the unneeded data out to lessen the impact of the trace and the amount of data you have to sift through. Now that you have trace data saved to a table. Create several stored procedures that you can use to identify the worst performing procedures by looking at duration (duration is displayed as milliseconds), SP:CacheMiss, SP:Recompile, Reads, and Writes. Create lists with the largest durations, the most Reads, the most Writes, and stored procedures that include the SP:CacheMiss and the SP:Recompile events. It might take you a few times to learn what is considered excessive Reads and Writes

SQL Performance Tuning

in your database, but if you see a stored procedure that it way above the average, it might give you somewhere to start. The other lists are easier to define for problem queries and can be worked on right away. Now that you have your hit lists of pre-production queries, long running queries, queries with excessive reads and writes, and queries that are recompiling, how do you proceed to optimize them?

Long Running and New Queries

For long running queries and queries that you have just created, the first thing you need to do is to produce an Execution Plan for that query. Using the Execution Plan you should: Look for anything that shows up in red. Query Analyzer will color icons red if it determines there are certain problems. Most of the time, red means that the statistics for the indexes being used by a part of the query are out-ofdate, but it can mean other things. Review any icons that show up red and determine how to fix the issues.

Look at how you are obtaining the data from the tables. Is the query doing a Table Scan? Can you do anything to change the Index or Clustered Index Scans into Index or Clustered Index Seeks? Can you rework your query to use the Clustered Index instead of a Non-Clustered Index? Simply moving the base data retrieval from a Table Scan to an Index or Clustered Index seek will usually solve the problem of a slow-running query. This is a very quick way to improve the performance of a majority of the problem queries you will encounter.

Look at the Cost of each query segment. A simple thing to remember is that the Cost of each segment roughly equates to the percentage of time it takes that segment to complete. Look at the larger percentages and see if you can optimize that segment of code any. The result may still mean that the segment remains the most costly, but the goal is to optimize that segment so

SQL Performance Tuning

it runs faster than before, not necessarily so that it becomes the least costly segment. If you find segments of code that seem to take a little longer than they should, use the following list to try to determine what is wrong in that segment. You should also use this list to optimize all of your code in general even though that code may not show up as a problem segment. Are there any cursors in the code? Cursors are a major performance problem and can usually be worked around. A simple temp table with an IDENTITY field can usually be used to replace the need for just about any cursor.

Can you reduce the use of temporary tables or the TABLE data type in the query? While these objects have their uses, some developers have a habit of using them when they are not really needed. Ask yourself if a JOIN can be used instead or if a different WHERE clause will eliminate the need for the table. If you can't eliminate the table, see if you can reduce its size with filters or use a table permanently created in a database so you do not have to take the performance hit of creating the table each time it is needed.

Do you have any statements that modify table structures? Just as creating a temp table in a query, modify tables in queries also cause a performance hit. Look at ways to work around these statements. A generic table may be able to be created that will server your needs.

Does the query return more data than is needed? Network congestion can cause minor problems with stored procedures. Try not to return more data or columns than is needed by the application. Also try to use SET NOCOUNT ON to reduce the usually unneeded row count that is return by all queries.

Are you using the sp_executesql system stored procedure instead of the EXECUTE command to run strings? The sp_executesql system stored procedure has a slightly performance advantage over the EXECUTE command.

SQL Performance Tuning

Look into ways to replace EXECUTE with sp_executesql.

Have you created any transactions within the query? If you create transactions in your code, make sure you can estimate the cost of that transaction. By knowing the cost of the transaction, you can estimate if it will normally take 1 second, 1 minute, or 1 hour for the transaction to complete. Steps can then be taken to simplify the transaction if the estimated cost exceeds a few seconds. Otherwise, you run the chance of introducing very slow queries that might even cause database blocking problems because of the time needed for the transaction to complete.

Keep the size of the procedure as small as possible by eliminating unneeded code. Ask yourself if that IF statement will ever run all of its branches -- if not remove them. Better yet -- create other stored procedures that handles the function of each IF branch. Also see if there are segments of code that you use over and over again which can be extracted and made into separate stored procedures. Finally, understand the uses of the CASE statement. CASE statements can sometimes eliminate large amounts of code if you know how to use them properly.

Use the new ANSI JOIN syntax instead of the old style joins. The new join syntax has a slight performance advantage over the old way of using the WHERE clause for a join. The new syntax also tends to have better readability and should become the norm in your Transact-SQL coding.

Some schools of thought will tell you to replace dynamic portions of your code with parameterized queries. This switch depends on your situation as I have found that in some cases, the dynamic query performs better even though the cached plan is not always used. This is up to your own research based on the query and your database.

Look for objects that are not called with qualified owner names. Each time the query is run without qualified owner names on the objects referenced within

SQL Performance Tuning

the query, the Optimizer has to hold compile locks on system objects until it can determine if the objects are the same as the ones in the cached plan. Qualifying the owner name will solve this problem and help with performance and blocking problems. Look at the data types being used in WHERE clauses and join statements. Any comparisons of data or joins should be done with the same data types. If the data types are different, then a slight performance hit is taken as the data types are converted before they are compared.

Recompiling Queries
Even if you have optimized queries by looking at the Execution Plan and the code, you may still have a problem that the vast majority of developers do not even consider. This problem is the performance cost taken every time a query has to create a new plan or recompile. Recompiling queries are usually the easiest to fix and can make a dramatic improvement on the performance of many applications. You should keep in mind as you look at the list of recompiling stored procedures that there are legitimate reasons that a stored procedure recompiles. It is just the ones that seem to recompile each time they are executed that you should worry about in the short term. You can look at the others later and determine if they are recompiling for legitimate reasons or not. Look for a mixture of DML and DDL statements. Do not mix the creation of objects and use of those objects in your code. Move all object creation to the top of your query and you should be able to avoid this problem.

Look for ANSI SET commands. Changing the status of SET ANSI_DEFAULTS, SET ANSI_NULLS, SET ANSI_PADDING, SET ANSI_WARNINGS, and SET CONCAT_NULL_YIELDS_NULL will cause the query to recompile each time it is executed. Make sure you really need to change these settings before you do so inside of a query.

SQL Performance Tuning

If you must have a cursor in your code, make sure you did not reference temporary objects in the cursor. Referencing a temporary object in a cursor will cause the query to recompile every time it is executed.

SQL Performance Tuning Compilation Process

When SQL Server receives a query for execution, its execution plan may already be present in memory (the procedure cache); if not, SQL Server will have to compile the query before executing it. The compilation process is divided into four parts: parsing, normalization, compilation and optimization. Parsing During this stage, SQL Server checks the query for syntax errors and transforms it into a complier-ready structure that it will use later to optimize the query. It does not check for object names or column names. Normalization At this stage, SQL Server checks all references to objects in the query. This is where we typically get the Object not found message when an object referenced in the query is not found in the database. SQL Server also checks to see if a query makes sense. For example, we cannot execute a table or select from a stored procedure. Bear in mind that while we can optimize select, insert, and update statements, there is no way to optimize if, while, and for operators. Compilation This is where we start building the execution plan for the query we passed to SQL Server. First, we create a sequence tree. The sequence tree is normalized, which includes adding implicit conversions if necessary. Also during this phase, if the query is referencing views, a view definition is placed in the query. If a statement is a DML statement, a special object is created called the query graph. The query graph is the object on which the optimizer works to generate an optimized plan for the query. This is the compiled plan that is stored in the procedure cache for reuse. Optimization

SQL Performance Tuning

SQL Server Optimizer is a cost-based optimizer, which means that it will come up with the cheapest execution plan available for each SQL statement. For each SQL statement to run, we need to use resources like CPU, memory, hard disk, etc. The cheapest plan is the one that will use the least amount of resources to get the desired output. For optimizing DML statements, SQL Server will test different indexes and join orders to get the best plan for executing the query. Your index definition helps optimizer by reducing/minimizing resource usage. If the index has a high selectivity then it is most suitable for optimization. Because a complex query will take into account all indexes and joins, there can be many paths to take to execute the query. In such cases, determining the best path for optimization can take a long time. The longer this process takes, the higher the cost that is involved. So first, a trivial plan is generated. This plan assumes that cost-based optimization is costly; if there is only one path for execution possible, there is no point optimizing the query. For example, when placing a simple insert statement into a table, there is no way that your indexes or join orders can increase optimization, so the trivial plan is used. For any particular query, SQL Server will use statistics to understand the distribution of data. Statistics are stored in the statblob column of the sysindexes table in each database. Join orders can also be optimized based on the number of rows that are fetched by each join condition.

SQL Performance Tuning

Conclusion
We have looked at the Execution Plan, analyzed the code, and look for recompiles; now what? Just because everything looks fine, doesn't mean that the query will run satisfactorily in every single scenario. Come up with all the different test cases you can think of for the query. How does it run against different sizes of data sets? How does it run with different sets of parameters? What happens if you run the query multiple times using multiple Query Analyzer connections? Whatever you can think of in your particular case, needs to be traced and analyzed. Don't just run the query once and say "I tuned this query and it is ready." Do the hardest thing in the world and convince your clients and managers that you need time to thoroughly test a new query or recently fixed query.

Reference
http://msdn.microsoft.com