SQL

"Why should DBA's support Azure?"

Shawn Weisfeld — Wed, 18 Aug 2010 11:39:11 GMT

Recently I heard a question asking "Why should DBA's support Azure?"

I thought I would muse a bit on it here.

IMHO it boils down to the question "What is your job?"
I ask myself this all the time. Is it my job to write really cool code or to tune that sql query? I don't think so. I think my job is to "solve business problems". Regardless of if that problem is provide accounting the ability to do their thing or give people the ability to purchase our products, IMHO, technology is the means to that end and we as IT professionals must not forget that. IMHO our focus needs to be providing the best solutions for our customers regardless if that is Azure, a local SQL Server instance, or a XML file. They are paying us to "solve their problems".

A historical perspective:

I would assume back in the day when we used computers with vacuum tubes, someone had the job of replacing them. I would also assume that when we used punch cards someone had the job of feeding them into the system. However we don't see positions for those jobs any more, and if I may be so bold, I don't think the industry is worse off because of it. I think that we have chosen an industry that is constantly changing, and it is a fact of life that technologies in the long run will come and go. The only constant is there will always be change, and IMHO we need to embrace it.

I think each of us needs to, every few years, look at our skills, and where the industry as a whole is going and pick a set of technologies that we feel complement our current skills and put us on a long term path that is consistent with the direction that the industry is going. Now I am not saying that Azure is going to replace the need for local DBs or DBAs. However if we constantly realign our skill sets with what the industry wants we will always be ensured that folks will come to us when they need their problems solved. IMHO, selecting IT as a career path means committing a lifetime of continual learning and continual change.

These are my thoughts what are yours? Please post them below in the comments area. . .

Could not open a connection to SQL Server

sweisfeld — Wed, 08 Jul 2009 16:44:01 GMT

Problem:

.NET application throws the following error.

A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)

Solution:

Check SQL Server to ensure that TCP/IP is enabled. Open the List of available “Protocols for SQL Server”. You can find this in the “Computer Management” Console under “Services and Application”, “SQL Server Configuration Manager”, “SQL Server 200X Network Configuration.” After you enable TCP/IP you will need to restart the SQL Server relational engine. You can do that using the “SQL Server 200X Services” option.

Using ROW_NUMBER() to eliminate the multiplicative effects of a join

sweisfeld — Fri, 03 Jul 2009 17:33:46 GMT

Problem:

It is a common situation to have a header/detail tables in SQL Server. Additionally sometimes you include a calculated value on the header table to make reporting easier. For Example lets say you had an SalesPerson table that had a column for total sales. Now you also have a SalesDetails table that lists every sale that the company made. The purpose of having the total sales column is that it eliminates the need to touch the larger SalesDetails table when you just need to get the total by sales person. This is a good thing until the user asks for a report that has a column for both the total sales of the sales person and a line for each of the SalesDetails. Maybe a picture will help explain.

So here is the setup. As you can see George has sold 5 items for $100 each for a total of $500.

Now when we are asked to produce the report that joins these two together we get the following result:

Now if someone who gets this report sums up the total sales column they get $5,000, the wrong answer. What we need to do is zero out the total sales on each row except for the first row. So our results will look something like this.

Now the user of this report will get the same answer if they sum up the total sales column and sale amount column. So now we know what we want how do we get there in SQL Server.

Well if we could number the SalesPerson Records by EmplID, then we would have the ability to zero out the records on all rows except for the first one. For example something like this. . .

Solution:

ROW_NUMBER() to the rescue. MSDN defines ROW_NUMBER() as “Returns the sequential number of a row within a partition of a result set, starting at 1 for the first row in each partition.” (http://msdn.microsoft.com/en-us/library/ms186734.aspx).

Here is my starting point, a standard join to put the two tables together.

As you can see the ROW_NUMBER function takes 2 arguments. The second is the ORDER BY, that argument tells ROW_NUMBER what order we want the results in. In this application this is irrelevant since each SalesPerson record will be the same. The first argument PARTITION BY tells SQL Server to restart the count every time it sees a change in a column or set of columns. In our case we want to restart the count for each SalesPerson.

So the next step is to zero out the TotalSales on all the rows except for the first one.

Yes! Now our users can sum up the TotalSales column or the SaleAmt column and get the same answer! Another happy report user = another happy developer!

SQL CLR: Query the file system to get a list of folders

sweisfeld — Thu, 11 Jun 2009 10:12:49 GMT

Today was a good day for questions, got one asking how to query the file system from sql server. See they have a list of network shares that they want to find out what folders are in each. SQL CLR to the rescue. . .

First lets write a Table Valued Function to get a list of folders. I chose a Table Valued Function so I can join it to a list of paths to search in sql server. Remember that since we are leaving the confines of the SQL Server to get data from the disk we need to elevate our permission, we are also impersonating a domain account as they have access to external resources.

here is the code for the Table Valued Function:

1: using System;

2: using System.Data;

3: using System.Data.SqlClient;

4: using System.Data.SqlTypes;

5: using Microsoft.SqlServer.Server;

6: using System.Collections;

7: using System.Collections.Generic;

8: using System.Security.Principal;

9:

10: public partial class UserDefinedFunctions

11: {

12: /// <summary>

13: /// Get a list of all the folders in a given path, for a given number of levels deep

14: /// </summary>

15: /// <param name="path"></param>

16: /// <param name="levels"></param>

17: /// <returns></returns>

18: [SqlFunction(FillRowMethodName = "FillRow", TableDefinition = "directory nvarchar(4000)", DataAccess=DataAccessKind.Read)]

19: public static IEnumerable Folders(String path, int levels)

20: {

21: WindowsImpersonationContext impersonatedIdentity = null;

22: List<string> results = new List<string>();

23:

24: try

25: {

26: //impersonate the user that is executing me in sql server

27: impersonatedIdentity = SqlContext.WindowsIdentity.Impersonate();

28:

29: //Get the data

30: GetDirectories(results, path, levels);

31: }

32: finally

33: {

34: //undo the impersonation

35: if (impersonatedIdentity != null)

36: {

37: impersonatedIdentity.Undo();

38: }

39: }

40:

41: return results;

42: }

43:

44: /// <summary>

45: /// Recursivly dig through the file system getting data

46: /// </summary>

47: /// <param name="results">List of the results</param>

48: /// <param name="path">Current Path we are looking into</param>

49: /// <param name="levels">Number of levels left to dig down</param>

50: private static void GetDirectories(List<string> results, string path, int levels)

51: {

52: //are we at the bottom of the query

53: if (levels > 0)

54: {

55: //append on the slash if we dont have it

56: if (!path.EndsWith("\\"))

57: {

58: path = path + "\\";

59: }

60:

61: try

62: {

63: //get a list of all the folders under the current path

64: foreach (string folder in System.IO.Directory.GetDirectories(path, "*", System.IO.SearchOption.TopDirectoryOnly))

65: {

66: //add each to the results, then dig down one more level

67: results.Add(folder);

68: GetDirectories(results, folder, levels - 1);

69: }

70: }

71: catch (UnauthorizedAccessException ex)

72: {

73: //If we dont have access to a folder, just report that back in the result set

74: results.Add(ex.Message);

75: }

76: }

77: }

78:

79: /// <summary>

80: /// convert one of our Enumerable objects into a data row

81: /// </summary>

82: /// <param name="obj"></param>

83: /// <param name="directory"></param>

84: public static void FillRow(Object obj, out SqlChars directory)

85: {

86: directory = new SqlChars(obj.ToString());

87: }

88: };

I created a table (SearchPaths) that contains a list of all the paths that I want to search:

Now I can just join the list of paths to search with the function and I have everything I was looking for:

If you are getting security exceptions and the like take a look at the slides from the CLR talk I gave a PASS Summit 2008, they should get you moving in the right direction.

http://drowningintechnicaldebt.com/blogs/shawnweisfeld/archive/2008/11/23/sql-pass-2008-talk.aspx

SQL Server: Who am I?

sweisfeld — Thu, 11 Jun 2009 08:48:30 GMT

Got a question today. How do I tell what user my sql server query is running as? Well you just ask.

Using the USER_NAME() function, without passing it an id, finds the name of the current user.

http://msdn.microsoft.com/en-us/library/ms188014.aspx

Using the SUSER_NAME() function, gets me the login identification name of the user.

http://msdn.microsoft.com/en-us/library/ms187934.aspx

Here is an example of each, using both a regular sql server login (left) and windows authentication (right)

Now that I know who I am, why not be someone else. . . In this example I change the context that I am running under with the “EXECUTE AS LOGIN” statement, then I revert back using . . . wait for it. . . the “REVERT” statement.

EXECUTE AS: http://msdn.microsoft.com/en-us/library/ms181362.aspx

REVERT: http://msdn.microsoft.com/en-us/library/ms178632.aspx

and they say getting to know ones self is hard. . .

SQL Server 2008 Access is Denied Error During Install

Shawn Weisfeld — Mon, 08 Jun 2009 15:33:00 GMT

Problem: when installing SQL Server 2008 you get an Access is Denied Error.

Here is the error you get during install. . . .

Fix: Check your Local Security Policy, you need to ensure that Administrators have “Debug programs” permission (Security Settings | Local Policies | User Rights Assignment)

More information can be found in the following articles:

https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=390424 (If you have had this error please follow this link and vote that it gets fixed in connect)

http://social.msdn.microsoft.com/forums/en-US/sqlsetupandupgrade/thread/ed4514fa-8d1d-4383-bbdf-fb06bfbad106/

http://support.microsoft.com/?kbid=2000257

Cross Tab Queries in SQL Server V3

sweisfeld — Tue, 10 Mar 2009 07:07:36 GMT

Using the example from my last post (Cross Tab Queries in SQL Server V2) I was asked to figure out how to make the query more dynamic. The only way to make it truly dynamic (i.e. return a different # of columns) would be to use dynamic SQL. However lets say you want to do a sliding window of 5 years for reporting. . . . Here is an example that utilizes a table valued function. . . . .

CREATE FUNCTION tvfTest
(
    @year int
)
RETURNS
@return TABLE
(
    ClientID int,
    year1 decimal(18, 0),
    year2 decimal(18, 0),
    year3 decimal(18, 0),
    year4 decimal(18, 0),
    year5 decimal(18, 0)
)
AS
BEGIN

INSERT INTO @return
SELECT [ClientID], year1 AS year1, year2 AS year2, year3 AS year3, year4 AS year4, year5 AS year5
FROM
(
SELECT [ClientID],
    CASE WHEN DATEPART(yy, [SalesDate]) = @year THEN 'year1'
         WHEN DATEPART(yy, [SalesDate]) - 1 = @year THEN 'year2'
         WHEN DATEPART(yy, [SalesDate]) - 2 = @year THEN 'year3'
         WHEN DATEPART(yy, [SalesDate]) - 3 = @year THEN 'year4'
         WHEN DATEPART(yy, [SalesDate]) - 4 = @year THEN 'year5'
         ELSE 'outofrange' END AS SalesYear,
   [SalesAmount]
FROM [Sales]) N
PIVOT
(
SUM ([SalesAmount])
FOR SalesYear IN
(
[year1], [year2], [year3], [year4], [year5] )
) AS pvt

RETURN

END
GO

You then call it like this:

SELECT *
FROM tvfTest(2006)

ClientID year1 year2 year3 year4 year5

1 100 200 300 400 NULL

2 150 250 250 250 NULL

Pivot/Cross Tab Queries in SQL Server V2

sweisfeld — Tue, 10 Mar 2009 05:32:00 GMT

Back in 2008 I blogged about Pivot/Cross Tab Queries in SQL Server (http://drowningintechnicaldebt.com/blogs/shawnweisfeld/archive/2008/01/21/pivot-cross-tab-queries-in-sql-server.aspx). I was asked for another example so here goes:
--Sample Sales Table CREATE TABLE [dbo].[Sales]( [ClientID] [int] NOT NULL, [SalesDate] [datetime] NOT NULL, [SalesAmount] [decimal](18, 0) NOT NULL ) ON [PRIMARY] --Added sample data SELECT [ClientID] ,[SalesDate] ,[SalesAmount] FROM [Sales]

ClientID SalesDate SalesAmount

1 1/1/2006 100

1 1/1/2007 200

1 1/1/2008 300

1 1/1/2009 400

2 1/1/2006 150

2 1/1/2007 250

2 1/1/2008 250

2 1/1/2009 250
--Create view to report Sales by Year by Client SELECT [ClientID] ,DATEPART(yy, [SalesDate]) AS SalesYear ,SUM([SalesAmount]) AS [SalesAmount] FROM [Sales] GROUP BY [ClientID] ,DATEPART(yy, [SalesDate])

ClientID SalesYear SalesAmount

1 2006 100

2 2006 150

1 2007 200

2 2007 250

1 2008 300

2 2008 250

1 2009 400

2 2009 250
--for sql server 2005 + --Pivot to report sales by client over years SELECT [ClientID], [2006] AS year2006, [2007] AS year2007, [2008] AS year2008, [2009] AS year2009 FROM ( SELECT [ClientID], DATEPART(yy, [SalesDate]) AS SalesYear, [SalesAmount] FROM [Sales]) N PIVOT ( SUM ([SalesAmount]) FOR SalesYear IN ( [2006], [2007], [2008], [2009] ) ) AS pvt ORDER BY [ClientID]

ClientID year2006 year2007 year2008 year2009

1 100 200 300 400

2 150 250 250 250

What do your error messages reveal?

sweisfeld — Mon, 09 Mar 2009 17:46:00 GMT

I know that most of you all know this already, but I figured it was worth a blog post since I just saw this from http://www.autosite.com. I got this very informative error message back when using their site earlier. . .

Query Of Queries runtime error.
Expected type STRING but encountered type NULL on the left of the LIKE condition
The error occurred in D:\INETPUB\wwwroot\content\Research\kbb\act_valueReport.cfm: line 74
Called from D:\INETPUB\WWWROOT\content\Research\kbb\index.cfm: line 21
72 :
73 : <cfmodule template="mod_kbbtable.cfm" width="#kbbModTableWidth#">
74 : <cfquery dbtype="query" name="qEquip">
75 : SELECT * FROM valueReport.Equipments WHERE GroupCode LIKE 'M' ORDER BY DisplayOrder
76 : </cfquery>

While it is important for us as developers to log this type of information, there is NO reason for us to show it to our customers. Now I know table names, physical paths, and other good stuff. In your asp.net applicaitons make sure you set your customer errors to “On” (http://msdn.microsoft.com/en-us/library/h0hfz6fc.aspx). You also want to make sure that you have logging so that you as the developer can triage errors.

Oh and here is a BONUS TIP: Don’t use Select *, in most cases you don’t need all the columns and there is no reason to have the DB send them all back. . . .

Let’s start a campaign to call out any website that we see on the internet that doesn’t properly hide their dirty laundry.

SQL PASS 2008 Feedback

sweisfeld — Tue, 16 Dec 2008 06:34:00 GMT

I wanted to share the feedback that I got from my presentation this year at the PASS Summit. (http://drowningintechnicaldebt.com/blogs/shawnweisfeld/archive/2008/11/23/sql-pass-2008-talk.aspx) Again thank you to all that attended, and a special thank you to those that provided feedback it is an invaluable tool to help presenters improve over time. (BTW these are all the comments, I did not filter them at all)

Overall 4.44 out of 5

What is your overall evaluation of the session? 4.5
Difficult topic to present. He did a good job.
Good content. Bonus content at the end took it to the next level without getting exotic. Wirht the time to sit through.
Interesting, but a little slow at the start.
Outstanding!
Speaker stands in the dark. Have some light on him.
This was the best session I attended. The presenter's pace was excellent and his lecture was well-thought out. He's obviosly comfortable speaking in front of a group.

How would you rate the usefulness of the information presented in your day-to-day environment? 4.08
Good selections.
The content was in an area we don't actively use, but don't want to avoid. This gave me a solid foot in the door as a DBA / Developer and exposed me to different facets to consider.
Samples should be more real world on what should be done in a CLR versus T-SQL.

How would you rate the Speaker's presentation skills? 4.5
Speaker is very smart about what to say.
Great speaking skills, kept the content flowing within the time alloted and managed to put in very valuable bonus content at the end. Great posative attitude.
He directed the audience well, even when we wanted to jump ahead of him.

How would you rate the Speaker's knowledge of the subject? 4.67
Obviously well versed on the topic, and for the most part able to take questions and resolve and answer that made sense.
My favorite part was that he showed errors we were likely to see and how to recover. No other presenter I've seen did this.

How would you rate the accuracy of the session title, description and experience level to the actual session? 4.5
Reletively hard to misunderstand this. Could have added the word "USING". NOt that there was a lack of space for it.

How would you rate the amount of time allocated to cover the topic/session? 4.5
A bit tight, but the content fit into the space allowed and the speaker stayed on topic to assure it worked.
And he did a great job of staying within his allocated time.
Great time management and ease of dealing with questions from the floor.

How would you rate the quality of the presentaion materials? 4.33
Good code examples demonstrated. Excellent use of realtime erros and troubleshooting.
stop showing the same thing on PPT and in code. it is not useful! You would save a lot of time.
He had all of his scripts pasted into his powerpoint for easy reading. Again, no other presenter did this.
Just leave most in power point. with just one or two real samples
Good examples; building on the go.

ClientID	year1	year2	year3	year4	year5
1	100	200	300	400	NULL
2	150	250	250	250	NULL

ClientID	SalesDate	SalesAmount
1	1/1/2006	100
1	1/1/2007	200
1	1/1/2008	300
1	1/1/2009	400
2	1/1/2006	150
2	1/1/2007	250
2	1/1/2008	250
2	1/1/2009	250

ClientID	SalesYear	SalesAmount
1	2006	100
2	2006	150
1	2007	200
2	2007	250
1	2008	300
2	2008	250
1	2009	400
2	2009	250

ClientID	year2006	year2007	year2008	year2009
1	100	200	300	400
2	150	250	250	250