The whole SQL world is an interesting place, part Database Admin, part Developer, and part system administrator all rolled into one neat package. Well, knowing some of them, maybe not so neat. Theirs is a difficult world because of the different areas they must master – back up the database, write cleaner and more efficient and secure code, come up with new queries, study for the new SQL 2012 exams coming out in June (oops, didn’t want to slip that in too soon).

SQL 2012 was recently released; in fact, the launch was held March 7th, 2012. There are three main editions of SQL 2012 available: Enterprise, Business Intelligence, and Standard, with the Business Intelligence being new to SQL 2012. In addition, the Compact, Developer, and Express editions will continue to be distributed without licensing or pricing changes. The new SQL 2012 is a cloud-ready information platform that will help you and your organization unlock breakthrough insights across the organization and quickly build solutions to extend data across on-premises and public cloud, backed by mission-critical confidence.

You will need to wait until June 2012 to take the new SQL Server 2012 certification exams. However, you can prepare now for the SQL 2012 certifications by ensuring you have earned (or will have earned) a SQL Server 2008 Professional-level certification. Exams on SQL 2008 that you take now will help when upgrading from a Professional-level SQL Server 2008 certification to a Professional-level SQL Server 2012 certification. If you do so, than you must successfully pass only three exams, rather than all five.

SQL Professional-level Certifications

The Professional-level SQL Server 2012 certifications will validate that you have the expertise to design, build, and maintain the next wave of cloud-ready database and information solutions. Two new Professional-level certifications will be introduced for SQL Server 2012: the Data Platform certification and the Business Intelligence certification. In order to earn one of the professional-level certifications, you will need to successfully pass five exams, and you will need to recertify every three years by taking a recertification exam.

Exam 70–461: Querying Microsoft SQL Server 2012

The 70–461 certification exam is intended for SQL Server database administrators, SQL implementers, system administrators and engineers, and SQL developers. It is expected that these individuals have two or more years of experience and are seeking to prove their skills and knowledge in writing queries on SQL 2012. There is a five day instructor-led course (10774A — Querying Microsoft SQL Server 2012) that will give you the base technical skills required to write basic Transact-SQL queries for Microsoft SQL Server 2012. This course forms the foundation for all the other SQL 2012 Server-related courses; namely, Database Administration, Database Development, and Business Intelligence. As an aside, many of the exercises in this course are SQL Azure-enabled – helping you prepare for use of SQL 2012 in the cloud.

Exam 70–462: Administering Microsoft SQL Server 2012 Databases

Database Professionals whose duties are mostly installation, maintenance, and configuration tasks will be interested in the 70–462 exam. These SQL 2012 professionals are the ones who will often set up SQL 2012 database systems and are responsible for ensuring that these systems operate efficiently. They also ensure that the data they store is backed up regularly, stored effectively, and that the data is secure from unauthorized access. The 10775A Administering Microsoft SQL Server 2012 Databases class is a five-day instructor-led course that focuses on teaching SQL professionals individuals how to use the new SQL Server 2012 product features and tools.

Exam 70–463: Implementing a Data Warehouse with Microsoft SQL Server 2012

Those SQL 2012 Professionals who are interested in Extract Transform Load (ETL) and Data Warehousing will be very interested in the five-day 10777A Implementing a Data Warehouse with Microsoft SQL Server 2012 instructor-led course. This course describes how to implement a Business Intelligence (BI) platform to support information worker analytics, create a data warehouse with SQL Server 2012, implement ETL with SQL Server Integration Services, and validate and cleanse data with SQL Server Data Quality Services and SQL Server Master Data Services. SQL 2012 professionals who take this course are most likely to focus on hands-on work creating BI solutions, including data cleansing, ETL, and Data Warehouse implementation.

This course will help prepare students for the 70–463 Implementing a Data Warehouse with Microsoft SQL Server 2012 certification exam.

Exam 70–464: Developing Microsoft SQL Server 2012 Databases

If you are a SQL 2012 professional who is interested in building and implementing databases across an organization while ensuring high levels of data availability, than the 10776A Developing Microsoft SQL Server 2012 Databases course is for you. This five-day class is aimed for those SQL professionals whose tasks include creating database files; creating data types and tables; planning, creating, and optimizing indexes; implementing data integrity; implementing views, stored procedures, and functions; and managing transactions and locks. If you are interested in learning about logical table design, indexing and query plans, and creating database objects including views, stored procedures, along with parameters, and functions. Also, if you are curious about common aspects of procedure coding, such as transactions, concurrency, error handling, triggers, and SQL CLR, than this is the course for you. This course will help prepare you for the 70–464: Developing Microsoft SQL Server 2012 Databases – of course, a good deal of hands-on experience will help as well.

Exam 70–465: Designing Database Solutions for Microsoft SQL Server 2012

The 70–465 Exam is for SQL 2012 database professionals who design and build database solutions for an organization. These professionals will be responsible for creating plans and designs for database structure, storage, objects, and servers. They will also be responsible for creating the overall design plan for the SQL environment in which the database solutions will be housed and run.

Exam 70–466: Implementing Data Models and Reports with Microsoft SQL Server 2012

The primary audience for this exam is Business Intelligence (BI) Developers. They are most likely to focus on hands-on work creating BI solutions including implementing multi-dimensional data models, implementing and maintaining OLAP cubes, and creating information displays used in business decision-making processes. The five-day instructor-led 10778A Implementing Data Models and Reports with Microsoft SQL Server 2012 course will aid with these tasks.

Exam 70–467: Designing Business Intelligence Solutions with Microsoft SQL Server 2012

The primary audience for this exam is the BI Architect. Business Intelligence architects are responsible for the overall design of the BI architecture. This is the framework used for data organization, information management, and the technology components that are required to build BI systems for reporting data (which includes how these systems relate to other data systems in use). The BI architect’s responsibilities may include BI solution architecture and design, the provisioning of software and hardware, data warehouse and data mart design, online Analytical Processing (OLAP) cube design, and Integration with Microsoft SharePoint or line-of-business applications.

Related Courses
Querying Microsoft SQL Server 2012 (M10774)
Administering Microsoft SQL Server 2012 Databases (M10775)
Developing Microsoft SQL Server 2012 Databases (M10776)
Implementing a Data Warehouse with Microsoft SQL Server 2012 (M10777)

In this post, we will explore just some of the new features under each theme to give a feel for the capabilities of the upcoming release. Other features and a pre-release version of the software are available on the Microsoft web site.

Mission Critical Confidence

AlwaysOn for High Availability and Disaster Recovery

Before SQL Server 2012, we had four options for High Availability (HA): Server Clustering, Database Mirroring, Log Shipping, and Peer-to-Peer (P2P) Replication. Each solution had its pros and cons.

SQL Server 2012 effectively combines the pros of each of these solutions with its AlwaysOn feature. With AlwaysOn, you will have the automatic failover of Mirroring at the Database level but with the added ability to create multiple secondaries for read-only load balancing. AlwaysOn will provide HADR and Load Balancing in one effective solution.

Project Apollo Provides Column Compression and Performance Gains

SQL Server 2008 introduced Data Compression at the Row and Page Levels to provide not only storage savings but also, surprisingly, performance gains based on less I/O. In SQL Server 2008 R2, the VertiPaq technology allowed massive column-level compression in conjunction with the PowerPivot add-in. In SQL Server 2012, the new column-store index, as part of Project Apollo, will provide the same massive column-level compression within the Database Engine itself. Data Warehouse queries against Star or Snowflake schemas will be optimized to give potential performance gains and storage savings in the order of 100 times improvement.

Organizational Compliance With Enterprise-Wide Security Controls

On the security front, new features in SQL Server 2012 include the ability to create user-defined server roles. This allows specific permissions to be applied to groups of logins instead of using the fixed server roles or having to give permissions directly to each login.

Another interesting security feature is the Contained Database that allows a database to be moved or copied to a new instance without having the dependency on login objects. Users can connect to the database without authenticating a login at the Database Engine-level. There exist some specific security threats that relate uniquely to Contained Databases (documented in SQL Server 2012 Books Online), but if you need a database to be easily portable, the benefits may outweigh the drawbacks.

The SQL Audit feature, which was introduced in SQL Server 2008 Enterprise Edition, allows Server– and Database– level auditing without the need for DDL or DML Trigger, giving a more efficient method of auditing that is built in to the Database Engine. SQL Server 2012 allows new features such as filtering audit output and now supports Server level audits for all SQL Server editions.

Breakthrough Insight

SSAS 2012: VertiPaq, BI Semantic Model and PowerPivot

SQL Server 2008 R2 introduced support for PowerPivot through Excel 2010 and SharePoint 2010 using its VertiPaq technology for massive column compression. This allowed advanced users to develop their own Business Intelligence solutions via Excel or SharePoint. The Vertipaq technology allowed literally hundreds of millions of rows of data to be imported and analyzed efficiently on the client via Excel and on the Server via SharePoint.

Based on the popularity of the technology, SSAS 2012 enables Vertipaq on the server-side within Analysis Services itself in a new configuration option named Tabular Mode that supports the BI Semantic Model (BISM).

Consistent data via Data Quality Services

SQL Server Data Quality Services (DQS) is new in SQL Server 2012 and provides a knowledge-based platform for data cleansing, including data correction and deduplication. The DQS Client application is intended for the end user to automate data correction in addition to an interactive mode.

Parallel Data Warehouse and Reference Architectures

When Microsoft acquired DATAllegro, its stated direction was to support Very Large Databases (VLDB) through a scalable appliance architecture using the combination of hardware and software to support databases in excess of 100TB. Now with SQL Server 2012, that vision is closer to reality with many hardware vendors supporting the Fast Track configurations and reference architectures. The Parallel Data Warehouse edition supports features such as Remote BLOB Storage and large-scale data partitioning while using up to 256 logical CPUs for high performance in the VLDB arena.

Cloud on Your Terms

SQL Azure is an exciting new SQL Server edition that introduces the cloud as a data platform for the future. SQL Azure now supports Open Data (OData) protocol for querying and updating data in the cloud. Project Juneau extends the Visual Studio 2010 shell to support SQL Azure and SQL Server applications using the .NET framework through standard data APIs such as ADO.NET, ODBC, JDBC, etc.

Conclusion

Microsoft SQL Server 2012 has many great new features that will allow you to develop higher performing, more scalable next-generation applications. Using the same architecture and management tools, customers will be able to smoothly upgrade their systems and skills based on the need for the new features and according to their own schedule.

Read operations (select) often oppose write operations (insert, update, and delete).

In a previous blog post, I discussed the value of indexing. The real value is to help us retrieve data quickly. No one likes to wait for data from the database. We want instant answers and indexes are the best way to facilitate that.

Well then, why not create a lot of them? Why not create an index to match the way we search or sort in each query? The answer is the negative impact on the server when we modify data.

I am going to way oversimplify this to make the point. Let’s look at some data.

Let’s say I have the following indexes and that they are all in ascending order:
Clustered index on Customer ID
Non-clustered index on Lastname
Non-clustered index on Zip

I’ve added the index order in blue to the table:

Now let’s say we hire a new person. In other words, we insert a row. We have one of two options — update the index, or let it grow out of date. If we update it, we will put everything in the proper order. Internally SQL Server considers where the rows really belong in the index. This takes work and adds overhead to the server. If we let it grow out of date, we might be looking for a row that we thought was in a particular order, but it really isn’t now.

We will update ours. Here’s what it will look like after the update:

We basically strike through the original value and reconsider where the rows belong. Let’s say that Sue Adams gets married and changes her lastname to Zachary and moves to the 32401 zipcode. This is an update. Now we need to redo the index order for the two non-clustered indexes:

We would have a similar situation if we delete a row. For example, let’s say that Amy Hamil asks to be removed from our customer list or is inactive for so long that we delete the row. We need to update the indexes again.

In our examples, we are considering a single row at a time. However, the reality is that rows are constantly being inserted, updated, and deleted. It’s not just one row we are considering — it’s hundreds, thousands, or even millions of rows. These processes are going on at the same time we report on the data. To optimize the read performance, we keep the indexes exactly up to date. However, at the same time of the read, when we modify data (if we are keeping the indexes up to date), we change the values of the position in the index due to the data modifications. Therefore, they are competing. If we don’t keep the indexes up to date, we might look in one place for the row when it’s really not located in that place any longer.

It’s also common for developers or dbas to create too many indexes. We might have a single query that is much improved by creating an index on a particular column. That’s great for the read performance of that query. However, modifications need to update that index. We maintain an index on a column that is part of every insert into the table but is searched only for that single query.

In summary, indexes improve performance if we search in the way we have indexed. Simultaneously, they potentially hurt us because of the overhead when data is modified.

We must carefully balance the needs of both. As our first step, let’s look at indexing.

Indexes potentially improve performance when we search and sort, but they potentially hurt performance when we add or modify data.

The two major types of indexes in SQL Server are clustered and non-clustered. SQL Server stores data physically in one of two ways. The first way is according to a clustered index, which means that the key of the index determines how the data is physically stored on disk, and the second way is called a heap. For example, let’s say I create an index on a column called customerid. Each row of the data would follow in order. In other words, customerid 2 would immediately follow customerid 1, and as new rows are added to the table, the index is updated sequentially.

Think of it this way, I have a filing cabinet, and as I place things in the filing cabinet I am mindful of the way I will retrieve them. In my case, I often lecture on a specific version of SQL Server, and  I have technical journals that arrive monthly. In my filing cabinet, I have the journals sorted by type, actual journal, and date. Right now, I am beginning to write and lecture about SQL 2012. My system works well in this case because I know that the “Denali” project (now branded as SQL Server 2012) is in the journals that were written around 2010 and beyond. The retrieval of the data is exactly how it’s laid out in my file cabinet.

What if I need to write about a particular feature in SQL Server? For example, let’s say I’m asked to lecture over all of the different versions of Reporting Services. The data isn’t physically ordered that way. We will come back  and use a non-clustered index for this.

First, we need to look at the other way data is physically organized. A heap is defined as “in no particular order”. Think of it this way, have you ever known (or been) a person with an incredibly messy desk? As paper comes into the office, it simply goes in the pile or various piles (I don’t keep my desk this way, I can’t function). I have always been fascinated by the people that do keep everything in piles and function perfectly well in what looks like chaos to me. Their data is in no particular order, but, amazingly, they can retrieve information quickly.

SQL Server is able to retrieve information quickly regardless of the clustered vs. heap storage. However, we need some other indexes to help. These are called non-clustered indexes. We can have indexes that are built on top of the clustered index or on top of the heap.

Say you travel to an unfamiliar city, and you have an address for your destination but no idea how to get to that location. You need a map or a GPS, right? You don’t know how to get to the physical location, but you know someone who knows. That’s the way non-clustered indexes work. The data is not physically stored according to the non-clustered index, but the non-clustered index points to the physical location — the heap or the clustered index.

Going back to the example above — writing about Reporting Services would be easy. I would simply have a lookup chart that lists the topic, and next to it I would have the journal name and journal date of the journals with Reporting Services articles.

When you run a query in SQL Server, the Database Engine physically has to retrieve the data you’ve requested. In order to do that, it must know the physical order. For efficiency, we want to store the data in an order that we will search (there are several considerations beyond the scope of this post, but for now, we are only considering the retrieval). If we request the data in a way that it isn’t physically stored, we want a way to look it up, thus needing a non-clustered index.

We only get one way to store the data physically at a time, a clustered index or heap. However, there are probably multiple ways that I search or sort the data. I might request a particular customer in one query, and in another query I might search by last name. In both cases, I need to physically retrieve the data. If my clustered index for the table is on customerid, the first query would use that. If I had a second query, a non-clustered index on last name could be used.

In summary, remember that indexes help us when we search or sort. We will see in another post how they potentially hurt performance on data modification.

Stop Using SELECT *!

Many people know they shouldn’t use SELECT* but don’t fully understand the impact. When you use SELECT *, you force SQL Server to go to the heap or clustered index one way or the other because this is where all of the columns are located — and you just asked for all of them. Oh, and all of the other people using SELECT * are going to the same clustered index or heap on that table, increasing contention for resources. Stay tuned for an upcoming post on indexing that will cover this in more detail. For now, just trust me. Please stop using SELECT *. You’re hurting database performance. The truth is you probably don’t need all of the columns. Let’s just say your query looks something like this:

SELECT * from customer

I’m going to put some numbers here just to help demonstrate the point. Let’s say customer has 2000 rows and 30 columns. Notice there is no WHERE clause here — so you’re accessing all 2000 rows when you run this query. Perhaps you need 25 of the 30 columns, but it’s easier to just type SELECT *. I understand.

Let’s say those five additional columns add up to 30 bytes. You’re accessing 30 bytes multiplied by the number of rows (2000) multiplied by the number of times the query is run — in other words, the number of connections / users / applications that run the query. By the way, you can drag and drop columns from the Object Explorer in the SQL Server Management Studio Tool to the query window, so you don’t even have to type the columns.

Use a WHERE Clause

The WHERE clause filters the number of rows processed and increases the chance that SQL Server can use an index. What’s the exact tipping point? Well, that’s a complex question. I’ve spent hours understanding exactly when the query would tip and use a scan instead of a seek, an index vs. the table. In my most recent study of one query with a pretty large table, it was just over 1%!! I was pretty shocked myself. As a very general rule, try to aim for a WHERE clause that yields less than 5% of the table. Of course, sometimes you need to access more than 5%, and you won’t have this option.

I Don’t Like “LIKE”

That’s just me. Well, no, it isn’t. I found more problems in queries dealing with the LIKE operator than I can count. Don’t get me wrong. LIKE is very powerful, and sometimes you need it. However, try to solve your query without the use of wildcards when you can. You reduce the chances that an index can be used for your queries. Especially if your pattern match is at the beginning of the statement, you reduce the likelihood that SQL Server can find a useful index.

Use Stored Procedures

When you use stored procedures, you give SQL Server the chance to use the same execution plan over again. SQL Server has something called the query optimizer. This internal mechanism figures out the best way to run your query on a cost based analysis. If you use ad hoc queries, SQL Server figures this out every time. For queries you will run over and over, just turn them into stored procedures. It’s easy.

If your query looks like this:

Select last, first from customer

You can turn it into a stored procedure by changing it to:

Create procedure usp_CustInfo as select last, first from customer

Use Your Indexes Wisely

Indexes only help you when you search and sort. They potentially hurt you when you insert, update, or delete. Follow these guidelines:

• Create indexes for columns that appear in your WHERE clause.
• Try to use index keys (columns that are part of the index) that are short and stable over time.
• Do not create indexes on columns that aren’t accessed in that order. Consider a filing cabinet that is organized by zip code. If you search by alphabet or topic, it really does you no good to organize by zip code. Indexes are useful when we access the data in that order. If I never access the data by zip code, I am simply wasting time to organize by zip code. Create indexes for the columns you request.
• Create indexes on columns that have high variation. It really does you no good (and in fact, does harm) to have an index on a column with very few different values. For example, an index on a column of gender yields approximately 50% of the table in a normal distribution of the population. SQL Server will disregard such an index because you are accessing a large portion of the table. You’re maintaining an index for something that will likely be unused.

Related Courses
Writing Queries Using Microsoft SQL Server 2008 Transact-SQL
SQL Server 2008 R2 for Administration
Designing and Implementing a SQL Server 2008 R2 Database

Database Administrators and Developers are tasked with making applications and upgrades work efficiently, and they must be able to integrate applications across versions. This matters because most companies cannot suddenly upgrade all of their servers and applications at once, so there is this type of mish-mash blend of versions.

I created this little chart to remind myself (and you) of some of the highlights introduced in each feature. As the next version (Denali) makes it way into the market, check back here for updates. Happy reading!

There are two standards that govern SQL (Structured Query Language) — ANSI (American National Standards Institute) and ISO (International Standards Organization). Microsoft SQL Server supports both of these standards. Like other database products, Microsoft SQL Server has its own dialect of SQL. This dialect for SQL Server is called T-SQL or Transact-SQL.

The basics of the select statement are the same from one database to another. However, some options of the statement vary from product to product. We’ll specifically examine SQL Server’s select statement, although it is virtually identical to other select statements.

SQL Server is one of many relational database systems. Relational databases store information in a series of tables (rows and columns) that are “related” to each other.

Let’s say we have a customer database. This database has a few different sub-topics — or entities like customer, order, vendor, product, and employee. Entities can be translated to the topic of the table. We’ll only work with a single table to keep our examples very simple.

Our table has the following structure and records (rows):

CUSTOMER

Let’s begin by discussing the parts of the select statement. Then we will select information from one table only.

The select statement is comprised of multiple clauses. These are:

SELECT
FROM
WHERE
GROUP BY
ORDER BY
HAVING

Each of these clauses has information that follows it. These are:

SELECT column(s)
FROM table / view
WHERE condition / expression
GROUP BY column(s)
ORDER BY column(s)
HAVING aggregate function and condition / expression

The only two clauses that are required in a select statement are SELECT and FROM.

The SELECT clause identifies which columns we want to retrieve, and the FROM clause identifies the name of the table we are using. We might have something like this:

Select lastname, firstname
from customer

This gives us the output:

Notice that the headings above the column data are not title case as they are in the table. This is because my query had them listed as lower case. Our select statement indicates exactly what we want to see in the output.

Now we’ll review the optional clauses.

WHERE

The WHERE clause is a filter for the query results and uses a condition or expression. We use the WHERE clause to specify our request for only particular rows.

If I want to retrieve only those individuals with the last name of Smith then I can add a WHERE clause.

Select lastname, firstname
from customer
where lastname = ‘Smith’

This gives us the output:

GROUP BY

The purpose of the GROUP BY clause is to group or cluster records together based on a column or list of columns. In our example, I also use an aggregate function that counts the number of rows specifying our criteria. I also created an alias for the heading of the column with the function.

Select State, count(state) as ‘Count of Customers by State’
from customer
group by state

This gives us the output:

HAVING

The HAVING clause works in conjunction with the GROUP BY clause. It’s very much like the WHERE clause since it works as a filter. However, it works as a filter on the aggregate group.

Select State, count(state) as ‘Count of Customers Greater Than 1’
from customer
group by state
having count(state) > 1

This gives us the output:

ORDER BY

The ORDER BY clause indicates the sort order for the query results.

If I want to put my results in alphabetical order by lastname, I can add an ORDER BY clause.

Select lastname, firstname
from customer
order by lastname

This gives us the output:

Notice that Ron Smith appears before Bill Smith in our example even though “Bill” comes before “Ron” alphabetically. This is because I didn’t order by firstname, only by lastname. To correct this, I can do the following.

Select lastname, firstname
from customer
order by lastname, firstname

This gives us the output:

We saw the basics of selecting data from a single table. Look for Part II where we’ll explore querying from multiple tables.

Related Courses
Writing Queries Using Microsoft SQL Server 2008 Transact-SQL
SQL Server 2008 R2 for Administration
Designing and Implementing a SQL Server 2008 R2 Database

The definition of a Star Schema is one in which all dimension tables are directly related to the fact table through foreign key relationships. It’s the simplest form of Data Warehouse design where complex dimension relationships are collapsed into a single layer of dimension tables. Looking at the diagram of the design shows a “Star” type figure with the fact table at the center, hence the name.

The Snowflake Schema gets its name from an extra layer or layers of dimension tables which are NOT directly related to the fact table. The extra layers make the star look more like a “Snowflake” supposedly although it normally takes a few egg nogs to see the similarity. The dimension tables are effectively normalized to yield additional relationships.

Normalized design is essential in high performing OLTP databases where data needs to be stored in one place and one place only, to ensure efficient updates and space savings. One exception is where we selectively choose to violate the rules of normalization in the name of performance costing redundant storage. This is called “denormalization” and should be performed with extreme care. With a Data Warehouse we can relax the rules of normalization a little more and denormalize as long as we buy off on the extra space needed. As always it comes down to performance versus storage.

The Star Schema effectively is a denormalized schema costing extra storage. Proponents would argue this is done for performance reasons and also emphasize the “keep it simple stupid” or “KISS” approach to database development just to wind-up the Snowflake gurus.

The Snowflake Schema is more normalized and saves space and redundancy. Skeptics would argue that it does this at the expense of performance degradation caused by excessive joins.

Of course, there’s a bit of Star and Snowflake in all of us. The typical Time Dimension in both schemas is really a collapsed snowflake-turned-star schema design with Year, Quarter, Month dimensions collapsed into a single table.

Some older analysis products actually required a Star Schema. Fortunately, Microsoft SQL Server Analysis Services (SSAS) now allows us a choice as it supports both schema designs.

To calm people down, I usually introduce the concept of a “Constellation” or “Galaxy” schema which actually contains multiple Star and/or Snowflake Schemas combined so we can look to the heavens knowing there is a common solution for all of us. And SSAS supports that too, just so we can all coexist. Thankfully.

The Type 2 SCD design requires that an attribute is identified as a historical attribute such that the system will keep track of the changes of that attribute so that ultimately analysis can be performed accurately across multiple dimensions including the Time dimension. The way this is done is through a “Surrogate Key” being generated in the Dimension table. Consider this design for the DimCustomer dimension table in the Data Warehouse.

CREATE TABLE [dbo].[DimCustomer](
[CustomerSK] [int] IDENTITY(1,1) NOT NULL,
[CustomerKey] [int] NOT NULL,
[FirstName] [nvarchar](50) NULL,
[LastName] [nvarchar](50) NULL,
[GeographyKey] [int] NOT NULL,
[StartDate] [datetime] NULL,
[EndDate] [datetime] NULL,
CONSTRAINT [PK_DimCustomer] PRIMARY KEY CLUSTERED
(
[CustomerSK] ASC
)WITH (PAD_INDEX  = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO

In this case, GeographyKey will be the “Historical Attribute” as we want to track any changes in its value for a particular customer. CustomerKey is the “Business Key” because it’s the Primary Key from the Operational OLTP Database, but is now an ordinary attribute in the dimension table. CustomerSK is the surrogate key which will be generated by the system using the IDENTITY property and will be the Primary Key of the dimension table. The actual value of the surrogate key is of little significance except that it is unique and will be used to form a relationship with the Fact table.

CREATE TABLE [dbo].[FactInternetSales](
[OrderQuantity] [smallint] NOT NULL,
[SalesAmount] [money] NOT NULL,
[DiscountPercentage] [float] NOT NULL,
[DiscountAmount] [float] NOT NULL,
[ProductCost] [money] NOT NULL,
[ProductKey] [int] NOT NULL,
[CustomerSK] [int] NOT NULL,
[OrderDateTimeKey] [int] NOT NULL
) ON [PRIMARY]
GO

ALTER TABLE [dbo].[FactInternetSales]  WITH CHECK ADD  CONSTRAINT [FK_FactInternetSales_DimCustomer] FOREIGN KEY([CustomerSK])
REFERENCES [dbo].[DimCustomer] ([CustomerSK])
GO
ALTER TABLE [dbo].[FactInternetSales] CHECK CONSTRAINT [FK_FactInternetSales_DimCustomer]
GO

To populate the DimCustomer dimension table we could use the following query to access data from the fully normalized Operational OLTP database:

SELECT     Person.Contact.FirstName, Person.Contact.LastName, Sales.Customer.CustomerID, Sales.Customer.TerritoryID
FROM         Person.Contact
INNER JOIN Sales.Individual
ON Person.Contact.ContactID = Sales.Individual.ContactID
RIGHT OUTER JOIN Sales.Customer
ON Sales.Individual.CustomerID = Sales.Customer.CustomerID

Note that the CustomerSK surrogate key is system generated and increments by one for each new row added to the dimension table. Note also that the StartDate and EndDate attributes are populated using the Slowly Changing Dimension transform or similar logic within the SSIS ETL incremental load process. For each customer move, the EndDate will be updated to reflect the date that the customer ceased to live in the old region (GeographyKey). A new row will be inserted for the same customer with the new region code and the EndDate set to NULL to indicate the current region. The CustomerSK surrogate key guarantees a unique Primary Key even though the CustomerKey has a duplicate value.

When the fact table rows are updated, the fact table load step within the SSIS ETL incremental load process may perform a sub-select to match the CustomerSK surrogate key with the correct customer row for the current location forming the correct Foreign Key value in the Fact table. If the incremental load is performed correctly in this way, the subsequent analysis will be accurate without any differences in end user requests. Here is the inner query of the fact table load query to populate the CustomerSK foreign key in the fact table.

(SELECT     CustomerSK
FROM          AdvWorksDW1.dbo.DimCustomer AS dc
WHERE      (dc.CustomerKey = Sales.SalesOrderHeader.CustomerID) AND (EndDate IS NULL))

This is the “magic” of the Type 2 SCD.

I developed and tested these code samples by adapting the AdventureWorks BI Sample available under  http://sqlserversamples.codeplex.com/

One of the many interesting features of Microsoft SQL Server 2008 R2 is Master Data Services (or MDS). This enhancement allows an organization to create a “Master Data Hub” which provides a “single source of truth” for the enterprise’s numerous data entities and hierarchies. The typical medium to large sized company has many disparate systems, some inherited from mergers or acquisitions, each using its own data source. These data sources can be aligned using a hub-and-spoke architecture to cross-reference the data entities into a central database that can be used for auditing, change tracking, and consolidation. It may even help you to build that Data Warehouse you’ve been thinking about.

Discovering the data specifications can be time-consuming, but when stored centrally and maintained with version control and change tracking, the business can benefit from this understanding of it mission critical data. When building a Data Warehouse, much of this data discovery needs to be performed, but instead of recording this information in an isolated spreadsheet, now the information can be recorded in a centralized database for all to access and contribute to. Through Web Applications, individual application groups can be held accountable for “their” data specifications through ownership, allowing all parts of the organization to contribute to the centralized “Master Data”. Ongoing Data Warehouse and Business Intelligence efforts can then benefit with the Master Data being readily available and up-to-date.

Master Data Services consists of the following components:

• Master Data Services Configuration Manager — used to setup the MDS Database and configure the Web application
• Master Data Manager — ASP.NET application used to manage your master data
• Master Data Services Web Service — used by developers to extend and customize the functionality of Master Data Services

Note that Master Data Services (MDS) is available on only the 64-bit editions of SQL Server 2008 R2 Datacenter, Enterprise and Developer. It is not supported on the 32-bit platform. It requires the .NET Framework 3.5 SP1 or later. For running the Web Applications, IIS is required with ASP.NET and Windows Authentication enabled.

Here is a good MSDN Blog on installing MDS.

Good luck!