Introduction

 Top Next

These eNotes are written for anyone wishing to improve their skills and knowledge regarding SQL Server.

Anyone who has already made simple beginnings into querying SQL databases using the SELECT statement,

and anyone who has gotten ADO.NET to work for them in an application where they used CRUD effectively.

The available VS solution for 'BEST company' will aid you with the ADO.NET part of this, and the ClickStart

eNotes entitled 'ClickStartSQL' will start you off with acquiring SQL Server 2008/5 as well as starting Management

Studio (SSMS).

In these notes we explore in more depth the use of T-SQL scripting, in order to make life easier for us all.

If you really wish to move yourself forward I suggest you buy some of the video training materials suggested

in the following few subsections.

I have watched in detail (more than once) both Paul Nielsen and Mark Long, and recommend them both. One

is better for getting you going and the other is better at taking you that much further. But both have something

to offer at all levels, it has to be said.

To get the MOST from these notes you need to have SSMS installed on your PC, and preferably version 2008.

Just about all the example material is shown in the SSMS environment, and you will be able to follow along, as

long as you also acquire the free download databases of AdventureWorks, and AdventureWorksLT (light).

Although the order of the chapters in these notes has some logical 'flow', in fact you can use and refer to many

parts independent of the others, if you are prepared to think a little ;-)

Good Luck, on your quest to learn more T-SQL and SQL Server in general.

If you have gotten going with SQL Server before this, then maybe now is a good time to learn some more about

the database environment itself - find out about DB objects, how to access them and how to manage them.

Please note that at this stage of learning T-SQL we are not yet ready to worry too much about DB security,

and restricting access to users, as well as maximising DB performance. We are really about being able

to get the most from T-SQL scripting so that we can query in a wide variety of ways, and get back the data

we require to solve our business problems.

To find out more about Database design and creation, as well as a whole other host of things to do with the SQL

Server DB then look out the 'ClickStartDB' eNotes which will be on their way into existence soon ;O)

See you in the next section.

PHH - July 21st. 2010.


Main Aims and Objectives

 Top Previous Next

This short section has come about due to my personal circumstances.

Having 'eased' my way into SQL and its use with ADO.NET in Vulcan some two years ago, I now

find myself needing to get deeply into Reporting Services (RS) for my clients.

As well as designing and testing new reports for RS, I am also asked to design and create script

files which will carry out certain data cleansing tasks on their SQL data which came over from well

used DBFs which were in use for some eighteen years - first with Clipper, then with Visual Objects.

Main Aim :

So my main aim in doing these notes is to be able to deign reports of a variety of formats, using a

suitable set of SQL scripting techniques, making sure I use SQL in the appropriate fashion at all times.

The detail of the report layout is covered in another set of ClickStart eNotes called 'ClickStartRS' but

these notes are to cover all the supporting SQL 'stuff'.

Some Objectives :

In order to reach our overall goal we need to be able to achieve a few objectives on the way.

Modularizing a task and designing and creating the modules is one such objective. The use of Views,

UDFs and SPs is quite essential in this task, so we need to know the dos and don'ts !

Obviously, managing these extra DB objects is an essential pert of the task too.

On other objective high on my list was to be able to develop a high level of skills with de-normalising

the many and complex table structure that my clients seemed to have acquired - for a number of

reasons - none of them to do with me.

Learning to merge their data using JOIN and UNION operators was crucial.

Also, the complex nature of the table field names meant that I needed to get to grips with the CASE

function. This is very flexible and powerful for overcoming some of the rather 'odd' table structure when

designing queries, and doing de-normalisation generally.

Finally, I wanted to develop much greater skills with Management Studio (SSMS) so that I can use it

in support of the design work I do with the report designer in BIDS (business intelligence development studio).

It is very useful to have SSMS open and in use on one screen, while working with RS in BIDS on another.

If you have similar needs to those discussed above then you will certainly benefit from reading and

studying these notes.

PHH - July 31st. 2010.


Welcome and 'How best to Use'

Quite honestly, if you have the time - or can make time - then work through the whole of these

notes with Management Studio (preferably 2008) on one screen, while viewing the notes in your

browser on the other screen. Having two screens (monitors) will help a great deal.

Because I have included a lot of images of the short test scripts, you can't copy and paste, but

this may be to your advantage as you need to gain experience at using the SSMS shell environment

at the same time as becoming familiar with the syntax of the T-SQL script itself. So get typing !

I also suggest that all users have downloaded SQL Server 2008 Express before starting work. As

well as the Server, download the Advance Services which may well now be part of the general

download of Express 2008. Advanced Services is Management Studio and Business Intelligence

Development Studio. Both of these are excellent, and the first is needed for the DB querying and

management side of things, and the BIDS is for developing report with Reporting Services.

There is a set of ClickStart eNotes for Reporting Services, it is called 'ClickStartRS'.

One thing you need to know and that is learning SQL takes some time and some effort, but it is

well worth it. You will need to keep coming back to sections of these notes to learn and try more

and different things.

So don't be in a hurry, no rush is required ;O))

I also suggest you look carefully through the next subsection on frequently used terms and words.

One thing these notes 'suffer from' and that is a lot of acronyms, that is things like SSMS and DB !!

See you in the next section.

PHH - July 21st. 2010.


Fashion, and how things change !

It would appear to me that when we come to SQL Server these days - around the 2010 year mark,

we have a whole raft of ancient and modern issues and sentiments to deal with.

By this I mean it is not always easy to see that some material is actually quite old, and was written

when things were very different in the world out there.

What was a good idea back then - pre 2000 - is not necessarily where we are all headed today.

Networks were much slower when the earlier SQL Servers were introduced, and the internet was not

even an issue or a reality.

One thing which was not on the fashion 'catwalk' was the use of 'Objects'. Now business objects, in

and throughout our application programming, is paramount to many of us developers.

New software technologies based on ORM (object related mapping) and LINQ (language integrated query)

mean some of us are wishing to keep our scripting within the application itself, and move away from

having large chunks of logic in stored scripts within the database.

ORM is in the shape of 'nHibernate' and Microsoft's Entity Framework - now at version 4 I believe.

Some things stay as they were ! :

Having said what fun and interesting things are on the horizon, and even in our midst, with ORM and

LINQ, we as paid workers - needing our clients - also find ourselves in a position where we need to

support legacy databases, with older legacy scripts and designs generally.

Quite often the third party software suppliers - for example SAGE accounting - will even specify an

older SQL Server version than is currently on the market. My clients have just had new SAGE systems

installed (SAGE 20 in one and SAGE 1000 in another) and yet they will not allow SQL Server 2008 to

be used as the backend data server.

Unfortunately that means we have to use Reporting Services for 2005 as well, and this means reports

are based on a different namespace, and that report designer tools used can't be the nice new shiny

ones with far better facilities.

To make matters worse, we see in the SQL scripts and objects put in place by third party software

houses, some very old techniques, and even a few bad practices.

So what do we do ? :

Well, first of all if we are aware of the situation it helps a lot. If we know that some of the quantity of good

learning materials for SQL Server 2005 are talking some old ideas, we can sift out what we want to hear ;-)

Some of the better 'stuff' like the Paul Nielsen books and videos actually revel in the new things coming

on-stream. Not all authors are so forward looking however.

A simple approach of mine is that I try to do everything new on the SQL front, with the latest tools. This

way I keep my skills and knowledge up to date, and know what to steer clear of as far as deprecated ideas

and methods are concerned.

Application Programming :

As far as I am concerned, when it comes to developing applications I try to use the latest techniques that

are currently out there - as long as they are mature enough to be relied upon.

I very much like the 'object' approach and will try and move my SQL work in that direction as soon as possible.

Certainly an abstracted data layer fits in nicely with the new data binding facilities we see within WPF.

So for me I will work from WPF, towards SQL, and am currently looking into ORM with nHibernate. Any reports

I do will be based on Server 2008, and hopefully VS2010 will have a Report Viewer (designer) which uses the

2008 namespace - not like VS2008 which unfortunately still has the older 2005 'flavour' due to release pressures

that MS suffered in the past.

The Future :

It will be interesting to see just how things get better and easier - our difficult task is moving our clients along at

sufficient a pace as to even try and keep up !

One thing I am sure of is that changes will keep happening and that we will need to keep learning, and then one

day, just around the corner it will feel like, some SQL stuff will feel as 'old hat' as DBFs did a year or so back.

Good Luck - and keep learning ;o))

PHH - July 31st. 2010.


Frequently used terms and words

SQL - Structured Query Language

[ not Sequential ! - thanks Oscar ]

TSQL (T-SQL) - Transact SQL

SS - SQL Server - sequel server

MSSS - MSSQLS - Microsoft Sequel Server

SSMS - Sequel Server Management Studio

SS8MS - SSMS for 2008

VO - Visual Objects

DB - database

DBA - Database Administrator

DDL - Data Definition Language

DML - Data Manipulation Language

RDL - Report Definition Language

AW - AdventureWorks (database) - the full and larger version

AWLT - AdventureWorks Light (database) - the cut down version

VS - Visual Studio

CRUD - Create Update Delete (INSERT, UPDATE & DELETE)

UDF - User-Defined Function

SP - SPROC - Stored Procedure

UDSP - User-Defined Stored Procedure

BoL - Books on line

MS - Management Studio? - not often

MS - Microsoft

WE - Windows Explorer

OE - Object Explorer

BIDS - Business Intelligence Development Studio

RS - Reporting Services

PP - Properties Pane

ORM - Object Related Mapping

LINQ - Language Integrated Query

PHH - July 22nd. 2010.


What is T-SQL ?

Books on Line (BoL) has this to say :

When you create an application with Microsoft SQL Server, the Transact-SQL programming language is the primary

programming interface between your applications and the Microsoft SQL Server database. When you use Transact-SQL

programs, two methods are available for storing and executing the programs.

1.) You can store the programs locally and create applications that send the commands to SQL Server and process

the results.

2.) You can store the programs as stored procedures in SQL Server and create applications that execute the stored

procedures and process the results.

It is not surprising therefore, that when people talk of using T-SQL they simply are referring to the whole process of

scripting with SQL Server statements and batches.

Obviously, Transact-SQL is the Microsoft 'flavour' of the extra functionality and features required to make the ANSI basic

SQL 'stuff' work at a developer and API level.

PHH - July 21st. 2010.


T-SQL reference material, MSDN

'Books on Line' (BoL) :

One thing which is easy and free, are that is the extensive notes from Microsoft, often called

'Books on Line' (BoL).

If you open SSMS and click the Help from the main menu, you will be able to set the filter as

shown below :-

=======================================================================

While writing these notes I have had my 'Filtered by:' setting at 'Express' as shown above.

There is a HUGE amount of stuff in there for reference, and I have tried to copy a few simpler bits

of the material into these notes when it helped to get us going.

Be warned however, not all of their BoL examples are good ones, and also there are a couple of

mistakes which I have found I needed to correct in order to gain the expected output, or even

remove errors.

However, generally, the BoL material is excellent for reference, but it can't be read as a book.

Hopefully, these notes and BoL used together should get you a VERY long way indeed.

Web Access MSDN :

Use the link below to start and investigate just what a huge amount of material is out there on the web for SQL Server.

web links - MSDN .....

================================================================================================================================================================

Using the screen above I searched on 'creating views' and go this :-

=============================================================================================================================================================

Which when followed up gave me this :-

==================================================================================================================================================================

It is up to you to look for information and articles, as well as just notes.

It is all out there somewhere ;0)

See you in the next section.

PHH - July 21st. 2010.


Learning materials

Books and Training Videos :

Below I will show you what materials I have purchased and used effectively during my own learning.

Apart from a few helpful emails from my colleagues in VOPS I have learned everything I know about SQL

from these materials - Oh! and lately 'Books on Line' from the Help menu in Management Studio.

Paul Nielsen has had a big influence on my learning, he is knowledgeable and even handed. He also

seems to thrive on change and presents new features well.

He encourages guys to move forward with the times.

Don't be put off by the size of these two 'Bible' volumes - they have lots of good examples, and are more

readable than 'BoL'. BoL is good reference material but it is not 'readable' like Paul's Bibles.

Paul Nielsen :

His home page on his Bible site ....

An the even larger (well more pages anyway) volume :-

51SBu5SS 2008 DB development Training Kit_

PHH - July 21st. 2010.


Other 'ClickStart' volumes

Using the 'ClickStart' web site :

The main ClickStart web site is to be found at :-

ClickStart web site - home page .....

The domain name is ClickStartVulcan.net and it has been up and running for a few years.

Over the years more volumes of eNotes have been added and recently a home page 'shift around'

took place.

Now the 'executive' is to be found when you log on to the domain name. The Executive is a web

publication from 'Help and Manual 5' (Professional) which provides me with an option to publish to a

'webhelp' format in place of the usual CHM one. So the learning notes are used as a web site and

are not downloaded as a large CHM file as was the case previously.

The consequence of this is that you will always have available to you the latest work I am developing,

almost as soon as I do it. No re-downloading, or using old and incomplete material. those days are

gone.

By having the executive I can add more material to the learning zone of the main site very easily.

New volumes can be added in one or two minutes, without much disruption to the existing material.

At the moment there are twelve links to eVolumes but the last one in the list needs a lot of work.

There is currently much material for dotNet guys wishing to follow in the Vulcan trail, as well as those

wishing to get into Visual Studio and Reporting Services.

Creating applications using Forms is extensively covered, and the WPF volume is reasonably substantial

and can help get guys into WPF quite a lot. The WPF eNotes work well without any need to have to

learn XAML first - it is all done with dotNet classes, and will be very familiar to you.

My focus as from now, and into the near future, will be to go deeper into WPF and all that it offers the

developer, use SQL Server and Reporting Services with it and using the flexible and easy to use Data

Binding within WPF.

I have started a push into ORM (Object Relational Mapping) and have got some way with NHibernate.

It is my intension to follow this up with a new volume of the eNotes soon.

I also intend to look in depth and detail into what documentation options the WPF platform has to offer us.

Good Luck with your learning and personal development.

*** Remember *** All of the eVolumes are 'growing' entities - they are not complete, they keep on growing,

and new ones need work to clean up initial note writing typos etc. However, my notes are available to me

and anyone visiting the site, almost as soon as I have written them. Usually, a new section or two gets

written each day.

PHH - August 5th. 2010.


Sample databases to use

From my PC system I just Googled as below :-

[ you may need to follow a couple of false trails first - as this DB is old hat and not directly supported by MS these days. ]

But version for 2008 and 2005 - full and light are out there somewhere !

download AW 2008 database ....

=================================================================================================================

download AW 2008 database .......

=============================================================================================================

And if you follow your nose correctly you will get this :-

============================================================================================================

AdventureWorksLT (light) :

==================================================================================================================================

CodePlex - DB downloads .....

===================================================================================================================================================================

===============================================================================================================

And when you are just about there you see this :-

====================================================================================================================

Notice that the 'light' form of the DB is 1.32 MB as opposed to 82.3 MB for the full version.

I would suggest you get both, as both are used in my notes and the full AW is used extensively in BoL.

Good Luck with downloading !

See you in the next section.

PHH - July 29th. 2010.


Acknowledgements

Thanks are due to a number of my VOPS colleagues for their help and support over many years now.

Mike Pitcher, Colin Newell, Ginny Caughey, Willie Moore, and Geoff Schaller who pushed me to get going.

Local colleague Andrew Watts (currently of Prosys in Cardiff, South Wales) has provided me with some

excellent ideas and approaches to SQL problems - thanks go to him !

I also owe Alwyn Jennings-Bramly a special mention as he regularly communicates via email over most of

what I do as development work - his ears are well chewed ;0)

Thanks also to Brian Feldman, Paul Piko, Robert van der Hulst and Chris Prygas for help and support with

just about everything Pearls and ClickStart based.

I hope you learn as much as you are hoping !

PHH - July 21st. 2010.


Words of Advice - from Paul Nielsen

===========================================================================================================================================================================================

SQL blog site .....

I suggest you read the above advice - KISS - keep it simple stupid !!!

I will look out the download to see it it helps at all with the section on "Backup and Restore".

PHH _ August 5th. 2010.


Chapter 0 - Database Creation

Before we start work on creating our own database with MS SQL Server we need to recognize that this product is

a VERY sophisticated one, and that there are many levels of complexity that we a basic users need not 'visit', or

certainly worry about for a long time to come.

In this chapter we will look at how to use both T-SQL script and the Management Studio (MS) 'shell' to do necessary

tasks related to making a database (DB) from scratch.

We will also look at how to 'sensibly' normalize our data, as well as making Primary and Foreign keys to relate one

table with another.

Although it is possible to store and retrieve data from a set of tables without keys or indexes, we will see how to use

these to make our DB solution better.

Keys :

Using primary and Foreign keys in an effective manner, allows us to more easily 'partition' or 'separate' our data, as well as

retrieving it more easily with JOINs.

Referential Integrity :

One feature of DBs with MS SS which is extremely useful to guys coming new to SQL is the way we can easily prevent

creating 'orphans' in our data tables.

By setting constraints correctly we can avoid the case, for example, where on order exists in a table without the customer

who made the order being in the Customer table. We will see how to protect - or preserve - our data 'integrity' by making the

necessary DB settings on our tables.

Indexes :

Once we have our heads around keys, relations and constraints, we can look at some performance enhancing aspects

of using indexes to improve our 'querying'.

It would appear from information from the Paul Nielsen SQL training DVD that for most of us we need not concern ourselves

about indexing, as long as we set our Primary Keys suitably, on each table we create. Also a correct (most suitable?)

choice of type of 'key' also makes everything work smoothly, and relieves us of further decision making.

For many guys like myself, using client's existing SQL databases, there is even less need to worry, as someone else is

responsible for DBA type work - I don't have 'personal' permission for changing and creating indexes, even though I may

have the security rights !!

For guys coming from VO and DBF technologies, the SQL world is quite different. It is SQL Server itself which chooses

the indexes to use when answering a query. It is said to have a 'query optimizer'.

MS SS is a very clever bit of software and we should leave it to do its own thing. What we have to do is find out, and then apply,

the correct way to set up our tables and also our PKs and related FKs.

Anyway, with EF4 (Entity Framework) we don't even have to make our own tables any more , so let us relax about the

whole business ;-0)) As long as we make our business objects then EF4 takes care of tables and other bits too.

Lets get started by making our first DB in the next subsection - first we need to checkout what DDL and DML are, and

what they can do for us - see you in those subsections ..............................

PHH - 9th. December 2011.


DDL - data definition language

The SDK documentation says this of DDL for Relational Database Management Systems (RDBMS) :-

The subset of SQL statements that define all attributes and properties of a database and its objects.

CREATE, ALTER and DROP are all part of the DDL set of commands.

TRUNCATE is another useful DDL command, as is USE.

These are used to manage the objects within a database (DB).

So DDL is for 'objects' !!

*** CARE ***

Using DROP in a script causes the object being dropped to be 'removed' from the database.

When the object is dropped all of its data is lost to the RDBMS - for ever !

Some useful information by Mike Chapple :

from :- http://databases.about.com/od/sql/a/sqlfundamentals_2.htm

The Data Definition Language (DDL) is used to create and destroy databases and database objects.

These commands will primarily be used by database administrators (DBAs) during the set-up and removal phases

of a database project.

TRUNCATE :

This removes all rows from a table.

The operation cannot be rolled back and no triggers will be fired.

As such, TRUCATE is faster and doesn't use as much undo space as a DELETE.

DROP :

This command removes a table from the database. All the table's rows, indexes and privileges will also be removed.

No DML triggers will be fired. The operation cannot be rolled back.

DROP and TRUNCATE are DDL commands, whereas DELETE is a DML command (data manipulation language).

Therefore DELETE operations can be rolled back (undone), while DROP and TRUNCATE operations cannot be rolled back.

USE :

The USE command allows you to specify the database you wish to work with within your DBMS.

For example, if we're currently working in the sales database and want to issue some commands that will affect the

employees database, we would preface them with the following SQL command:

USE employees

It's important to always be conscious of the database you are working in before issuing SQL commands that manipulate data.

ALTER :

Once you've created a table within a database, you may wish to modify the definition of it.

The ALTER command allows you to make changes to the structure of a table without deleting and recreating it.

DROP - more details :

The final command of the Data Definition Language, DROP, allows us to remove entire database objects from our DBMS.

For example, if we want to permanently remove the 'personal_info' table that we created, we'd use the following command:

DROP TABLE personal_info

Similarly, the command below would be used to remove the entire employees database:

DROP DATABASE employees

Use this command with care! Remember that the DROP command removes entire data structures from your database.

If you want to remove individual records, use the DELETE command of the Data Manipulation Language.

CREATE :

Installing a database management system (DBMS) on a computer allows you to create and manage many independent

databases.

For example, you may want to maintain a database of customer contacts for your sales department and a personnel

database for your HR department.

The CREATE command can be used to establish each of these databases on your platform. For example, the command:

CREATE DATABASE employees

creates an empty database named "employees" on your DBMS.

After creating the database, your next step is to create tables that will contain data.

Another variant of the CREATE command can be used for this purpose. The command:

CREATE TABLE personal_info (first_name char(20) not null, last_name char(20) not null, employee_id int not null)

establishes a table titled "personal_info" in the current database.

In our example, the table contains three attributes: 'first_name', 'last_name' and 'employee_id'.

That's the Data Definition Language in a nutshell.

Giving things a try :

For many of us, we will already be familiar with Use, Create, Alter and Drop - to a greater or lesser degree -

however, I suspect that you, like me, have not seen TRUNCATE in use before.

If you have seen it then you should not be reading these notes I suspect ;-)

Anyway, fun apart, let us see how the DDL command 'TRUNCATE' operates.

Checkout this below - and don't worry, I have a script to repopulate the Table once it is empty !

Study the following image carefully :-

=================================================================

In the above image the 8 and 0 refer to the 'SELECT' statements.

The 'TRUNCATE' does not produce a line in the messages, unless run on its own as shown in the

next image below :-

=====================================================

Now let us click the other tab - called 'Results' to see what we get from the original three sectioned

query statement - check this out carefully :-

===============================================================================================================================================================================================

Yes, just as we wanted, the eight lines before the 'TRUNCATE' statement finally shows as zero lines.

Also you need to remember that using TRUNCATE on a larger Table will be better from a performance

point of view. Don't use DELETE on a large table when you could use TRUNCATE - if only selected rows need

deleting that is different, here we are discussing the whole lot.

Time to move on, we will see DDL commands used many places during our exploration of the rest of this Chapter.

See you in the next subsection ............................

PHH - 16th. December 2011.


DML - Data Manipulation Language

There is a subset of the SQL statements which are used to change and manipulate the information stored

within the rows in our SQL tables.

DML stands for 'Data Manipulation Language'.

The commands in the DML subset are :- INSERT, SELECT, UPDATE and DELETE

Mike Chapple has something useful to say in his article at URL :-

http://databases.about.com/od/sql/a/sqlfundamentals_3.htm

Because we will be using all these commands so much throughout this chapter we will say only a little about them here.

INSERT :

The INSERT command in SQL is used to add records to an existing table.

SELECT :

The SELECT command is the most commonly used command in SQL. It allows database users to retrieve the specific information they desire from an operational database.

UPDATE :

The UPDATE command can be used to modify information contained within a table, either in bulk or individually.

Each year, a company gives all employees a 3% 'cost-of-living' increase in their salary.

The following SQL command could be used to quickly apply this to all of the employees stored in the database:

UPDATE personal_info
SET salary = salary * 1.03

DELETE :

You'll find that the syntax of this command is similar to that of the other DML commands.

However, be very careful as it is VERY easy to script things such as whole groups of rows get 'deleted' in error.

Luckily, if you use script correctly, the DELETE command may be 'rolled back', unlike the TRUNCATE of the DDL.

Just be EXTRA careful when doing any large scale data 'changing' operation.

Time to move on, and see how to create a new table - see you in the next subsection ....................

PHH - 16th. December 2011.


CREATE a new DB

Using Script :

Well, believe it or not, this is one thing which is very easy to do.

In the image shown below we have a script waiting to be run in a MS query pane, and the existing DBs are

shown in the Object Explorer (OE).

Check this out below :-

========================================================================================================

In the above image notice that the list of DBs in the OE pane does not yet include 'MyClickStartTest'.

So lets us now highlight the full script and choose to click the 'Execute' button - see below :-

=======================================================================

After the execution do not expect the new DB to appear in the list in the OE pane.

You will probably most likely need to refresh the OE by right clicking the 'Databases' node in the 'TreeView'

pane of the OE, and using the context menu as shown below :-

===========================================================

Sometimes it may appear itself, and will then be placed at the bottom of the DB list, but when you do a 'Refresh'

it will appear in the A-Z position of an alphabetic sort.

This is what I now have in my MS shell - see the following illustration :-

=======================================================================================================

What else do we have in our expanded DB entry in the OE pane ?

Let us see shall we ?

=================================================

In the image above I have expanded the 'Tables' node so that we can see that we do not as yet have any

'user' tables.

We will make our own first table in the next subsection. For now, let us see how we can make a new DB using

the GUI of the MS 'shell'.

Using the MS 'shell' :

Lets us close down the OE pane so we have all DB node collapsed.

By right clicking on the 'Databases' node we get the context menu, from which we will select 'New Database...'

See below for how this looks :-

============================================================

We are given a dialog screen when we do the selection, as shown beneath :-

====================================================================================================================================================================================================

All I have done is type in a name for our new DB - the rest was done by the 'shell' or IDE.

Notice the path where the DB will be placed - this is very common for MS SS operations of this type, whatever the

version (or 'flavour') of the Server itself.

Note that for security and access reasons, when I place an MDF file from elsewhere, I place it within my Windows 'User'

folder structure - just be aware that you can cause yourself grief when copying and moving MDFs yourself.

So let us click the 'Add' button at the bottom of the 'New Database' dialog screen.

This is what I see - view the image below :-

==============================================================================================

And please notice that although the new DB is shown in the OE pane, it is not in alphabetic order until we

do a 'refresh' - or close and open MS again ;-0)

Doing a Refresh I now see this below :-

==============================================

You may be aware that from the context menu which appears when various nodes in the OE pane are selected, we often get a

choice which includes the word 'Script'.

The 'Script' option will give us full scripting details provided by the 'system' - see below for the DB case :-

===============================================================================================================================

So then, we should expect to see something small, and / or short, since we typed very little to make the

first one called 'MyClickStartTest'.

Below is what we get when we ask to see the script to create the Database - be prepared to be 'alarmed' !!

Check this out below :-

=================================================================================================

Wow! I hear you say - and that is what I think too ;-0)

I would appear that possibly the default values are being applied, just in case - just to make sure.

We certainly did not need to do all this stuff when we made our DB way above, at the start of this subsection.

Okay then guys, now that we have a new DB, let us move on to some design considerations, and making our first table.

See you in the next subsection ..............

PHH - 9th. December 2011.


CREATE a first Table

*** IMPORTANT *** please note this fact >>

Warning Message:

Saving changes is not permitted. The changes you have made require the following tables to be dropped and re-created.

You have either made changes to a table that can't be re-created or enabled the option Prevent saving changes that require

the table to be re-created.

This is by design and can be quickly fixed in Management Studio by un-checking a property.

To fix this in Management Studio, go to Tools -> Options then go to the Designer Page and un-check "Prevent saving

changes that require table re-creation".

http://www.bidn.com/blogs/BrianKnight/ssis/52/sql-server-2008-designer-behavior-change-saving-changes-not-permitted-1

Many thanks to the above web link, and Brian Knight.

=================================================================================================

Okay then guys, let us start to make a Table or two for our new database (DB).

Using GUI tools :

To our so far 'empty' DB called 'MyClickStartTest' we need to use the GUI tools from the context menu to

allow us to easily make our first Table.

The Tables we make are often called 'User' tables in the context of the DB itself. As you will see in other parts of these

eNotes there are 'System Tables' which are used by the MS SS system to manage the DB.

Study the following image :-

=====================================================

When we choose the 'New Table' option we get a new 'tab' in the shell which looks like the following :-

[ * note * we do NOT get a dialog screen as you may have expected ! ]

====================================================================================================================

Although you can't see it (because I did a screen capture at slightly the wrong time) there is a flashing cursor

within the input box of the 'Column Name' column.

We are to supply a set of column names, along with data type, and also say if we wish to allow the data column to

have NULL 'settings'. I say settings as 'values' is not really the NULL way of talking.

If we allow NULLs then a null signifies that a value is so far not associated with the data cell of the column being specified.

Good performance, and clear DB scripting (and coding) comes from handling NULLs correctly. Do not be afraid of them, just

learn to treat them as they were intended - this was a good piece of advice I got from a VO/VN Guru !!

What are we aiming at ? :

Before we get 'sucked into' discussing details about data types for any of the columns we add to our table, let us first of all look

at an existing table - and how it appears when expanded in the OE pane - study the following image closely :-

============================================================

In the above image we can see that seven user columns are in the 'ClickStartTest' table of the 'GBPRODUCTCODES' database.

We can see a bunch of column names as well as data types and also if they are 'nullable'. Yes, we often see and use the

word 'nullable' meaning that the object in question can take a NULL.

The image shown above has all of its 'user' columns set to 'nullable' - if they were 'not nullable' they would show 'not null' at the end.

This means that the one column placed in our table by the system - called RECID - is the only one to NOT allow nulls. This

is simply because in the absence of a suitable PK field the system uses this field for 'tracking' - management stuff.

You will often (always?) see a column called 'rowid' or 'RECID' (or similar) in a table without a suitable PK or UNIQUE constraint.

Examining the 'New Table' tab in more detail :

Okay then, lets start entering some information, and call our first column 'PersonID' - and then look at the data type values

available in the drop list.

Here is a start, and what the tab looks like while I am trying to select 'int' as my data type for 'PersonID' :-

=============================================================

The whole range of values for the data type selection are show in the images below :-

================================================================================

[ ** note1 ** I have duplicated last value of each drop list so as NOT to miss any out ! ]

{ .. also note that these are for MSSS 2005, and there are others for version 2008 .. }

*** NOTE2 *** The 'n' before any text or string data type signifies 'Unicode' or two bytes for National

Language variations. These days we all ought to use Unicode for new designs - for many

sensible reasons.

Here is a small sample table which is complete enough for us to save it - but before we do

take a look at the structure :-

==========================================================

So much of what we see is common sense - but we allow Date of Birth (DoB) and 'Salutation' to be NULL is we

so wish. We may not know if a person is Miss, Mrs, or Miss, and we will often not know their DoB.

Also I should have used 'nvarchar' in place of 'varchar', this was a mistake as now I can't use Unicode for names

with multi byte characters - we must put this right in real DBs we design.

Before we move on lets see the rest of the Tab, the area below the table 'Design'. In the following image we will

see the complete Column Properties tab which displays at the bottom of the New Table design tab.

Check this out :-

===================================================================

You may be interested to know that you can re-set (or second choose) the data type from within the properties tab,

AND, that the drop list displayed may be re-sized as shown below (see the bottom LH 45 degree 'corner' handle) :-

[ .. LH stands for Left Hand .. ]

==========================================================

If we close down the expanded sections as much as possible, while still having something to see, we get

the following to view :-

========================================================

Okay then, lets move on to save our table.

Saving the Table :

Right click the tab itself (yes, I know you have to 'know' this - can't go looking for it), then

you will be given a context menu - see below :-

===================================================================

Then when you see the following dialog screen, provide a suitable name.

Like a lot of things "SQL database", try to get the naming right the first time, so you don't have to change things "down the line".

In the following image we are naming the new table 'MyPersons' - yes, a stupid name for any real and productive DB, but a good

one for demo and testing, and writing learning / training eNotes.

Check this out :-

==============================================================================

And once you have saved the new name, the tab will be suitably labelled, as seen below :-

===========================================================================

And, the structure of the new and growing DB will look like the following :-

==========================================================================================================================

Well, so far so good, but what if we need to make changes ?

Editing, Updating, and Developing an existing Table :

Okay then, at first this subtitle does not seem a bad one - BUT - should we be making changes at all. Also,

just what changes can be made once the table is in use ?

Well, as usual I am not stuck for words and/or advice ;-0) I suggest that you make few changes at all after your design

phase of the new DB project. AND, certainly you make NO table 'column' changes after you start to populate with real data.

While a table is empty you have more flexibility to add new columns, and change existing details. But I have found that

some times the MS shell gives messages about needing to drop and remake the table. Not a great idea when it may contain

necessary data.

This section is NOT intended to be any bunch of 'tips and tricks' on "How to" as far as Table structure design is concerned,

BUT, it is really a firm message to you about thinking carefully over your DB design, and getting things right at the start,

- BEFORE - you do any data population.

Even the names of your tables need very careful thought, never mind the names of columns.

Adding to our existing Table structure :

Okay then, we have a simple enough structure to our Persons table, let us add a few extra columns for the address details.

See what I got here after adding a few lines of extra column data, all basic stuff :-

==========================================================================================================================

Well, I did warn you ;-0) I have not figured out when or why this happens, or which extra columns (and data types?) can

be added without problems.

[ *** note *** possible solution found at top of subsection - just now need to see what happens with existing data. ]

Lets try scripting the table to a query pane, and then adding our extra columns as extra script lines - noting carefully the

position of the separating commas.

Below is what I typed at the query pane after MS SS gave me the existing script to add to :-

=============================================================================================================================================================================

Please note that an extra comma needs to be provided for the last line - in our case after the DoB line.

And in a similar way we need to refrain from adding a comma to the last line we add - so no comma

after 'PostCode'.

If we highlight and execute the whole script we get 'success' and then after a refresh in the OE we see

the details as shown in the above image.

To add to our possible confusion we can see that if I try to add one extra line of type 'int' (or INTEGER)

we seem to get success where previously we had failure - see below :-

===========================================================================================

We seem to be able to add an extra simple data type line to the end of our table specification BUT when

we try to insert it in the middle somewhere we get back our error box from MS SS - see below :-

[ .. remember this is now solved see very top of subsection .. ]

=============================================================================================

But if I then Drag and Drop the line from the middle back to the bottom, the Save from the Tab right click seems

to work just fine - see below :-

=======================================================================================================

*** NOTE *** This problem has been fixed by using Tools and Options to change a property = see top of this section.

However, we may need to be careful of losing existing test and set-up data. We need to do an experiment - or two!

Summary so Far :

Plan and think well ahead, then you will spend less time 'fighting' the system afterwards ;-0)

Get your table names correct, as too the column specifications (names and data types), even for your first 'save' of each

table structure.

Once you have a table structure you will normally wish to set Keys and Relations, but this is covered in later subsections.

Data Types :

Time to move on and look in more detail at the possible Data Types on offer to us.

See you in the next subsection ......................................

PHH - 9th. December 2011.


Adding Rows - Test Data

Okay then guys, we need to be able to add some test (or demo) data to one of our Tables.

The reason for doing so is to be able to test out thoroughly all parts of our database (DB) design.

It is recognised that even with good planning we may well need to alter some of our table structure after

we have already created it, for these reason we need a small amount of test data to experiment with, well

before we think of porting in our main data set.

I suggest that you work with a small amount of test data in each of your tables, before you even think of

inputting real data.

As thing unfurl below, we will see if changing table structure (ALTER) will lose us any existing table data.

For the moment we will work with our test Table called 'MyPersons' - see how it currently looks below :-

=============================================================================================================================================================

And the Table structure is as follows :-

=============================================

[ .. notice the null and not null declarations / definitions .. ]

So how do we add rows of new data ? Also be aware that because we have not as yet got a Primary Key

set on the field 'PersonID' with seed value and increment also set, we will need to provide a value, it will not

carry out 'auto-incrementation'.

Look at the following image to see one way, that where we add something for every field, and also keep to

the same order as set out in the column structure of the Object Explorer (OE) pane.

Check this out for yourself :-

=============================================================================================================================================================

In the above image we can see a new line at row position 8 and this line has our inputted data values showing clearly.

Note that after the INSERT operation I had to do a 'SELECT * ' in order to list the table contents as shown above.

Variations on the 'format' of the INSERT statement :

While we are adding some test data for experimental purposes, we may as well examine the different ways that

it is possible to use the INSERT statement.

First of all we need to simply appreciate that all field (column) values which are declared as 'NOT NULL' in the

Table definition need to be supplied - otherwise we will generate an error.

However, having said that we are then at liberty to change around the order of the fields within the INSERT statement.

Before we play around with the order, let us look at, and study carefully, the image below.

This is a shorthand way of inputting every value in the Table structure, and in the Table's fixed (defined) order :-

In the image shown above we can see that we now have two identical rows, and this also means a duplicate ID value.

This would not have happened if we had PK set on the 'PersonID' column, with other suitable settings, as we will see

later in this subsection.

Leaving out all but the necessary fields :

Okay then you guys, let us try and cut things down a little. Let us tray and ONLY supply values for the 'NOT NULL' columns.

This means we can ignore six (6) fields of those we have used above - those allowing NULLs.

If we carefully re-order the script 'text' and comment out suitable parts we end up with the following - check it out :-

===========================================================================================================================================================

Once again, in the above image we see the returned result set to a 'SELECT * ' type query, after the successful insert.

There is one thing to notice extra to what we may have expected, and that is we have only five (5) NULL entries. This

is because we currently have a default constraint set on the 'Salutation' field, that is where the 'Mr.' has come from.

Now let us be brave, and change around the order of inserting, I will rearrange each of the three active lines.

Checkout the following detailed image :-

==========================================================================================================================================================

And once again we have an extra row of data in our new, and growing, test Table. Obviously,

the data items have gone into the correct fields / columns since this latest row is identical to the

previous one - which was added before we re-organised the INSERT statement.

Changing some existing values :

Okay then, this subsection is really all about the 'test phase' and development of a new DB, with associated suitable

Table structures.

We now have a perfect situation to see an example of what the 'Edit Top 200 Rows' GUI option is all about.

Here is what we see when we right click the 'MyPersons' table node in the OE pane :-

======================================================

And after selecting we see this new tab in our MS 'shell' :-

[ .. note the RHS has been 'chopped' ! .. ]

================================================================================================================

From the 'Edit' tab grid we can enter new rows, delete existing ones, and change detail in each specific cell. Of course

we need to keep to the existing underlying rules for our table, so we can't place NULLs where they are not allowed,

and also we must stick with any CHECK constraints also.

What I was thinking we could do with the Edit tab, was change the 'PersonID' values in the later entries so we have

unique values in the data set. This is the sort of thing that I feel the 'Edit Tab' option is all about - original design and

adjustment of the Tables in a new DB - or even just a Table added at a later stage to an existing, and well used DB.

Before we use 'Edit Tab' :

Let us take an opportunity while we have it - let us try and change the Table design so that we make a PK on the

column called 'PersonID', just to see what happens.

Check this out below :-

========================================================

And a new Query Tab is made looking like this below :-

==============================================================

Right click in the 'Design' part of the tab to bring up the context menu - as seen below :-

==========================================================

Select the option to set a PK constraint - as I have done in the following image :-

============================================================

Now the BIG test. Let us see if we can 'save' this using the top tab context menu option.

Here is an image of just before I tried to do a 'save' :-

===================================================================

Oh! and here is what I get after making the selection from the GUI :-

================================================================================================

=============================================================================================

=============================================================================================

Well, of course we should get this as we have a few 1241 values entered into the 'PersonID' column.

However, can we use the Edit tab to change this directly ?

Let us try, and see just what happens :-

=============================================================================================

The GUI environment does not seem to help, we still get a problem, as seen above.

And if you study the following images (3) you see that we can't 'delete' from the GUI tools either !!

==================================================================

Although it looks as though it is going to work (see next image), it does not :-

==============================================================================================

And here is the 'brick wall' that we seem to hit - check this out below :-

=============================================================================================

So, we seem stuck - BUT - can we script our way out of trouble ? Let us see.

What if we write a script to try and delete all of the '1241' entries ?

Study the following few images carefully :-

========================================================================

Paul Nielsen in his DVD training (and others) suggest it is always good practice to use a SELECT statement first, before

we do the DELETE, just to check that we actually have the correct data set - the one we are after.

Now the following is what we have after running the DELETE script - check out the following :-

==================================================================================================================

Yep! all our 1241 row entries have gone - been removed.

Now we can return to the 'Edit Tab' and make the PK changes - note you may have to first close and reopen it. This

whole process is rather a 'life experience' one, a bit like driving a car on the road ;-0) A lot is just done by 'feel'.

Anyway, I have eventually got my PK settings to 'stick' and here is my refreshed OE pane :-

================================================================

Notice in the image above that we have a few changes to the OE pane node entries.

Those (three in all) in the Keys, Indexes, and Statistics nodes are due to us having a PK set on the 'PersonID' field.

Below I have used the EditTab to directly enter a row of values for a new Person record.

Because the first column is now a PK field the values are 'greyed' out as these are auto-generated.

Study the following image carefully :-

====================================================================================================================================

In the above image notice that the seed value of '2000' has been used automatically by the SQL Server.

[ .. you will see in the section on Keys why we have apparently lost a record for '2000' .. ]

Adding Multiple Rows By Script :

When we are still at the test and development phase of our project it can be very handy (useful) to have a

single script which adds a whole bunch of rows to our new table.

This way we can always DROP and CREATE a new structure, and then populate the columns and rows again

quite easily.

For this part of the notes we will simply study the script to 'delete' all rows and then repopulate them from the

script we have written. In the subsection on DDL and the 'TRUNCATE' command we will see another way of

doing the deleting.

Study the following image which is of a script I have checked out thoroughly at this end.

===========================================================================================================================================================================================================

The Message tab below the Query pane shows the following :-

==============================

Which is what we might have expected - 8 lines deleted, then eight times a single insert operation, followed

by 8 lines of selected dataset return.

Saving our Efforts :

It is a good idea to save this script as we have put some effort into its writing or creation !

Use the top tab context menu as shown below :-

============================================================================

This brings up the dialog input screen shown below - note I have made a new folder for our scripts we create for

this eNotes project - see below :-

Any subsequent 'saving' either through the tab GUI menu, or by keys - 'Ctrl+S' - will just do a 'silent' save update, as in other

Windows applications.

Summary :

We are now in a position to be able to delete all our data rows if we need to. We can then DROP the table and re-CREATE it

with similar or different column structure etc..

If we make any significant changes to the structure then we will need to change the script shown above. Remember, the

order of the columns is taken as the natural order in our table, and we need to supply ALL field values unless we change the

script text drastically.

This whole area is a bit of a 'catch 22' one, we have an INSERT script which works for a set table structure - BUT - we may

well be changing that structure !! You will just have to live with that ;-0)

It seems even MORE necessary to me that we try to get our table structure correct at the design and creation stage, yes,

do some testing with minimal data, BUT, do not commit yourself to any reasonable amount of data until you have things

all correct and tested. And you know you are then not going to change table structure any more.

Time to move on, see you in the next subsection .......................

PHH - 17th. December 2011.


data script

use [MyClickStartTest]

go

DELETE [dbo].[MyPersons]

where [PersonID] is not NULL

;

GO

INSERT into [dbo].[MyPersons] VALUES ('SP7 3ZX','..','Jimmy','Fixit','n/a','m/t','13, Front Street','Sidepan Alley','1800-01-02',9,17,'1989-03-25');

INSERT into [dbo].[MyPersons] VALUES ('NP19 1QX','..','Mary','Dorrit','n/a','m/t','26, Golden Avenue','St. Julians','1800-01-02',29,13,'1977-05-17');

INSERT into [dbo].[MyPersons] VALUES ('NP20 4WT','..','Billy','Merrier','n/a','m/t','39, Green Meadow','Malpas','1800-01-02',29,13,'1965-09-21');

INSERT into [dbo].[MyPersons] VALUES ('NP18 2GT','..','Anne','Wilder','n/a','m/t','69, Brownlea Drive','Pill','1800-01-02',29,32,'1973-11-02');

INSERT into [dbo].[MyPersons] VALUES ('YN88 6TY','..','Harry','Hodge','n/a','m/t','59, Yellowstone Park','Whynot','1800-01-02',29,32,'1999-07-11');

INSERT into [dbo].[MyPersons] VALUES ('LS13 2YR','..','Eva','Harrison','n/a','m/t','133, Nicely gardens','Hayhoe','1800-01-02',29,32,'2002-07-11');

INSERT into [dbo].[MyPersons] VALUES ('DL1 1AA','..','Mickey','Mouse','Mr.','m/t','1, Starthere Ho.','Disneyland','1800-01-02',21,14,'1952-07-11');

INSERT into [dbo].[MyPersons] VALUES ('PP1 0AA','..','Donald','Duck','Prof.','m/t','Pond Place','Field View','1800-01-02',NULL,NULL,'1952-03-18');

GO

select *

from [MyClickStartTest].[dbo].[MyPersons]

--INSERT into [dbo].[MyPersons] VALUES ('','..','','','n/a','m/t','','','1800-01-02',,,'')

--INSERT into [dbo].[MyPersons] VALUES ('','..','','','n/a','m/t','','','1800-01-02',,,'')

--INSERT into [dbo].[MyPersons] VALUES ('','..','','','n/a','m/t','','','1800-01-02',,,'')

PHH - 18th. December 2011.


Data Types - Fields & Columns

The Problem :

It would appear to me that selecting from the list of data types, at the time we need to design

our new DB Tables, is a bit of another 'Catch 22' situation.

By this I mean it is often the time we have little or no experience of SQL Server, and also ADO.NET

and the dotNet code generally needed to cope with a SQL database layer, and yet it is also the very

time we need to exercise plenty of experience too ;-0)

We need to know which data types to use to at least make some suitable basic tables with which we

can play around (or experiment) with.

It would be nice to be able to use commonly used data types in a meaningful way, but we also need to

keep away from some of the more advanced ones, which may also be more difficult for us to handle without

plenty of experience.

It is all a matter of balance - sensible, but not 'over the top'.

A Possible Solution :

When I first came to this problem, or dilemma, a few years back, I decided to restrict myself to data types (DT)

that would do the most common of things, and ones I could find in other sample DBs like AdventureWorks.

Here are a bunch I seem to have used successfully since then :-

int

bigint

datetime

bit ... Boolean

numeric(9,3) ... decimal is similar

money

Unicode :

nchar(n) ... 1 through 4000 (fixed)

nvarchar(n) ... n (1- 4000) | MAX (variable)

[ .. use in place of those below whenever possible .. ]

non-Unicode :

char(n) ... n (1- 8000) (fixed)

varchar(n) ... n (1- 8000) | MAX (variable)

So if you look carefully at the above I think you will see that we can try to use only the first eight (8) and in so

doing we ought to be able to do most things quite well.

Please realise that I am NOT saying that you ought to use only these, I am saying if you have little experience of

all things SQL Server, then stick with these 8 data types while you gain some working experience.

Another Consideration :

There is another side to selecting and using SQL data types, and that is when we have to move the data back and

forth, to and from dotNet variables where we are using .NET data types.

My approach is to try and make life easy for myself whenever I can - by looking ahead, and NOT using a 'strange'

SQL type just because it is there.

To see the best way to convert from returned data in a 'dataset' to dotNet objects in code, you need to refer to the

ClickStart eNotes on ADO.NET - in ClickStartSQL, and ClickStartADO, or even ClickStartST (for Small Trader).

One aspect to look into is the differences between the 'decimal' type in SQL and that with the same name in .NET

Lets get Going :

Since these eNotes are intended for beginners, and those with limited experience of SQL Server, we can get going

with what we have seen in our top 8 data types.

Out of Interest :

Since we are here talking of SQL data types, let us find and record which are available to us from the MS GUI tools

when we use 2008 R2 'Designer' :-

[ .. our connection is to a 2005 SQL database and so these only apply to that version - version 9.0 .. ]

===================================================================================================

** Note ** if you compare these with those shown in the last subsection you will see we have extras, a handful

of extras in fact. I will leave it up to you to find out which ;-0) The extras are there because we are using a

connection to a version 10.0 SQL database, the version of MS 'shell' does not come into play !

Be Aware :

The 'timestamp' data type is synonymous with 'rowversion' and is not what you may think it is !

Each database has a counter that is incremented for each insert or update operation that is performed on a table

that contains a rowversion column within the database.

This counter is the database rowversion.

This tracks a relative time within a database, not an actual time that can be associated with a clock.

Which is often what we require as developers.

Interfacing with ADO.NET code :

Just so that we know where we may be headed with all this SQL stuff, let us look at some dotNet code in Vulcan

which shows how data from a returned dataset from the MS SS is handled, and converted into .NET data types.

Checkout carefully the following details in the image below :-

=======================================================================================================

We need to be aware that if NULLs are returned then we have to handle these specifically.

Lines 265 through 269 show what we can do to trap and handle a NULL for a T-SQL datetime data type.

Line 266 on its own will throw an exception when the returned value is NULL, so line 268 is executed in that case

and we get a 'default' value of '1900-01-01' for our 'Edited' property.

This side of using SQL needs to be looked at in much more detail.

It is time to move on, see you in the next subsection .................

PHH - 11th. December 2011.


Primary Keys

No doubt you have heard that an SQL database is a 'Relational' database.

Well, in order to relate one table with another we need to have 'keys' set up.

Once we have keys we can use them to relate various tables with one another.

Let us look at how the Object Explorer (OE) pane looks in Management Studio (MS) for our recently made

'MyPersons' table, see below :-

====================================================

As you can see in the above image we seem to have no other information visible other than column names

and data types for each column. Oh! and of course whether each field is 'nullable' as well.

As you study this image please notice the bunch of tree 'nodes' at the bottom - ones named 'Keys', 'Constraints',

and 'Indexes'. We will come back to these in a little while.

In comparison with this detail shown above, let us now look at another table which has been designed by a professional

colleague of mine - a table which is used by my client on a 'day to day' basis.

The 'SS' stands for 'Script System', as the DB is all about patients, and their 'Prescription' orders.

Checkout below the extra icons for PK and FK - [ 'Primary Key' and 'Foreign Key' ]

====================================================

As you can see in the above image, we have two fields (or columns) marked as 'key' fields.

One - with a golden key icon - is the 'Primary', and the other - with a silver key icon - is a 'Foreign' key.

You will see in a later example that it is possible to have more than one 'Foreign' key, but ALWAYS only a

maximum of one 'Primary'.

In fact we could have a table with no Primary key but multiple Foreign keys. It all depends on the business

situation being modelled.

Since we have here a nice simplified example of a table, lets open up the other nodes to see what is to be

seen - nodes like 'Keys' and 'Indexes'. Study the following image carefully :-

=========================================================

We will look into the meaning of all of this stuff in due course. For now just mentally make a note

that Keys, Constraints and Indexes, will often be set in our tables.

For most of us Triggers and Statistics are of a much lesser importance for many, or most, tables we create.

*** Note *** in our example above, the second line of the 'keys' node is probably the one we need take most

note of, simply because it tells us that we are relating the 'SurgeryID' column in this table, with the Primary

key in table named 'SS_Surgeries'.

We had better take a look at what this other (related) table looks like in the OE pane - check this out :-

===============================================

So we can see that there is a Primary key in the above table to match the one used as a Foreign

key in the earlier 'SS_GPS' table. [ .. GP stands for General Practitioner, or 'doctor' .. ]

[ Because this professional guys does thing 'right' he has both fields (columns) called the same name - 'SurgeryID' -

in the two tables being related. ]

Examining the Script for Keys :

While we have a chance to do so, let us examine the script for the 'SS_GPs' table, to see what the MS SS

system produces when we use the GUI context menu options.

Checkout the way to do this below :-

=========================================================================================================================

When we take the above action we get a query tab which seems very complicated at first.

Let us examine what we get first, and then worry about how we may do things for ourselves, and our

new tables, afterwards.

Here is what we get, shown below in the image of the script - study it carefully :-

=========================================================================================================

Okay, I did warn you, it seems as though some of the complexity of the script comes from the requirement

of needing an 'Index'.

Also when we ask the MS SS system to script for us - for an existing object - we often get the text of the script

VERY poorly formatted, and therefore hard to read and follow.

In the text script above I have tried to tidy things up a bit !

Getting at Just Parts of this :

It may help if we see that using the OE pane, and selecting nodes in the 'keys' node, we can script just for the one

selected key - see below :-

=================================================================================================================

By scripting just for the primary key we see the following :-

=========================================================================================================================

Okay, I hear you say - "this still looks complicated, and a bit confusing".

Well, lets do something about it in the next subsection.

Removing the Confusion and Complexity !? :

Now that we have seen what 'Keys' look like in the OE pane, and how there are constraints and indexes set too, we are

probably best to start with our own simple table, and after making a second one, try and relate the two using suitable keys

and the likes.

There is nothing like 'doing' a task to learn how to do it properly, and well !!

Let us return to our own DB called 'MyClickStartTest' and in the OE pane right click on the 'PersonID' node in the 'Columns' list

of the 'MyPersons' table - checkout below :-

======================================================================

This actually brings up the same 'designer' tab as if we had right clicked the chosen table node and selected

the 'Design' option as shown below :-

=============================================================

The result of either / both of these operations is the following - the Designer :-

[ .. or Table Designer .. ]

=============================================================

Yes, you are right, we had this same 'designer' when we were adding our columns originally.

From the context menu (shown below) select to set a PK on selected 'PersonID' field :-

============================================================

And below we see that the icon now shows for Primary Key, next to the 'PersonID' entry.

============================================================

If you are following along you may like to see what happens if you try to set a Primary key (from the context menu)

on another of the columns - you will soon realise that the GUI system is NOT allowing you to set two !

Where Now ? :

'History' ? - Well now, because the 'system' seems to be keeping me from re-saving the changes I am trying out on the 'MyPersons'

table, it seems much easier for us to go and make a new table, correctly called 'Patient' this time. ( Notice the absence of an 's'. )

As well as setting a Primary Key we will also supply a couple more bits of information, and then let the Management Studio

and MS SS systems do as they please ;-0)

Start a new table and make your entries as shown below - including setting the PK :-

============================================================================

Right click in the tab at the top when you have set the Identity and the Increment and Seed values.

You should see this below :-

===========================================================================================

Then after refreshing the tables node in the OE pane you should see the following :-

=========================================================

You can see that the 'system' has done a few things for us, set the PK on the column we require, placed an

entry in 'Keys', and also set an index as well.

So how does our automatic scripting now look for our new Patient table ? Checkout the details below :-

==================================================================================================================

Okay, as you can see we have used the GUI tools to make a few 'sensible' settings and the rest has been done for us.

When I say 'sensible' I mean that we did not need understand the GUI in any complex way - just to say use the PK as the

identity and set auto increment on, going up by one each time, and starting at 100. That was hardly rocket science ;-0)

Entering Data to the Patient Table :

To test and try out our new and empty table, as far as some basic data entry goes, let us use the context menu option

from right clicking the 'Patient' table node in the OE - see below :-

======================================================

This then brings up the following - I call it the 'Edit tab' :-

====================================================================================================================================================================================================

Notice that there is a 'grey colour' to the first column 'NULL' text - also it can't be edited. As usual

'greyed out' means it is not an option, or can't be changed / selected. remember it is being managed

by the PK system / constraint.

So move to the others, one at a time, and make some entries as shown below :-

================================================================================================================================================

As you tab along the row and make entries, the red exclamation mark (said 'pling') is left to show that the newly

entered data has not as yet been saved - it has not been persisted.

Saving of the data while using this MS 'Edit' screen only happens when you the user moves to another row. At that

point the 'dirty' row is saved by the system - or should I say that a 'save attempt' is made by the system.

Checkout the following image which was captured just after I moved my mouse pointer to the last row :-

=============================================================================================================================================

Notice in the above record entry that I have added cell contents for each and every field or column, but there are a

couple I could have skipped - we will try this in a few seconds.

Also, notice that automatically the value of 100 has been entered as our 'PatientID' data - remember that this was

our 'seed' value from when we created the table.

Let us see what happens for the second data row - see below :-

===============================================================================================================================================

To see what would happen I left out a mandatory field. The 'PostCode' column is set to not allow NULLs.

When I tried to move off the row being entered, the system tried to make an effort to save the new record (or row)

and an error was generated in the SQL Server, the error code was returned to MS from the Server, and now MS is

handling the message box etc..

There appears to be an unfortunate consequence while working with the 'Editor' tool in MS. Can you see that there

appears to be an incremental value 'used' for each attempt to save the row.

Study the following image to see we do not now have a value of 101 - check this out below :-

================================================================================================================================================

And now making an effort to cause multiple 'failed' attempts to save the fourth data row, we can see below that we have

lost entries 104, 105 and 106.

=========================================================================================================================================================

I am not totally sure what would happen if we did a similar thing using ADO.NET and sending incorrect data rows

from our Vulcan.NET code, but you do need to be aware of this behaviour. Certainly as it shows itself at a

MS level - using the GUI tools.

My prediction is that the very same behaviour would show itself when we generate a 'save' error from our ADO.NET

code from our apps. The Server is reacting to a script sent to it in a query statement, and both MS and our apps work the

same way - they send the same script - so we should expect the same result.

One thing that is good, and that is we don't get duplication of PK values.

Notice that the DoB is displayed as a full 'DateTime' value, which indeed it is ;-)

As a final trial at data entry, let us go back to 'PatientID' of 103 and try to enter an incorrect date for the DateTime value.

Checkout the following :-

================================================================================================

And correcting day of the month (for July) from 37 to 27 we get a good save as shown below :-

======================================================================================================================================================

Lets now move on to Foreign key specification and use in our DB generally.

See you in the next subsection .......................

PHH - 9th. December 2011.


Foreign Keys

Before we can use a Foreign Key (FK) in our sample DB we need to have a new table which is suitable for

the purpose.

In a simple way of looking at FKs we can think of them as PKs from other tables.

[ .. and other tables includes 'SELF' as we will see much later in the 'Family Tree' example .. ]

So for example, our 'PatientID' field in the 'Patient' database, which is set to be a Primary Key (PK), can be

used in other tables as a Foreign Key (FK).

If we are to make an 'OrderHeader' new table, we can set it to have its own PK (so each new order has a unique

reference), but also have a column containing a Patient reference. If we set this to be a FK then it must have a

matching PK in another table. We will also point to the table in question.

Enough of using words to explain, let us see what this looks like in images of the MS shell.

Study the images shown below - look at them carefully :-

================================================================================================================

In the above image we have made a new table called 'Carrier'. In the UK a 'Carrier' is the company who delivers the

goods - like 'UPS' for example.

Here is the Column property information for the PK field - which is the 'CarrierID' field :-

==============================================================

Notice that I have seeded at 3000 and for increments of 10.

This means we can now 'relate' to this PK from another table, where we make one key the FK as

shown in the following images.

Just before we see how to set Foreign Keys (Fks), let us make a suitable new table called 'OrderHeader'.

Below is what it looks like with its PK set - check this out :-

=================================================================

Once again notice the seed value and the increment.

Setting the FK Relationships :

The FK relationships (also called constraints) are set at a Table level. Unlike the PK setting which

was done after clicking the respective field / column required.

Study the following images, where a dialog screen has been produced by right clicking in the top section of

the 'Designer' and not a particular field 'row'.

=========================================================================

After selecting the option for 'Relationships' we see this dialog screen - shown below :-

====================================================================================================================

We now need to use this dialog input form to allow us to set the required relationships for both the Patient

and the Carrier.

If we select the 'Tables and Columns Specification' input we get a set of 'Ellipses' at the Right Hand Side (RHS) to click.

See the following image :-

After clicking the ellipses once we get the next image seen below,

================================================================================================================================================

We then have to use the drop list controls to select what we are after - see the drop list contents

shown in detail below :-

=================================================

=================================================

After selecting the correct options to make a 'CarrierID' FK relation for the table in focus (new one will

be called 'Carrier') we should see the following shown below :-

===============================================================================================

After clicking the OK button we should have completed the entries for 'CarrierID' and the main Relationships

dialog will be like that below :-

==================================================================================================================

You can probably work out for yourself that in the above image 'Table_1' is present because we have not

as yet saved the new table under the name 'Carrier'. That will come soon.

First we need to make a second relationship - that for the patient using the 'PatientID' field. Hopefully we will just

need to copy what we have already done above, but select different columns and table name.

Here we have suitable input data and are half way through the creation :-

==================================================================================================================

Oh! dear, we were doing so well with our tasks above, BUT, we seem to have hit a snag when we

look into the details of the following image :-

====================================================================================================================

Yes, this will hit us all (and hurt us), at some time of our coding and scripting life, so we may as well learn now ;-0)

If we are to set a Foreign Key (FK) relationship then we MUST have the data types in both the PK field and the

FK field exactly equal, identical in fact. If they are string fields they must also be the same length.

Above we got an error because we had an 'int' type and a 'bigint' type - no, it was no good thinking they are both of type 'integer'.

So we seem to have learned a GOOD lesson quite easily, as I have quickly corrected the problem, we now have 'bigint'

as the data type in Patient as well as Carrier tables.

Having corrected the mistake we now get the following image when we try to save the second relationship - that for 'PatienID'.

Study the details below :-

===================================================================================================================

*** Remember *** the 'Table_1' object name will be replaced by our choice ('OrderHeader') when we save this new table.

Let us now do that - checkout the image below :-

[ .. note to save the new table right click in the 'tab' at the top of the screen .. ]

========================================================================================

Replacing, the suggested object name with our own and clicking the OK button we get the

following image shown on our monitor :-

===============================================================================================

Okay, after a bit of a panic I see it is a helpful screen and not a 'problem' one like another above which looked similar to this.

The YES button was kind to us and all appears to have worked as hoped - no error messages !

And now when we inspect the OE pane we see the following :-

[ .. remember to do any 'Refreshing' of nodes if required .. ]

==========================================================

Yes, in the above image we see a new table called 'OrderHeader', which has one PK and two FKs.

Finally we have this :

And now, after just a few (well a large handful really) of mouse clicks we are able to get the MS shell to print

out the script for our Carrier table - the new one we made with both Primary and Foreign keys.

Looking at the image below it appears to be just VERY similar to that we started with from the SS_GPs table in the

last main subsection on PKs.

=========================================================================================================================

And VERY finally, if we expand a few more nodes in the OE we see this :-

=======================================================

Once again, VERY similar to what we got way back from the 'SS_GPs' table of our real DB example.

Time to move on - see you in the next subsection ..................

PHH - 10th. December 2011.


Seeing our DB objects

Now that we have made a few Tables and also set some FK constraints, we need to see what has been

placed in the 'sys.objects' table - we are using a 'view' below.

Note that the WHERE clause is to show us the entries directly relevant to us and what we have done. There

are another 58 rows which are connected with the 'system' side of things.

** Obviously, PK is for 'Primary Key' and U is for 'User Defined' table - see below :-

==================================================================================================================================================================================================

*** Note *** we can get a lot of this by using the special 'view' created for us by Microsoft in the schema

section of our DB called 'INFORMATION_SCHEMA' - this is to be found in 'System Views'.

=====================================================================================================================

And also the following is useful :-

============================================================================================================

And finally, yet another possibility - checkout below :-

===========================================================================================================================

You will see more of the use of these twenty (20) 'views' later in this section.

Let us move to the next subsection, see you there .....................

PHH -11th. December 2011.


Adding data - Relationships in place

Let us do some testing on our newly created tables, and their relationships.

See if we can see just what facilities the 'referential integrity' provides us with. What protection do we have ?

At the moment we only have data in our 'Patient' table, even though it is only a few records.

Let us display the state of the 'Patient' table as it now stands - see below :-

=======================================================================================================================================================================

In the image above we have used the right click context menu option to show the existing data rows.

Let us now provide some 'Carrier' information. We can do this by selecting the 'Edit Top 200 Rows' (Edit tab)

from the options in the context menu, when we right click the 'Patient' node in the Object Explorer (OE) pane.

Study the following image carefully :-

===========================================================================================================================================================================================================================================

Look carefully at the above image and notice that the 'NULL' in the first column is 'greyed' out and is not editable.

After we add our first data record without incident, or error messages, we see the following :-

===========================================================================================================================================================================================

Notice that the 'CarrierID' value is that of the seed we chose in the last subsection.

Let us now try to add a couple more rows :-

=================================================================================================================================================================================================

The above image shows that our incremental value of '10' is being applied as we would expect.

We should have enough record data to do our test with 'OrderHeader' - with what we see above.

So we need to bring up the 'Edit tab' for the table named 'OrderHeader', do this using the context menu,

see below for a visual clue :-

==============================================================================================================================================================================

Before we start to attempt to add data rows to our 'OrderHeader' table we need to appreciate a couple of points.

One is that the auto-increment will handle the numbering of the PK field, the column called 'OrderID'.

However, we need to provide the FK fields - 'PatientID' and 'CarrierID' with suitable existing values - ones which

already exist in our other tables.

With this in mind let us try to create a few rows of data. Below is our first successful line of data :-

=======================================================================================================================================================

Okay then, I have attempted to make no mistakes and we now have this data in our table :-

=================================================================================================================================

Success - so far so good, but we now need to test out the limits of what is possible, and what is not.

Trying to Break 'things' :

Q1 : Can we add a FK which does not exist ?

Let us try and add a line for Patient '106' which does not exist - remember it was 'skipped' over when we had a few

erroneous data entries.

Below we will give this a try, first the line before we move off it, and from which the attempted 'save' will take place :-

================================================================================================================================

And then when I click the mouse outside of this new row we see the following message appear on the screen :-

====================================================================================================================

The image above is our first visual proof that we now have 'Referential Integrity' in operation - due to the Relationship

we set up with our PK and FK for 'PatientID' in the two tables 'Patient' and 'OrderHeader'.

So if I now correct our data and provide an existing patient unique reference, we ought to see the 'OrderID' has skipped

an allowable value - as it did in earlier test doing a similar thing.

Here is the proof of what we have done :-

=================================================================================================================================

And YES, we have a successful INSERT (or save) but the 'OrderID' has skipped a value due to the previous error.

Partial Summary :

When we have a PK / FK relationship in place we can't add a value to a FK field (or column) unless it already exists.

We could use missing ID values (under certain circumstances) as a measure of how many erroneous attempts to save

data records are taking place. In an ideal world we would see NO missing values, as no errors would take place. Also

there are better ways to log errors - it was just an idea ;-0)

So the PK value is at the top of the referential tree, without that we can't refer to it elsewhere, when we need FK values

they must exist in the 'master' table - the table where the PK resides.

Obviously, we can re-use PK values in FK fields - so we can have many different orders for the same patient, and hence

'PatientID' can appear many times in our 'OrderHeader' table, in the 'PatientID' field - or column.

It is also quite common for more than one FK column in a single table to be related to the same PK column.

*** Note *** in a real DB situation the names of the fields holding PK and FK values are very likely to be named differently.

So for example 'customer' and 'acc_code' are the same unique ID in my client's DB - so be aware of this common 'real

life' situation.

Removing Data - Referential Integrity :

So far we have seen how we can't insert FK values which don't exist in the 'master' table where the PK exists.

Now we need to see what happens when we try to remove a PK data record for which there exists at least one FK entry.

Please realise that we could easily have the same FK field value in many tables, so a patient 'ID' FK relationship can be

in place for more tables than the 'OrderHeader' table.

Using the GUI :

For a start, let us try and use the GUI tools, as this is easiest for us to test out.

In the 'Edit tab' for the 'Patient' table let us click the '1003' line entry and try the 'Delete' option from the context menu.

Remember the PK field of 'PatientID' is at the top of the referential tree, there are existing references below this level.

Checkout the following :-

=================================================================================

After we click the menu option we see the following message box :-

==============================================================================================

Obviously the first thing the 'shell' is doing is to check that we wish to attempt a delete operation at all - before

the SQL script is sent to the Server.

So what happens when we say 'Yes' to this ? See the following image :-

====================================================================================================

Okay then, just as we had hoped, the referential FK 'constraint' is being applied to protect our data.

Let us see if we can get some similar results with scripted queries from Management Studio.

Scripting a Delete :

Before we get to Deleting or Replacing our PK values, it is recommended by 'those that know' that we first

do a Select - see below :-

======================================================================================================

Now that we know we are pointing ONLY at the record we require, we can change the syntax to try out

a 'DELETE' operation - see below :-

========================================================================================================================

So studying the image shown above, we see that we can't remove the row with script either.

Which is NOT surprising since the MS shell will have sent the same script when we used the GUI tools ;-0)

Remember that all actions achieved at the MS SS end of things are brought about by SQL query statements

being sent - from any software application we use, or even build ourselves.

And if we query the Server immediately after our attempt to delete, we can get the error message code as follows :-

===================================================================

There is information later in these eNotes about errors at the server. To get the correct result the query must be sent as the

very next query statement after the one which caused the error.

Check out error 547 later in the subsection on error codes, or indeed ask the Server for the basic information using a query !

Try this out :-

=================================================================================================================================================================

Although the text is intended to have parameter values substituted for 'place holder' strings, we can still read the bare bones of it.

If you are good at languages, try missing off the fourth script line, the one with the second part of the 'where' clause.

Also notice that the 'message' table is held at a high level that our database - it is in the master one, so it is not unnecessarily

repeated over and over in others.

Going a Step Further :

Okay then you guys, we need to do a bigger test - one in which we see if the referential integrity is maintained if we try to remove,

or delete, a whole table - but a table which would leave 'orphans' in another table which would remain in the DB.

In our example above we have tried to remove a single row from the Patient table - the one used in the single data record in the

'OrderHeader' table. But what if we try to DROP the table altogether ?

Lets try it, using both GUI tools and query script.

Study the image details shown below :-

=====================================================

After choosing the context menu option - 'Delete' - we get this below :-

And now when we click the 'OK' button we see this below, as a result :-

The only way to clear this dialog from the screen is to press the Cancel button.

Now let us try and use some query script - you may wish to study the last of the subsections in this chapter, it is specifically

about dropping Tables and Databases.

============================================================================================================================

Which leads to the following script - which I have tidied up as far as screen formatting is concerned :-

=========================================================================================================

[ .. the 'N' is just casting, or forcing, the text string to be of type Unicode - a double byte .. ]

The script reads as this - if in the DB called 'MyClickStartTest', we have an object called 'Patient' of type 'User defined' table,

then delete it. Remember DROP for a DB object is the same to us as deleting it - removing it.

Now let us see what we get when we try to run it - checkout this below :-

==============================================================================================================

Well, just what we expected to see really.

If we ask for the error code we see the following :-

=====================================

So the error code from the Server for this situation is number 3726 - but remember you must NOT send any other query statement

before requesting the error code to be returned.

Even sending the error query a second time will give us a zero return result - meaning no error was detected or found for the

very previous query statement execution.

Why is this ? Well, quite simply because the first request for the error code worked as it should, returning the 3726, and so this

query worked correctly ! Checkout the image below :-

=============================

Time to move on, see you in the next subsection ...........................

PHH - 11th. December 2011.


Some Design Considerations

This is a hard section to write, so I have done my best and put down everything I thought may be useful for

you to read and consider - but I am not an expert, just a user.

Remember - all the 'stuff' describing your data, the DB relations, and the restriction on individual values, ought

to be in the database and not in your code. These all belong together ! keep it all in the DB.

Put as much as you can into the DB and use Views, Triggers and Programmability (Functions and Stored Procedures)

as much as it makes sense to do so - Oh! and Constraints too.

What you DON'T place at a DB level is anything specific to your application, if it will affect others using the DB who

are not using your app., but a completely different one. So adding specific Functions, Views or Stored Procs is OK

in done carefully and with thought.

Suitable Example :

It is probably best to have a function find the last order for a specified customer, than you write this in dotNet code.

Here is my example - see OE pane details below :-

==================================================

And although the script within the function is not perfect, it is worth looking at - see the following image :-

[ .. Remember that 'GO' belongs to the MS shell, and is not part of T-SQL scripting .. ]

================================================================================================

And we can use the function in our queries like that shown below, even though this is not a fast way of doing things

when we have lots of patients in a returned data set.

I am only using this as an example of what may be done at a database level, rather than making performance comparisons.

In the following image I have made the data difficult (impossible?) to read for confidential reasons - check this out :-

================================================================================================================

So in the above returned data set we can see at least two customers who have never placed an order.

*** Out of Interest *** If the above was my DB and I had the ability to make all changes to it, I would have a field in a customer

related table, which would accept the 'LastOrderDate' value at the point of the order being placed / made. Maybe the good

use of a 'Trigger'.

This way I could refer to it whenever I chose, and without any performance penalties. Then we would be able to use this

value in queries for reports MUCH faster than the way we see things above.

How Many is Too Many !? :

Another problem I see regularly is where one table has by far too many fields / columns.

As a rule of thumb I would say to try and keep to less than 50 columns. On my monitor, while using Management Studio,

I can fit just under 50 in the OE at any one time.

I still personally feel that 30 max is probably a better number to aim for. Hopefully most of your tables will not have LOTS of

fields in them.

There is a time when having too many fields in tables can cause problems, or possibly rather 'performance' issues.

When we need to make complex queries using many JOINs then the MS SS system can end up with data records of hundreds

and hundreds of fields in length. This is the case with my client's DB I use every day.

It is only at the point where the returned data set is finally 'made up' (or put together) that fields are excluded it would seem -

so carrying huge numbers of columns unnecessarily seems rather pointless - almost silly !

This situation of MANY fields in a JOINed data set (de-normalised) is a thing to check out if you are doing 'sorting' on large numbers

of records - certainly worth a test or two.

It is my feeling that the above problem exists in my clients DB simply because the designers did not fully understand the way

SQL Server works under the hood. If they had fully appreciated the use of JOINS in queries then they would have designed things

differently I am sure.

Organising is the name of the Game ! :

Keeping the field names the same in different tables - for the same piece of data - is a good practice.

I have a real life (large) example where naming fields is something like this ... 'acc_code' in the Accounts table.

Checkout the following image before we discuss other related issues :-

=========================================

In the above image the field called 'acc_code' is the same as the 'customer' field in the main Patient table.

Personally, I would prefer it if they were both called 'CustomerID'.

It seems to have been a style of the developer of this system that field names are prefixed with Table related abbreviations.

And as you can see they are not even consistent as far as 'case' goes.

Oh! there is yet another 'style' of design (in the above image) - that is adding 'fds' to the field name to show who added it. I am

not sure if I like all this stuff, even though I can see where they are coming from.

To my mind we should name a field / column by a text string which is related to what it is, and stick with that. After all there are

ways in SQL scripting where we can provide a temporary name for the table so 'A' or 'AC' for accounts help us keep track of things.

===========================================================================

So in a more complex JOIN clause we can keep track of which field is from which Table.

See the following example :-

======================================================================================

[ .. 'S' is for Sales Ledger and 'AC' for Account .. ]

So there is a lot that could be put right in the above DB and it situation - this is only possible at design stage - so get it right

the first time around ;-0)

You could even go as far (writing queries) as doing that which I have done below :-

===========================================================================================================

So for a data set in Reporting Services (RS) the fields 'CustomerAC' and 'CustomerS' would come through into the report

environment, where they may be used - AND understood better.

However, we can always rename any field in RS at a returned data set level, so that is always another alternative.

Now let move to another important issue Upper and Lower case !

Case Sensitivity - just in case !! :

When you are dealing with more than one DB, and also dealing with Reporting Services, as well as Vulcan.NET and often C# as well,

there are often many places where 'case sensitivity' does matter.

It is so easy to be caught out that I have a survival rule for myself - that is use case correctly at all times, even when writing email

messages to the IT Manager. If a Table name is 'Account' and not 'account' then that is what I use, everywhere.

Some SQL databases can be set to be case sensitive and others may not be. I believe you can even download both versions of

the 'AdventureWorks' database.

So save yourself frustration and time - be VERY precise with your casing, and typing of your code and script.

Cluttering up your DB :

It would appear that even many professionals out there in the wide world do not see an SQL database as an object.

And, as an object containing and managing many other objects.

It would also appear that some / many think of tables as files - as things were in the old days of DBF and the likes.

After sitting here researching, and being a user of SQL for well over three years it seems crucial that we ONLY include in

our DB those tables which are necessary for the correct operation of the DB itself. In fact this goes for all DB objects, not

just Tables.

It is not the physical size of the tables and their data which are the problem - apart from when we come to make Backups,

and transport these to other PC's, systems, and hard disks / storage devices - but how some much unnecessary 'clutter'

affects the DB tools we need to use.

I have a client DB of over 6 thousand tables, and I suspect that less than a thousand of these are current and in use. The

rest are dead, empty or archives, and this causes me many problems.

One thing I regularly come across can be seen in the following image - check this out below :-

===================================================

I would guess that 13 of the above tables are 'archived' tables. By this I mean they are copies of other tables, and as such are

not now used in the standard way the DB functions, or works.

I suppose I am trying to say that in the old days of DBFs we would have a folder named 'Archive' or 'DataBack', and into that

we would place such duplicated files - those which now have their data in Tables, and are still part of the DB object.

Since I am NOT advocating losing old data copies, made before major data 'cleaning up' operations, I will suggest a much better

way to 'archive' table data.

Study the following image carefully :-

===========================

Notice in the above image we have a DB called 'SafetyWarehouse' - this is my answer to copying and duplicating Tables and

their data. The other DBs are all large client DBs and need table copies making from time to time.

Below is an image showing the tables added to my DB - set aside specifically for data archiving :-

================================================================================================================

If you look closely at the above details you can see that we have copies of tables which retain their main original name,

and onto which we have added the date and time reference to within one second.

This allows me to be able to copy any table just before data cleansing operations, into the separate DB called SW,

where SW stands for 'SafetyWarehouse'.

This way we do NOT lose the data and also we can re-run the operation (as long as it is not within 60 seconds 'max' of the last one)

over and over again.

I suppose if you are keen to use this SW database for more than one DB and tables have similar names, we could change the

archiving rules (syntax of script) so that the DB name (or a shorthand) could be added to the front - see what I mean in the following lines :-

instead of "AccountDec 10 2010 10:14AM" we could have "SAGE_AccountDec 10 2010 10:14AM"

and also "pel_stockmSep 10 2011 8:25AM" in place of "stockmSep 10 2011 8:25AM"

Since the two different DBs are 'SAGE1000' and 'pelican'.

The main benefit of this separation of function - working DB and archiving DB, is in the management of the DB, both internally

by the MS SS and also for me working with unnecessarily large list of objects in the OE pane of MS. I have sat at times for 30 seconds

or more while the 'Tables' node is being opened - particularly when I have already got other large lists open.

These GUI and management problems are not something that the maverick designer / developer sees or takes into account. But

I certainly do. I spend a lot of time (many hours each week) in MS designing query statements for Reporting Services, and this sort

of issue gives me regularly a lot of 'grief'.

Temporary Tables :

While I am having a moan about 'Table Bloat' I will also mention the apparent tendency of the system programmers, or code

developers, to use permanent tables in place of temporary ones.

If it is so necessary to make a permanent table for a short term task, then at least they could name the table in such a way

as to easily identify them as 'expendable' - meaning we could easily see them from a query, and then DROP them from a

stored procedure which we could run once a week, or over night / weekend.

To my mind all this stuff is simply about thinking ahead at design and coding time. Some of these issues are simple enough to

avoid, BUT, difficult to correct afterwards.

Mixing data values and 'meanings' in one column :

This is also a common mistake, and one which causes real problems when queries need to be designed and run.

I mean that one field / column has data entries signifying different 'things' - yes, when in fact multiple columns would sort

out the problem very nicely.

Recently I came across a problem where the user / designer had one field in a data record related to a place name. The

name of the place represented (called 'Type') either a Province, County, City, or Town - the value 0,1 2, .... represented the these possibilities

and so 'North East' would have a value of '0' in the 'Type field, and 'Yorkshire' a value of '1' in this column - Leeds a value of '2' and so on.

Obviously, Leeds, Bradford, York etc. would have type values of '2' because these are Cities - Rotherham and Barnsley a value of '3' because

they are towns. We could go further, but I think you see what I am describing.

The problem with mixing data and type meanings like this is apparent when someone comes up with an apparently simple

request of putting all the Counties into alphabetical order, or just listing the Cities.

It will be much easier to create and structure queries on such data if there were separate columns for Region, County, City,

and Town. A Boolean ('bit') field for each would allow much easier scripting of queries. And on top of this if separate

columns were supplied for which County a City or Town belonged to, and one field for which Province/Region a County

belonged to, then all would be much nicer indeed ;-0)

Probably we need to see the two extremes - the Bad and the Good, we can forget the plain Ugly for the moment !!

=================================================================================================================================

The example above is taken NOT by any way of criticism - we often are left, or stuck, with table design done by others

sometimes years ago, and we just have to get on with things. I know, I do it every day.

So we are using this example from Kevin simply because it is a good example of how we can improve things, or make them better,

simply by using some good thinking and planning at an early stage in our DB design.

Here is the original table of his :-

Now look how simple it is to get the same order (as the image before the last one) when we have an extra column of relevant data :-

=============================================================================

Although this is still not as explicit as I would like, at least our extra column of data values helps us script a

suitable query for sorting, much easier.

I will see if I can come up with my own 'English' version of this which does go much further as far as added suitable

columns are concerned..

And now our own English Example :

A very Common Mistake :

It would appear from my somewhat limited experience that it is a common fault with DB design to incorporate

more than one type (or bit) of information into one column - almost as if the designer was trying to save space by

reducing the number of columns he/she uses.

It was a practice in the past for fields to contained 'coded' cell contents - let us say we have a table for Contact

and one column for Type. In field Type you may see 'C' (for customer), 'S' (for supplier), 'SH' (for shipper), 'N'

(for nurse), 'GP' (for doctor), 'Internal' (for internal company use) and other untidy stuff too - like NULLs and spaces,

and mixed case letters and often I also see spurious stuff like a post code or part of a telephone number.

The problem has arisen from making the field a string field of at least 8 characters in length, so that 'INTERNAL' may

be used. But by doing this a whole 'can of worms' has been opened up. Even for one 'type' I see things like :

int, INTERNAL, internal, INT - and even the odd i or I

I would much rather see separate columns (Boolean/Logic) in nature for 'Customer', 'Supplier', 'Shipper', 'Nurse',

'Doctor', 'Internal', and 'Other'.

Then we can have a Supplier who is also a Customer, and more likely to be used, we can set constraints if we so wish.

Note we have a 'catch me all' with 'Other', so there is always a check-box which can be selected.

I feel that quite often we see this 'sloppy' design simply because the developer has started to create the DB before they

have really thought about the problem thoroughly enough.

It is certainly not a good idea to leave the string data input for users to decide - a check box for each is much preferred.

In fact this is just how the designers of the SQL Server system seem to have done much of their stuff - see below for part of

the 'sys.tables' structure :-

===============================================================================================================================================================================================

And here some more examples from the 'sys.indexes' table - see below :-

=================================================================================================================================================================

And looking into the 'sys.views' table we see the following details :-

===============================================================================================================================================================================

In each of these above three images I have moved the scroll bar to the Boolean section I wished to show you, as

there are other fields as well.

We need to remember that Boolean values are stored as a 'bit' data type in the SQL Server database itself. This

means that you need to select 'bit' as a data type in the Table Designer of Management Studio.

Kill the Cursor :

DON'T use cursors - in all but a tiny few cases there is a better way using set algebra - do not grab for a cursor

solution because your knowledge is lacking - go learn more about full use of the SELECT statement.

CURSOR use degrades the performance of the Server

Naming Tables :

If you have a table in which you place a new Customer, then call the table 'Customer' not 'Customers' (with an 'S').

Try to make a table for each of your business objects - those in your dotNet code solution.

So I would tend to have a 'Product' table, and also an 'OrderHeader' table, as well as 'OrderItem' table.

Similarly, 'Surgery', 'Doctor' and 'Nurse' as well.

Naming Columns :

Try to keep your PK and FK field / column names identical from one table to another - this helps a lot while scripting JOINs.

This is often a legacy problem - old DB designs and Tables added through time by many different guys, so we get many

different styles and naming conventions.

However, it doesn't have to be like this - simply look to your PK fields and when making a new FK field in a new table, reuse

the same column name. This way we get for example, 'CustomerID' in both tables.

Keeping it general :

Make the names of your tables and column names 'general' ones.

If you have something like 'August2011' then you are doing it wrongly - the date should be a 'cell value' in a data row.

Here is a part of a 'Stock' table in a real life SQL database sold by a globally know accountancy software vendor - this stock

table has 192 columns at least (far too many in my view of things) and many of the field names are NOT general, but

specific - take a look :-

================================================================================

It would appear to me that the specific names of table columns shown in the above image refer

to periods in the year - this is a very old fashioned way of doing things and is certainly NOT object

oriented in its approach.

Any information encoded into a column or table name suggest STRONGLY that your design approach is wrong !

The information should be in the data - and data in a generally named table, in a generally named column.

Without me looking much closer into the above situation, it would appear it would have been much better to

have a table called 'StockPeriod' and for one column to be 'ProductID' which is a FK to the same named column

which is a PK in the 'Stock' table.

Then we can have fields for :-

'period'

'month_to_date'

'year_to_date'

'issue_qty'

'issue_val'

'demand'

'dspare'

[ .. there will be need for another couple of fields so the row can be correctly identified - like 'ProductID', and some others

for the date the row was created, and by whom it was created etc. .. ]

** Note ** there is no need to have 'period' in the column name as the table is effectively for 'Period'.

One reason I recognised the above for what it was, is because I was instructed to make a VO app in the past (using DBFs) work

in a similar fashion :(

I certainly would not do this now, no more than I would hard code my business objects in Vulcan with property names containing

specific information. Since I would to have a strongly typed list of items in my objects, then that is how I would design my tables

and columns.

PHH - 13th. December 2011.


Archiving Tables to 'SafetyWare' DB

In the following image you can see the details of a script which I use regularly for my clients.

The top part is to handle the new and existing table in the 'SafetyWarehouse' database - the part at the bottom,

between the BEGIN / END is where I place the script to make the changes to the live data in the Table within

the live database.

======================================================================================================================================

If you adapt the above script for your own purposes it can be VERY helpful to you.

Good Luck, and be careful ;-0)

In the child folder/file from here there is the text of this script - for you to Copy and Paste into your MS query tab.

See you in the next subsection ......................

PHH - 19th. December 2011.


working script

--/*****************[ do NOT change between lines of 'stars' !!! ]*****************/

--/$$$$$$$$$$$$$$$$$[ copying and renaming 'stockm' file ]$$$$$$$$$$$$$$$$$/

use [SafetyWarehouse]

go

IF EXISTS (SELECT * FROM sys.objects

WHERE object_id = OBJECT_ID(N'[dbo].[MyTableXYZ13]') AND type in (N'U'))

BEGIN

DROP TABLE [dbo].[MyTableXYZ13]

select 'Our DUMP Table - [MyTableXYZ13] - was found and DROPPED !!'

END;

go

--/* --- the script below works! --- */

select *

into [SafetyWarehouse].[dbo].[MyTableXYZ13]

from [pfhl].[scheme].[stockm];

go

--/* ----------------------------- */

declare @retcode int;

declare @NewTableName nvarchar(max);

set @NewTableName = ltrim(rtrim('stockm'+CAST(getdate() as nvarchar(26))));

EXEC @retcode = sp_rename 'MyTableXYZ13', @NewTableName, 'object' ; -- rename a specified table

--/* --- should get a zero for Success ! --- */

--/* in the script to be constructed below --- */

--/* only do the UPDATE if value is actually '0' !!! */

if @retcode = 0

--/*********************************************************************************/

BEGIN

/* --- make changes ! to 'stockm' --- */

/* --- with only Competitor product prices !! --- */

update S

set S.price = U.New_Price

from [pfhl].[scheme].[stockm] as S

/* --- the following table contains the recently updated Competitor prices --- */

/* --- table is named as in the third square brackets --- */

inner join [pfhl].[dbo].[_phh_tPriceUpdateComp] as U

on S.warehouse = U.Warehouse and S.[product] = U.[Product_Code]

--=============

select @retcode

END;

go

--/*===================[ End of data changing routine ]===================*/

PHH - 19th. December 2011.


Adding more Tables - Normalisation

Once again this is a 'thorny', or difficult topic to address.

Many 'old school' guys seem to run a mile when the word 'Normalisation' is mentioned.

For me, I think it ought to be embraced with both arms - BUT - not overdone !!

Try to structure your tables so no business object specific field data is in more than one place. If we remove

data duplication then we can manage our data much easier. Duplication of data is most often eradicated (removed)

by sensible use of 'Normalisation'.

==================================================================================

Normalisation is the process of designing your tables such that any data item only appears ONCE in your DB.

There is a sensible level to which the degree of normalisation makes a good working DB.

For example, do not have a specific 'PostCode' value in your data more than once. If it is there more than once, then

you have created a situation which can only cause confusion and/or error at a later stage.

Have data like address, postcode, and phone numbers, in one table - possibly the 'Contact' table - so for any

updating you only have to visit one place.

It makes sense, to many coming new to SQL, that we should have a name, address, and telephone number in the DB

just once - BUT - they may have to have a hard think about placing all such 'contact' data in one table.

Personally, I like the idea of having a 'Contact' table, where we can have a PK field called 'ContactID' and another

few columns which are Boolean (bits) which represent the possible types of contact.

I am sure many of you would be happy with a Contact Type 'code' which is stored in one field / column, but I think

this is old fashioned, and not how MS is structuring its tables for the master DB in MS SS it would appear.

So I would try and include single and separate 'bit' columns for 'Patient', 'Shipper', 'Doctor', 'Supplier', 'Customer',

'Supplier', 'Employee', and so on etc..

Lots of 'Normalisation' is fine, however, there is a case for making your DB more 'usable' by having the a data value

in one table when the source of the value is in another.

For example we may have a 'Number_of_Items' field in the 'OrderHeader' table when we actually have the same

number of lines (or rows) in the 'OrderItem' table. Instead of calculating this each time it is required we can

store it once - at order creation time - and edit time if edits are allowed on orders.

Another example of this is the 'LastOrder' and 'LastOrderDate' for a customer. These too may be stored along

with the 'Customer' record, when in fact the invoice number and its date are held in another place as well. This

could be a good case for using a 'Trigger'.

Some would call the above a case of 'sensible' de-normalisation - but be aware that you do not let your design

principles slip, just for an easy life as you see it - or the 'quick fix' !

I would suggest your seriously consider what is common elsewhere, making a 'Contact' table into which you

place the name, address, and phone details of all other types of 'folk' - folk like those for Company, Surgery,

Doctor, Customer, Shipper, Delivery Firm, etc. etc. as a good place to test and try out normalisation as it make

come to help you and your DB designs.

Keeping it general :

No table you create should have anything but a general name.

So one like 'Sales2011' should not exist. This is probably more likely to happen with stock, when management

wish to track monthly levels - so no 'StockValues2011' either.

Too Many Fields / Columns :

Make another / more tables

Living with Business Objects :

In our code we have BOs - Business Objects - if we have designed things right for our business solution then very

often the BOs will translate very easily into tables within our DB.

I have made a STATIC class (or two in fact) in Vulcan for working with ADO.NET and the business objects of a

particular solution.

The methods required (made by us) for one of these layers matches up with our BOs and so makes data access

to and from the SQL Server and the database very easy indeed. We send scripts just like those we can hand

script in the Query panes in Management Studio.

Entity Framework 4 :

PHH - 13th. December 2011.


Setting Relationships - Data Integrity

This short BUT very important subsection is all about 'Integrity'.

Referential 'Integrity' and data 'Integrity'. We will see what the SDK documentation has to say on this.

Some of what we see and read below applies to us, even with some simple and straightforward DB, other stuff is

for more complex and advanced situations.

However, from a users point of view I would say that it is a matter of letting the data Server do the job of

'housekeeping' as far as the data is concerned. Then any software application which uses the data in the

database will not need to have any concern over relationships, it will just use the DB and its data.

So the role of managing the data - in its widest sense - has been given over to the Server itself, and as

long as the DB is suitably set-up we will get good performance at a data security / integrity level.

If we set the Relationships and Constraints at a DB level, and let the data Server manage these, then all

other application using the DB will be 'protected', 'secure' or 'safe'. Many such words will be appropriate,

but 'integrity' is really what this is all about !

Let us do some research in BoL (Books on Line) and SDK documentation :-

Overview of Referential Integrity :

Referential integrity is a system of rules that ensure relationships between rows in related tables are valid and

that you do not accidentally delete or change (already) related data.

You can set referential integrity when both of the following conditions are met :

- The matching column from the primary table is a primary key, or has a unique constraint.

- The related columns in the foreign table have the same data type and size (checkout string fields).

When referential integrity is enforced, you must observe the following rules :

- You cannot enter a value in the foreign-key column of the related table if that value does not exist in the primary key

of the related table. However, you can enter a NULL in the foreign-key column.

For example, you cannot indicate that a job is assigned to an employee who is not included in the employee table,

but you can indicate that an employee has no assigned job by entering a NULL in the 'job_id' column of the employee table.

- You cannot delete a row from a primary key table if rows matching it exist in a related table.

For example, you cannot delete a row from the jobs table if there are employees assigned to the job represented by

that row in the employee table.

- You cannot change a primary key value in the primary key table if that row has related rows.

For example, you cannot change a row's job_id value in the jobs table if there are employees with that job_id in the employee table.

Referential integrity preserves the defined relationships between tables - when rows are inserted or deleted.

In SQL Server, referential integrity is based on relationships between foreign keys and primary keys, or between foreign

keys and unique keys, through FOREIGN KEY and CHECK constraints.

Referential integrity makes sure that key values are consistent across tables.

This kind of consistency requires that there are no references to non-existent values and that if a key value changes,

all references to it change consistently throughout the database.

When you enforce referential integrity, SQL Server prevents users from doing the following :

·Adding or changing rows to a related table if there is no associated row in the primary table.

·Changing values in a primary table that causes orphaned rows in a related table.

·Deleting rows from a primary table if there are matching related rows.

User-Defined Integrity :

User-defined integrity lets you define specific business rules that do not fall into one of the other integrity categories.

All the integrity categories support user-defined integrity.

This includes all column-level and table-level constraints in CREATE TABLE, stored procedures, and triggers.

Data Integrity :

Enforcing data integrity guarantees the quality of the data in the database.

For example, if an employee is entered with an employee ID value of 123, the database should not permit another

employee to have an ID with the same value.

If you have an 'employee_rating' column intended to have values ranging from 1 to 5, the database should not accept

a value outside that range.

If the table has a 'dept_id' column that stores the department number for the employee, the database should permit

only values that are valid for the department numbers in the company.

Two important steps in planning tables are to identify valid values for a column and to decide how to enforce the integrity

of the data in the column.

Data integrity falls into the following categories:

- Referential integrity

- User-defined integrity

- Entity integrity

- Domain integrity

We shall continue below, to see what the last two of these refer to.

Entity Integrity :

Entity integrity defines a row as a unique entity for a particular table.

Entity integrity enforces the integrity of the identifier columns or the primary key of a table, through UNIQUE indexes,

UNIQUE constraints or PRIMARY KEY constraints.

Domain Integrity :

Domain integrity is the validity of entries for a specific column.

You can enforce domain integrity to restrict the type by using data types, restrict the format by using CHECK constraints

and rules, or restrict the range of possible values by using FOREIGN KEY constraints, CHECK constraints,

DEFAULT definitions, NOT NULL definitions, and rules.

Okay then, we have enough to think about so far, and some of this stuff should apply to us all, right from the beginning of

any planning we do regarding our databases.

For some of this stuff, many of us may never come to, but we need to know of the general existence of Relations and Integrity.

Summary :

It would appear more than ever that we need to plan our DB carefully. It goes much deeper than making up the names of a

few tables and their column names too.

Three things to get right, Planning, Thinking ahead, and a bunch of suitable and useful 'Initial Tests' !!!

It is possible to write ADO.NET code solutions in Vulcan or C# which do not employ any PK and FK constraints, but I don't

recommend it.

If you do write such code you will invariably be writing code into your solution 'stuff' which you could leave to the data Server, if you

took the time to set up your DB, Tables and Columns correctly.

Why re-invent the wheel ? Anyway, MSSS does it much better than us.

Scripting some Common Key Constraints :

Primary Key :

Let us return to our test DB and the few tables we have - see Object Explorer (OE) below :-

===========================================

Note the 'Carrier' table and associated nodes in the above image.

If we use our existing 'Carrier' table and examine what the system Script is we see the following :-

=============================================================================

The above script makes a new table as well as setting the Primary Key (PK) and an index on the same column too.

If we had not created a PK initially, when the table was first made, we could use the following script afterwards :-

============================================================================

Yes, it would appear as though the Management Studio 'system' has added a couple more attribute / property values.

But basically the scripts do the same thing - set a PK with a suitable index on the 'CarrierID' field.

Foreign Key :

Let us now use our existing table called 'OrderHeader' to see some script for Foreign Key (FK) declaration.

First checkout the OE pane details :-

=================================================

Let us right click the first node representing a FK, that of '...Carrier' and this is what we see shown below :-

===================================================================================================================================================

If we can make good sense of the PK script far above, then we should be able to make sense of the FK

script lines just shown - immediately above.

I think you will find it useful and constructive to view the script produced by the MS system when we use the GUI and

context menu tools to display the script to create the 'OrderHeader' table, complete with its PK field and also its two

FK fields.

Checkout this following image :-

=======================================================================================================================

It is interesting to read carefully through above script, it seems as though we are simply making a new table with

a PK and associated index, and then to the existing table adding a couple of FK constraints, one by one.

So to 'Summarise' for FKs, we simply need to know how to script such that we 'ALTER' an existing table, and we

appear to have two parts to our script.

Personally, I am not quite sure I have my head around these table alterations, but I will keep working at it, and at least

we have script that works for us.

I am tempted to use the MS GUI tools whenever I can, that way I don't get mistakes introduced. I will then copy and paste

the script text into ADO.NET methods I write later in my Vulcan code.

Time to move on, see you in the next subsection .......................

PHH - 9th. December 2011.


Setting Constraints

First of all let us discuss what 'constraints' are and how they apply to our DB and its Tables - and

indeed Columns too.

Here is what the SDK documentation has to say - the Books on Line (BoL) stuff within Management Studio.

Constraints let you define the way the Database Engine automatically enforces the integrity of a database.

Constraints define rules regarding the values allowed in columns and are the standard mechanism for enforcing integrity.

Using constraints is preferred to using DML Triggers, rules, and defaults.

The query optimizer also uses constraint definitions to build high-performance query execution plans.

We will now move on to see different constraint types.

Classes of Constraints :

SQL Server has a few different class of constraint. These can be listed as :-

- NOT NULL

- CHECK

- UNIQUE

- PRIMARY KEY

- FOREIGN KEY

It is possibly the 'CHECK' constraint which needs most explanation and examples shown for us to study. We

will look at this in more details later in this subsection.

First let us look at some PK and FK stuff - we have seen a bit of this previously no doubt.

CREATE TABLE part_sample

(

  part_nmbr int PRIMARY KEY,

  part_name char(30),

  part_weight decimal(6,2),

  part_color char(15)

);

CREATE TABLE order_part

(

  order_nmbr int,

  part_nmbr int

     FOREIGN KEY REFERENCES part_sample(part_nmbr)

        ON DELETE NO ACTION,

  qty_ordered int

);

Column and Table Constraints :

Constraints can be column constraints or table constraints.

A column constraint is specified as part of a column definition and applies only to that column.

The constraints in the previous examples (above) are column constraints.

A table constraint is declared independently from a column definition and can apply to more than one column in a table.

Table constraints must be used when more than one column must be included in a constraint.

CREATE TABLE factory_process

(

  event_type int,

  event_time datetime,

  event_site char(50),

  event_desc char(1024),

CONSTRAINT event_key PRIMARY KEY (event_type, event_time)

);

Objects and Collections :

This is one area in which the dotNet coder should be comfortable, and feel at home, or familiar!

Okay then, the DB is an object, and holds other objects of its own.

Let us use script to find the Table objects we have made so far in our demo DB - see below :-

======================================================================================================================================

[ ** TIP ** Oh! it is probably a good time to mention that the Create and Modify dates refer to the table 'structure'.]

{ -- It is not possible to see when a table last had an update or insert using the 'Modify' column :( -- }

Notice that each table has an 'object-id', this identifies it within the DB itself.

So if another object has a parent with a 'parent-object-id' similar to one of these, then it can be said to be a child of that parent object.

In other words each table will have a number of 'child' objects - or children.

It seems to be a good time to look at the 'sys.objects' table (actually a 'view') to see if we can see any patterns etc.. - checkout below :-

==============================================================================================================================================

In the above image we see that the 'parent_object_id' values are in four groups - one for each of our User Defined Tables.

So we can think of each Table as having a 'collection' of Constraints. The implication of this is that we need to use GUI

or scripts to 'Add' new Constraints to the collection, or to remove an existing one (or many).

It would also appear that once the Constraints are in the 'collection' we can then effectively turn them on/off. They do

not need to be removed in order for them to be non-applicable, or not applied.

Going a Step Further :

In fact, now that we know that '2089058478' is the object identity for the 'MyPersons' table we can find the row entries

for the indexes currently set for this same table - see below for details :-

[ .. here we are using the 'view' called 'sys.indexes' .. ]

====================================================================================================================================================

This relates to what is shown in the OE pane - see the following image :-

================================================================

Just before we move on, let us look at how this parenting and 'collections' idea may apply to our columns.

Now that we know our table 'MyPersons' is an object with an identity of 2089058478 then we can use this fact to find out

which columns belong to our table (via the 'view' called 'sys.columns') - see the following script generated image :-

========================================================================================================================================================================

Well, we have a bunch of twelve (12) which appear to be correct ;-0)

Enough of this for now, but look out later for script which ADDs and DROPs objects to 'collections', particularly on a Constraint

level of things.

Lets move on.

Care of Indexes Set - data entry UNIQUE :

In the subsections to follow we will study an example of how an Index set on the Table influences data allowed to be INSERTed

into the table as new rows.

In the rest of the scripts for the Table and it Constraints we apparently can't find out why !? More later.

Using GUI tools :

While examining the world of 'Constraints' we will also see how we need to be at least 'familiar' with the GUI environment

of the Management Studio 'shell' for setting and dropping Constraints.

When using the GUI we will gradually open up the scripting necessary for us to do things ourselves, most of the time.

One reason why I think we need to become familiar with the GUI tools is that there appears to be no easy way using just

the OB pane to see when some Constraints are applied, and when not.

Let us move on to explore the GUI tools in MS right now.

See you in the next subsection ..................

PHH -11th. December 2011.


Using the MS GUI tools

For this next part we will be using the 'MyPersons' table of the 'MyClickStartTest' database, both

of which were created for demo purposes and learning about MS SQL Server and T-SQL.

Below is shown the details from the Object Explorer (OE) pane for this DB :-

=======================================================================

We also really need to see the GUI list of columns, just to see that no PK and FKs are set.

Check this out :-

================================================

Obviously we can also see that there is no entry in the 'Keys' node as the +/- sign is missing.

Now let us use a simple script to find the value of the 'object_id' for our table called 'MyPersons' - see below :-

======================================================================================================================================================================================

PHH - 15th. December 2011.


DATA related

Dropping, Creating, and Altering :

PHH - 15th. December 2011.


CHECK

Data Values - CHECK constraints :

==========================================================================================================================================================

==================================================================================================================

============================================================================================================

*** NOTICE that *** the script portion 'WITH NOCHECK' is used to prevent the constraint from being validated against existing rows,

and to allow for the constraint to be added.

So existing data is ignored, or left as it is, and the new Constraint added, so it can be applied to new data.

=======================================================================================================================================================================================

===================================================================================================================================================

=========================================================================================================================================================

======================================================================================================================================================

=========================================================================

======================================================================================================================================

===========================================================================

=====================================================================================================================================================

=====================================================================================================================================================

============================================================================================================================================================

=========================================================================

============================================

=======================================================================================================================================================

=======================================================================================================================================================

===========================================================================================================================================================

======================================================================================================================================

=======================================

=====================================================================================================================================================

=======================================================================================================================================================

========================================================================================================

=====================================================================================================================================================

====================================================================================================================================================

======================================================================================================

=====================================================================================================

PHH - 15th. December 2011.


DEFAULT

=====================================================================================================

===============================================================================================================================================================

=======================================================================================================================================================================================================

=============================================================================================================================================

=======================================================================================================================================

=============================================================================================================================

=======================================================================================================================================

=========================================================================================================================

There are no 'C' in this list - that is no CHECKs.

PHH - 15th. December 2011.


NOT NULL

PHH - 15th. December 2011.


UNIQUE

Let us start with a good piece of advice, or a smart observation.

*** CARE *** an Index may be set which forces 'unique' entry of data BUT there is nothing to show in the OE or

Table script itself.

=============================================================================================================================

I have included this because in my experimenting with Constraints I have come across the problem first hand.

Obviously, if we need to have a column set to 'UNIQUE' then we ought to make it a more 'specific' fashion.

Here is another piece of good advice :-

When you create a PRIMARY KEY constraint, a unique clustered index on the column or columns is automatically created

if a clustered index on the table does not already exist and you do not specify a unique non-clustered index.

The primary key column cannot allow NULL values.

When you create a UNIQUE constraint, a unique non-clustered index is created to enforce a UNIQUE constraint by default.

You can specify a unique clustered index if a clustered index on the table does not already exist

=====================================================================

Multiple unique non-clustered indexes can be defined on a table.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Now here is some other VERY useful information on the UNIQUE constraint :-

You can use UNIQUE constraints to make sure that no duplicate values are entered in specific columns that do not participate

in a primary key.

Although both a UNIQUE constraint and a PRIMARY KEY constraint enforce uniqueness, use a UNIQUE constraint instead

of a PRIMARY KEY constraint when you want to enforce the uniqueness of a column, or combination of columns, that is not

the primary key.

Multiple UNIQUE constraints can be defined on a table, whereas only one PRIMARY KEY constraint can be defined on a table.

Also, unlike PRIMARY KEY constraints, UNIQUE constraints allow for the value NULL.

However, as with any value participating in a UNIQUE constraint, only one null value is allowed per column.

A UNIQUE constraint can be referenced by a FOREIGN KEY constraint.

=====================================================================

Enough for reading, let us start to do some scripting, and makes some experiments to see how this stuff works.

PHH - 15th. December 2011.


Referential Constraints - Keys

Primary and Foreign Keys (PK & FK) :

As well as data related constraints we can have referential ones too.

We call PKs and FKs 'constraints' as well. These constraints have to do with referential integrity as opposed

to the actual data values themselves.

PHH - 15th. December 2011.


Setting Indexes

*** BEWARE *** Indexing looks to be a large and complex topic. Even 'complicated'.

I suggest we look to find out enough to help us along with our early SQL work, and learn what NOT to

do to impair any performance of Server and DB alike.

So once again we need to know something, but not everything !!

Before we go into detail of what, why, and wherefore etc., let us first see some of the contents of the

system table (via its 'view') called 'sys.indexes'.

Checkout the following image detail :-

========================================================================================================================================================

In order to make the above results set from the query, we had to set the 'object_id' to a high value as it

would appear that there are many others (with lower vales - less than 100) which seem to be totally for

system use, and don't make any sense to me and my mind !

At least the above returned rows seem related to what we have done so far in our DB and Table creation.

And that includes the one with 'sysdiagrams' in the name - as I did click the relevant node in our OE pane

I seem to recall ;-0)

See the following image for this 'diagrams' related index :-

====================================================

Okay then lets more on to the next part - that where we understand more of the details in the 'Setting Relationships' section.

Understanding more - the index 'properties', of field values :

When we were examining scripts to set PKs on our tables in the 'Setting Relationships' section previously, we saw a lot of

ON / OFF values for strangely named fields, or columns.

If we now look to the right hand side (RHS) of the 'sys.indexes' table we can relate back to some of these - see below :-

===============================================================================================================================================

Try and relate some of the above 'logical' or Boolean (bit) switches to the following script we saw earlier :-

===========================================================================

Maybe we will be able to understand more when we look at other information on indexes in the SDK documents.

General Point to Understand :

Before we go much further, we all need to understand is that there is an awful lot of SQL stuff for us to learn before we

need worry too much about indexes, and their fine points.

We should be able to manage very nicely with default settings on any index we create, at least for a long while as

we learn T-SQL and much more.

I will listen once again to the DVD section from Paul Nielsen's training video disks to see what he has to say.

For the moment lets make do with some useful SDK documentation material - see below :-

This topic describes major index creation tasks and provides implementation and performance guidelines to consider before you create an index.

Index Creation Tasks :

The following tasks make up our recommended strategy for creating indexes :

1. Design the index.

Index design is a critical task. Index design includes determining which columns to use, selecting the index type (for example, clustered

or non-clustered), selecting appropriate index options, and determining filegroup or partition scheme placement.

2. Determine the best creation method.

Indexes are created in the following ways:

A.) By defining a PRIMARY KEY or UNIQUE constraint on a column by using CREATE TABLE or ALTER TABLE

The SQL Server Database Engine automatically creates a unique index to enforce the uniqueness requirements of a PRIMARY KEY or UNIQUE constraint.

By default, a unique clustered index is created to enforce a PRIMARY KEY constraint, unless a clustered index already exists on the table, or you specify

a unique non-clustered index.

By default, a unique non-clustered index is created to enforce a UNIQUE constraint unless a unique clustered index is explicitly specified and a clustered index on the table does not exist.

Index options and index location, file-group or partition scheme, can also be specified.

An index created as part of a PRIMARY KEY or UNIQUE constraint is automatically given the same name as the constraint name. For more information, see PRIMARY KEY Constraints and UNIQUE Constraints.

B.) By creating an index independent of a constraint by using the CREATE INDEX statement, or New Index dialog box in SQL Server Management Studio Object Explorer

You must specify the name of the index, table, and columns to which the index applies.

Index options and index location, file-group or partition scheme, can also be specified.

By default, a non-clustered, non-unique index is created if the clustered or unique options are not specified.

To create a filtered index, use the optional WHERE clause.

3. Create the index.

Whether the index will be created on an empty table or one that contains data is an important factor to consider. Creating an index on an empty table has no performance implications at the time the index is created; however, performance will be affected when data is added to the table.

Creating indexes on large tables should be planned carefully so database performance is not hindered. The preferred way to create indexes on large tables is to start with the clustered index and then build any nonclustered indexes. Consider setting the ONLINE option to ON when you create indexes on existing tables. When set to ON, long-term table locks are not held enabling queries or updates to the underlying table to continue. For more information, see Performing Index Operations Online.

Permissions :

Requires ALTER permission on the table or view.

User must be a member of the 'sysadmin' fixed server role or the 'db_ddladmin' and 'db_owner' fixed database roles.

Creating Indexes from Script :

*** Remember *** we are learning to create indexes more from a sense of interest and learning, than from necessity.

If we set up PKs and FKs as previously covered in earlier subsections, then we will have done all what is reasonably required

for the pretty standard SQL database.

First let us use our earlier 'test' table called 'MyPersons' and remove the PK set on it.

===================================================================================================================

We can try to DROP the index using script (as above) or the GUI tools, as shown below :-

Select below the modify option in the context menu :-

=================================================================

Right click the first line of the Designer tab, where the golden key shows the PK for 'PersonID' is placed.

======================================================================================================

After selecting the 'remove' PK option - and remembering to select the relevant 'Refresh' options on the OE nodes,

we should see the golden key has disappeared - been cleared - see below :-

=============================================

Now we can use this table to experiment using other type of script in order to create other types of indexes,

and on other fields, and even multiple columns as well.

Creating a simple non-clustered index :

Let us start by making a simple non-clustered index on the Date of Birth column - see below :-

=================================================

Once we have an index we can add extra script which will DROP it first, when we need to re-create it.

The following script works well, for both DROPping and CREATing an index - check it out :-

==============================================================================

I have annotated this script so it is clearer to understand - see below :-

==========================================================================================

Yes, the text string 'IX_MyPersons_DoB' is the name of the index object. And yes, we could have called it anything

but it would be little use calling it "Santa's little helper" ;-0))

Now we can see in the refreshed OE pane the presence of the new index - see below :-

============================================================

Note above that Statistics are also present, and MS SS uses these to help it manage the index and when it

chooses to apply one in order to solve the query sent to it. Once again, leave the Server to sort itself out.

Creating a simple non-clustered composite index :

So what if we wish to index on multiple fields ? Use more than one data column ?

This is not a difficult task, hardly any more effort that a single column - check this out below :-

==============================================================================================================================

And if we refresh the OE pane nodes we see the following :-

=================================================================

*** REMEMBER *** it is NOT up to us as users, or even DBA, to select which index is used by the Server in answering

any query sent to the Server itself.

All we do is provide indexes we think will be useful - if we know what we are about, that is. I would leave well alone !!

Now then, here is a script of what we have just created - supplied by the GUI tools in the MS 'shell'.

===================================================================================================

Yep! in the above image we see that we now have two fields specified between the braces above the WITH clause.

If we wished we could have more, and we could change the SAC / DESC specification as well.

Creating a unique non-clustered index :

And finally for this subsection, we can create a UNIQUE index. One that will only allow single entries of a field

value where the field is specified as the index column.

======================================================================================================================================================================

Yes, I know, if we were to specify DoB as a UNIQUE index this would not be very clever, as the chance

of two people having the same birthday is often quite likely in a group of persons, customers, patients and

the like.

Oh! from listening to Paul Nielsen and his DVD it seems a good idea to use a 'BIGINT' for a PK and let the

system make its own index on this - then any newly added data does not cause problems with splitting data

pages when trying to store the new information.

All such new data is just added on the end of existing stuff, and few complications arise, if any at all.

So don't be tempted to use "Surname+Firstname" for any people base table - use 'PersonID' instead an make it

a large integer - 'BIGINT'.

Obviously you can use alternative naming with no penalty in performance, like 'PatientID', or 'CustomerNum' etc..

Time to move on, see you in the next subsection ................

PHH -12th. December 2011.


Dropping - DB, Table, Constraint

Databases :

To remove a Database completely from the computer - hard disk and filing system - we use the MS SS command 'DROP'.

So 'DROP' means remove, delete, completely get rid of, both a DB and also a Table.

You can think of 'DROP' as referring to the fact that we are talking of objects and not files.

Tables are objects within the DB, and the database is an object as far as the Server is concerned.

Do NOT get mixed up, or confused by, 'Detach', which is a term applied to associating an existing DB with the Management Studio

'shell'. When we 'attach' an MDF we make a link to the database via a selected server instance. So a 'detach' can be undone

by an 'attach', but this is definitely NOT the case with a 'DROP'. And there is no such operation as 'Ctrl+Z' to undo the 'DROP' operation.

There is no 'ROLLBACK' option either. This will discussed in other sections.

Obviously we can do a DROP for a DB with both a script 'query' and also the GUI operations within the MS shell itself.

By Script :

In the image shown below the script was supplied to me by using the GUI option to display the script.

It is VERY useful to get into the habit of using the 'Scripting' options within the context menus of the MS shell.

Here is what I did to get the script :-

======================================================================================================================

And after tidying some comments line stuff, here is the script we will run to drop our DB - shown below.

Check this out :-

======================================================================================================================================================

So all I need do now is highlight the script text and select the 'Execute' button :-

==========================================================================================================================================================

When we do a DROP manually, by script, it seems as if we always need to do a 'refresh' for the OE to catch

up with what has happened.

In the image below I am forcing a refresh :-

=======================================================

And now the OE pane seems as we would expect - no more 'MyClickStartTestTWO' database.

=============================================

We now need to prove that the files have been completely removed by Windows from the hard drive.

If we use Windows Explorer (WE) to view the folder deep within the 'Program Files' folder we see the following :-

======================================================================================================================

Notice that we now only have 'MyClickStartTest' left and the other one has 'gone' ! - Disappeared - is no more ;-0))

Before we move on :

Let us take a few moments out before we move on - let us see how we can find and list all the possible DBs we have.

Check this out below :-

=======================================================================================================================

And now we can see our available databases, what about listing our Tables ( 'User defined' ) ?

Once again we have an easy way to script a query request - see below :-

===============================================================================================================================================

Okay then, some of this may prove useful in the future, lets move on .............

By the GUI operation :

Let us now investigate the removal - or 'DROP'ping of a DB using just the GUI tools, and context menu options.

We will not use script directly ourselves - but of course we fully realise that the MS shell is just sending script to the Server

for us, as all SQL Server operations are done by scripting, and query statements.

So to get the context menu up all we need do is right click the correct DB node in the OE pane - see below :-

=====================================================

When we select to DROP (or remove) a Database we need to make sure that there is nothing on our Desktop which is using

the DB. And that often is true for part of the MS shell too.

Study the following image and also notice the two check box options at the bottom :-

If you neglect to check the last option - 'Close existing connections' - you may find you get an error message as a consequence

of trying to do the 'DROP' operation.

Anyway, in the case shown above it worked in my MS 'shell', as the following images show :-

=============================================

As well as the DB not showing in the 'refreshed' OE pane list, it has also gone from the 'WE' listing of the data

folder in the Program Files sub-folder - see below :-

========================================================================================================

In the next part of these notes we will see that a whole DB, one with Tables and more, may be equally EASILY deleted

using the 'DROP' command.

For now let us move on to Tables themselves, DROPping them one at a time ;-0)

Tables :

Here we will look at what we may loosely call deleting or removing tables. The SQL term is 'DROP' as it is for a few

other objects within the database itself.

Like with DBs, when we DROP a Table it is actually removed in total - there is no going back, it is gone for good !

PHH - 9th. December 2011.


Constraints

PHH - 15th. December 2011.


Changing Table Structure

Okay then, we need to do some experiments on changing Table structure.

By 'changing' I mean adding and removing columns, changing data types of existing columns, and

making and removing Key constraints. And probably a few more things too.

Let us start by seeing the current structure of our test Table, called 'MyPersons'. See image below :-

============================================

Let us see if we can simply remove the two columns called 'Address3' and 'Address4'. To do this we need

to bring up the Designer tab using the context menu, as shown below :-

======================================================

And when a suitable selection is made from the menu options we see the following :-

==========================================================

I have now used this GUI approach to removing columns - correctly termed 'DROPPING' columns - on both

of these two address fields, and all seems to be fine after saving using the context menu from the top tab, as

well as refreshing the nodes in the OE pane.

Study the following image for its full details :-

===========================================================================================================

Obviously, the bigger question is do we still have data in our table rows. To check on this we need to query.

Here is what I have just done :-

=========================================================================================================================================

Data is present !

Well, all seems OK so far, but what if we now insert a column - check this out :-

===========================================================

The result of the above operation as to place the new column (actually a row in this representation) in

the wrong place, see below :-

=========================================================

As the above image shows, we can drag and drop the row, representing or column, into

another position - that indicated by the horizontal black line.

The image below is the result after the drop - of the DnD operation.

=========================================================

So now I just clicked in each input cell in turn and entered a suitable value.

Note I have selected to NOT allow NULLs. See below :-

==========================================================

What happens if we try to save these new table structure settings using the Tab context

menu 'Save' option ?

Below I an doing the 'save' selection :-

==========================================================

Oh! dear, we have a problem. Checkout the wording in the LONG error message - which

does not seem to display easily.

======================================================================================================

============================================================================================

It would appear that our problem is due to use having data rows already entered into our table.

We will come back to discuss and explain the problems caused by having data, and wanting to insert a new column

which has a setting to NOT allow nulls.

For now we need to set 'allow nulls' to true, and then retry. This seems to work OK and we see the following after saving

from the tab context menu :-

=====================================================================================================================================

Let us now try moving some of these columns around - changing their position.

Here is the final rearrangement I will try to save :-

==========================================================

And that appears to work also. Study the following image in detail :-

========================================================================================================================================

Adding some extra columns :

Let us now try to add some extra columns (or rows in our Designer tab) - we will call

these 'Description' and 'PersonalComment' - see the next two images below :-

=========================================================

This and the next column will both be 'nvarchar(MAX)' data types, 'n' meaning Unicode string data where

two bytes are available for each character - hence we can think of 'n' or 'N' fro National - as a country's language

is often associated with it nationality.

===========================================================

So what happens when we try to save these changes while both have settings to NOT allow nulls ?

===============================================================================================

Once again we seem to be having problems because we have existing data in the table of which we are trying to

change the structure.

So we will have to allow nulls for now until we clear the data, then we should be able to change the table and column

settings back to NOT allow nulls.

Study the following carefully :-

==========================================================

Having re-set the 'allow nulls' we can 'Save' and get the new data grid layout after sending a query script

to the Server.

Checkout the following information :-

===========================================================================================================

However, there is another way to approach this problem. If we add a value for each of the cells in the 'Description' and

'PersonalComment' columns, we would be able to make the changes then I feel.

Lets give that a try, using the 'Edit Top 200 Rows' option - see below :-

=======================================================================

In the editor tab it was easy to make the cell content changes. And you can get into a

rhythm so that Copy and Paste work well for us.

Here are the results after a select query :-

===================================================================

UPDATE - Alternative way to set cell and column values :

Instead of using the GUI tools we can quite easily script the data changes. See below :-

===================================================================

This results in the following data rows - check out the image below :-

===================================================================

And so we can also change the second of our new columns - see below for details :-

=========================================================================================

And why stop at our two new character fields, let us try and set our 'Joining_Date' cell values where they are

as NULL for the current state of the table data - see the following details :-

==========================================================================================================

And while we are at it we may as well change the nulls in the 'Salutation' column as well - see below :-

==================================================================

Which produces the following when we query with a select :-

===========================================================================================

Re-setting the 'Allow Nulls' on certain fields :

Okay then guys, if we have no more NULLs to worry about, we can now turn our attention to setting the

options on three of our fields as we originally wanted some time back - above in this subsection.

Returning to the Designer tab we can now un-check the boxes for these fields - one of which is the

'Joining_Date' field illustrated in the following image :-

==========================================================

If we look at the OE pane and refresh all the necessary nodes, we see the following :-

=========================================================================================================================================

As you can see for yourself, we seem to have eventually gotten to where we wished to be way back

in this section.

Summary :

Try to do all your table structuring without any or much data, if you need to make all but the smallest change

then remove your existing test data using TRUCATE (DDL) or DELETE (DML).

In fact it is well worth having a script which does the bunch of INSERTs to help the developer in these situations.

That way script can be re-run and help reduce typing by a large fraction ;-0)

We will look later at such tricks and tips.

Time to move on - see you in the next subsection ...........................

PHH - 17th. December 2011.


Dealing with NULL - Some Aspects

This is a 'thorny' subject and one which can be confusing to the new guys - or more correctly I should

say that 'it is confusing' to the new guys - all of us !

I like to think of things like this - NULL is not a value, in fact 'null' is the absence of any value.

However, a table 'cell' or a specifically designed data type can be NULL simply because it has no meaningful

value.

If we are using SQL data tables then we MUST focus on NULLs at some stage, and we need to find a way of

dealing with them in a suitable fashion - one which does not cut across what others do.

We must NOT develop our own 'cock-eyed' way of doing things 'Null'.

Let us start by looking at some SDK stuff to help us :-

========================================================================================================

And then there is a little more helpful stuff from the SDK blurb - see below :-

========================================================================================================

In many ways we can think of NULLs as being fine while they are in the data table cells. While they are there

they cause us little harm, but our SQL query statements must take then into account in a correct and acceptable

manner.

However, even if we have all of the above correct, it is often when we need to handle the returned data sets that we

get into NULL difficulties. We will discuss more on this later, but for now we need to study the following SDK information,

which describes a namespace which can really make our coding lives easier.

Check this out :-

=============================================================================================================

I gather from reading some of the earlier SDK blurb above, that we need to be careful in the semantics of our

statements and words, so I will make it clear to you that when I say 'nullable' I simply mean that a variable can

be made to take 'null' - as opposed to one which can't.

So for both writing to and reading from SQL tables the SqlTypes can be most helpful to the coder.

Although I have only used MS flavoured SQL server, I would think that these could still be employed with

other 'flavours' of the DbFactory approach.

If you wish to have a data grid in GUI and have a DateTime column which can take NULLs, then it may be best

to give this a type of SqlDataTime, depending on the grid type you are using.

Just remember, it is NOT the fundamental data type which is NULL, but the 'container' which can accept 'null'. Just

in the way a cell in a table or grid 'contains' something, think of the SqlDateTime variable instance as a container.

Now let us stop theorising, and look at some real coding situations - see below :-

Dealing with Returned DataSets - converting rows to custom objects :

Very often in our applications we need to turn returned data rows from the SQL Server back into business

objects, before continuing with our business logic.

In the following image we see a typical example of this - study the lines starting at 796 and 828 in particular :-

==================================================================================================

For instances of my own custom objects I usually prefer not to have NULLs and more often than not I

design my code so that all of my BO properties have values, even if these are default ones set in the

constructor.

There may be better, or more ideal ways to do this, but at least I feel I have gotten around the potential

error situations, and am being reasonably true to both 'DateTime' and NULLs.

In the code shown above I am only likely to get NULLs because of errors due to development, and runtime

crashes during the development phase itself. I know, I have had my share !

Once my code is finished and thoroughly tested I should have values for my DateTime properties.

HOWEVER - alternative style of coding :

You do need to be aware that there will be many other scenarios in which NULLs are required in memory

as well as in the SQL database. After all a NULL shows the absence of a meaningful value, and that in

itself can be useful.

So to anyone needing to use this style of coding I suggest you look closely at using SqlTypes to handle the

NULL situation correctly, and without errors being thrown. This will probably mean you including them in

your custom class definitions.

At least now you are more aware of the possibilities and can at least try and solve your own problems from

a more knowledgeable standpoint.

Time to move on, see you in the next section ................

PHH - 13th. September 2011.


Using System Views - DB Management

Often we may think, or talk, of accessing the 'system' tables when in fact we are using 'Views'.

There is a host of 'System Views' as can be seen directly from the Object Explorer (OE) pane itself.

======================================================================================================================================================

In order to produce the image above I have set a filter on the 'System Views' - the complete list

is very long indeed.

Having spotted the 'sys.databases' entry in the list, let us try and do something with that.

Here is the structure of the 'view' - or at least the columns we will get returned :-

========================================================

We will not look at this data as yet, we first need to see how we can use a 'view' to see all the 'views'.

Seeing the Views :

In the following image we can see how one view works for us and another gives us nothing returned in the data set.

It would appear that 'sys.all_views' is the one we need to use - see below :-

================================================================================================================================

*** Tip *** At the end of the subsection we can see an easy way to get to all the 20 views the user needs to access, this is

just one of those that will be made easier.

Seeing the Databases :

In order to see which DBs we have available to us in the MS shell - as these have been attached - we need to do the following :-

[ .. Server below is an instance of 2005 Developer .. ]

========================================================================================================================================================================

And if we connect to a different server instance - that of Server 2008 Developer R2 we see this :-

===============================================================================================================================================================

I have to admit I don't yet know how to swap between the two instances open (connected) in my OE pane.

The image below shows what I have in front of me :-

=====================================================================================

I would like to be able to 'point' at one Server and run the script for seeing databases, and then 'point' to the

other connection and then run the same script to get a different return set - much in the way we can 'point' at

DBs by using the 'GO' command word.

I will ask the News Group to see what they have to say !

For now lets move on.

Seeing the DB objects :

If we use the following script on a large existing DB we do get a lot row entries to study / view.

My client's DB shows almost 36 thousand rows. Checkout the following image :-

=====================================================================================================================================================

[ ** TIP ** one good reason for not having duplicate Tables for data archiving purposes !! ]

{ .. remember each table has a host of columns which are also objects .. }

Seeing Your Tables :

[ .. only seems to contain U - user defined tables ! .. ]

The script used in the image shown below will only show us 'User' tables (called User Defined) and NOT system tables. this

at least cuts the number down a lot.

=============================================================================================================================================

Let us now move on to see some column information.

Seeing the Columns :

In a similar way to that above when we found Table rows, we can find Column rows - see the following image :-

====================================================================================================================================================================================

In the above image I have highlighted the row from which we start to see columns made by me - the user.

Up to row 375 these seem to have been made by the system, for its own purposes.

Now to look at indexes.

Seeing the Indexes :

Once again we see the first bunch of rows are for indexes made by the system for its purposes, not the user's.

In the image shown below we see the first 72 to be of 'system' creation type. Checkout the image details below :-

======================================================================================================================================================================================

Lets move on to 'Constraints'.

Seeing your CHECK constraints :

It is VERY useful to be able to see in one place just which 'Constraints' we have applied.

In the following image we choose to see the CHECK constraints and can view their 'definition' as well. Checkout the following :-

==============================================================================================================================================================================

Now what about some other types of 'Constraint' ?

Seeing your KEY constraints :

In order to gain this information we need to use the 'sys.key_constraints' view (or R/O system table).

Study carefully the image below :-

=============================================================================================================================================================

So what of Procedures and Functions ?

Seeing Stored Procedures :

Well, I have found out how to see the Stored procedures (SProcs ) and this is shown below :-

==============================================================================================================================================================

As of now, I have yet to find out how to find similar details for Functions in the 'Programmability' node of the OE pane.

I will work on it ;-0)

*** Solution *** see the end of this subsection where we look at the view called 'INFORMATION_SCHEMA_ROUTINES'.

Seeing your Triggers :

Although many new guys to SQL, including myself really, will not use Triggers until they move from being new to SQL Server and become

slightly more experienced, we still hear of Triggers, and need to know what they are and how to find them - or at least 'view' them.

Below is the script and the results - study them carefully :-

===========================================================================================================================================

And now we can see the easier way to do some of this 'viewing' !!

'Safe' Seeing your Complete DB Information :

Well, would you believe it ? There are twenty (20) 'views' at the start of the 'System View' node in the OE pane

which do all the very stuff we need to do, and have been trying to achieve above - selecting and using the system

views from the body of the whole set.

Look at the image shown below to see that although the names look rather scary or intimidating, they are in fact

our friends when it comes to accessing DB information at our own 'user level'.

By that I mean we are not experts or gurus - just users, and some of us quite new to SQL Server 'stuff'.

Check out the following details :-

====================================================================================================================================================================================================================

So we only need look through these twenty 'view' names to see what may be useful to us.

Let us now try and see what the 'Table_Constraints' view has to offer us.

Table Constraints :

Below is how this 'view' looks in the OE pane, when expanded :-

====================================================================

Now let us use the 'view' as we would a table - to get the row information.

Checkout the following image detail :-

=============================================================================================================================================

I think you will now agree with me that there is a HUGE amount of useful information available to us, and that

information is about our DB itself, and the settings which we have applied.

MS SS is not just about our own data records. We can script queries which allow us to manage and oversee

the database itself.

Yes, obviously, sometimes it is easier to use the GUI tools available to us in Management Studio, but at least

we have the choice.

See our Functions (and Procedures) :

One benefit to the new user oriented 'views' in the 'INFORMATION_SCHEMA' is that the one for 'ROUTINES' gives

us a list of both Stored Procedures and Functions.

Study the image shown below :-

=====================================================================================================================================================================================================

As you can see in the above image we even have the 'definition' of the routine in one field called 'ROUTINE_DEFINITION'.

So if we wish to get my own functions in the 'pelican' DB (I always these days add '_phh_' to my names) we can do the

following in the query tab of MS - study carefully :-

[ .. whoops! by using 'ft' we are only seeing the table functions - those which return a 'table' .. ]

==================================================================================================================================================================================================================

*** Note *** the 'escape' sequence character is defined by the script to be '\' the back-slash.

This defined character is then seen in front of the '_' underscore character, to change its behaviour from a

wild-card to just the normal character itself.

In the above image the field called 'ROUTINE_DEFINITION' is quite wide, and because of how I do things for my Functions,

the first few characters of the definition does nothing but specify who, what, when etc..

Now time to move on, see you in the next section ...................

PHH - 16th. December 2011.


Chapter 1 - the SELECT statement

*** Note *** It may be useful if you look ahead two subsections to remind yourself what an

'Expression' is - as this will be referred to many times while discussing the SELECT statement.

======================================================================

Paul Nielsen has suggested that in order to get the most from T-SQL we all need to have a deep

understanding of the SELECT statement and what it can do for us. He strongly suggests that we

improve our skills with it till they become finely tuned !

What does SELECT do ? :

Basically it allows the developer and DBA to choose which columns and which rows are returned in

a dataset, as well as their order and how they may be grouped and aggregated.

The SELECT statement is made up from a number of possible 'clauses' - most of which are optional.

When you are dealing with any real life DB the crucial 'thing' to understand is that before many of the

clauses can take effect, the process of de-normalisation has to take place.

You should realise by now that NORMALISATION in a relational database is VERY important indeed.

Tables are designed around normalisation but this does make it more difficult for humans to read the

data as we may have done in DBFs or flat files. It would seem that we only get any readable, or

understandable rows of data if we join together data from many tables.

So at the heart of the SELECT statement is the FROM clause, and it is within the FROM clause that

we cause de-normalisation to happen.

Once again, Paul Nielsen has suggested that in order to be able to get the most from the SELECT

clause a developer or DBA needs to have a high level of skill with the JOIN clause, in all of its various

forms.

Expect to have to JOIN a handful of tables for any common task, do NOT run away (or hide) from

using JOINs - they are your friend ;-)

Once we have a fully useful dataset after de-normalising, we can subsequently open up the power and

flexibility of the SELECT statement by the use of the CASE function in just about any place in the

statement we choose. Paul Nielsen recommends we also become expert with the use of the CASE

function. See chapter 15 to get more skilled with its use.

Looking at a Working Example :

It is expected that anyone using and reading these notes will have had some experience with using SQL

previously. If the following SELECT statement frightens you then you ought to go back and use the other

volume of ClickStart eNotes, called 'ClickStartSQL'. This will ease you in to using SQL more gently.

The example illustrated below is being shown because it contains a number of important elements of the

SELECT statement, as used in real life queries. Study the script carefully :-

==============================================================================================================================================

If we look at the execution plan for this query, the one created by the SQL Server, we see what is

done first. I have to confess that I am just starting to get into viewing such plans, at first I was just

looking for success - now I seek understanding ;O))

In the image shown below, read it from right to left, following the arrows :-

Even a quick view, or scan of the image shows that all the required JOINs were carried out first. And

if that meant the server had to make and use indexes, then that is what happened.

[ The impact of any missing or not available indexes has nothing to do with me - I only do the reporting

for my client ! ]

The simple point being demonstrated here is that the FROM clause 'performs' first, and gathers together

the complete de-normalised dataset - all rows and all columns - as specified by the JOINs.

Only when the FROM clause is 'done' does the rest of the statement start to operate on the data.

The filtering of rows, and the selection and creation of columns, and then the ordering of the rows in the

returned dataset, all take place after the process of de-normalisation.

Don't be shocked when you realise that the main dataset so created will have MUCH information duplicated

throughout the large table - that's just the way it is ! If we wish to have tidy normalised data tables, then we

have to HUGE and de-normalised returned sets. 'Catch 22', seems to be the phrase to use here !?

What to do next :

You have probably realised when coming to learn SQL that we seem to need to know a lot of different things

all at the same time - which is of course impossible ;0)

Well, if you don't know enough about JOINs then I suggest you have a look at chapters 3 through 7. And,

do not panic or worry, JOIN and UNION clauses are not as bad, or as difficult, as they may first appear. It

is all quite logical when explained correctly - then you have 'blast-off' !

If you don't wish to look into JOINs any deeper at the moment, then read the subsections of this chapter,

which are all based on work I have done after I got simpler things to work for me with SQL.

Finally, if you have little or no experience with SSMS (Management Studio) then go and get some - plenty

in fact. Go read the ClickStartSQL eNotes and try our some of the exercises early on in the notes.

Whenever I am using SQL (with anything) I have SSMS 2008 open and do all my development and testing of

my queries within this environment. And SSMS 2008 also allows me to work on SQL Server 2005 databases.

See you in the next subsection when we look at 'batches'.

PHH - July 21st. 2010.


the 'Batch'

'Books on Line' (BoL) has this to say regarding 'Batches' :-

A batch is a group of one or more Transact-SQL statements sent at the same time from an application to SQL Server

for execution. SQL Server compiles the statements of a batch into a single executable unit, called an execution plan.

The statements in the execution plan are then executed one at a time.

Each Transact-SQL statement should be terminated with a semicolon. This requirement is not enforced, but the ability

to end a statement without a semicolon is deprecated and may be removed in a future version of Microsoft SQL Server.

A compile error, such as a syntax error, prevents the compilation of the execution plan. Therefore, no statements in the

batch are executed.

A run-time error, such as an arithmetic overflow or a constraint violation, has one of the following effects:

·Most run-time errors stop the current statement and the statements that follow it in the batch.

·Some run-time errors, such as constraint violations, stop only the current statement. All the remaining statements

·in the batch are executed.

The statements that are executed before the statement that encountered the run-time error are not affected. The only

exception is when the batch is in a transaction and the error causes the transaction to be rolled back. In this case, any

uncommitted data modifications that are made before the run-time error are rolled back.

For example, assume there are 10 statements in a batch. If the fifth statement has a syntax error, no statements in the

batch are executed. If the batch is compiled and the second statement then fails while running, the results of the first

statement are not affected because it has already executed.

A useful example :

SQL Server provides statement-level recompilation. That is, if a statement triggers a recompilation, only that statement

is recompiled and not the whole batch. This behavior differs from SQL Server 2000. Consider the following example that

contains a CREATE TABLE statement and four INSERT statements in the same batch.

CREATE TABLE dbo.t3(a int) ;

INSERT INTO dbo.t3 VALUES (1) ;

INSERT INTO dbo.t3 VALUES (1,1) ;

INSERT INTO dbo.t3 VALUES (3) ;

GO

SELECT * FROM dbo.t3 ;

[ *** remember 'go' belongs to the SSMS environment ]

First, the batch is compiled. The CREATE TABLE statement is compiled, but because the table dbo.t3 does not yet

exist, the INSERT statements are not compiled.

Second, the batch starts to execute. The table is created. The first INSERT is compiled and then immediately executed.

The table now has one row. Then, the second INSERT statement is compiled. The compilation fails, and the batch is

terminated. The SELECT statement returns one row.

Rules for Using Batches :

The following rules apply to using batches:

1.) CREATE DEFAULT, CREATE FUNCTION, CREATE PROCEDURE, CREATE RULE, CREATE SCHEMA,

CREATE TRIGGER, and CREATE VIEW statements cannot be combined with other statements in a batch.

The CREATE statement must start the batch. All other statements that follow in that batch will be interpreted

as part of the definition of the first CREATE statement.

2.) A table cannot be changed and then the new columns referenced in the same batch.

3.) If an EXECUTE statement is the first statement in a batch, the EXECUTE keyword is not required. The EXECUTE

keyword is required if the EXECUTE statement is not the first statement in the batch.

For more information than this, consult BoL and other MS sources - like MSDN. For now this is quite sufficient for us to

work with at the level we are needing to script.

PHH - July 21st. 2010.


the 'Expression'

Here is what BoL has to say :

An expression is a combination of identifiers, values, and operators that SQL Server can evaluate to obtain

a result. The data can be used in several different places when you are accessing or changing data. Expressions

can be used, for example, as part of the data to retrieve in a query, or as a search condition when looking for

data that meets a set of criteria.

An expression can be any of the following :

·Constant

·Function

·Column name

·Variable

·Subquery

·CASE, NULLIF, or COALESCE

An expression can also be built from combinations of these entities joined by operators.

In the following SELECT statement, for each row of the result set, SQL Server can resolve 'LastName' to a single

value. Therefore, it is an expression.

USE AdventureWorks;

GO

SELECT LastName

FROM AdventureWorks.Person.Contact;

An expression can also be a calculation such as (price * 1.5) or (price + sales_tax).

Expressions involving character strings and dateTime :

In an expression, enclose character and dateTime values in single quotation marks.

In the following SELECT statement, the character literal B% that is used as the pattern for the LIKE clause must be in single quotation marks:

USE AdventureWorks;

GO

SELECT LastName, FirstName

FROM AdventureWorks.Person.Contact

WHERE LastName LIKE 'Bai%';

GO

PHH - July 23rd. 2010.


Row Filtering - the WHERE clause

The way we control which rows are returned to us (or not) in our returned dataset from our query,

is by using the WHERE clause.

If we do not include a WHERE clause then all rows are returned - so not filtering takes place.

The WHERE clause can't create rows, but the JOIN clause may - depending on how it is used.

Although the WHERE clause is extremely important, and has great effect on the datasets we get returned,

it is quite easy to get to understand and use effectively. Many of us will have been used to filtering in

our previous data environments.

'BoL' also thinks this is straightforward as it has only this to say :-

WHERE - Specifies the search condition for the rows returned by the query.

[ WHERE <search_condition> ]

< search_condition >

Defines the condition to be met for the rows to be returned. There is no limit to the number of predicates

that can be included in a search condition.

The 'Tricky' bit ! :

Although the idea is easy enough, and the design of scripts with successful WHERE clauses is not difficult,

the full specification of the details in BoL is rather daunting or off-putting !

As with all Boolean and logical condition 'things' it can get to sound almost impossible. So I will try my

best to make it sound simple.

For a single 'search condition' the row is included if this condition evaluates to TRUE.

For multiple conditions it depends how they are grouped. If we have a series of single conditions 'linked' by

the OR operator, then as soon as one of these evaluates to TRUE the row is marked for inclusion and the

remainder of the script line is ignored.

If the multiple conditions are 'linked' by the AND operator then all must evaluate to TRUE for the row to be

marked for inclusion in the returned dataset. As soon as one evaluates to FALSE the row is marked for

exclusion and the rest of the line is ignored.

If like me you have used filter conditions elsewhere you will know that mixing NOT with AND as well as OR

can get to be a bit of a nightmare, then you know you need to tread carefully, as well as using parentheses

when at all in doubt !

More than ANDs and ORs :

In the WHERE clause we can use more than just Boolean operators, and you may notice within the a single

'search condition' we may have CONTAINS and EXISTS.

To scare yourself look at the syntax of a 'search condition' - BUT - don't panic, it is not that bad really !

====================================================================================================

After the panic of thinking we can't understand anything within the image shown above, we see a few words

which look interesting to us - like CONTAINS, EXISTS, LIKE, BETWEEN, and IN. To name but a few ;0)

A Suggested Way Forward :

This is one area of SQL in which a few good working examples can teach us a lot, without the need to use

too many words to explain things in detail. I suggest we look at a number of these useful 'arguments' from

above as we look at examples which we may choose to copy, adapt, and follow.

Example 1 :

In the example we saw at the start of this chapter here is the simple filter statement :-

WHERE        Y.invoice_customer = '99999999'

So any row is included in the return dataset if the character (string) column 'invoice_customer' contains

data which is eight '9's and nothing else.

Example 2 :

Here is another example, and one in which we have two Boolean expression within the 'search condition'.

For the search condition to be TRUE both the Boolean conditions have to be TRUE.

In the image below notice that we are using tests for IS NULL, as well as IS NOT NULL.

=========================================================

Also notice that the returned data is a scalar quantity rather than a table or list. This is because we have used

an aggregate function to count the number of rows in the set specified in the statement.

Example 3 :

Moving on to include a 'DateTime' field we can see this real example below :-

==============================================================

So in the above example, for any row to be marked for inclusion in the dataset all four of the Boolean expressions

much evaluate to TRUE.

Now if we are interested in a 'dateTime' field being in a date range, we need to use something like what is shown below

for example 4.

Example 4 :

Once again the image shown below is of a live query, and there are three rows returned.

Notice that the test on the 'P.Pers_CreatedDate' is for it to be within the range. The BETWEEN is inclusive of the DateTime

range values specified. Be warned however, you need to check out how to make sure you get everything within the range you think

you are specifying. The dateTime value has a time component to it as well as the date, so check out the use of dateTime values in

other parts of the ClickStart eNotes.

*** in the example below, if I run the query at 10.30 am on June 13th. 2010 and the date being tested is the right date but has a time

component of 2.30 pm (14.30) then the row will be marked for exclusion !!!

It may be more desirable to take 'getdate()' and use this to create another dateTime value with a time component of 23:59:59 but with

the same date componet. This way the range would probably be as wide as you may expect.

An alternative to this is to take each data value and force the time component of it to be 00:00:00 - it is up to you, and you need to

choose wisely.

==================================================================================================================================

Example 5 :

In this fifth example (shown below) we see the use of IN to allow us to specify a number of values, any one of

which can be used in a successful test - as NOT IN is also allowable.

=======================================================================

In the image above, the example is still using a series of AND operators for the whole WHERE clause. This

means that for a row to be marked for inclusion all six Boolean conditions need to be TRUE.

During the query execution, as soon as one condition is found to be FALSE then the row will be marked for

exclusion, and the Server will move on to the next part of the overall evaluation.

Example 6 :

Now let us look at mixing ANDs and ORs - we need to take care !

=================================================================================================

In the image above, the example evaluates the 'double' Boolean expression to see if it is TRUE.

Notice the parentheses enclose the following :-

( acc_fds_pelinurse is not null or acc_fds_dob is not null )

And this gives a vastly different row set returned to when these parentheses are not included, see belwo

to see the alternate version and how thousands of more rows are now included in the return set :-

=========================================================================

So the moral of the tale is be very careful with mixing AND, NOT, and OR, as well as where you place the

parentheses in your script lines.

Example 7 :

Let us now move on to see how EXISTS can be used to improve our filtering.

BoL says :-

EXISTS - Specifies a subquery to test for the existence of rows.

[ Returns TRUE if a subquery contains any rows. ]

The example shown below is of a very useful type. There will be many places in our querying where we can

benefit from such a filter condition. I for one will look out for places where I can employ this syntax.

Study the details shown below :-

================================================================================

*** NOTE *** The example code from BoL does not work for me as it is (out of the box), because of

a lower case 'd' in 'a.ContactId' - my DB must be set as case sensitive !?

Here is what I started with at the beginning of this short section above :-

=============================================================

This is at least a second occurrence of me trying script directly from the SDK documentation, only to

find a small error - as if someone is free typing from notes. So be warned !

At least my notes are of working code and images of script and code are to prove it. The

downside from you point of view is that you need to enter the code manually - but then you need all the

practise you can get ;O))

Example 8 :

Now probably our last example for the moment let us look at LIKE, this can be

a very useful tool for those of us wishing to design filter conditions :-

===========================================================

And if we wish to push this a little further then we can do as follows :-

==================================================================================

The reason I chose 'Ki' and not just 'K' was because the full dataset return was quite large :-

===================================================================================

I hope you have noticed that we have been using in our last few examples, a WHERE clause within

another WHERE clause !

Surprising how fast we move on in our learning ;0)

See you in the next subsection.

PHH - July 24th. 2010.


Sorting - the 'Order By' clause

This is a very useful clause and one which is relatively easy to script and use.

As usual we will look at what 'BoL' has to say about this :-

'ORDER BY' - Specifies the sort order used on columns returned in a SELECT statement. The 'ORDER BY' clause is

not valid in views, inline functions, derived tables, and subqueries, unless TOP is also specified.

*** Note ***

When 'ORDER BY' is used in the definition of a view, inline function, derived table, or subquery, the clause is used

only to determine the rows returned by the TOP clause. The 'ORDER BY' clause does not guarantee ordered results

when these constructs are queried, unless 'ORDER BY' is also specified in the query itself.

Syntax :

For most of us developers, for most of the time, the syntax shown below is quite sufficient for our needs :-

' ORDER BY fieldname1 ASC /DESC, fieldname2 ASC /DESC, fieldname3 .......... '

The general format is what we old guys used to call a multi-key sort back in Z80 and Tandy days !!

Note that the default order is ASC but it seems to me sensible to always specify what is required, as it makes

the script easier to read as well as understand - AND - it is hardly onerous in its requirements ;-)

One final remark is that it is actually 'expressions' and not just 'fieldnames' which sorted, so the following

is more correct :-

' ORDER BY expression1 ASC /DESC, expression2 ASC /DESC, expression3 .......... '

It seems that we will gain most be seeing some examples at work.

Lets do that now.

Examples :

Let us use the AWLT database and look at the table 'Customer' - FYI, my DB does not appear to be set to case sensitive.

Study the script below to see a simple sort being applied :-

=======================================================================================================================================

As you can see in the above image, the rows seem to have been returned in order of last name A to Z.

Let us now try to present a final row order of last name then first name, and so see so useable results

we need to remove our previous filter, and look at 'Miller' - see below :-

===============================================================================================

In the image above look at Miller and notice that the first names are in order Z to A.

However, if we make an expression form concatenating last and then first name as shown below,

we get both fields in ascending order, that of the default :-

=============================================================================================

Finally for this example, we can combine a name and and identity - both is descending order.

See below for details :-

=================================================================================================

Looking at the data rows above, we seem to have achieved our goal.

Collate :

If you special needs of an order of characters based on other than the basic 26 letter alphabet of the USA and UK,

then you can specify your requirements in the COLLATE phrase of the ORDER BY clause.

Consult the 'BoL' section named 'Using SQL Server Collations', as well as 'Collation and Unicode Support'.

I think we have covered enough here for you to be able to sort well enough to do most things you require.

And, don't forget, if at all possible keep things simple. Complex 'orders' like logic, can often produce something

which is unexpected, simply because you did not think things through fully. This is less likely to happen with a

simple order or set of multi-key specifications.

See you in the next subsection.

PHH - July 24th. 2010.


TOP, DISTINCT, PERCENT and more

What can 'Throw' the Beginner :

If you are like me then you will have noticed ( and probably scratched your head over too) that the very

start of the SELECT statement seems to be where we choose our columns to be in the returned dataset.

Well, like many other things in life, there is a 'strange' twist to the story.

It is possible to include some extra 'bits' in our code, before our column specification, which actually affects

the row set returned. Odd isn't it ? just when you think you are understanding the structure of the 'thing'.

Limiting the Rows Returned :

You may have noticed that one context menu option in SSMS when a data table element from the OE pane

is right clicked, is to 'Select Top 1000 Rows' as shown below :-

=======================================================================

So running this query and then looking at the query pane and returned messages tab we see this :-

==========================================================================================

Notice that the 1000 figure (or value) is a limiting one, if there are not this number of rows then we

get the whole set.

Changing the TOP value to '6' we get this :-

=============================================================================================================

Let us now restrict the columns to only be 'ManagerID' as follows :-

=========================================================================================

Okay then, so we have six records, but what if there are a few others which also have the same value !?

This is when we need to use the 'WITH TIES' qualifier phrase, but this needs to have an ORDER BY clause

as you will see later on. lets see what happens now :-

=========================================================================================

Sure enough we get a total of eight rows returned, and that means there must be seven rows with a 'ManagerID' cell

value of 3.

Let us remove the 'Order By' clause to see what happens :-

================================================================================================================

And if we wish to have a percentage number of rows returned we can try this :-

..............................................

..............................................

========================

As you can see we get a total of eighteen.

And what if we wish to try the 'WITH Ties' qualifier phrase ?

................................................

................................................

========================

As you can see there are four more rows all with a value of 7.

Final Throw of the Dice :

Okay guys, I am now going to do something for fun - it was a question from Willie Moore at the first

'DevFest' session I ever attended !

I remember the question of how to get to the next five records after the first five. Well look at what we have

below and this may answer a few question along those same lines :-

================================================================================

In the image above we have got the whole set and from these we are looking at the first five.

However, in the image below we are getting the top five after sorting :-

=================================================================================

But if we use the top 5 as a set and restrict through the WHERE clause we can do this :-

==========================================================================================================

It does not take much imagination to be able to adapt this to find the next 10 top rows starting after 40.

See below for this solution :-

=============================================================================================================

Well guys, its an answer - or two - but I don't think I will get any prizes from Willie for a speedy solution ;O))

Using ROWCOUNT :

I was going to show you this BUT the Microsoft SDK documentation (BoL) suggests that we all stop using it

in favour of using TOP. So let us forget the idea.

Appearently ' SET ROWCOUNT 10; ' executed before our query above would limit the returned set to ten rows

no matter what we changed our values to (as long as they were for more than 10).

Only setting the system setting back using ' SET ROWCOUNT 0; ' would turn off the option. So it appears to be

a type of environment setting, and not part of the query or batch itself.

Yep! Dangerous I feel.

Unique Values with DISTINCT :

[ Eliminating duplicates with DISTINCT. ]

BoL has this to say :-

The output for statements involving DISTINCT depends on the collation of the column or expression on which

the DISTINCT is applied.

This seems to be a complicated way of saying that each of the columns specified for output is used to do a match

against those to be returned. If the new row being checked is not found in the existing set for return, it is marked for

inclusion, as well as also checking the subsequent rows.

No other columns are part of the duplicate 'check pattern', only those specified after the SELECT DISTINCT phrase.

*** NOTE *** Don't get caught out by wishing to have some extra information returned, and adding a column or two

to the one you wish to be unique. This will most likely cause duplication somewhere in the returned data set, but

most likely in the column you started out with as the only selected field :-) I know, I have been there !

Example 1 :

Let us look at a couple of simple SELECT statements to see the difference in the data returned :-

====================================================================

Yep! a total of only seven.

Let us add just another field, you know, just for a little more information ;-)

===================================================================

And as I warned we have a whole heap of other rows - simply because we are now looking at the pair

of fields, and the pair is matched against all the others.

If we are right then we could also predict that if we reduce our statement to only include the 'LastName'

column then the number of unique rows would fall - lets do this now :-

=================================================================

The approach to not getting more than you bargained for, is to do some thorough testing in SSMS before

you commit yourself to anything 'serious'. Either adding script lines to application code, or running CRUD

'stuff' against your clients database ;O))

See you in the next chapter.

PHH - July 24th. 2010.


As defined in 'BoL'

TOP clause / expression :

Specifies that only the first set of rows will be returned from the query result. The set of rows can be

either a number or a percent of the rows. The TOP expression can be used in SELECT, INSERT,

UPDATE, MERGE, and DELETE statements.

Some places in BoL it refers to TOP as being a clause and in others as being an expression.

However, it also has this to say :-

The select list can also contain the following keywords that control the final format of the result set :

- DISTINCT. For more information, see "Eliminating Duplicates with DISTINCT".

- TOP. For more information, see "Limiting Result Sets by Using TOP and PERCENT".

So above it refers to TOP and DISTINCT as keywords.

DISTINCT keyword :

The DISTINCT keyword eliminates duplicate rows from the results of a SELECT statement.

If DISTINCT is not specified, all rows are returned, including duplicates.

.....................................

For the DISTINCT keyword, null values are considered to be duplicates of each other.

When DISTINCT is included in a SELECT statement, only one NULL is returned in the results, regardless

of how many null values are encountered.

All - default :

For compatibility with the ISO standard and other implementations of Microsoft SQL Server, the ALL keyword

can explicitly ask for all rows. However, there is no need to specify ALL because it is the default.

The Select List :

The select list defines the columns in the result set of a SELECT statement. The select list is a series of

expressions separated by commas. Each expression defines a column in the result set. The columns in the

result set are in the same order as the sequence of expressions in the select list.

These ideas and statements above may help clear things up as far as terminology and verbal expression is

concerned.

PHH - July 27th. 2010.


Predicates

The dictionary says this about Predicate :-

Logic - something that is affirmed or denied about the subject of a proposition.

Then 'BoL' says this :-

Predicate - Is an expression that evaluates to TRUE, FALSE, or UNKNOWN.

Predicates are used in the search condition of WHERE clauses and HAVING clauses,

the join conditions of FROM clauses, and other constructs where a Boolean value is required.

[ In places in these eNotes I have used 'Boolean expression' when perhaps I should have used

the word 'predicate'. I will try to correct this as I go back through the text and remove typos and

the likes. ]

PHH - July 27th. 2010.


Subqueries, and Correlation

Think on this TRUISM :-

We can't live without subqueries if we are to do anything at all useful with SQL Server.

Subqueries can be of a type that is simple, and stand alone. That means that within the main query where the

subquery appears, there is no connection between the outer and inner queries. They are NOT correlated.

However, a subquery can be a 'Correlated Subquery' where there is some interrelation of expressions from the

outer and inner queries.

'BoL' has this to say about simple subqueries :-

Many queries can be evaluated by executing the subquery once and substituting the resulting value or values

into the WHERE clause of the outer query.

And this to say about Correlation :-

In queries that include a 'correlated subquery' (also known as a 'repeating subquery'), the subquery depends on

the outer query for its values. This means that the subquery is executed repeatedly, once for each row that

might be selected by the outer query.

Let us start be examining the first and 'simple' case.

Simple Subqueries :

A subquery is a query that is nested inside a SELECT, INSERT, UPDATE, or DELETE statement, or inside

another subquery.

A subquery can be used anywhere an expression is allowed.

A subquery is also called an inner query or inner select, while the statement containing a subquery is also

called an outer query or outer select.

[ *** Many Transact-SQL statements that include subqueries can be alternatively formulated as joins.

Other questions can be posed only with subqueries.

In Transact-SQL, there is usually no performance difference between a statement that includes a subquery

and a semantically equivalent version that does not.

However, in some cases where existence must be checked, a join yields better performance. Otherwise,

the nested query must be processed for each result of the outer query to ensure elimination of duplicates.

In such cases, a join approach would yield better results. *** ]

Paul Nielsen states in his training material that he tries to use JOIN whenever he can.

Let us now run a query to demonstrate this - see below :-

=================================================================================

In the query shown in the above image, we have effectively asked for all products to be listed where their price

is the same as the product with the name 'HL Mountain Frame - Silver, 42'.

This works because there is one price for the single item which is named and so the expression evaluates to

a scalar value. The subquery must return a value which 'makes sense' to the place in which it is used within

the main or outer query.

However, in the following 'adapted' form of the same query, we are finding the average price of all products

with a name starting with 'Road' and the listing only these. See below for the details :-

=================================================================================

I am not sure the example above makes a great deal of business 'sense', but it does to illustrate the idea of

a query within another query.

It also serves to illustrate another important point which will come in very useful when we move on to Correlated

subqueries - and that is, can the subquery be run separately, on its own, and work by returning meaningful data !?

If we use the last example to highlight and run the subquery we get the following :-

================================================================================

It would appear to be an independent query alright, and supplies a scalar value as the 'ListPrice' comparison in the

outer query.

We will return to this type of test later when we come onto the Correlated form of the subquery.

Correlated Subqueries :

The entry in my small dictionary for 'correlate' says this :-

>> - either of two interrelated things ... closely and naturally related ....to be in, or to into mutual relation. <<

'BoL' has this to say :-

Many queries can be evaluated by executing the subquery once and substituting the resulting value or values

into the WHERE clause of the outer query.

However, in queries that include a correlated subquery (also known as a repeating subquery), the subquery depends on

the outer query for its values.

This means that the subquery is executed repeatedly, once for each row that might be selected by the outer query.

Let us now take a look at an example based on the AW database, and one which has also been based on an

example from the 'BoL' documentation - I have made changes however, and have taken the example further also.

Let us look at this example in its simplest form - study the script below :-

====================================================================================

Notice that the inner query uses a value from the outer (or main) query, that of 'Ord.SalesOrderID'

We can prove that the inner query is dependent on the outer query in order to run correctly, by trying

to run the inner query on its own - by itself - see below :-

==========================================================================================

Remember this type of test, it will prove to be useful later.

And finally, taking this example a step further, here are two subqueries with the second one finding the

average unit price :-

========================================================================================

Note that although the subqueries are independent of each other, they are both dependent upon the

outer query to be able to be evaluated.

Example 2 - of another Correlated Subquery :

*** personal comment *** I find these 'BoL' examples a bit weak I have to say, but they will do for now!

Once again we will use the AW database and try to list out the full names of sales employees who have

a bonus value of exactly 5000.00 - see below for the working script :-

==============================================================================

Notice that the DISTINCT clause causes no duplicates to be output, each name will appear only once.

It may surprise you that an 'IN' operator is use and not an equals, but it is conceivably possible for one person

could have more than one bonus if the suitable data existed, hence the safety to test if 5000 is in the list of

values returned.

Lets finish this example by testing the inner query as seen below :-

==================================================================================

Sure enough, the inner query relies on the outer query, and can't be evaluated alone.

I think we ought to see if we can now come up with some better examples of own own, or from some other

place at least ;-)

See you in the next chapter.

PHH - July 27th. 2010.


INTO - very useful

PHH - August 25th. 2010.


making a copy of a table structure

Enter topic text here.


using with a 'Safety' database

================================================================================================================================================================================================

PHH - August 25th. 2010.


UPDATE and INSERT

INSERT - adding extra Rows to a database :

*** Note *** Although in the following subsection we will look at adding data rows to tables, and changing field / column / cell data

values as well, this is NOT the sort of thing I suggest you do 'regularly' on any live or important database. It is all too easy to harm and

damage the existing data, and tried and tested software 'screens' ought to be used - yes, GUI stuff.

So with the idea that safety of your data is firmly in YOUR hands, let us see how to script changes to the DB data.

First of all let us see a script to add about ten rows of new data to our table - see below :-

===========================================================================================================================================

You will have noticed that we have almost 50% of the code as IF statements, which do an IMPORTANT job of checking

to see if the new line of data already exists in the DB.

Basically the IFs stop multiple entry - more than a useful check to have in place I feel.

Now we can see in the next illustration below, what sort of rows appear in our table - check this out :-

===============================================

So now that we know how to add a few lines, let us now move on to focus on how to change them.

UPDATE - changing field / column data within a Row :

Once again, it has to be emphasised that doing data changes this way (on the fly) is NOT good practise, and

later in the section we will see what disasters can easily happen when we do things in this 'raw' manner.

First of all let us see an illustration of how a single line (row of data) can be changed - see below at line 10 :-

===================================================================

Yes, in the image shown above '111' and 'Doug Sanders' has been added to the row in question.

Now let us see how we can achieve this in T-SQL script - check this out below :-

============================================================================================

Notice in the script shown above that we have only one 'SET' command, even though we are changing

the values of two fields. Also the individual 'updates' are separated by commas.

So this one update statement makes only the changes on one line or row - as below :-

====================================================

If we wish to change them all then we need to repeat the process another none times, which is what the

IT Manager actually did.

CAUTION - possible catastrophe !!! :

Even if we get our script correct, it is possible to lose ALL our data in the table - YES, I said all :((

Before I demonstrate how easily this can happen I will first make a copy of the Table to another table name.

Below is the script to make a new Table called '_phh_tTest4Stephen', which then will have duplicate data items.

Checkout the scripting as it is useful to know, and use at times :-

================================================================

So the new table will have duplicate rows. So let us use this new table to see just what damage we can

cause by little slip-ups at this 'raw' level of doing things.

When I use Management Studio (MS) I frequently highlight sections of script before I click the 'Execute' button.

But as we all know it is easy to drag the mouse in such a way as to miss parts of what we were hoping to highlight.

In the following illustration we see that the user has missed off the last line from the highlighting process.

Study the details carefully :-

===============================================================================================

Oh! dear, in the image shown above we see the message say 2970 rows affected, so what has happened.

Well, if you are an optimist you will now say - all has not been lost, one of my rows is correct.

However, the pragmatist will say you have lost 2969 rows of good data - OUCH !!!

The following image shows some of the many duplicate field values :-

==============================================================

For this sort of reason, I suggest that you do not work 'on the fly' with T-SQL scripts like this in MS.

Use a much more controlled way to edit data and do dat entry - using GUI screens in dotNet applications etc..

If you MUST do some data cleansing then do as I describe elsewhere in these eNotes - use a 'SafetyWarehouse' database,

into which you copy the Table to be 'cleansed' before you make any lasting changes.

We do this at a local client site and all has gone well so far - but there is always a chance of errors !! So take CARE.

*** Aside *** Oh! I suspect you have realised by now that there is not a 'Ctrl Z' facility for scripting changes ;-0))

Time to move on, see you in the next subsection .........................

PHH - 23rd March 2012.


Chapter 2 - 'GROUP BY' - rows

Put quite simply, this clause is probably the most helpful to most people, most of the time ;-)

It can be used in a myriad of different scenarios to get us all very useful information.

Here is what BoL has to say :-

'Group By' - Groups a selected set of rows into a set of summary rows by the values of one or more columns or expressions.

One row is returned for each group. Aggregate functions in the SELECT clause <select> list provide information

about each group instead of individual rows.

Below we are going to explore with SSMS the simpler form of the 'ISO-Compliant' syntax. the more complex

syntax options are best left to third party tools - like Reporting Services (RS) - since they wrap up the complexity

and hide stuff with wizards and the like.

*** note *** this may be one of the most useful clauses, but it can also be one of the more complex ones to

apply in its fullest form - so tread carefully ;0))

Basically, the 'Group By' allows us to count the occurrences within a column, or field.

Let us take a real life situation - one which I have just completed for a client.

It is probably best to see the query statement and the results pane, before we discuss any further detail. So

study the image shown below :-

=======================================================================================================

Notice that the 'Group By' clause is using column 'acc_fds_accstatus' which is the account status.

[ ... in the Accounts table made by a third party software house called 'FDS'. ]

The results tell us that there are just four different (unique) entries in the status column and their totals can be

seen in the column headed as 'Count'. Obviously there are few 'Inactive' customer accounts, and vastly many

more 'Active' ones.

Well, when I say active that is part of our problem - we seem to have 'A' and 'Active' entries meaning the same thing -

and my clients wants to tidy this up as part of a 'data cleansing' exercise.

In another table - of Sales ledger for customers - they have another column which represents status, and

typically to large legacy databases, the field name is different, here it is 'analysis_code3'.

Here is a similar 'Group By' which applied to a single table - see below :-

==================================================================================================

In the image above you can see that we are filtering out any row with an entry of '99999999' in the column

'invoice_customer' - to them and their data this means ignore patient customers.

We can now go on to look at some apparently simpler, or straight forward examples - BUT - the above

example shows just what we as developers face when coming head to head with a real life DB system.

I can't change their table structures or their column names etc. I just have to make it all work ;O))

A Simpler Example :

Now that we have seen a real life example, let us look at one from the Adventure Works (light) database.

Study the image below, before we discuss any detail :-

.....................................................................

.....................................................................

=============================================================================

In the image above we are using the cut down version of the AW database. And the table we

are using is the 'Product' table.

We are therefore finding the full set of unique category IDs at the same time as counting them.

Now in this case finding the unique category entries is not too special, since you would expect them

to run from 1 to 'N' where 'N' is the largest and last entry. But there are other times when this is more

spectacular !

In the image below we will look at the 'Address' table of the AWLT database. We will group by the 'City'

column, study the results and the syntax of the statement :-

===================================================================

Remember that BoL says that we can only have the column (field) being used for grouping, and the aggregates of

other fields. One row will be output for each subgroup.

In the image above we see that we have most customers living in Toronto, then London, etc. etc.

This is probably in the form that will be most used by developers, except that quite often we will wish to join more

than one table.

PHH - July 21st. 2010.


ISO-Compliant syntax

PHH - July 22nd. 2010.


Group Filtering - the HAVING clause

In order to see the effect of the HAVING clause, lets first look at a simple example of grouping

using the full AW database. This is based on stuff in the BoL but I have made some changes

to suit our discussion.

========================================================================

In the image above we are using the SalesOrderDetail table to add together the line item values for

order items on the same order. So we are calculating order totals in effect.

I think I would have used 'OrderTotal' in place of 'SubTotal' !

We are only looking at the first few rows of some 31 thousand (plus) returned in the dataset.

Now let us look at our slightly 'adapted' syntax below, and what it produces :-

=========================================================================

In the image above we have run a query which has an extra line with a 'HAVING' clause included.

This new clause acts as a filter for the 'group by' output - much as the 'where' clause filters on the

standard 'row' output, or selection, from the 'FROM' clause.

So to control what is placed in the query 'return set' we use 'FROM ... and WHERE ....' as well as

'GROUP BY .... and HAVING ...'.

The 'Order By' clause is always last as it is applied to the returning dataset.

Don't forget that the alternative syntax shown below does exactly the same thing as that above :-

===========================================================================

As does the following also :-

===========================================================================

Placing the name of the output column (field name) in single quotes seems to make no difference

to the output results. Juts be aware of the possible syntax variation.

PHH - July 22nd. 2010.


Chapter 3 - denormalising with JOIN

Study this 'Truism' :-

"At first the JOIN can seem a threat to new guys coming to SQL - but in fact it is your friend ! "

Frequent use of JOIN is a basic necessity to use SQL correctly.

Study how to use JOIN, and understand all the detail, and your life will become MUCH easier.

Why JOIN ? - What is it all About ? :

Simply put, JOIN is required because of normalisation of the data within correctly designed SQL Tables.

It is much easier to maintain data when it is split up (normalised) into separate and different data tables,

then as some of us may have done in previous DBF days and make long records with many fields

which remained empty, or worse, where data was repeated across second and third tables or rows.

Let us take a simple example close to my heart ;-) Healthcare, and patient details.

If we have a record (or row) for a single patient, it is possible to put their details such as address and doctor's

surgery into one patient record.

But we have forgotten to take into account that a number of our patients will have the same surgery, and even

the same GP (doctor).

As well as this a few patients may need more than one address, for when they go to stay with relatives and

the likes. Even temporarily go into a nursing home.

Consequently, if done correctly in a normalised SQL database, we would be best to have a separate table for

Surgery details, one for Doctor details, and another for Address details. The Patient table would be more

of what we have become to know as a 'Header table.

So what of the Consequences ? :

Well, now that we have gotten our data nicely separated out, and we only have one place to visit to change

details for a Surgery, or Doctor, or a single place to add or change an Address, we need to think of what needs

to be done when we require full details on a single chosen patient.

In order to get the information on a chosen patient we need to acquire information from a number of separate

data tables - this is where the JOIN comes into its own. We JOIN the necessary tables using identity fields

which we designed into our database when we created it. So we would have a SurgeryID, and a DoctorID, as

well as an AddressID. Obviously we have a Patient ID as well.

The consequence of all of this is that we need to use JOIN to bring together all the various different information

on the patient, and use the IDs to point to the correct data rows which we will add together, or de-normalise.

The process of gathering together related information for one 'thing' is called de-normalisation, and is a MUST

if we are to gain the benefits that normalisation has to offer the developer, DBA, and ultimately the client.

The Necessary Evil !? :

Until you get confident with all forms of JOIN, and how to make them work for you, then you can see them

as a 'Necessary Evil' ;0))

Once you know how to script your queries so that you get all the rows you require, and not extra part duplicates

and the likes, you will see them as your friend indeed.

How did I make the Breakthrough ? :

Quite simple to answer, I bought Paul Nielsen's DVD video training box set from Total Training, at a very reasonable

price indeed - and I suggest you do too (see details in the Introduction).

I am an image guy as I am sure you realise by now, so it was these 'hand's images from Paul which did it

for me. If I show you the images, then you will have to buy his DVD set to get the wisdom of his words.

Guess what this is ? :-

Yep! This is an INNER JOIN ....... ;O))

And now this :-

Obviously, a LEFT OUTER JOIN ........ ;O))

And once again this :-

No prizes for this, the RIGHT OUTER JOIN ....... ;O))

And you can draw your own hand and finger image of a FULL JOIN, just make sure you have the 'free'

finger out for each hand !

The INNER JOIN is often scripted as simply JOIN.

Having watched his video training material (a few times) I prefer to use the full terminology as personally I feel

this and a few other 'shortcuts' are plain sloppy!

The INNER JOIN is covered in the next subsection, so see you there.

[ I fully recommend that you purchase the DVD training set - you will revisit it many times, and learn a huge

amount too. I am still using it almost two years down the line. ]

PHH - July 21st. 2010.


The INNER JOIN

'BoL' has this to say :-

Inner joins return rows only when there is at least one row from both tables that matches the join condition.

Inner joins ignore, and eliminate, the rows that do not match with any row from the other table.

[ INNER JOIN is sometime written as just JOIN - I will use 'INNER JOIN' in these notes. ]

Merging data :

The term 'merging' data is a general or 'loose' term, and simply means putting together data from separate

tables either to make longer rows with many more columns, or to make a longer pile of rows by stacking

one data set on top of another.

You will see the UNION in a later chapter, this is 'stacking' of data and the columns remain fixed, but the

number of rows increases.

In this subsection we will look in detail at the INNER JOIN where generally we will see extra columns being

added to an existing row, the extra columns come from the second table in the join.

Below is a 'non hand and finger' diagram / schematic of the INNER JOIN :-

A restricted set of rows from the first table is returned with extra columns from the second table - the rows

included are determined by the 'joining condition' - better known as the 'ON' phrase, or sub-clause.

Suitable Examples :

It is time for us to look at some suitable examples to see just how this comes together in our scripts.

We are going to use the AWLT database and the first two tables are 'Address' and 'CustomerAddress'

which are shown below in the OE pane :-

===========================================================================

In the image above we can see that both tables 'AddressID' columns with one being a primary key,

and the other being the foreign key.

First of all let us look at the returned results from a simple selection of all rows :-

====================================================

Now let us combine data from the two tables using an INNER JOIN - see below :-

=================================================================================================================================================================================

Immediately we can see that we have a lot more fields or columns.

In fact we have the same number of columns as the two tables combined minus one. The minus one

is due to the fact the 'AddressID' field is in both tables, but is included in the JOINed set one once.

Yes, one of the first things that you have learned is that JOINs make very large merged tables, and large

meaning WIDE in this context - as well as deep, depending on the data.

Don't worry about the size etc. - let the server do its own thing ;-)

Once your SSMS tests show that you have the correct INNER JOIN then restrict the selection of the columns

to suit your problem.

Looking at the image below, we see an all too familiar situation when JOINing, that of not correctly specifying

a required column - simply because the field name is in both tables being referenced - see below :-

====================================================================================

We can correct this simply by qualifying the 'AddressID' field with the 'CA' alias of the first table.

See below for what has been changed :-

=============================================================================================================================================

Although this works fine (and we see what we wanted to return), and SQL Server must workout a few

things for itself, it is a good practice I feel to always qualify the column name with the alias.

If nothing else it helps us as developers and scripters, particularly when we get to JOINing many tables,

which we must eventually do if we are working with a real life database.

Here is what I like to do with the qualified field names, however I usually drop the use of 'AS' when aliasing

the table names. It is shown here to let you see a working alternative - see below :-

=============================================================================================================================================

Also please realise that I have gotten used to quoting the full database name and schema, simply because

with my paid work I am frequently JOINing tables from different databases. It works, and it is very clear, so

I always do it as a matter of practice now.

More Rows than you Expected !? :

It is VERY easy to get totally focused on what you are trying to achieve with the JOIN and forget the possible

detail of you data structure !

By this I mean you seem to get three or four times the number of rows you thought you would get, and have

forgotten the 'rules' of the INNER JOIN' itself.

For every row in the first table which has a TRUE joining condition with any row in the second table, we get a

resultant row.

Let me explain this more fully, by considering a customer ordering system.

For each single order 'header' row (with a unique OrderID value) we are likely to have more than one row in the

order 'detail' table. So when we JOIN on the condition that "O.OrderID = OD.OrderID" we will get in our returned

data set a row for each of the order detailed rows, and each of these will have common field data from the orders

table.

This is just the consequence of de-normalisation, don't worry about it, the server will cope - BUT - don't get

caught out by the multiplicity. There are times when in fact you need to 'beef up' the joining condition simply

because you don't wish to have more than one row. This will be up to you to choose a pair of fields (or more)

which when compared and placed together with an AND will point to only one JOIN.

Many good examples, and a chunk of experience gets us around these pitfalls ;-)

Example 2 :

Staying with our previous example, we can JOIN a third table which will give us the extra information about

the customer's name and salutation.

See the image below, and notice that we now have some column aliases too :-

============================================================================================================================================================================================

We still have 450 rows however, so we are not being caught by ill thought out joining conditions.

Example 3 :

For this example where we will de-normalise a number of rows from the row in focus in the first table,

we can use the AWLT database once again. This time we will concentrate on the orders and the item

lines belonging to each order.

There will be an order header row, and then at least one related row in the order details tables, to match

up with the identifier fields, called 'SalesOrderID'.

First of all lets take a look at the tables and their column specs in the OE pane :-

==========================================================================

In the image above the only common key field is the 'SalesOrderID'.

Let us use this field to make a 'joining condition' from the header table to the details table.

When we have got a successful JOIN we can examine the rows to see how many we have. First

of all we need to see just how many rows are within the header table - see below :-

===================================================================================

As you can see we have 32 rows of header information.

If I now make the first attempt to do an INNER JOIN and retain all the possible columns with their

original names, we get the following - a surprising number of rows indeed ! :-

[ *** note *** we could use the syntax 'select H.*, D.* ' to chose the columns - more in next section ]

===========================================================================================

Yes, in the image above we see multiple rows from each sales order, except 71917, and there may be

others - BUT - the majority have many, and some quite a lot !

In fact we are in a position to use our SQL knowledge to help us out, lets try a 'Group By' to see just

where all the 542 items lines have come from.

Study the script below to see that we indeed have 32 header rows responsible, but six of them alone

account for 300 rows in the final dataset returned :-

========================================================================================

So to finish this off, lets try and restrict the columns to something we may use in a simple report.

Study the script below to see what we have :-

============================================================================================================

Okay then you guys, here is a test for you. Modify the above query so that you get only orders which have

included in them item lines with a single quantity ordered.

*** Hint *** If you start off by getting 128 then you are on your way, but notice some orders have more than one

line which is single. Amend your code to get an answer of 29 ;-)

If you are really moving ahead you may be able to total the value of the single item lines for the whole dataset. This

means not just the one single line per order number. It appears to be $50,176.25 or £ or Euros ....

Good Luck ! - see you in the next section.

PHH - July 26th. 2010.


Chapter 4 - using OUTER JOIN

'BoL' has this to say :-

Outer joins return all rows from at least one of the tables or views mentioned in the FROM clause,

as long as those rows meet any WHERE or HAVING search conditions.

All rows are retrieved from the left table referenced with a left outer join, and all rows from the right table

referenced in a right outer join.

All rows from both tables are returned in a full outer join.

At first 'sight', or 'thought', or first "whatever", you may feel after the last section that you don't need any

other sorts of JOINs in order to solve your business problems. I understand if you do feel like that, however,

there are a number of 'classic' types of problem which are VERY nicely solved by Left / Right

and 'Full' JOINs - so read on ;0)

SQL Server uses the following ISO keywords for outer joins specified in a FROM clause :

- LEFT OUTER JOIN or LEFT JOIN

- RIGHT OUTER JOIN or RIGHT JOIN

- FULL OUTER JOIN or FULL JOIN

If we are to cover the outer join fully in our explanation we need to explain what the LEFT OUTER JOIN (LOJ)

is and also what the RIGHT OUTER JOIN (ROJ) is, as well as how these are related to each other.

Unfortunately in my view the LEFT OUTER JOIN can be referred to as 'OUTER JOIN' - this is 'sloppy' I feel

and we all benefit from specifying fully and correctly which JOIN we are using. After all we are only saving

the typing of a couple of characters - so why even think of being 'sloppy' !?

LEFT OUTER JOIN (LOJ) :

If we refer to the schematic diagram below we see we have all the rows from the first table, that on the

left, and any data from the second table on the right where there is TRUE joining condition. As with the

previous INNER JOIN, multiple rows can be created if more than one row in the right hand table gives a

TRUE evaluation for the joining condition.

If there is a FALSE evaluation for the row in focus, for the joining condition applied to each and every row

in the second table, then the extra column values from that second table (added to the first table's set

of columns) are all set to NULL.

Relating the LEFT and RIGHT OUTER JOIN :

It does not take a 'super brain' to realise that if we had the above diagram on a transparent sheet of material,

and then viewed it from the opposite side, then left becomes right, even though set 'A' is still set 'A'.

The good news is that this is not complicated, and many gurus would suggest (as Paul Nielsen does in fact)

that you can get by with only ever using the LEFT 'flavour', most of the time.

I figure that since the relation becomes easy to understand, the more we think about it, then we ought to

be able to apply the RIGHT OUTER JOIN when it makes our scripting and life a bit easier. And I have come

across a situation exactly like this in even my limited experience ;-)

So here it is :-

"Table A LEFT OUTER JOIN Table B" is exactly equivalent to "Table B RIGHT OUTER JOIN Table A".

This may not look or sound any BIG deal, but, there are cases and scenarios in which it is not easy

to change the script so that we put table B first (before A) so we can use LEFT in place of RIGHT.

*** Careful ! *** Am I being lazy ? Paul Nielsen said you can always design your queries to avoid ROJ and

simply use LOJ.

See the notes in the following appropriate subsection, for a good example of this.

A Good Example of a LOJ :

We will use the AWLT database to look at the business 'problem' of finding all the customers who do not have

an order in the sales order tables. We don't need to bother with a date range for this simple database, but that

would probably be the case in a real life scenario. "Who has not made a purchase in the last three months ?".

Obviously, we will need to involve the two tables 'Customer' and 'SalesOrderHeader', and on top of that we need

to include in our returned dataset ALL customers whether they have made an order or not. We can then filter

them accordingly, once we have our dataset.

If we think about the problem for a moment, then if a purchase has been made, there will be a row somewhere

in the 'SOH' table which will point to the customer in focus. If no purchase has been made then no row will

match up on the 'CustomerID' unique reference. If we use a LOJ to join the tables then fields for the sales order

will be filled with NULL entries.

After we have made the dataset to be returned, all rows with NULL values in the SalesOrderID field will designate

a customer not making a purchase.

Let us now see the OE entry for the relevant parts of the two tables :-

===============================================================

===============================================================

Notice in the images above that the 'CustomerID' is a key field in both the tables and can therefore

be used in a 'join condition'. Since there will be single entries in the Customer table for all keys

(we would hope!) then we will get no multiple rows being created.

Don't Worry about Numbers of Columns :

Before we see the script to start to solve our non-purchasers problem, I need to inform you that you

need not concern yourself with the numbers of columns being added into the rows to be returned in

the dataset - let the clever SQL Server 'engine' worry about those things, it is VERY good at sorting

itself out ! Think modern, don't mess !!!

Okay then, let us see our first LOJ in action - study the script below :-

===========================================================================================================

========================================================================================================================================

In the images above notice that the syntax of " select C.*, S.* " means we get a double entry for 'CustomerID'.

However, the two 'split' images of the returned columns do show that all do appear to be present.

In the image of the script above, you will see that we had included in it a WHERE clause which filters out

rows with NULL values for SalesOrderID - Yep! you are going to say we appear to be throwing away the very

rows we were after - and you would be right ;-)

So our script above actually finds the list of customers who made orders. Do be aware however, that if some

made two or more orders then they would appear in more than one row. So we would need to find distinct rows.

First of all lets see what we get as a total row count for the above code :-

=================================================================================================

So our first question should be, are these all single row entries, or do we have some duplicates ?

It would appear from a cursory look down the list as though they are all singles - BUT - if we had a lot more

than 32 rows we would need another way of finding out - so lets include the DISTINCT predicate into our

SELECT statement.

Study the image below :-

================================================================================================

It would appear that we were right - 32 single entry rows, one purchase per customer.

Back to the Non Purchasers :

Fine, we seem to be doing Okay with our LOJ set, now we need to filter out the customers making any purchase,

and set our filter condition in our WHERE clause such that it includes only NULL values for SalesOrderID.

Note with this business problem we don't need to worry about multiple entries, as we would hope that any

one customer is only included once in the Customer table - else we have our DB designed wrongly ;O))

In the script below we are excluding purchases :-

================================================================================================================

Notice all the NULL values for the columns coming from the second table - that on the right.

Making a few minor adjustments we get this below :-

================================================================================================

===============================================================

So we have 408 customers having not made a purchase.

Remember when you create this sort of solution in real life you will need to use a data range as well as

what we have done above.

This is a good example of the sort of script that may get rolled up into a function (UDF) and we may revisit

this problem again in Chapter 10.

Now onto another good example, using a common problem.

Example 2 - Products not 'Moving' ! :

Moving on from focusing on customers, let us now focus on products.

Let us try to answer the business questions of which product items are not moving off the shelf ? What is

not selling ?

And then the question, which are our best sellers, in order of best first ?

The two Tables we need are 'Product' and 'SalesOrderDetail' - both once again from the AWLT database.

Here are the table column details needed for joining :-

===================================================

=========================================================

From the images shown above we can see that 'ProductID' can be used to make a join condition.

Since we are focusing on the products and using the information in the 'detail' table to tell us about our products,

we need to do as we did previously - left and right tables with a LOJ - so study the code carefully :-

The first thing you will notice in the following image is that we definitely do get multiple rows for most products :-

=================================================================================================

Now when we include the DISTINCT predicate and re-run we get this :-

=================================================================================================

And our rows fall from 695 to down to 295 in total - which is in fact the total number of products ;-)

Now in our image shown below we are returning to all the columns available, but are switching the order in the columns selection.

Note we are also filtering via the WHERE clause so that we get just products without a sales order.

================================================================================================================================

If you are following along with the scripting in SSMS you will see that 153 rows are returned by this query.

This therefore means that of the 295 total products 153 have not sold any in the sales details we have to hand.

So lets us check what we get when we reverse the filter logic :-

==================================================================================================

But we know that many products appear in more tha one sales order, so we need to use DISTINCT once again

to reduce the returned dataset :-

==================================================================================================

================================================

This looks right, 142 plus 153 make 295 on my fingers ;0)

So all we need do now is group these to see what we have as totals - BUT - wait ! Surely we can solve this

problem a lot easier since we have all the sales information on product in the 'detail' table :-

===================================================================================

So the moral of this is don't get carried away with joining, think about your data and your table structure.

Only make a JOIN if it is necessary.

Finally, if we further wish to know which products sold simply a single item then we can filter on the group

summary with our HAVING clause - like so :-

=====================================================================================

Notice that we can't use the alias of 'Total' in the HAVING clause as this only applies the the final

dataset being returned.

Enough of the useful LOJ for now, lets move on.

See you in the next subsection, when we get to turn things around !!

PHH - July 21st. 2010.


the RIGHT OUTER JOIN

Some experienced and 'quality' training guys would say that you can always write your script so

that you use a LEFT OUTER JOIN in place of a RIGHT one. It is just a matter of getting your table

data sets in the correct order - one before the other.

However, there may be times when you have existing and tested script to which you just need to JOIN

another table, and you want this to be after your existing script, and not before - we will see this later.

First of all, we had better prove that we can use LEFT in place of RIGHT and get the same results.

Writing LEFT as RIGHT and vice versa :

Question to answer - is 'A' LEFT OUTER JOIN 'B' the same as 'B' RIGHT OUTER JOIN 'A' ?

or writing this in its shorter form, is 'A' LEFT JOIN 'B' the same as 'B' RIGHT JOIN 'A' ?

We will use the AdventureWorksLT database and start with the sales orders table and join with the customer

table, much as we did in the last section - however, we will only filter those rows not matching, those with a

NULL value for SalesOrderID - see below :-

[ SORRY ! - in the examples below I use SOD as an alias for 'SalesOrderHeader' !! please live with it !!! ]

=====================================================================================================

==============================================================

So this returned dataset to our query means that 408 of out customers did not place an order.

Notice the syntax for all fields of table alias 'C' followed by all columns of table alias 'SOD'.

Now lets see if we can get the same results by turning around the query and using RIGHT instead.

Study the script below and the results :-

======================================================================================

Sure enough, the new code still gives us our 408 returned rows.

Also note that although I turned just about everything around left to right, it was not necessary to change

the join condition, so I have put it back as it was originally - see below :-

=======================================================================================

There is nothing too complex or complicated here I feel, just need a minute or two to think about what

we have done, and which table (or data) set it is which will provide ALL of the rows.

Let us now move on to see where we may use RIGHT JOIN instead of turning the code around.

Using RIGHT OUTER JOIN :

Okay then, lets try something similar, but different.

Let us do and INNER JOIN of the sales order header table to the detail table so that we get all order

lines for each and every product line sold in the period.

Here is the script :-

=================================================================================================

But what if we now wish to have a list of all products which were not used to make up the detail on any

of the orders in the period we are examining ? We need to join in such a way as to get ALL rows from the

extra table we ill use, the Product table.

To my mind it is easy to do a RIGHT JOIN instead of changing our tried and tested script shown above.

So here is my version of the list of 'not used' products in the period :-

===========================================================================================

Now I know that some of you will say that it is just as easy to change the lines around - BUT - my

point is that I have already built up and tested the other JOINed set (and usually this can be at least

three or four JOINs), so adding to the end of the list feels better to me, especially when I know what

Paul Nielsen's fingers on his right hand mean in the photo !

So what is the BIG deal ? For me I will use RIGHT OUTER JOIN when it is easiest, but start from

LEFT when I am starting to solve a problem from scratch.

To be honest, if we other features of TSQL like Common Table Expressions then life is easy anyhow.

The Truth ! :

I have just made a genuine effort to change the above script around to use LEFT JOIN in place of the

working RIGHT JOIN - and found that it was not as easy as I though because I could not just move

my product table to the front of the FROM statement. So after a bit of a re-jiggle I ended up with

this below :-

[ ... this is WRONG !! ... ]

===================================================================================================

============================================================

And after some more messing around and changing and testing I get this :-

=======================================================================================================================

So, although I will win in the end and get a working solution, I found it much easier to use RIGHT JOIN

in the very first place.

I will keep going, just as a learning exercise ;-)

First a fresh cup of tea !!

Okay guys, I have an answer, and it is quite simple really, so we have learned something. Before we go

into any details, here is the working script :-

======================================================================================================

In the image above it would appear that although the subquery can be highlighted and run on its own etc. the

specification of the columns needs to be done with care. In doing the OUTER JOIN the Server does not like

the situation when there are two columns in the table set with the same name.

If we comment out all the columns from the Header table its works OK, and gives 153 rows.

Finally - Job Done :

I think I have sorted it - changing the order can be a bit like the LOGIC lines for Boolean, the name of the

'operation' needs to change too.

Study the script below to see that we get the required answer as long as we have two LEFT JOINs. This

does make sense BUT we have changed what we had been building up and testing :(

====================================================================================================

Summary :

Enough messing for me guys, I will use RIGHT JOIN when in suits me ;O)

I think no more should be said about this now except don't be afraid to understand and apply both LEFT and

RIGHT forms of the (OUTER) JOIN when your script requires it. I don't know why the gurus suggested otherwise

in the first place.

See you in the next subsection.

PHH - August 1st. 2010


the Full JOIN

For a FULL JOIN we have no problems with which way round we have our tables, it does not

matter as the results are the same - 'A' FULL JOIN 'B' is the same as 'B' FULL JOIN 'A'.

In one sense we can think of a FULL JOIN being both a LEFT JOIN and a RIGHT JOIN both

happening at the same time.

We get all the rows of 'A' and all the rows of 'B' no matter whether there is any match on the

joining condition we supply. Also the rows that do satisfy the join condition give the number

of rows determined by number which satisfy the condition, and that is both 'A' to 'B' as well

as 'B' to 'A'.

Trial and Testing :

Although it is easy enough to script an example of this, it is generally harder to test and show

results, as quite often we can't change our sample data because of existing constraints. More

on this a little later ..... first some results :-

Let us show an example with the Product and 'SalesOrderDetail' tables in AWLT.

==================================================================================================================================

In the image above we have created a results set from a FULL JOIN of the two tables.

The results shown are the only three for product identity values present anywhere in the joined

dataset where there is not a matching ID in the product table.

Note I had to force this by editing the 'ProductID' values in an existing row, after turning off the foreign

key constraint in the relationships dialog for the 'SalesOrderDetail' table as shown below :-

===========================================================================================================================

I don't recommend you do such things as this ;-)

But it allowed me to change three fields to 13,26, and 39 as you will see in our results.

So what of the other part of the FULL JOIN dataset ? Well, as you can see below, we have a bunch

of 154 rows where there is no sales detail for the product in the original table, so NULLs have been added.

We have seen this previously in our LEFT and RIGHT exercises and examples. So now we have one table

of results with rows which are fully populated with data in each field, ones which have NULLs in the product

fields, and ones which have NULLs in the sales detail fields.

No, we don't have rows with all NULLs ;O))

Recommendation :

Don't you screw your database by doing what I did above - find a better example if you can, or just take in

the fact that this is possible, when you have suitable data on which to carry it out !

See you in the next subsection.

PHH - August 1st. 2010.


the Cross JOIN

This is rather academic, apparently, unless you know what you are about and you have a definite need

for it.

I recommend you look at Paul Nielsen's video 'stuff' to see how he has used it.

This is hardly likely to be needed by you in weeks and months (years?) to come - so don't worry !

See you in the next, and VERY important subsection, on SELF JOINs.

PHH - July 26th. 2010.


Chapter 5 - self JOIN

This is really a 'special case' of the INNER JOIN, but it does help us solve a classic

type of problem - related to the inheritance tree structure. This is commonly used in

manufacturing of products, as well as personal inheritance scenarios.

Basically the 'self' JOIN is when the same table is used twice - as both the Left and

the Right table in our schematics.

Below we will look at an example from 'BoL' as it uses the AW main database, and the

example is quite a good one, and one which is likely to be able to suggest to us other

similar scenarios which may help our work.

Study the script carefully :-

==================================================================================

Looking Ahead - to Solve an Immediate Problem :

By adapting the script we have developed we can get a list of unique products as follows :-

=================================================================================

====================================================

As you ca see we have 152 rows returned each with a unique 'ProductID'.

BUT - if we now wish to simply count the rows in our our returned set, to produce a scalar value, and

return that, how do we do it ?

After a couple of failures I wished to have a clean and tidy way to script the solution, and came up with

the following - which is both easy to read and follow, and it works nice too ;-)

Study the script in the image below :-

===========================================================================================================

Lets work backwards, the answer is correct !

The select statement which runs and uses the CTE is very small and easy to read. Obviously 'MultProd' acts

as a table would.

The CTE is easy to script and is basically the body of our JOIN statement which we developed and tested above.

Look out for more on CTEs in chapter 12, they could prove very useful.

Other SELF JOIN problem areas : a Classic !! :

In Paul Nielsen's video training he describes a good example of when and how to use the SELF JOIN.

Unfortunately, I have not got his SQL database to demonstrate, but we will see what can be done to rectify

this later.

Describing the necessary tables :

There is a main table called 'Person' in which a few details of each individual is held, including their names.

There is then a second table called 'Family' in which each person has a record and within the record is one

field for 'mother' and another one for 'father'. Held within these two columns are 'PersonID' values which (if

known) refer back to the 'person' table. If the father or mother is not known then a NULL is held instead of

an existing ID.

Each person can have only one mother and one father, so only two columns are required. However, one

mother can have many children, as indeed can one father.

The interesting questions posed of such a database include those like list all mothers along with their children.

And list the sets of twins. Also whose are full siblings and who are half siblings - or if you want to be sexist, who

are half brothers and who are full sisters etc. etc. These sorts of questions abound, and we have not gotten to

aunts and uncles, as well as cousins ! ;-)

Or even, who are grandmothers and who are grandfathers ?

The 'Self Join' will serve us well in answering these questions. I think I have talked myself into making my own

database, as I can't find the one Paul shows in his DVD course, anywhere on his website or even any other

out there after a short search.

The exercise will do me good ;-)

Okay then here is the first output from our new and growing family tree - see below :-

[ for now just notice that we have joined the same table together three times - well 3 tables and 2 joins ! ]

==================================================================================================================================================================

Okay this stuff can be very confusing if we do not label the columns correctly, so I better do that now,

as well as restrict the column output to only meaningful stuff.

However, I can read the second row to say that person with an ID of 2 was born in Newport, their mother

was born in Durham, and their father in Sunderland.

Okay then guys, I feel as though I am making progress, see this for an improvement :-

=================================================================================================================================================

Well, if nothing else it does read a bit easier. Problem is these sorts of problems are a little tricky

to get your heads around - its not the SQL stuff.

From Parents to Siblings :

Having pulled together information of both parents, it is time to change around our focus and look at

brothers and sisters, or as we will know the whole process - by siblings.

Before we get out groups of fancier names etc., let us just see if we can write a query which finds in one

single row a pair of siblings, and do this for all persons with a sibling in the database.

Study the code below to see how we can once again do a self join to solve a problem which would be otherwise

difficult to solve :-

=============================================================================================================

It is probably too early in these notes to go much further but we can see if we can get out some friendlier

names rather than 'PersonID's. It is also probably best to order by the Mother ID so we get the pairs of

siblings grouped together :-

======================================================

And after a bit of fiddling about we get this :-

======================================================================================================================

This is looking better, and when we do more TSQL we can make sure we get rid of the duplicates ......

Oh! that seems easy, we just change our WHERE clause to 'P.PersonID < S.PersonID" - we will

incorporate this in our further development stuff in Chapter 26.

In the section on UDFs we could always design and write a function to find all the children to one

specified mother, and then work back from there - but first things first, at least we are on our way

and making progress ;O))

See you in the next chapter, when we will look at merging data with the UNION operator, and more !

[ When you have learned more good stuff go to Chapter 26 to see more developments with the Family Tree. ]

PHH - July 27th. 2010.


the Family and Person tables

In order to have suitable data on which to do some 'SELF JOIN's I had to make a couple

of new tables. I added these to the AWLT database for convenience - my convenience ;-)

Here is the structure and some contents of both the 'Person' and 'Family' tables.

=======================================================================================================

==========================================================================

==================================================================================================================

==================================================================

Although this is far from complete, at least we have samples of twins and siblings of three

at least a few times. And on top of that we go back more than a century.

If I am to improve on this it will be to record the family name of all females, their surname / last name

at the time they were born - in the UK we call this the 'maiden' name.

We shall have to see ;0)

When you have read further, and learned a lot more, revisit the Family Tree problem in Chapter 26. Here

you will find some useful, and instructive functions.

PHH - August 5th. 2010.


Chapter 6 - the UNION operator

Let us start by seeing what 'BoL' has to say :-

Combining Results Sets with UNION :

The UNION operator allows you to combine the results of two or more SELECT statements into a single result set.

The result sets that are combined by using UNION must all have the same structure. They must have the same

number of columns, and the corresponding result set columns must have compatible data types.

By default, the UNION operator removes duplicate rows from the result set.

If you use UNION ALL, all rows are included in the results and duplicates are not removed.

The result set column names of a UNION are the same as the column names in the result set of the first SELECT

statement in the UNION. The result set column names of the other SELECT statements are ignored.

- The number and the order of the columns must be the same in all queries.

- The data types must be compatible.

A Simple Explanation :

The operators UNION, EXCEPT and INTERSECT are all similar in that they operate on datasets which are

structurally similar, or even identical. So we are dealing with bunches of rows of potentially the same table.

So UNION adds the two bunches together, either uniquely or the whole of each bunches.

[ Unique means all columns identical - one row to another. ]

Then EXCEPT makes a return set which is all the rows which are NOT common to both.

The INTERSECT operator returns a bunch of rows which ARE common to both sets.

'BoL' has this to say on EXCEPT and INTERSECT :

Returns distinct values by comparing the results of two queries.

EXCEPT returns any distinct values from the left query that are not also found on the right query.

INTERSECT returns any distinct values that are returned by both the query on the left and right sides of

the INTERSECT operand.

Time for some suitable examples I feel.

See you in the example subsections which follow.

PHH - July 27th. 2010.


UNION example

Okay you guys, lets get straight to work and make a working example just to see how things

work with UNION, EXCEPT and INTERSECT.

If we specify a good example here and use it to explain the UNION operator, then we can do

something similar to explain the the other two.

Let us use the AdventureWorks full database as it has a spread of dates for the sales orders.

If we use the 'SalesOrderHeader' table we can make ourselves some script which returns a dataset

with order in the date range supplied - see below :-

==================================================================================

Setting Our Task :

We will find ourselves two sets of product items, one for each of two months, say January and March of

the year 2003.

We are then going to pose a few questions like :-

- what is the unique list of items which were sold in either of the specified months ?

- what is the list of items where each product was sold in both of the separate months ?

- which products were only sold in either the first of the second month, but NOT in both ?

Let us keep to these three for now.

So far we have found 'SalesOrderID' values for the date range, and now need to join with the 'SalesOrderDetail'

table so that we can see products. See below :-

======================================================================================

So if we use this joined table to provide two similar structured datasets we can then merge them with

the UNION operator as shown below :-

========================================================================================

===========================

As you can see from the above images, we seem to have unique product value entries and a total of 83 of them.

Let us make a small change and write 'UNION ALL' in our script - what does this do ?

See for yourself :-

========================================================================================

Wow ! we get thousands more rows.

And, it isn't until we get to row 26 that we get onto the second product, that with 'ProductID' = 708

see below :-

===============================

So if we use UNION without a qualifier then we only get unique entries - set 'A' and set 'B' are merged together

but with only one similar row entry - no duplicates. This is applied to the whole set of columns, even though in

this example we have only got a single column.

Using Multiple Columns :

If we change the above simple example to something a little more complex, we can see the effect of using two

columns. We will also include in our returned sets for merging, the territory identity, as seen below :-

========================================================================================

This time when we comment out the 'ALL' we get single row entries for a combination of both the

field values.

====================================================

As you can see we now have a total of 351 rows.

Taking things further :

Don't forget we can keep using the same UNION operator over and over again, merging a new dataset

with what has been merged previously - see below for a simple example :-

=======================================================================================

What does this tell us ? Well that the difference between 83 and 99 extra 'unique' products were sold

in the month of May. Or to put it another way, a list of 99 products which were sold in either of the three

months specified.

======================================================================

Moving on to the next subsection we will see the effect of the EXCEPT and INTERSECT operators.

See you there.

PHH - August 1st. 2010.


EXCEPT and INTERSECT

The slight problem with writing SQL learning notes is that we need suitable data to be present in our

test / demo databases before it can be used to show the various points we are trying to highlight.

I tried the previous section 'simple' example with one column, but the data did not make it a suitable

example - we got 83 in each month it would appear - so lets go back to our double column example.

Study the script and image shown below :-

=======================================================================================

=================================

This results set means that 72 different rows of ProductID and TerritoryID combinations are in one or

the other of the merged sets, but not both.

What of the INTERSECT operator ? :

What about the set which is common to both, can we find that with INTERSECT ? Let's see shall we :-

=======================================================================================

=============================

Multiple Operations - EXCEPT and INTERSECT :

Just as with the UNION operator, we can have more than two sets being merged, one after the other.

So let us see what we get when we first EXCEPT the three months sales by territory, and the INTESECT them.

See below for the results :-

======================================================================================

So we have 72 drop to as low as 5 rows. However, I do realise that we are probably losing a mental 'grip'

on what it is we are doing ;-)

This is the reason a good example is the best way to learn !

Still, to finish off our test session, let do the INTERSECT with the three datasets :-

======================================================================================

And we have 204 drop to 198 - once again the meaning of this is rather lost on me, even though

I can make the script work with these new operators.

Overall Summary :

What I can say from my limited experience, is that when you come across a good place to use the

UNION operator, it does make life so much easier ;-)

It can mean that instead of having an overly complicated WHERE clause, we can simplify and make two

or more simpler filter conditions and merge the results with a UNION - or two. Much easier.

As yet, I have not had occasion to use EXCEPT or INTERSECT, but there will be a time, and at least

by doing these test examples I am ready and prepared for the time it does happen.

Remember, however, the datasets MUST be similar, in column number, order, and data type.

Now time to move on to the next section - see you there.

PHH - August 1st. 2010.


Chapter 7 - Hiding the JOINs - 'views'.

*** Remember - a 'View' is a stored query. ***

So any JOIN complexity that can be included in a query can be encapsulated in a 'View'.

A 'view' is much easier to use elsewhere than the full original query.

Keep you 'views' general, so that their use can be made specific with filters via the WHERE clause.

==========================================================================

Many of us are application programmers and developers, and as such we are used to 'views' being

a way of seeing data in different ways - usually with respect to a data grid control in GUI.

Different views of the underlying data can have filter and sorts applied so we see what we wish to

see, in the order that we require. Well 'views' in SQL can bee seen in just a similar way - they hide

the underlying complexity of how the data is assembled.

I will show below an example from a real life client situation in which a number of JOINs are encapsulated

in a view. This view is then used in related functions and then one of the functions in a dataset for a report.

Guys who like modular programming will also like this approach. Each block of script which is rolled up into

a view can be thoroughly tested and checked out, so that subsequent use is more reliable and robust.

If you are unfamiliar with 'VIEWs' then it may be worth you first having a brief look at the following

two chapters, 8 and 9.

Below is a real example of a VIEW I use in a report situation - study the script and see we have five tables

combined with four INNER JOINs :-

=========================================================================================================================================================================================

In the image above, I produced the return data set by highlighting the SELECT statement from within the

view script and clicking the 'Execute' icon.

Once we have the script saved as a view, we can produce a returned dataset with a much simpler script.

See the image below for the 'simple' way to do this :-

=====================================================================================================

So anywhere we need to use the returned dataset we can use this simpler script and SELECT statement.

Look at my real example below, where we see a UDF (User Defined Function) based on this same simplified

script, when we need the full returned dataset we had above.

Notice that it makes the script much more readable, and hence easier to maintain in the long run :-

====================================================================================================================================================================================================================

And to complete the 'encapsulation' we can see below the script in a dataset within the Reporting Services 'shell' of BIDS

and the UDF is used to supply a return dataset based on the two date range parameters and the underlying view within the

UDF.

Once again notice how much more readable this is than when having all the script of JOINs and more placed together :-

==================================================================================================

Notice that in the image above the 'Order By' clause comes after the view and UDF. We can't put ORDER BY within either

of these unless it is the special case of determining which rows are TOP etc. - and then it does not operate in the way a

standard 'Order By' clause does anyhow.

So finally to the report itself. Now we have go so far with the underlying scripting of the report, we may as well see the

results, even though this is from within the designer environment itself.

In the image below notice the input parameter controls at the top of the Preview pane :-

=======================================================================================================================

In the image above the From / To dates are passed into the UDF when the report is run, and this determines the returned

dataset.

Now we ought to move on to the next subsection where we need to consider the consequences of using views in this way.

If a report is to be based on an existing view and UDF then we need to manage the situation !

See you in the next subsection.

PHH - July 21st. 2010.


Managing your 'views'

The Ups and the Downs ! :

Okay you guys, lets take stock of things !

It is great to be able to write modular script via Views and UDFs, as well as Stored procedures - BUT -

there is a possible downside to this approach.

If for example I am designing a report which uses a View (or two) as well as a dependent UDF as well, then

I need to make sure that these objects do not get removed from the database otherwise the report will no

longer function. It will be an 'orphaned' report :(

When we design our DB and reporting system so that it is nicely modular in structure, it is then also our

responsibility to have a process (or system) to make sure we manage our modules, and don't remove or

damage any so that other parts of out DB system cease to function.

There are a number of things we can do to 'manage' the situation, some are small and some more obvious.

Making a comment in Script :

One easy thing at design stage is to include a comment within the View and UDF to say which report is to

use the object.

Keeping track of things in a Table :

Is this necessary - or are there tools to help ?

Surely any Report is designer outside of the DB itself and so it will be up to the developer to 'manage' !

We need to think out a useful structure for keeping tabs on what is required by what !

Helping ourselves - What NOT to Do :

One simple thing which will certainly make a significant contribution to help us in our management task, is

not cluttering up out folders in out DB schemas with lots (or any) trivial and transient objects.

The training material I have watched clearly advises against making one-off Views and UDFs to solve short

term problems. We are advised to make each and every such object a general one which will be applicable

for a while ahead, to get short term answers to problems use SELECT in the SSMS shell.

By not making short term objects, we don't then need to manage them - just supervise them !

Or to put this in 'man-in-the-street' parlance - " if you don't make a mess, you won't have to clear it up ! " ;-)

Having a Test and Demo Database :

It can't be stressed enough how useful having a demo / test / development DB is for keeping the main one

free from such possible debris.

Use the "View Dependencies" option in SSMS :

At a number of places throughout these notes I have referred readers to SSMS and the facility found there

for users to right click all User-Defined database objects to bring up the context menu with the 'View Dependencies'

option. This allows us to see

PHH - July 29th. 2010.


Chapter 8 - Views, Functions & Procedures

You will find a lot of information about what Views, UDFs and Stored Procedures are, and what you can

do with them - BUT - not a lot about what you can't do with them.

Or probably to be more precise it is hard to find suitable information comparing the three as to what each

can't do compared to the others.

Oh! I hear you shout, this seems all very unclear and complicated - well - it is not. Slightly obscure it may

be be complicated and complex it is NOT !

Views and Functions (UDF variety) :

Although we can change data in the tables of the database - if the user has the correct permissions etc., the

objects themselves (Views and UDFs) can't change the database directly. By this we mean things like creating

and dropping tables - in fact anything which changes the database itself, or its settings.

Running a SPROC from a UDF is another matter, it is only the permissions which then matter.

Stored Procedures :

Unlike Views and UDFs the SPROC (Stored Procedure) can make changes to the database and it settings.

Obviously, if changes to the DB can be brought about, then changes to the data in tables seems small fry ! ;0)

So SPROCs can change just about anything, to run them the user needs to have suitable permission.

Schema :

Remember, objects within the 'dbo' schema can be used by all. So any Stored Procedure residing in the 'dbo'

can be run by most (if not all) users and that also means ones which change the DB itself.

Making some experiments in SSMS :

Let us start with trying to make a very simple Stored Procedure which creates a table - nothing flash, it does not

even check to see if it already exists. See below :-

=====================================================================================================================================================

As you can see in the image above, it seems to work OK and the SPROC is placed in the OE pane.

However, when we try a similar thing with a UDF we get the following happening :-

===================================================================================================================

However, if we as much as comment out the script line to create the table we get success as shown below :-

==================================================================================================================

And the new function appears in the OE pane - see below :-

=========================================================

Finally lets try a VIEW along similar lines :-

============================================================================================================

If I comment out the create table line I need to have a simple content for the VIEW so here is something

suitable :-

==============================================================================================================

And now we get a success and a new entry in the OE pane after a refresh of the Views folder :-

=============================================================

Summary of Experimental Results :

Okay then, lots of fun, and we saw lots of things happen as well as not happen !

But what did it all mean ?

I read it as even when trying to create View and UDF objects in the default 'dob' schema, we are not allowed

to include T-SQL command words which cause a change in the underlying database.

However, we can use CREATE TABLE with a Stored Procedure, and I would assume DROP TABLE, and a

few other associated commands as well.

This will apply to Temporary Tables as well as permanent ones.

Still to be Done :

We need to explore for ourselves the use of a SPROC from a UDF and View - where a DB change is made as

a result of the SP action.

PHH - July 21st. 2010.


DROPping Tables

===========================================================================================================

=======================================================================================================================

====================================================================================

PHH - July 29th. 2010.


Chapter 9 - Views

=============================================================

Think on a 'View' as an SQL query stored under a specified name in the database.

Similarly a 'Stored Procedure' is really a 'batch' which has been pre-compiled and

stored under a specified name.

=============================================================

As usual, let us start out by looking at what 'BoL' has to say about the VIEW :-

A view can be thought of as either a virtual table or a stored query.

Unless a view is indexed, its data is not stored in the database as a distinct object. What is stored

in the database is a SELECT statement.

The result set of the SELECT statement forms the virtual table returned by the view.

A user can use this virtual table by referencing the view name in Transact-SQL statements the same

way a table is referenced.

Views and their use can be a subject for hot debate - some love them, some hate them, and many

I am sure misuse them !

However, after some research, and listening to Paul Nielsen, it seems as though we can use them in at

least one positive way - which does make all our lives easier ;-)

He says he likes to use the view to hide complex JOINs and de-normalise tables so that other less experienced

and knowledgeable folk can use them as tables in their own querying.

Some more able clients like to use tools like Report Builder 2.0 to create their own reports, and this is when

a few well crafted views can prove very useful indeed.

Let us now consider the concept of views, types of view available, and scenarios for using views. We will

also try to consider some suitable examples.

A view is a virtual table whose contents are defined by a query. Like a real table, a view consists of a set

of named columns and rows of data. [ Unless indexed, a view does not exist as a stored set of data values

in a database. ] The rows and columns of data come from tables referenced in the query defining the view

and are produced dynamically when the view is referenced.

Type of View :

Apparently we can create three types of view - standard views, indexed views, and partitioned views.

Standard :

Combining data from one or more tables through a standard view provides us with most of the benefits of using views.

These include focusing on specific data and simplifying data manipulation.

Indexed :

An indexed view is a view that has been materialized.

This means it has been computed and stored.

You index a view by creating a unique clustered index on it.

Indexed views dramatically improve the performance of some types of queries.

Indexed views work best for queries that aggregate many rows.

They are not well-suited for underlying data sets that are frequently updated.

Partitioned :

A partitioned view joins horizontally partitioned data from a set of member tables across one or more servers. This

makes the data appear as if from one table.

A view that joins member tables on the same instance of SQL Server is a local partitioned view.

[ *** Note *** The preferred method for partitioning data locally is through partitioned tables. ]

When a view joins data from tables across servers, it is a distributed partitioned view.

Distributed partitioned views are used to implement a federation of database servers.

A federation is a group of servers administered independently, but which cooperate to share the processing load of a system.

Forming a federation of database servers by partitioning data is the mechanism that lets you scale out a set of servers to

support the processing requirements of large, multi-tiered Web sites.

General Summary :

Mostly we will focus our attention on getting the most from the 'standard' type of view.

However, we will take a look at what the indexed type of view has to offer the basic developer and DBA.

For most guys reading these notes (and this includes me) the third type, the 'partitioned' view, is probably a

view too far ;-)

Let us now move on to see how to create and use the most basic of views - see you there in the next subsection.

PHH - July 24th. 2010.


How to Create

Once again let us start with the entry from 'BoL' and create a standard type of useful 'View'.

You may be interested to note that the script in the BoL did not work on my system 'out of the box' !

It appears that the case sensitive 'issue' bit once again and I had to change 'ProductSubCategory' into

'ProductSubcategory'. Then it worked OK as you can see below :-

{ The changes in layout a capitalisation and naming of aliases was my doing! ]

==================================================================================================================

This then places a new object in the View folder in the OE pane - see below :-

================================================================

To Focus on Specific Data :

Views let users focus on specific data that interests them, and on the specific tasks for which they are responsible.

Unnecessary or sensitive data can be left out of the view.

For example, a view 'vBikes' for the AdventureWorks sample database would let a user see the names of all bicycles that are

currently in stock. ( See above for details of how to create this view ... )

The view 'effectively' filters out all fields from the Product table except Name, and returns only names of finished

bicycles - instead of bicycle components.

Myself, I would prefer not to use the term 'filter' as I think we tend to think of filtering rows with a WHERE clause.

I would say that the above view only returns the Name column, and ignores the others. But this may be semantics !

Naming of 'Views' :

It appears to be accepted convention that the names of views start with a lower case ' v '. So in the example above we

have ' vBikes'. If it had been my example and not BoL I would have named it " _phh_vBikeNames ".

The image shown below shows how in my clients DB I have named my designed and created views.

*** note *** ' phh ' are my initials and ' fds ' is a thitd party software house.

=================================================================

Running our Views :

This is really quite simple, as we can use ' vBikes ' as if it were a data table.

Study the query pane and the output shown in the image below :-

..........................

..........................

=================================

In the image above the default order is ascending, but we can make a few changes to our query and

get this shown below :-

==================================================

*** NOTE *** The view does not allow an 'Order By' clause generally. This needs to be supplied in the

query using the view - unless a predicate of TOP is being used (special case).

Some experienced authors of books and videos would suggest that our views ought to be kept general, and

not unduly limit rows or data, and certainly NEVER answer specific queries.

So it is OK to have a view to provide a unique list of bike names, but not ones for 'blue' bikes and another

for red ones.

If we are after a list of Blue bikes we can adjust our query to that below :-

=======================================================

You can see in the image above we are listing out only 'Blue' bikes. Note that because my

flavour of AW database is case sensitive, the same query with 'blue' gives no hits as shown below :-

=======================================================

Anyway, the main point of this is that we are best to keep our view general, and then make a UDF

(User Defined Function) to handle the choice of colour.

We will deal with creating UDFs in the next chapter, for now lets just make the T-SQL script work with

a local variable.

Study the code below and the results for 'Silver' :-

=====================================================

In the image shown above it is obvious that we can supply any chosen colour and have the returned query

dataset to match the 'input'. We will see how to 'wrap' this as a UDF in the next chapter - the type of UDF

is called an 'in-line' UDF.

Although not a complex query, view vBikes in the AdventureWorks sample database lets users focus on

specific data without having to construct the JOIN clauses that are required to produce the view.

Views can simplify how users work with data.

You can define frequently used joins, projections, UNION queries, and SELECT queries, as views - so that

users do not have to specify all the conditions and qualifications every time an additional operation is

performed on that data.

For example, a complex query that is used for reporting purposes and performs subqueries, outer joins, and

aggregation to retrieve data from a group of tables can be created as a view.

The view simplifies access to the data because the underlying query does not have to be

written or submitted every time the report is generated; the view is queried instead.

Summary :

many different views can be made in this standard form using script similar to that shown above in the

'CREATE VIEW' example.

Simply any SELECT statement which works (and doesn't have an Order By clause) will roll up into a view.

Lets go and see the next subsection - or briefly visit the UDF section to see how to create a useful function

based on this bike colour problem.

See you soon.

PHH - July 24th. 2010.


Modifying Data in Tables

PHH - July 24th. 2010.


Scenarios for Using

PHH - July 24th. 2010.


What NOT to do !

PHH - July 24th. 2010.


Viewing the System - with Views

[ *** CARE *** take notice of the case of the schema and view name, and use upper and lower as it is displayed ! ]

===========================================================================================================================================================

===============================================================================

==============================================================================================================================================================================

=================================================================================================================================================

PHH - September 9th. 2010.


Chapter 10 - User Defined Functions

In the last chapter (number 9) we saw the need to 'wrap' a view into a UDF so that we could have a

general function to take a character string - representing a colour name - and apply this in a filter

condition to the return from the vBikes view in the AdventureWorks database.

Well guys, here is the T-SQL code we tested back in that chapter, slightly adapted so that the variable

comes in the 'shape' of an input parameter.

Take care as the code is case sensitive (the AW database is set to be case sensitive it would appear)

and that means @Colour is not the same as @colour - it gives zero rows returned if this error is made.

Study the code below to see how easy it is to make a simple in-line UDF :-

=====================================================================

The image above shows that when we run the CREATE script section we get a 'success'.

Refreshing the 'Functions' folder (right click for context menu) the following is seen in the

Programmability folder structure :-

====================================================================

Note that the parameter now correctly shows a lower case ' c '.

And another reminder to ourselves, we can't include an 'Order By' clause generally - see below :-

===========================================================================================================

So we need to add the 'Order By' as part of our query in which we use the in-line UDF as a table.

[AdventureWorks].[dbo].[_phh_fBikesWithColour]('Silver')

acts as a standard data table in our query statements. So the image below is how we use our new function :-

===========================================================================================

The output from the use of our new UDF shows that we still have some Silver bikes for sale ;-)

Lets move on and look at the next subsection.

See you there.

PHH - July 24th. 2010.


Limitations - take care !!

Read this First and Save Yourself Time ! :

I wished I had found and read the limitations below before I tried a number of tasks which 'hit the wall'

for the reasons listed below :-

=============================================================================================================================

From these points itemised above, the restriction of not using Stored Procedures seems the biggest problem to me.

I need to think some more on this before I give any more advice.

[ http://www.informit.com/articles/article.aspx?p=31724&seqNum=2 ]

More useful info. .....

======================================================================================================

PHH - August 22nd. 2010.


Different forms of UDFs

Understanding "User-defined Functions" (UDFs) :

Like functions in programming languages, Microsoft SQL Server user-defined functions are routines that accept

parameters, perform an action, such as a complex calculation, and return the result of that action as a value.

The return value can either be a single scalar value or a result set.

Benefits of UDFs :

- They allow modular programming.

You can create the function once, store it in the database, and call it any number of times in your program.

User-defined functions can be modified independently of the program source code.

- They allow faster execution.

Similar to stored procedures, Transact-SQL user-defined functions reduce the compilation cost of Transact-SQL

code by caching the plans and reusing them for repeated executions. This means the user-defined function does

not need to be reparsed and reoptimized with each use resulting in much faster execution times.

CLR functions offer significant performance advantage over Transact-SQL functions for computational tasks,

string manipulation, and business logic. Transact-SQL functions are better suited for data-access intensive logic.

- They can reduce network traffic.

An operation that filters data based on some complex constraint that cannot be expressed in a single scalar

expression can be expressed as a function. The function can then invoked in the WHERE clause to reduce the

number or rows sent to the client.

Components of a User-defined Function :

User-defined functions can be written in Transact-SQL, or in any .NET programming language. For more information

about using .NET languages in functions, see CLR User-Defined Functions.

All user-defined functions have the same two-part structure: a header and a body. The function takes zero or more

input parameters and returns either a scalar value or a table.

The header defines :

- Function name with optional schema/owner name

- Input parameter name and data type

- Options applicable to the input parameter

- Return parameter data type and optional name

- Options applicable to the return parameter

The body defines the action, or logic, the function is to perform. It contains either :

- One or more Transact-SQL statements that perform the function logic

- A reference to a .NET assembly

====================================================================================

Specifying Parameters :

A user-defined function takes zero or more input parameters and returns either a scalar value or a table.

A function can have a maximum of 1024 input parameters.

When a parameter of the function has a default value, the keyword DEFAULT must be specified when calling the

function to get the default value. This behavior is different from parameters with default values in user-defined

stored procedures in which omitting the parameter also implies the default value.

User-defined functions do not support output parameters.

[ They are 'themselves' the return value - scalar or table ! ]

We will look at input parameters after we have gotten some simple examples to work.

AND - all this will make more sense if we go and look at a few good examples.

Let us move on now to the next subsection in which we explore what UDFs can do for us.

PHH - July 24th. 2010.


Naming of

Enter topic text here.


Scalar UDFs

As usual, let us start be seeing what 'BoL' has to say about Scalar Functions :-

Scalar Functions :

User-defined scalar functions return a single data value of the type defined in the RETURNS clause.

For an inline scalar function, there is no function body; the scalar value is the result of a single statement.

For a multi-statement scalar function, the function body, defined in a BEGIN...END block, contains a series of

Transact-SQL statements that return the single value.

The return type can be any data type except text, ntext, image, cursor, and timestamp.

The following example creates a multi-statement scalar function. The function takes one input value, a 'CustomerID',

and returns a single data value, the aggregated value of the purchased items - with all orders for that customer.

Let us use the AWLT database for this example of ours. Because we are dealing with order total values we can

simply stick with the Header table and don't need the Detail table for this problem.

Study the script example below to see what it does :-

==================================================================================

In the image above we are currently outputting a very small table, but we can simplify this to one value later.

For now we need to see what happens if we try customer with ID of 13 - see below :-

================================================================================

As you can see in the image above, when we supply a 'CustomerID' which is not in the Header table we get

no results, so the case of getting a NULL total value will need to be taken care of in our UDF it would seem.

Study the image below and see the script which makes (and remakes) the new function :-

========================================================================================

After running the above script and getting a 'success' message, we need to look in the OE pane and

refresh the Functions sub folder to see our newly stored function :-

============================================================

Let us see how we may use this new UDF in a query - just a simple one to start with :-

=======================================================================================================================

And if we add a filter WHERE clause we can do this :-

=======================================================================================================================

Yes, I know we are apparently using the same function twice for each row, but we can worry about that later - lets just get

things working for now !

Testing for No Orders :

At the moment we have not tested the UDF in a way which will throw at it customers with no orders. What we need to do

is to use the Customer table in a similar manner to that in the above query. Lets do it now :-

=========================================================================================================================

Not surprisingly we get the same results set returned - BUT - we did not get any errors thrown up ;-)

What we need to do now is see which customers have a zero value, this is a better test of things from

our current point of view :-

======================================================================================================================

So lets try to get our single answer to how many customers have spent zero cash in the period, by looking ahead

to a useful bit of scripting from chapter 12 - let us use s simple CTE - Common Table Expression - see below :-

=======================================================================

And this appears to be correct when I look at the bottom of the results set from the previous test - when we got

the long list.

Notice how we can simplify our script be placing the subquery outside of the main query script.

Now it is time to move on to most exciting part of the UDF 'stuff', how we can return a table.

See you in the next section - after a nice fresh cup of tea ;-)

PHH - August 2nd. 2010.


Table Valued UDFs

Both forms of the 'Table-Valued' function return a 'table' data type.

The table returned may be used in a query anywhere a tabular results set is expected.

These UDFs appear to be a real advantage when writing T-SQL and were not always available in SQL Server.

Now they are here I intend to get the very most from them ;-)

I think it will be best if we go straight into the subsections where we deal with the two formats of the

'Table-Valued' UDF - these are termed 'In-Line' and 'Multi-Statement'.

Lets hope a few good examples will make these more accessible to all of us developers and programmers.

See you in the examples ......

PHH - August 2nd. 2010.


In-Line

Table-Valued Functions : - In-Line :

User-defined table-valued functions return a table data type. For an inline table-valued function, there is no

function body; the table is the result set of a single SELECT statement.

The following example creates an inline table-valued function. The function takes one input parameter, a

customer (store) ID, and returns the columns ProductID, Name, and the aggregate of year-to-date sales as

YTD Total for each product sold to the store.

Since this example from 'BoL' is clear and easy to follow, we may as well use it for our first example, before

we branch out ourselves, and make our own examples.

*** notice *** that we are using the 'Sales' schema so the function will only be available directly to those with

access rights to this same schema.

=============================================================================================================

After running the script we should refresh the Functions folder in the AW folder within the OE pane, and see this

output - don't forget to switch from looking at, and displaying, the AWLT to the AW, or it is difficult to find the new

UDF - how do I know ? I will leave it to you to figure out ;O))

======================================================

And this is how we run the UDF - see below :-

===================================================================

==============================================================

I am sure there are lots of places where we can think of using such UDFs, and in many ways they are really

CTEs (Common Table Expressions) with possible input parameters.

This means they can simplify our main query scripts as well as allowing us the flexibility of inputting variables,

or parameters.

*** REMEMBER *** Don't forget our good practice with stored DB objects - keep then of a general nature.

Example 2 : Doing our own :

Okay then you guys, we have let 'BoL' get us going but now we need to branch out ourselves.

Let us design and make a UDF which will return a table of customers who have had delivered (ordered) as

specified product in the period of the data in the AWLT database - yes, we need to swap back once again.

In doing this example remember I screwed up the data slightly when I edited the codes to show the FULL JOIN

in action, so if you get a few slight differences then that is why [ I just noticed 'ProductID' 13, 26, & 39 - these were

placed by me in the DB after turning off the enforced checking !! ]

===================================================================================================

It is not difficult to script a UDF around this trial query and we should get something like this shown below :-

[ sorry ! there is a missing 'LT' in the third line before '.ufn_' ]

=====================================================================================================

========================================================

And having made and stored the function as a DB object we can now run it as below :-

================================================================================

And choosing another product we get this :-

===============================================================================

Okay you guys, we are just about to do something VERY interesting, and that is use the UDF in the column

specification clause - see below :-

========================================================================================================================

In the image above we are using the whole subquery with parameter 'ProductID' as the value for the returned third

column called 'Qty'.

Going further - as well as tidying output :

As we did in the last section on Scalar UDFs let us use the CTE syntax to make things easier for ourselves and also

make the main query much clearer to read too.

In the extended example below we have applied a filter so that only non-zero values are returned as well as moving the

ORDER BY clause from the sub query level into the outer query. You need to remember at all times that the 'Order By'

clause needs to come very last of all as it is applied to the data set to be finally returned for the main query. Yes, I know,

that is that special case about 'TOP' but we are talking in the general at the moment.

Okay enough of the talking, lets see how this looks :-

===============================================================================================================================

I think you will agree, it is rather neat, and an interesting way to do things ;O))

Now you had better read the CTE chapter when you get a moment, first of all, let us move on to the next section to see

how we can merge table data to return one table, but use a few in the process of compiling the return dataset.

See you soon !

PHH - August 2nd. 2010.


Multi-Statement

Multi-Statement User-Defined Function (MS-UDF) :

Here is what 'BoL' has to say :-

For a multistatement table-valued function, the function body, defined in a BEGIN...END block, contains a

series of Transact-SQL statements that build and insert rows into the table that will be returned.

The example in 'BoL' seems a bit complex to be our first to follow, so I will need to go and have a think

about a good example of our own, and how to explain it.

What we simply need to know :

Often the training material and notes in BoL are too focused on getting over the complete technical details

of what a feature in T-SQL actually is, rather than briefing the 'new guy' as tit what it can do for him in his

work 'from the off'.

Effectively the MS-UDF allows us to return a full table of data which we have built up from scratch from

nothing. Effectively making a temporary table within the UDF and passing this back on return to the

calling script. This returned 'Table' can be used in our script by placing the UDF call in-line with our calling

script. This is a great advantage over the SPROC, as this can't be used in-line.

Example 1 :

Time for us to look at an example I feel. A good example (or three) will help us see the power and flexibility

of this function.

I have taken an idea from Paul Nielsen's 2005 Red Bible of SQL and adapted it to the database we have

available to us. We will use AdventureWorks (full version).

Although the following example is a little forced (or false) it serves the purpose of getting us going with MS-UDFs.

Let us see the script which will do the task, and then we can talk through it - study the detail carefully :-

=====================================================================================

Notice first of all that we are using two different existing data tables, from two different DB schemas,

Production and Sales.

The table type we are constructing 'on the fly' as it were, is called @Price and this has to be defined at the

top of the UDF script. We are using three columns for our 'found' data, an INT, a DATETIME, and a MONEY.

Notice also that the first part of the table is the product item list values, whereas the second part of the

table is a set of average sales prices for any product item found in the sales table.

The list price for each product item has a date associated with it, BUT, the average price of selling does not,

so we need to make this NULL.

Running the MS-UDF :

Since the function when used, or called, is effectively a Table then we can use it just as a Table in a query. Here

is a first example :-

========================================================

Notice I have displayed the results set at the junction of the two merged data sets from Sales and

Production.

Or we can filter out those we don't want with a WHERE clause :-

=========================================================

Remember the column is now called 'ID' and not 'ProductID' as we specified this in the UDF itself.

Alternative way to specify the UDF script :

While we are at the stage of closely examining the syntax of the UDF itself, let us see how we could

have scripted this, so we can get a feel of just what is really happening - study the following carefully :-

===================================================================================

In the image above notice that both the order and the number of fields provided to the INSERT statement

may be changed.

As long as the data types are correct and the SELECT column specification (list) fits the field name list

in the INSERT column parameter list, then everything should work okay.

Note that any column / field not specified will have a NULL provided, and that suits our purpose here, so we

don't have to place the NULL ourselves as we did earlier in the first scripting.

Finally, notice that the names of the fields declared in the Table declaration are those which are 'exposed' to

us when we need to use column names in our queries - see below :-

=========================================================

Let us now move on to another example.

Example 2 :

Let us now try something slightly different, but very interesting I feel.

So far we have not combined or compared data from tables in two different databases, but now is

the time to try ;-)

Let us try to write a MS-UDF which will reside in the AWLT database, but will make up a table of

customers from both the AWLT database as well as the AW full database.

The fields we will try to get for each customer are just basic ones of customer identity and an address

line or two as well. This is slightly easier to do for the AWLT database than the full AW version so lets

start the easy way ;0)

Below is a trial query to see if we can join up the necessary tables in the correct fashion :-

============================================================================================================================

Please note that in this and the following with the full AW database I gradually build up the query a

section at a time, and test it out to see it does as I require.

Now lets look at the one I have built up using the full AW schemas and tables :-

[ Because the information is normalised across more tables we need to do more joins ... ]

Study the script below, and see how it provides some suitable data for out UDF :-

=========================================================================================================================================================

If you are not used to all these joins then please DON'T panic, as it is far easier than it looks !

I am not that familiar with the AW database in detail, so I had to go looking through the most likely

schemas, and the most likely tables, then pick out the suitable primary keys and foreign keys.

Once again I built up the query, a JOIN at a time, and kept adding a field or two to the column list

after SELECT.

Putting these Queries Together :

It would appear as though we now have potentially two suitable sets of customer data which we can

attempt to merge together into one table using a MS-UDF. We just need to be careful as to which fields

we use to wrap the information into a single table - both which fields we use in the temporary table which

is exposed to our query, as well as those from our twp UDF subqueries which we map to the temporary table.

Lets give this a go, see below for what I come up with after my testing :-

[ One thing I did learn and that is to make your character fields plenty wide enough for existing data or errors !! ]

==========================================================================================================

In the image above you can see that we have two sub queries which access data from different databases.

The temporary table - as such - has a structure which has all the fields / columns present in both of the two

returned datasets, and as previously, some can be ignored if required.

The care I had to use in designing the script for the UDF was mainly on getting the order of the fields for the

table in the parentheses the same as in the related SELECT statement.

I spent the most time in testing and editing, trying to overcome an error which I caused by not specifying the

character field wide enough for 'FirstName' - I suppose I am used to DBF technology where truncation would and

the developer not even be aware of it !

So learn from this - make your string columns wide enough - and a bit more ;O))

Running the new UDF from a query statement is easy enough - see below :-

================================================================================================================================================================

Or we can see how the two data sets have merged with this query :-

=========================================================================================================================================================

Just looking at the above information it would seem that the 'CustomerID' values are quite different between AWLT and AW.

However, it now seems appropriate that we add a field which contains 'AW' or 'AWLT' to indicate which

database the row came from.

Let us see if we can make the appropriate changes to our UDF. Below I will only show the script lines which

have changed to do the task as required :-

======================================================

==============================================================================================================

==============================================================================================================

And here is a small portion of the results set from this newly 'ALTERED' MS-UDF :-

====================================================================================================================

A useful point to notice is that once you have created the UDF for the first time, it is a good idea to change

CREATE into ALTER, as you are likely to need to make frequent changes as your development takes place.

I would not like to guess how many times I re-saved the changes using ALTER while I go the MS-UDF to do

as I wished.

=======================================================================================

*** Since writing this ***

I have developed and tested a MS-TV-UDF for the Family database first started in the Self Join section.

Now we have a fuller exploration section in Chapter 26, and from a Function there I have produced this for us to use as

a good example of a "MSTVUDF" - see below for details :-

=========================================================================================================================

===========================================================================================

Notice in the above script that we no longer need the UNION operator as we have a table data type into

which each pert of the results set can be placed as the dataset is built up.

=====================================================================================

Before we leave UDFs we need to go to the next subsection and do a short 'bit' on schema binding - that is

after some work on Input Parameters and 'Correlated UDFs' too.

See you there.

PHH - August 2nd. 2010.


Input Parameters

Input parameters of UDFs are handled differently to those of Stored Procedures.

It would appear that the same number of parameters needs to be supplied in the UDF call, as were originally

defined when the UDF was created.

If one or more of the parameters has a default value set in the script, then in order to make this take effect the

word DEFAULT must be provided.

I think this calls for some suitable examples !

Example 1 :

Let us start simply - and add three integer values together.

Here is a script which has three default values set, at 1,2, and 3 :-

======================================================================================================

Lets call the function within a simple query statement - see below :-

============================================================================================================

It is not the result which is really of interest, or even the input values, but more what is about to happen

when we provide no values at all - see below :-

===============================================================================================================================

Now the return result is important as it ought to be 1+2+3 .... the sum of the default values, as defined previously.

Or to get my favourite number - that of thirteen - we do this :-

=================================================================================================================

And if we supply no parameter input values we get this :-

====================================================================================================

And just to be thorough we do this test as well :-

==================================================================================================================

Summary :

So we MUST provide the same number of parameters as the UDF was originally defined with, or scripted with. We

can't supply fewer or more.

If a default value has been declared then we can provide DEFAULT as the input, but can't miss it out.

Finally (for example 1) :

If we ALTER the UDF and neglect to define a default value for @Z we get this when we try to invoke a default value :-

=================================================================================================================================

It would appear that @Z is taken as NULL since the default is not defined, and so any result including it in the computation will

also be NULL. So take care with defining default values and their subsequent use.

Obviously, if we provided one or more of our input values as NULL the same result would be returned - NULL.

Below is what the OE pane shows after the changes :-

=================================================

Now time for another example I feel ;0))

Example 2 :

Example 3 :

PHH - August 3rd. 2010.


Correlated UDF - the APPLY

The APPLY command is new in SQL Server 2005, and it can be used with Table-Valued UDFs.

There are two forms to the APPLY command, these are :-

- CROSS APPLY

- OUTER APPLY

Paul Nielsen says this :-

" ... the 'cross apply' has a confusing name because it operates more like an inner join than a cross join. "

" The outer apply command operates much like a left outer join. "

The Cross Apply :

Although we are going to design and make our own example, based on the AWLT database, let us first look

at a query script sample from Paul's 'Bible' which will help to demonstrate what we have happening :-

SELECT C.LastName, Code, DateBegin, Tour

FROM Customer C

CROSS APPLY fEventList2(C.CustomerID)

Order By C.LastName

In the query above, for each row that is being processed in the main query, the Table-Valued UDF is run with

a different parameter and the data from the main query will be joined with each and every row of the dataset

returned by the UDF. If no rows result from the UDF use, then nothing is added to the returned dataset of the

main query - as it the calling value was ignored.

I think it is now time for us to try and make a working example, even if it is not the best, or most useful, example

on the planet ;-)

Example 1 :

Before we get to explain what our example will do for us, let us take a look at the following test query in SSMS :-

===================================================================================================

In the image above we have singled out a customer called 'Abel'.

We have then found from all the order item lines in each and every order for 'Abel', the list of products, and

then made sure we have a unique entry in a list of products with names.

If we roll this into a UDF which accepts as a parameter the LastName, and then returns a table of product

with product name, we can then have something suitable to do a correlated 'cross apply' test query.

Study the details in the image shown below :-

=================================================================================================================

If we have done our homework correctly we will be able to use this in a simple query as shown below :-

===============================================================================

=======================================================

So when we use the UDF with 'Abel' in a single query we get 43 rows of uniquely named products.

It is then quite easy to wrap this into a correlated query as shown below :-

===========================================================================================

==============================================================

Any eagle eyed person who knows the AWLT database will spot that we now have more rows than we had

individual item lines, so we must have duplicate names, which will multiply up the product 'cross apply'.

The way to correct this would be to use the 'CustomerID' in place of the name. let us try this :-

================================================================================================

===================================================================

That looks a bit better since there are 542 rows in the SalesOrderDetail table - "what of the other three"

I hear you ask, well I did warn you about turning of integrity etc. when we went to edit the Customer

numbers in the details table, and make a 13,26, and 39 .... I think that has returned to haunt us ;O))

However, we may see something in the next section immediately below this, even if we never get the three

lines of row details because they belong to customers who don't exist !

The Outer Apply :

Well, now that we have a Table-Valued UDF we can simply use it in another query statement, but using

the OUTER APPLY command in place of the 'cross' variety.

Let us see what we get :-

============================================================================================

===========================================================

As you can see we now get one row at least, for each and every row in the 'calling' table dataset.

And if we adapt the query so as to see only the non-matches we get this below :-

==============================================================================================

===========================================

So 409 individual customers did not have any product items at all. Yes, I know that there are other ways of

finding that out, it is just that we are trying to understand what we have done with our new found commands.

Obviously, if we change the script to 'is not NULL' we get back our 539 rows - still with the 'haunting' 3 missing !!

I think this is enough to get us on our way, let us move on.

At least we have seen some scripting options which seems as modern (is some ways) as the dotNet to which

many of us have become accustomed.

Some other parts of SQL Server and T-SQL feel and show their age in my humble opinion ;-)

PHH - August 3rd. 2010.


Schema Binding within UDFs

Let us see what 'BoL' has to say :-

Schema-Bound Functions :

CREATE FUNCTION supports a SCHEMABINDING clause that binds the function to the schema of any objects it

references, such as tables, views, and other user-defined functions.

An attempt to alter or drop any object referenced by a schema-bound function fails.

These conditions must be met before you can specify SCHEMABINDING in CREATE FUNCTION :

- All views and user-defined functions referenced by the function must be schema-bound.

- All objects referenced by the function must be in the same database as the function. The objects must be

referenced using either one-part or two-part names.

- You must have REFERENCES permission on all objects (tables, views, and user-defined functions) referenced

in the function.

You can use ALTER FUNCTION to remove the schema binding. The ALTER FUNCTION statement should redefine

the function without specifying WITH SCHEMABINDING.

PHH - August 3rd. 2010.


Chapter 11 - Stored Procedures

*** Remember *** Using SPROCs we can change the state of a DB.

By this we mean not just the data in tables, but also 'objects' of the database itself.

=============================================================

A Stored Procedure is really a 'batch' which has been pre-compiled and stored under a

specified name.

Similarly a View is an SQL query stored under a specified name.

=============================================================

Modes of Use by Developer / DBA :

VIEWs are for seeing our data as 'virtual' tables, for hiding all operations which merge the data.

Hiding the JOINs !

UDFs are for doing some data manipulation - possibly using VIEWs - which then supply a scalar

value or a table value, which can then be used directly in queries statements.

Procedures are for doing real 'programming' tasks where larger 'chunks' of code can be registered and

stored in the database, for faster use when required. Stored Procedures (SP) should be designed

and used which are of a general nature. SPs may be getting a little old fashioned for general use !?

Like VIEW and UDF, the SP should not be of a specific nature, make it as generally applicable

as possible, by using the parameter input to good effect.

Any stored DB object needs to be managed, so planning and designing what is stored is part way

to managing the stored objects !

====================================================================

Example :

Let us start with following through a small but useful example.

Below is shown the 'Programmability' section of the Object Explorer (OE), with the subsection

'Stored Procedures' open and listed :-

================================================================

Now let us enter a short statement to create a Stored Procedure - called SPROC.

I have entered the script in the editor and run both of the two batches shown below :-

===========================================================================

If we have been successful - as the 'Messages' tab suggests, then we ought to be able to see a new

entry in the OE when we have done a 'refresh' as shown below :-

===================================================================

And sure enough the new SPROC is shown in the OE pane and can bee seen below :-

============================================

[ *** note *** the SP goes into the 'dbo' schema because this is default and no other schema was specified. ]

[ ... to see more about schemas look at chapter 19 ... ]

Running the Stored Procedure :

Now that we have our procedure stored, we can think of running it.

There are two ways of running the SPROC from the query pane - see below for examples :-

=============================================================================

Notice that both methods work when we have two batches, however do be careful as the following

causes an error :-

==================================================

But placing an 'EXEC' before the second statement in the batch does this :-

=======================================================

You need to try this out for yourself, to get a feel of what can and can't be done.

Other Forms of Procedure :

To do anything much with SPROCs we need to be able to use more input parameters, as well as

specifying a range of different types of return values.

Before we go off into other subsections to see what we can do to make our own types of SP, let us see

just what main forms exist in SQL Server.

There are three, and one is no longer recommended for use :-

- User-defined Stored Procedures - [ for us to create and use, there are two types ]

- Extended Stored Procedures - [ avoid using, will be removed in the future ]

- System Stored Procedures - [ provided for our use ]

For the two forms which are still in use, we will look in detail at how to best use these in our work with

SQL Server, in the next subsections

See you in these shortly.

PHH - July 21st. 2010.


When and Where to use

SQL Server is a mature product, it has been going strongly for well over ten years.

The world has changed a lot since the time it was designed and first used.

Many training materials, both book and videos, have been written and created some time ago.

SQL Server itself has had some important and major changes and additions since it was first put

on the market.

For guys like you and me, one of the hard things is to extract from the support material just what is

going to be most use to us in today's world. What should we use most, and what should we keep

clear of, or use little.

Quite often the support material tells us what can be done technically, but does not take that personal

view that we are hoping to hear - like "give this a miss in 2010 and use 'XXX' instead".

One thing that has taken a while to 'come through' to me over the two years I have been trying to get

to grips with SQL is just this - there appears to be some parts of it which seem 'old fashioned' and even

quite awkward to use in today's world of dotNet application developers.

Stored Procedures is a topic I have been agonising over for quite some time, I have even left it till

quite late in the writing of these eNotes. To put it frankly, SPs don't seem to be to me as useful as I first

thought they would be. UDFs I can grasp immediately, and think they are fantastic, but SPROCS I still

don't fancy too much.

Today's World (2010) :

Much faster networks, and ones with much broader band width, including even the internet, is just one

aspect of change which affect the style of use of SQL features.

We also need to take into account the quality and flexibility of ADO.NET and the modern trend in application

software development with LINQ and ORM. The trend in this modern world is to have objects throughout

the application, and to have your business solution layered, so that we have a data abstraction layer.

LINQ (Language Integrated Query) is going towards having the script in the application code so that on

compilation the developer knows if it is likely to be correct - or not.

ORM (Object Relational Mapping) builds objects directly over our tables - or even vice versa. But one

thing which seems to me to be happening is that the emphasis on pushing code and logic into chunks of

script which is then stored in the user's database, seems to be in decline.

For the past few months I have been working with clients whom have had recent SQL installations which

are based on a world wide name of an accounting and stock tracking package. One has the smaller version

and the other the larger version of the application offering from SAGE - Sage200 and Sage1000. These are

built of very different software technologies but both use SQL Server 2005.

Believe it or not the Sage200 is the more modern one built on dotNet 2.0 technology. I believe the other is

C or C++ under the hood.

When I look at the Programmability folder of the OE pane and then the Stored Procedures sub-folder, I see

very few SPROCs at all, 13 in the '1000' flavour and 12 in the '200'. Hardly over-use when the systems are so

large.

For you interest the more modern '200' system has 18 UDFs 6 more than SPs, and the older '1000' has no

UDFs at all.

I suspect (reading the object names) that most of the UDFs in the modern C# supported system are from the

customisation (quite large) of our original patient healthcare provision system. So the professional programmer

guy is using UDFs over SPs

In fact using the properties dialog for the objects we are discussing I can see from the created date that this

guy programming in C# made most of the UDFs for the '200' version of our Sage implementation.

Application Programming & Reporting :

I have spent the last year or more focusing on using SQL Server in applications written in dotNet using Vulcan,

with a little C# thrown in when I needed it, as well as trying to move into designing reports for a client using

Reporting Services with MSSS 2005.

Both of these aspects of using SQL Server don't require me to focus on housekeeping in the DB or suing scripts

in real time to create and drop tables, and other objects. So what I am about to say is qualified by the needs I

have brought to the table.

When using the client DB for the larger SAGE1000 system I am currently mainly focused on designing reports,

although I do some data cleansing too.

The existence of many tables (thousands), residing in two different DBs, and the fact that there are many columns

in most tables, often with complex names - non of which seem to be the same in other tables, even though they hold

the same data - means that using 'Views' to hide the complexity of the table joins is great!

I also find that using UDFs is also very helpful, but so far I have been surprised that I don't seem to have had a

need for using SPs - I thought at first that this was due to my lack of experience, but I am now thinking that it may

simply be that I have no need for what I am trying to do.

My application development style also would not seem to lend to using SPs as I would wish to keep any DB object

tidying script in the app. code and visible within VS too.

Where to Go from Here ? :

Well, I think we should know what Stored Procedures are, and what they can do, as well as try out a few useful

examples to get a good feel for what is going on.

However, I think we should also keep one eye on the fact that there are other features and facilities in SQL Server

2005/2008 which may be the instrument of choice as far as we are concerned.

My Personal Tip :

I would suggest looking at Views and UDFs to see how to use them sensibly, and how to get the most from them

in your application development and report designs.

These seem rich areas for use and application.

PHH - August 4th. 2010.


Naming of

This is an issue which is either nothing or something - it all depends on your situation ;-)

I have found that working with a client database (or two) that when a number of third party software

houses do work on the DB itself, it is best to have a good naming policy for the site.

This does not have to be complicated, or unwieldy, just consistent and simple to apply and understand.

Following my client's requests I now adopted the following 'naming policy' :-

Any database object I create starts with '_phh_' where these are my initials between underscores.

Next is an indicator of the type of object, so we have ' v ', ' f ', ' sp ' and so forth. Since I have not yet made

a table I can't say if we would add an indicator. However, tables should only be made by those responsible

for database design, and that is not me on this occasion ;-)

Finally I add sufficient words or part words to identify the purpose or function of the object.

Lets take a real example - see below :-

==================================================================

The highlighted object is a 'View' I made and it provides a dataset which is the age and date of birth for any

active patient.

You will notice 'fds' this is the software house who made the SAGE installation and integration for my client.

You will also notice that they seemed to have a few variations on the naming syntax, and this may reflect the

fact that they seem to use a number of self employed 'consultants' who come in for a day or two at a time :((

Let us now look particularly at stored procedures - here is what I now have for one of my clients main DBs :-

==========================================================

In the image above you can see a few items in the OE pane which I have created. And yes, you may spot a

couple which seem to be not quite along the lines of what I just said above - BUT - it is my development

system and some of these objects will not get as far as their 'demo' system at Cardiff !

Finally, just to reinforce what I have said so far, here is an image of part of their other main database :-

=====================================================

Obviously the image above shows functions, or UDFs.

Schema used :

Here may also be the right place to say that any object created is placed in the ' .dbo. ' schema unless there is

a special need to do otherwise.

Placing it there (in .dbo. ) means that the object is available throughout the database. This would be different if

in fact security was an issue.

However, my feelings are that third party software houses servicing small to medium sized companies, tend to

leave the security solution rather simple ! I can see just one schema in the large DB and that is called 'scheme' !!

It seems like a suitable place to stop criticising other peoples work ;O))

See you in the next section.

PHH - September 7th. 2010.


User-defined SPROCs - examples

Once again let us start by seeing what 'BoL' has to say :-

Stored procedures are modules or routines that encapsulate code for reuse.

A stored procedure can take input parameters, return tabular or scalar results and messages to the client,

invoke data definition language (DDL) and data manipulation language (DML) statements, and return output

parameters.

In SQL Server 2008, a stored procedure can be of two types: Transact-SQL or CLR.

Although this is seemingly useful information as it stands, it does not mention the SPs can be nested, or

chained. So one SP can call on another.

Or that the SP does not have to take any parameters

Also the phrase about using DML statements is one important 'thing' which makes the SP stand out from

a UDF or VIEW.

To most application programmers and developers the procedure is familiar, and we like modular design !

Let us now look at what 'BoL' has to say on the two types of UD SPROC (UDSP) :-

Transact-SQL :

A Transact-SQL stored procedure is a saved collection of Transact-SQL statements that can take and return

user-supplied parameters. For example, a stored procedure might contain the statements needed to insert a

new row into one or more tables based on information supplied by the client application. Or, the stored

procedure might return data from the database to the client application. For example, an e-commerce Web

application might use a stored procedure to return information about specific products based on search criteria

specified by the online user.

CLR :

A CLR stored procedure is a reference to a Microsoft .NET Framework common language runtime (CLR) method

that can take and return user-supplied parameters. They are implemented as public, static methods on a class in

a .NET Framework assembly.

Since these eNotes are focusing on T-SQL we will not follow up the second type of UD SPROC but leave it that at

least we know of its existence.

Stored Procedure Basics :

... they can :-

- Accept input parameters and return multiple values in the form of output parameters to the calling procedure

or batch.

- Contain programming statements that perform operations in the database, including calling other procedures.

- Return a status value to a calling procedure or batch to indicate success or failure (and the reason for failure).

The benefits of using SPROCS stored in the 'back-end' DB as opposed to script stored on the local client PC are :-

- They are registered at the server.

- They can have security attributes.

- Users can be granted permission to execute a stored procedure without having to have direct permissions on the objects

referenced in the procedure.

- They can enhance the security of your application.

- Parameterized stored procedures can help protect your application from SQL Injection attacks.

- They allow modular programming.

- You can create the procedure once, and call it any number of times in your program.

- They are named code allowing for delayed binding.

- This provides a level of indirection for easy code evolution.

- They can reduce network traffic.

Okay you have heard the Microsoft hype, and in many ways these are positive points, however the world has changed and

now we are in the world of LINQ, and that is also from Microsoft, I suggest that they are actually using some of these points

'in reverse' as it were, to sell to us LINQ technology.

ORM and 'Entity Framework' technology also muddy the water as to how much or little we need to use SPROCs.

[ Object Related mapping - ORM ]

So I will take the sensible and pragmatic approach to SPROCs - they exist, and they offer some benefits to me as a developer,

and so I will take the positives, but not go over the top on use (or abuse) !

Actually, this same practical approach guides me with use of VIEW and UDF.

Before we get into making our own SPROCs we need to read these rules from 'BoL'.

[ VERY important things in red, lesser stuff in green ... necessary info. in blue ... ]

Rules for Designing Stored Procedures :

... they include the following :

The CREATE PROCEDURE definition itself can include any number and type of SQL statements, except for the

following statements. These cannot be used anywhere within a stored procedure.

CREATE - AGGREGATE, RULE, DEFAULT, SCHEMA, FUNCTION, TRIGGER, PROCEDURE, VIEW

ALTER - FUNCTION, TRIGGER, PROCEDURE, VIEW

SET - PARSEONLY, SHOWPLAN_ALL, SHOWPLAN_TEXT, SHOWPLAN_XML

USE - database_name

- Other database objects can be created within a stored procedure. You can reference an object created in the

same stored procedure as long as it is created before it is referenced.

- You can reference temporary tables within a stored procedure.

- If you create a local temporary table inside a stored procedure, the temporary table exists only for the purposes

of the stored procedure; it disappears when you exit the stored procedure.

- If you execute a stored procedure that calls another stored procedure, the called stored procedure can access

all objects created by the first stored procedure, including temporary tables.

- If you execute a remote stored procedure that makes changes on a remote instance of Microsoft SQL Server,

those changes cannot be rolled back. Remote stored procedures do not take part in transactions.

- The maximum number of parameters in a stored procedure is 2100.

- The maximum number of local variables in a stored procedure is limited only by available memory.

- Depending on available memory, the maximum size of a stored procedure is 128 megabytes (MB).

Taking a Practical Approach :

Although I am trying not to hind any information from our eyes, I am aware that we can have too much at a time

when all we wish to do is get some results ;-)

However, because the behaviour of SPs and how they can be used by us, is deeply entwined with security and access,

we need to be aware of most of the rules, even if some seem many a mile away from us and our needs.

The next section from 'BoL' is necessary for us to know, as it is about the limiting 'scope' of SP use depending

on the way we use schema names.

Qualifying Names Inside Stored Procedures :

Inside a stored procedure, object names used with statements (for example, SELECT or INSERT) that are not

schema-qualified default to the schema of the stored procedure. If a user who creates a stored procedure does

not qualify the name of the tables or views referenced in SELECT, INSERT, UPDATE, or DELETE statements

within the stored procedure, access to those tables through the stored procedure is restricted by default to the

creator of the procedure.

Object names used with all Data Definition Language (DDL) statements such as CREATE, ALTER, or DROP

statements, DBCC statements, EXECUTE and dynamic SQL statements should be qualified with the name of

the object schema if other users are to use the stored procedure. Specifying the schema name for these objects

ensures the name resolves to the same object regardless who the caller of the stored procedure is. If a schema

name is not specified, SQL Server will attempt to resolve the object name first using the default schema of the

caller or the user specified in the EXECUTE AS clause and then the dbo schema.

Don't panic, I suspect that all this is not as much of a problem as it sounds ;0)

If we are not 'sloppy' and fully name our tables and other objects, then we will be well on the way to success.

Also, if we don't need to use other named schemas, then stick with the default of 'dbo' - till at least you gain

some valuable experience.

Getting Information About Stored Procedures :

In the image shown below I have adapted the script included in 'BoL' in a number of ways.

Study the image details and we will discuss my changes later :-

===============================================================================================================================================================

First of all I have made the table names full names, to include database and schema. This means

that it does not matter which database name we have in the earlier 'USE' statement.

Secondly I am also listing the name of the SPROC.

Thirdly, we could filter out the SPs which are of the system variety and in the 'dbo' schema - see

the following image first :-

===============================================================================================================================================

It looks as though we need to get a lot more experience with this sort of system object processing,

but at least we have made a start, and below we are seeing something more like what we as users

have created in the SPROC department.

[ don't ask where the 'spCheckChangeSproc' went - I suspect the '_' character is a wildcard in the LIKE operation :( ]

========================================================================================

Okay, enough of the fun .... ;O))

So let us look at some good examples of how we can use SPs to make our data access life easier.

Example 1 :

In this example we will look at a couple of things at once.

As well as seeing how to create and use a very simple Stored Procedure we will see also what issues can be

thrown up if a suitable name is not chosen 'wisely'.

Naming Stored Procedures :

We recommend that you do not create any stored procedures using 'sp_' as a prefix.

SQL Server uses the 'sp_' prefix to designate system stored procedures.

The name you choose may conflict with some future system procedure.

If your application uses nonschema qualified name references and your own procedure name conflicts with a system procedure

name, your application will break because the name binds to the system procedure, not your own.

A user-defined stored procedure that has the same name as a system stored procedure and is either nonqualified or is in the dbo

schema will never be executed; the system stored procedure will always execute instead.

The following example (shown below) demonstrates this behavior.

[ Using an explicit schema qualifier also provides a slight performance advantage. Name resolution is slightly faster if the Database

Engine does not have to search multiple schemas to find the procedure. For more information, see Executing a Stored Procedure. ]

First of all, lets see if we can use a full 'name' for our SP, one with a database qualifier, as well as a schema qualifier.

============================================================================================================================

No ! apparently not, and this makes sense from a security point of view I suppose.

Now lets try just the schema qualifier :-

===========================================================================================================

That seems to work just fine, and so after a 'refresh' of the SP folder in the OE pane we get this :-

==================================================================

So now we need to try running (or executing) the SPROC - see below for alternatives :-

==================================================================

Well, both of the qualifying forms seems to work well, and the results are the same for each, as well as what would

be expected from the script we used.

However, what happens if we chose to execute the unqualified form of the script ? - see below :-

==============================================================================================================

We appear to get some output, but some very different details from that previously.

It looks from the output as if we are running a system SP - see looking in the relevant folder we see this :-

============================================================================================================

Now what if we try to run a script to remove the SP - lets try :-

==================================================================================================================

Once again we can only work with the database in focus, so this form below works :-

========================================================

And after a refresh, here is the OE folder :-

[ it seems more important to refresh after a delete, as OE reports a SP exists when it doesn't ! ]

===================================================

Finally notice that we can't qualify a SP execution with a DB qualifier either :-

==============================================================================

Summary :

Don't use 'sp_' as a prefix to your SPs - I use 'uspPHH_' for mine.

[ 'usp' for 'User-defined Stored Procedure' ]

Although we have seen how to use a named schema, it is probably wisest to use the default of 'dbo'

unless you need to do otherwise - as may be dictated by your clients.

Finding Out - Does the DB Contain a Named SP Already ? :

Before we go any further, we need to know how to find out if a specific (named) SP already exists.

In the example show below I have added my own extra script lines over that shown in the BoL notes.

It makes sens to have the script report back as to whether it exists or not - well at this level of our work

when using SSMS at least ;0)

====================================================================================================

Let us now add a small section of script to create the 'same named' SP - see below :-

========================================================================================================

Example 2 :

But it will only be when we run this again that we find out if the SP exists - yes, I know we can

refresh and look in the OE pane :-

===================================================================================================================================================================

In the image above, we see that because we still have the DROP in the IF / ELSE, then we have

actually removed the object from the DB.

Be careful, it is easy to get mixed up in your head between the script running to create the SP and

the execution of the SP itself.

Before we move on and improve and change this example from 'BoL' let us run it - see below :-

=============================================================================================

So the following will work as a batch :-

===================================================================================

Okay then, enough of this VERY simple example, lets move on to just a 'simple' example ;o))

Example 3 : passing input parameters :

For this example we will modify the example in BoL so that we get a range of people. The range

will depend on the input parameters for the start and finish of the 'last name' values - see below :-

.......................................

.......................................

==========================================================================

In the image above notice that we input two values, both surnames (last names) but that the input variables are

specified and so the order in which they are supplied does not matter - see below for a reverse order input :-

=========================================================================================================

However, keep thinking about what you do as the following is no use at all ;O))

=======================================================================================================

However, if you wish to 'trap' the silly input error then we can change the SP script to be as shown below :-

==========================================================================================================

Notice in the image above we have run the newly re-created SP with the erroneous input data.

Moving on, lets see how we can set some default values for input parameters, and then supply (or not) these

parameters.

Once we have added this input flexibility to out armour we can then turn our attention to output parameters.

Example 4 : setting input defaults :

The whole point of providing default values is to give more flexibility about provision of input parameter values.

In the example below we have adjusted our previous example (3) so that both input parameters have now

been supplied with defaults - study the script below :-

=================================================================================================

As you can see by the first test execution, we run the Sp without input of any kind.

The results are all the surnames from the start of 'C' to the very end of 'EZZZZ'.

In the image shown below we have tried three more versions of input and they all work and supply the

information you would expect - if one parameter is supplied and which one not specified then it is taken as

the first one.

================================================================================================

Finally, even with this extra modification the error trap still functions :-

============================================================================

Example 5 : Passing Output Parameters :

To complete our adaptation of the HR example, and last names etc., let us now count the rows

of details found (number of people) and pass back as a parameter that total or count.

Study the code below to see what can be done :-

=================================================================================================

And to run this SP we need to write script like so :-

===============================================================================================

Enough for now, let us take a break - a nice cup of tea sounds good ;-)

See you in the next subsection.

PHH - July 30th. 2010.


Limitations of Use

Although the SP can output data of many types, including a tabular dataset, it can't be used in

an 'in-line' fashion ! [ .. whereas functions may be ... ]

We dotNet coders and developers have gotten quite used to using a method name in our expressions

when we require the returned data from the method.

If we wish to use 'in-line' values as part of our expression then we need to use UDFs. This is probably

why so many folk today seem to like them ;-)

I suggest you go and study UDFs in some depth - if you haven't already - to see how useful the 'in-line'

feature is, and can be to your scripting.

Sorry for this rather 'sad' restriction on SPs - probably shows their age !

Have a look at the subsection a little further on, called 'Return and Outputs'.

See you in the next subsection.

PHH - August 4th. 2010.


Specifying and Passing Parameters

Although we have already looked at a few examples in the previous section, we now need to branch

out to make some meaningful examples of Stored Procedures (SPs) ourselves.

In this section we will look particularly at how information may be passed into the SP - how this is both

scripted and also handled at runtime.

Also it is important to say from the very beginning, that there is no point in showing examples of SPs

which would be better scripted and saved as UDFs.

If script can be put into a UDF then it is probably best to make one in place of a SP, simply because

there is more flexibility in how the UDF can be used in queries.

So although we may 'force' the issues somewhat in our examples in these related sections, we will try to

keep to demonstrating what SPs can do and UDFs can't.

Our Example No. 1 :

[ ... incorporating temporary tables ... ]

Our first example will use a temporary table (which is not allowed in UDFs) and will not accept any input

parameters - that will come in the next example.

Since at the moment we have very little else than this 'brief', let us decide on a name for our SP, and I will

suggest "sp_FindAllChildren".

What our SP will do is use the FamilyTree database to find all the data 'pairs' which are parent and child.

Although I later use a view to do this, it can be instructive to make an SP in this stage of our learning.

First of all lets see the query script which will be wrapped into our first SP - see below :-

==========================================================================

Notice in the image above that the script does not require any 'external' data as such. No other values

are required, just the instructions on how to construct the table return dataset.

Here is a start to our new SP, so far we only handle the first part of the dataset, so study carefully the

script of what we have so far :-

=================================================================================

Although the above works in both that it is stored by the DB engine in the database itself, and that

it runs when queried, it could be put in a better format for our purposes, so stud this new form shown

below :-

=================================================================================

As you now see we have a temporary table which we make in our script, as well as a SELECT statement

effectively for the SP itself, so we get a dataset displayed and not just the number of lines affected.

So lets see if we can now include code for our second part of the required dataset !?

Here is what saves and works - along with the results :-

=================================================================================

In the results pane we get a long line of parent / child rows which total 38 in number, below

is just the top and bottom of the list - as a sample :-

.............................................................

.............................................................

================================

I realise that this is slightly 'forced' and that we could have gotten out a results set with the same structure

in an easier way - BUT - it is showing us how to create and use a temporary table which is one thing we are

allowed to do in SPs which we can't in UDFs. It will prove useful at other places in our solution spaces ;-)

Our Example No. 2 :

[ ... passing parameters ... input variables ... ]

For the sake of simplicity, and keeping things as straight forward as possible, we will adapt our existing example

in order to see how to pass in parameters - into our SPROCs at runtime, for them to use the input data during

the execution of the script.

Remember, SPROCs are stored batches. so the parameters as like local batch variables, and they also have a

leading '@' character to remind us of their function / use.

Okay then, let us look at how to run our new SPROC, with a single input parameter :-

=====================================================================

Notice in the image above we have the name of the parameter followed by its value.

And below is the amended script of this new SPROC :-

====================================================================

If you study the image above you will appreciate that it is not difficult to adapt the script to add a second

parameter which we can use to control the upper limit of the person identity for whom to for children.

Once again lets start by seeing now to use the SPROC at runtime, and how to supply the data values :-

==================================================================================================

In the illustrating image above, I have purposefully placed the high limit before the low limit, just to show

that the order does not matter when we specify values in this manner.

More on supplying input data a little further down this example no. 2, but for now let us look at the script

to handle our two limiting values for our '"RangeOf Children" SP. In fact it is really more a rang of parents

which is specified, but remember this 'stuff' is all exploratory so that we get a feel for what SPs are as well

as how they can be used.

Study this script carefully :-

===========================================================================================

No doubt you have noticed in my images that by the presence of 'alter' in place of 'create' that I

frequently take more than one attempt to get my SPROC correct. I often develop it with a few cycles

of try and see, or do and test etc. etc.

We can execute the stored procedure by simply supplying values without variable names - BUT - we

then have to be careful about positioning.

Let us say the ordinal position of the data input is the same as the ordinal position of the declared variables.

There is a one to one correspondence.

Study this set of trial executions :-

=====================================================

So position does matter ! Obviously we get no results for parent IDs between 39 and 13 !!

Okay, you could adjust your SP script and do a test for this in the code you create, BUT, that

is not the point of our discussion at the moment.

Now, lets try a couple of other tests, what happens if we supply no data values into our parameter list ?

==================================================================================================

Oh! that may be annoying as you may wish the SPROC to run with default values if no others are provided.

In that case how does the following work and NOT cause any errors to be thrown ? :-

======================================================================

The simple answer to us getting good and suitable results sets from each of the three queries shown

in the image above, is that we have previously scripted our SPROC to have default values defined.

Look at the following section of script to see how to do this :-

================================================================

All that has simply been done is that a value has been supplied after the declaration.

We can provide default values to any or none of our parameters. And any parameter which has a default

value defined, may be left out from the input value specification.

However, you then need to be careful how you specify those which are provided. I have to say I dislike the

ordinal fashion (it seems old fashioned to me) as it courts disaster - the use of parameter name followed

by value is my preferred way, as the parameter order does then not come into play.

A much more robust approach I feel ;-)

Finally, on input parameters, even with or without defaults, we can then do some initial cross checking of

input data to possibly detect and deal with more subtle variations than can' be handle by the methods shown

above. In all we have many methods to get things as we would like them to be ;0)

See you in the next section on Returning values, as well as Outputting values, from the SPROC itself.

PHH - September 6th. 2010.


Return and Outputs

It is important that you understand just what the return value is, and is used for, as well as any

restrictions on the way the output values can be used in scripting other 'stuff'.

By other 'stuff' I mean how we may use SPROC output variables and values in Reporting Services

scripts for Datasets, and in designing and creating other SPROCS - yes, do remember, we can

execute an SP from another SP, which is one thing you can't do in functions (UDFs).

Our Example No. 2 - 'augmented' :

To get our first glimpse of how OUTPUT parameters work with stored procedures, let us take our

existing "sp_RangeOfChildren" SPROC and add a couple of lines.

Study the images below which are sections of the script in the SP :-

=============================================================

======================================================================

In the two small images above we can see that we have added a parameter called @RowPairs for outputting

the value of the number of rows in the returned dataset.

Notice how the value of '@RowPairs' is provided by the SELECT command on the temporary table.

Getting values back :

It is now important that we see just how to write the syntax to get the value returned to us when we execute

the SP. Study the image below very carefully :-

=========================================================================

And there are a variety of ways to script this as you can well see in the images below :-

=========================================================================

====================================================================================

The best way to get a feel for this is to make your own tests to fully explore all the possibilities

available to the DBA and developer.

One thing you can't do as things stand, and that is miss out the association of the OUTPUT variable.

=============================================================================================

And of course if we have a mismatch of data types we get a 'nasty' result too :((

====================================================================================

I think we have done enough to extend our knowledge and skills on OUTPUT parameters, and how

to use them in an execution call, we now need to move on to the next section where we will look in

some detail on how we can execute a SPROC, from where, and what the syntax may be, depending

on how we intend to use the data.

RETURN :

Before we move on however, we need to say that RETURN is not to be confused with OUTPUT of values

from a stored procedure.

When we talk about a SP returning a value, then literally it is an integer value which is by design supposed

to be an indicator of how things went in the SP execution.

If all goes well then we ought to return a value of zero, ' 0 '. Otherwise it is up to us to make up a list of

value and description pairs to be used as reference.

To follow this through a bit more thoroughly, let us take an example from BoL to see what is going on.

Study carefully the script in the image below :-

===============================================================================

By following through the script in the above image we can see that the SP returns a value of zero or the

other values applicable to the error in the DB engine. These codes are well documented.

Finally, we need to see how to script things such that we can get our hands on the RETURN value.

Here is a test script for the 'SampleProcedure' SPROC shown above :-

===============================================================================================

==============================================================================================

Notice how the declare local variable @ReturnCode is used to pick up the value coming out of the SP.

It is probably time to move to the next section where we see the different ways of executing a SPROC.

See you in the next section.

PHH - September 6th. 2010.


Executing

We may all be aware that when we wish to run a stored procedure we do an 'EXECUTE' or that we

use the command EXEC.

It is not just the case of getting a SP to run, but where and how we place the clause (or phrase) in the

calling command that really affects the results of our scripting, and how effective we may be in the way

we write our scripts and batches.

So how we EXECUTE will effect our style of scripting - so we need to do our homework both carefully

and thoroughly.

However we do need to realise that Execute is a wider issue than this, and that we can even execute a

character string.

EXECUTE (Transact-SQL) :

BoL has this to say :-

Executes a command string or character string within a Transact-SQL batch, or one of the following

modules: system stored procedure, user-defined stored procedure, scalar-valued user-defined function,

or extended stored procedure.

Before we get deeper into running SPROCs and even a UDF, let us see how to use a string of characters.

Here is a suitable example from BoL using the AW database :-

=====================================================================

And now we can see how to run a stored procedure :-

========================================================================================================

So EXEC or EXECUTE can be used, as well as neither if the line is the first in the batch - personally I wish to

forget this option as it can only lead to problems. I consider it sloppy coding in any language to use special

cases, or one-offs. And as you know by now in scripting I like 'INNER JOIN' and don't use JOIN on its own.

To Return a value :

As we discussed in the last section we can script the EXEC command so that we gain a return value for use.

Here is the BoL example (modified!) :-

==============================================================================================================

Note we are not going to another (remote) server in my adapted script.

Execution from a Variable :

This is variation on the command to execute gives the developer some much needed flexibility in his/her scripting

style :-

==========================================================================================================================

But, don't get too carried away as the following does not work :-

================================================

But a little thought on this and a change or two gives us this success :-

==================================================

RECOMPILE to change execution plan :

Another useful technique is that to 'force' a change of execution plan for a SPROC.

Sometimes the developer / DBA may be aware that the input data will cause a radically different set of data to

be processed when the SP is run. When this is the case we can force the DB engine to recompile the script and

form a new execution plan.

Here is a start, the example from BoL using the AW database :-

==================================================================================

Using DEFAULT values :

And now we look at a variation on the way we can specify the default input parameters.

In the last section we looked at using default values, but did not get to specifying them on the command line.

Study the following image to see both what is possible, and what works :-

==============================================================================================

Executing a UDF (scalar) :

Study the following script and results, to see how we may run and scalar function.

This is a UDF from the AW database :-

==============================================================================================

However, one style of scripting to keep clear of is this below - it does NOT work !!!

===================================================================================

It appear as if when we use EXEC or EXECUTE we can't include the input in parentheses.

Strange, but there you are ;0)

How to use Stored Procedure calls within our other TSQL code :

PHH - September 6th. 2010.


Errors as well as Try / Catch

We have already seen in passing in a previous section how we can retrieve the error code from

the database engine using the @@ERROR function.

Let us remind ourselves of how to use this by looking at a simple example in SSMS :-

==================================

================================================

[ ... using SET commands ... ]

PHH - September 7th. 2010.


Further examples

[ ... using SET commands ... ]

[ ... calling other SPs ... ]

[ ... using Try / Catch ... ]

PHH - September 6th. 2010.


Dependencies

'BoL' has this to say :-

Before you modify, rename, or delete a stored procedure, it is important to know what objects depend on the stored

procedure. For example, changing the name or definition of a stored procedure can cause dependent objects to fail

if the dependent objects are not updated to reflect the changes that have been made to the stored procedure.

============================================================

===========================================================================================================

============================================================================================

PHH - July 31st. 2010.


System SPROCs

PHH - September 6th. 2010.


Executing at Startup

'BoL' has this to say :-

Best Practice :

There is no limit to the number of startup procedures you can have, but be aware that each consumes one

worker thread while executing. If you must execute multiple procedures at startup but do not need to execute

them in parallel, make one procedure the startup procedure and have that procedure call the other procedures.

This uses only one worker thread.

PHH - September 6th. 2010.


Chapter 12 - Common Table Expressions

A common table expression (CTE) is basically a temporary 'view'.

After being defined as a CTE this 'view' is used later in the relating query.

CTEs can be used in the same way simple sub-queries can be used.

So using CTEs make the main query much easier to read and understand.

Each single and separate CTE can be run and tested thoroughly, prior to use.

More than one CTE may be included in the same query.

- Unlike subqueries, CTEs can't be nested - no CTE is to reference or use another.

- Like simple subqueries, CTEs must be self contained - they can't reference the main or

outer query.

Summary :

So if each CTE can be tested separately, and return a suitable results set, then it can be

used within the main query.

For correlation you need to use subqueries and not CTEs.

On a personal level, I have started to use CTEs a lot, and find them very convenient to use, and my

scripting stays clear, and easy to read.

Example 1 :

Time for an example to show what this is all about. Let us return to the 'vBikes' example from

earlier work with 'views'.

Study the script below to see how the larger subquery has been removed form the layout of the main

query and how it is therefore more readable :-

===================================================================================================================

In the image above notice just how simple and 'clean' the main query looks.

Also notice how the parameter for the CTE is also an alias for the actual column(s) exposed for

use by the main query.

It will interest you to see that if we highlight the simple subquery within the CTE we get it to run and produce

suitable data - see below :-

[ Obviously this is a superset of the data we filter with the WHERE clause. ]

====================================================================================================================

*** CARE ***

When we are making and sending a 'batch' we need to make sure that there is a semi colon ' ; ' before the start of the

WITH line in the script. Just so that we have an obvious end to the previous statement.

This is one of the few times that ' ; ' is required, but always using it as a statement terminator is a good practice. Both

Microsoft and Paul Nielsen recommend this practise.

Now that we have had success with our first CTE lets move on to a further example in the next subsection, where we

explore more detail of how the CTE can be used to make our life easier.

See you there ;0)

PHH - July 31st. 2010.


Simple CTE example !

Let us start our research into CTEs be looking at a simple example taken from BoL

that uses the AW database.

Basically this simple use allows our main (or final) select statement to be kept nice and

clean, and basically 'simple' - study the code below :-

==============================================================

I have slightly adapted the BoL entry just to make it a little more interesting ;0))

In the next section we will move on to start to see what other benefits we may gain from CTE use.

See you there.

PHH - September 3rd. 2010.


A little more useful ?

This next example (again from BoL) is possibly a bit more useful than our first 'simple' example.

Here we are seeing how a CTE may be used to limit counts, or indeed averages - and in another

but similar example even sums.

Study the script in the image below :-

============================================================================================

Although this works OK, and makes sense, it is NOT a very useful or even meaningful example ;0))

Once we have a feel for CTEs - how they look, and how they appear to work - we can try and provide

a few better examples from my recent client work. But first things first.

Lets move on to see how we can write a SELECT statement to reference a CTE more than once.

See you in the next section.

PHH - September 3rd. 2010.


Referencing more than once

Once again we will start with an example from BoL because it uses a DB which is available to

any user and reader of these notes - and download is both FREE and easy.

We are using AW once again.

In the example below it will not surprise you to see the CTE used more than once, it just 'feels

right' by this stage of our research work.

Study the following code and returned results set in the image below, and once again I have

made a few small changes to make things a little more interesting :-

==================================================================================================================

Taking things further it is not difficult to see that with a suitable example (meaning we have correct data)

we could also have 'self joins'. Let us see if I can find an example from my family tree DB.

The "FamilyTree" database, and 'Self Joins' :

Let us first of all see the structure of the two tables in this simple DB :-

===========================================================================

Okay then, I have just been doing my homework, and getting the database to be available to us in the

tree view of the OE pane, as well as constructing an interesting and working CTE which 'self joins'.

First of all my DB looks like this :-

===========================================

Notice it shows the tables we looked at above, but now they are in their own DB, which was not the case

before this ;-) Just been doing some exporting of tables using the SSMS shell tools.

And now for the interesting bit, study this code below to see two self joins in action :-

=======================================================================================================================================================

and here are the remaining RHS columns - with one column overlap :-

===================================================================================================================

Notice that Laura (in the blue highlighted row) has now got mother details (Julie) and father details (Philip) added

to the end of the original CTE record / row.

So we have effectively join three instances of the same CTE.

For more on the Family Tree database and related useful queries, see Chapter 26 towards the end of the notes.

Let us now move on and we will take CTEs further in the next section.

PHH - September 3rd. 2010.


Recursive use to show many levels

Well, once again I am starting out with following an example from BoL because it uses a standard

database, that of AW.

However, I am not sure at all that it is doing what I recognise as recursion :((

Whoops !!! - must need a fresh cup of tea - YES, just seen how the recursion is working !

In my words - its "joining onto itself from within" having first added one to the level.

I think like all recursion, it will take me a while to get this fully into my brain, none the less, it is interesting

to see in action - so study carefully the image shown below :-

======================================================================================

Once again I have made some small changes to improve the interest 'value', I nearly said 'level' but

under the circumstances that is not a good idea ;0))

Also note that the commented line seems to be the only difference in two separate recursive examples on BoL.

Try un-commenting this script and running it again to see your difference in the results set.

To be fair to both you and myself, I think this is the first part of TSQL which will need some time to sink in. Most

of the rest has been quite straight forward, if not complex - meaning made up from lots of smaller parts etc.

See you in the next section where we may get our brains stretched even further !

PHH - September 3rd. 2010.

Corollary :

If you are developing a recursive query and you are uncertain as to its correct behaviour, then you can provide

yourself with a bit of a safety net so that you don't get stuck in and endless loop.

See the image below to check out the value of the MAXRECURSION setting, and also choose a value at which

you think the recursive process ought to have completed. If you guess too low then you get an error - see below

for this as well :-

=================================================================

Changing the MAX value from 13 to 3 gives us this error :-

=======================================================================================

PHH - September 2010.


Going that little bit futher

Let us build on the previous example we used from 'BoL' to see what else we can gain from

our recursive CTE and main query.

In fact the notes in BoL are a bit of a mess, as certain bits point at the wrong example, etc. etc.

So I am trying to keep the best bits from it, as well as tidying the ends !

The example shown below builds on that from the last section but this time the 'DirectReports' CTE

provides more information than it did before.

Study carefully the script shown in the image below :-

=============================================================================================================================

And once again I have tweaked the code in some small ways so we get more meaning from it.

Remember to add the OPTION for maximum recursive level if you are in doubt of your code.

Now let us progress to the next section and learn some more very useful 'stuff' about CTEs.

See you there.

PHH - September 3rd. 2010.


Anser to our Great grandchild problem ?

Okay you guys, we may be getting somewhere, in looking through this next example ( listed

as 'H' in the 'BoL' notes ) I think I may have hit on a smart way to script a query so we can find

the Great grandchildren of any specified person in our Family Tree database.

But, first things first.

Let us study the script for the CTE and main query which finds the product sub-components for

item '800'. Whatever that may be - ProductAssemblyID = 800 that is.

Here is the script, study it carefully :-

==================================================================================================================

To be honest, there is no substitute for reading through this time and again, as well as trying out the

script in SSMS, and making small changes and running it again etc. etc.

I think after revisiting this problem that I now see a way to adapting this script so that I can make a similar

one to handle grandparents and great grandparents for people in the family tree DB. Keep you fingers crossed !

Family Tree :

Well, I will be honest with you, I have had a go at my grand parent idea, and have got some answers which are

correct but incomplete as such. I think my family tree problem may be more complex that the parts one we saw

earlier.

Below I will show you that I have managed to specify a person (Andrew Gething) and trace his mother and then

her mother. Before we see if this can be expanded to get all parents and grandparents we need to review what

we have achieved.

Study this code carefully - it is my first recursive success, even though it is only partial.

================================================================================================================

In the returned results set I can confirm that Andrew's mother is definitely Annabel, and her mother is Jean. So

Andrew's grandmother is Jean, on his mother's side.

It later on a Friday and my mind hurts, but before I have a break let us see what we get for Theo, identity 30.

=====================================================================================

Well, yes this is good as far as it goes, Theo is the son of my niece, and so his mother is my niece Susie,

and her mother my sister Kathleen, so her mother, and Theo's Great Grandmother, is my mum !

I still have no idea how to get all Theo's great grandparents how. We need to sleep on this one - BUT - we

are getting recursion to work for us, so we have achieved something.

Oh! would you believe it ? I have just scrolled down the BoL notes to see the final example they give, and

it would appear that they are covering the very problem - "Using multiple anchor and recursive members".

Let us leave this at the point of our partial success and follow through to a full conclusion in the next section.

See you there.

PHH - September 3rd. 2010.


The final answer to grandparents etc. ?

It would appear that once I get my Family Tree data correctly entered this recursive CTE seems to

be giving the correct results.

There is sufficient evidence to convince me - who knows the people involved - so I will make a record

of it here for you and others to copy and adapted etc. etc.

Study the script carefully, and if you wonder why it looks different to the code in the BoL example in the

notes it is because they used old fashioned syntax for what we now call INNER JOIN.

I have change the syntax to the modern way of doing things so that it is easier to read and fits in with the

sections in chapters 3,4 and 5 on joining.

=========================================================================================================================================================

Notice the 'Tree Levels' which indicate ancestral 'levels' in the family tree.

So 0 is parent, 1 grandparent, and so on. This seems to be just fine, but I now need to focus on

more data entry.

Good luck with your CTEs ;0))

See you in the next section

PHH - September 3rd. 2010.

Corollary :

If you think having two anchors for our recursion was cool, how about three ?

I just realised that we could also include the person for whom the list of ancestors was generated,

along with a zero 'TreeLevel' value and 'one' for the parent levels.

So study the new style CTE script below :-

==========================================================================================

Because this code produces some duplicates I have had to change the body (or main) select

statement to give on overall acceptable and suitable results set. See this new script in the

image displayed below, along with the results which look OK to me :-

===================================================================================================================================================

Yes, I know some of the ancestors are missing ;-) It is simply because I need to add more data

but at least we have a few more since the slightly earlier sections in this chapter. like Theo's dad

Richard, he will appear soon !

Now, how about a more complex case of finding grandchildren this way ? Since our data only has

mother and father fields associated with a person, we need to go about it in the same way, but

different, if you get my drift ;0))

See you in the next section.

PHH - September 4th. 2010.


Really grand children this time

Okay guys, time for a confession - I have explored this problem over a few hours, and have eventually

found an elegant solution which is nothing like the one I first attempted to make work for me ;-)

I was to try using a function (UDF) from within the recursive CTE, one which would supply a short list

of children - in PersonID format / data type.

I have now scrapped the idea in favour of making and using a view, which simply provides a list of all

family people with a child. Anyone with more than one child is in the list for each child they have.

This means we can use the 'view' as a table, and the whole CTE becomes less complicated and much

easier to read.

Let us study below what I have gotten to work, as far as the 'view' is concerned :-

===========================================================================

When studying the image shown above we notice that it is made in two parts, and then these parts are

joined together (merged is the correct term) by a UNION.

The image above shows the script to alter the view (simply use 'create' for your first attempts in place of 'alter')

as well as the test query to check it out.

So in subsequent script we can use 'ParentChild' as if it is a table.

In the image shown below we can see how this simplifies the CTE as we could have only one anchor if we chose,

and also only one recursive section. The gender (mother / father) issues has been resolved in the 'view'.

Notice that I added a second anchor condition so that the person themselves is added with a zero level - this

could be removed if so desired.

Study this script below very carefully :-

================================================================================================

And just in case anything 'odd' is going on I have re-run the query with a different input parameter. In both

cases the results seem correct to me.

below is the 'cross check' query :-

==================================================================================================

So far so good, but we have not really named our scripts very accurately. What we have created is a script

which finds all the 'Off-Spring' or descendants of a specified person in the Family.

If we wish to have easy to use scripts for Grandchildren and/or Great grandchildren, then we ought to wrap

up this script into something more suitable.

Let us look at a start to this approach in the following image - note however that we have not as yet limited

the returned dataset to just grandchildren :-

==================================================================================================

Testing this with a suitable query we get the following :-

===============================================================================================

To make things as they should be, when we ask for the grandchildren of Ashley (identity 8) we ought to

limit the output to only those with 'TreeL'evel values of 3 - let us see the results as they are now for Ashley,

and then make the changes necessary to the UDF to correct this :-

=================================================================================================

Let us first look at the amended script, and the altered UDF as it now is after making the changes :-

==================================================================================================

And now a test to see that we get what we are after :-

================================================================================================

So you see we now only have the grandchildren, and so we could write a few other UDFs for simply

children, or great grandchildren, and a few others too. Obviously we would tidy up the data supplied,

remove a few columns etc. etc.

PHH - September 4th. 2010.


Simplified ancestors recusion

Having worked on the off-spring problem in so much detail we can see if a similar approach can be

taken to the ancestor problem we have visited previously, in earlier sections.

In fact, without having tried it yet, I think we can use the same parent-child 'view' as we did for the

last section on grand children - when we developed such a simplified script.

And sure enough, after just a few minutes we have a working version to show here. This script as

it is at the moment is called 'Ancestors' and may be used to create other more specific scripts

later.

Let us study what we have created and what the results look like :-

==============================================================================================

Having selected Laura (my daughter) who is identity '1' the results show me and my wife (her parents)

and then the level 2's which are her grand parents - Jean, Peter, Henry and Eva.

So the '3's are Laura's great grandparents.

Using this script as a basis we can now make a new UDF called "GreatGrandparentsOf" as shown

below :-

[ first some good results ..... ]

===============================================================================================

And now the UDF script which was used and saved after an execution in the SSMS shell :-

================================================================================================

*** for the first time change 'Alter' to 'Create' ***

I will show you my extra UDF called "GrandParentsOf" and let you change the one character required as

such, required to make it :-

==============================================================================================

Okay, we have had some success with CTEs and recursion, but I am feeling I have had enough for now

of children, and old folks, as well as myself - AND the wife ;-)

Let us move on, see you in the next section.

PHH - September 5th. 2010.


Chapter 13 - Managing DB Objects

PHH - July 21st. 2010.


Dependencies

============================================================================

==================================================================================

=======================================================================================

PHH - July 31st. 2010.


Checking the Database

DBCC CHECKDB () :

Do a pretty thorough check on the chosen DB using this function / facility.

'BoL' - Checks the logical and physical integrity of all the objects in the specified database by performing

a suitable set of operations.

.......................................................

<< too much to list here ..... >>

.......................................................

=================================================================================================================

...............................................

...............................................

================================================================

PHH - July 28th. 2010.


Which Objects exist ?

Enter topic text here.


Metadata functions

================================================================================================

PHH - July 28th. 2010.


Identifying an Object

=====================================================================================

PHH - July 28th. 2010.


Finding a 'View' definition

First by Script :-

===========================================================

Now by SSMS Menu Options :-

===============================================================================================================================

====================================================================================================================================

PHH - July 28th. 2010.


Listing Columns in a Table

====================================================================================

PHH - July 28th. 2010.


System Tables

===========================================================================================================================================================================

PHH - July 28th. 2010.


Chapter 14 - Create, Alter & Drop with SSMS

PHH - July 24th. 2010.


Chapter 15 - the CASE function

In this chapter we are going to look in depth at the T-SQL CASE function.

This provides a great deal of extra flexibility to how we can use the SELECT statement.

Many of use will be familiar with what we call 'the Case statement' from experience in programming with

languages like VO and dotNet languages - and even some from before that !

However, we must first of all realise that the CASE programmers are used to was a multiple switch, but a

switch which allowed us to chose the logic path of our program code. So the action of our program depended

upon the outcome of the switch statement.

In T-SQL however, the CASE function still acts as a multiple switch, but the outcome is an expression. We can

think of the expression as being a piece of data as it is usually a field / column, or a function of one or more

fields / columns. So it is 'value based' and not logic flow based.

Later in this chapter we will see how this gives us the developers much power and flexibility in how we design

SELECT statements, and how we can build in sufficient options to the specification of a required column.

Remember, columns we place in a SELECT statement can be calculated fields, and this is one area alone

where CASE comes in very handy ;-)

If I were to try and simplify what a CASE is, and what has already been said about it, I would simply say that

it allows us to turn one 'expression' into another.

So if we have an input 'expression' we generate an output 'expression'.

So we need to remind ourselves of what BoL tells us about 'expressions' :-

An expression is a combination of identifiers, values, and operators that SQL Server can evaluate to obtain a

result. The data can be used in several different places when you are accessing or changing data.

Looking at Books on Line (BoL) it has this to say about the CASE function :-

==============================================================================================================

Once again, notice that the text keeps referring to expressions and results - so we are talking values, or 'things' the

end up as values. Not logical flow of program control.

Before we go on to examine examples in detail, in the following sections, let us see the general forms

of the CASE function, as seen in BoL :-

======================================================================

The first 'simple' format of this switch typically takes a field or column data value, and depending

on the value itself, provide an output value to be used outside the function.

So for example, the data content of a single column (for the row in question) could determine which data

value is used in the rest of the statement. We will see examples of this very soon.

On the other hand, the second format of the CASE function examines a set of Boolean expressions, and

depending on the first 'TRUE' evaluation, provide an outcome value.

In both forms of the function the examination stops when the first 'TRUE' evaluation is found.

If no 'ELSE' argument is included then it is possible that no comparison operation evaluates to TRUE, in

which situation the outcome would be NULL.

So if you don't want your evaluation to end as NULL you had better include a suitable ELSE argument.

Summary so far :

To give us a handle on what we may have in front to us with the CASE function, let us say that the first

'simple' form allows us to examine one data field and provide an outcome based on the value contained

within it, whereas the 'searched' form allows us to examine the data contents of a whole set of possible

columns and provide an outcome value based on what is found in the many fields.

This may be a simplification - BUT - it allows me as a developer to see how I can use the CASE function

to solve two very different types of problems I face on a regular basis.

Now let us move on to see some real life data examples of these.

See you in the next section when we view possibilities of using the 'simple' format of CASE.

PHH - July 21st. 2010.


the 'simple' CASE function

As a good place to start, let us take and adapt the 'simple' example from BoL.

I have changed and added to this so it reads better, and also shows the field content of the input

expression for the CASE function. See below, and study the details :-

========================================================================

Notice that when the value of the 'ProductLine' field is NULL we get the ELSE argument coming

into play, and the 'Category' field is set to 'Not for sale'.

It is interesting to comment out the ELSE argument and see the results we then get - see below :-

====================================================================

In this situation it is best to provide an ELSE argument, and this is probably true for most of our scripting.

*** Remember *** You need to carefully order the 'when_expressions' as only the first one evaluating to 'TRUE'

will be used to determine an output value. Without careful though it is possible to have a 'when_expression' which

is never reached, simply because it is always covered by an earlier one - be CAREFUL !

Our own 'Silly' Example :

Let us move on to explore more, and make our first own (somewhat 'silly') example ;-)

Let us use the AW light (AWLT) database, and the 'Address' table.

We are going to generate a new column called 'weather' based on the country where the address refers to. In my 'silly'

example it will be 'rainy' in the UK and sunny everywhere else.

Look at the script and results details below :-

==================================================================

And if we wish to add an entry for Canada we may have something like this below :-

======================================================================

Although this is a very useful function for determining output column data, in some ways this is

also restricting because we need to supply a fixed text 'when_expression' for comparison. If for

example we wanted to use ' like (%King%) ' then we need to use the alternative format for the CASE

function, see the next section.

Staying with the above example for a moment, once we have specified a column then the comparison

value needs to be of a similar data type - but the outcome value may be different, see below :-

=======================================================================

I will leave it to you to try changing the text values and generate errors ;O))

One final change and test :

Before we move on to the 'searched' format of the CASE function, let us try using an expression as

an input argument in place of our field value that we used above.

Look at the image below to see how we have used our 'silly' example to show this :-

=================================================================================

I hope you find some better examples than that used above ;0)) But it does illustrate what an

expression as input can look like.

Let us now move on to the slightly more complex 'searched' form of the CASE function.

See you in the next section.

PHH - July 23rd. 2010.


the 'searched' CASE function

As previously, lets get the ball rolling by using an example from Books on Line (BoL).

In the example we shall use below the CASE function is a set of Boolean expressions, each associated

with an output value. The list of Boolean expressions are evaluated in order first to last, and the first

time a 'TRUE' is found the function finishes by returning the associated expression, or value in this example.

If non of the set of Booleans evaluate to TRUE then the ELSE argument is used to return an expression.

The image below is an example I have slightly modified, study the CASE syntax :-

=====================================================================================================

So in the image above we have a new field (sometimes called a calculated field) which is not part of the

data table itself, and which is a string or character field.

Notice that the data type of all the possible output expressions (values as shown here) are the same, and

are character based.

However, we can easily adapt this so that the output value is an integer, to represent a price 'band'.

=====================================================================================

In the example shown in the image above, we have integer values for the 'price band. I have basically

used a negative to show obviously that we are not now dealing with strings any more.

What we can't do is mix the data types of the possible output expressions.

So in the following image we get an error because the ELSE argument specifies a sting when all the

other 'when_expressions' (values in this case) are integer :-

===================================================================================================

And pushing our example just that bit further, we can change our output values into expressions using

a local variable (see more on local variables in the next chapter), but the data type of the output expressions

is still type integer. See the adapted code below :-

=====================================================================================================

As a final remark regarding this example, it is useful to know that we could be using more than one column

value as part of the Boolean expression. So instead of using 'ListPrice' in all of them, we may use 'CostPrice'

in some others, and even 'SellingPrice' in another. Obviously this all depends on the table data, and whether the

Boolean 'test' expressions make 'business logic' sense.

Let us now move on to another example so we can learn more about 'searched' CASE functions.

Using the CASE function in the WHERE clause :

In the example shown below using the Product table from the AWLT database we are using two separate

fields and combinations thereof to determine the filter condition.

In my slight 'false' situation below, using the price and cost fields in the product table, we can see that we can

have a CASE function used as the value or expression in the right hand side of the WHERE clause condition.

In the image below try to see the WHERE clause as saying :-

" WHERE (StandardCost < 'some_calculated_value' ) "

Study the details below :-

This rather 'set-up' or fictitious example at least actually filters out some of the rows. So it does work ;-)

Of the 295 total rows, we only select 197. The 198 in the bottom RHS is a sum of the 1 and 197 from the

pair of statements in the batch.

One final point to make is that the script 'WHEN StandardCost < 100' actually applies to the situation

in which StandardCost is 50 or more and less than 100, simply because the first Boolean expression, if it evaluates

to TRUE, will cause the CASE function to stop any further evaluation and return a value. So the second WHEN line

will only get evaluated if the first Boolean expression is false.

I think you will be able to see the rest of what is going on, certainly if you have application programming skills

and experience.

PHH - July 23rd. 2010.


places to employ it

The CASE function is a special Transact-SQL expression that generates an output expression.

Since it appears to be able to generate any valid expression then it may be used wherever an expression

can be used.

This idea of a CASE function being effectively a standard expression is supported by the following form

Books on Line :-

An expression can be any of the following :-

·Constant

·Function

·Column name

·Variable

·Subquery

·CASE, NULLIF, or COALESCE

An expression can also be built from combinations of these entities joined by operators.

PHH - July 23rd. 2010.


Chapter 16 - Using local Variables

BoL has this to say :-

Local Variables :-

Allow you to store data for use as input in a later Transact-SQL statement. For example, you can code a

query that needs different data values specified in the WHERE clause each time the query is executed.

You can write the query to use variables in the WHERE clause, and code logic to fill the variables with the

proper data. The parameters of stored procedures are a special class of variables.

Transact-SQL has several ways to pass data between Transact-SQL statements.

One of these is by use of Transact-SQL local variables.

A Transact-SQL variable is an object in Transact-SQL batches and scripts that can hold a data value. After the

variable has been declared, or defined, one statement in a batch can set the variable to a value and a later

statement in the batch can get the value from the variable.

For example:

==============================================================================================================

The Lifetime of the Variable :

The variable lasts only as long as the batch that it is within. Look at the code below to see what we get

when a 'go' is placed prior to the SELECT statement :-

=============================================================

In the image above, there are two batches and the variable is declared in the first one. So when the second

batch is run the variable no longer exists - hence the error message !

Now if we take the use of local variables that step further we can get a range of employees. Study the image

show below of this adapted statement :-

=============================================================================================

*** Notice *** The second commented 'go' I was expecting to cause a problem and still give us errors.

However, I think this is due to me using SSMS 2008 and not 2005. It would appear to me that the code

behind SSMS 2008 has been augmented and that a 'go' at the start of a line within a commented section

no longer is picked up by the pre-processor as a batch terminator.

I will leave it to you to try this out, but I am sure that the training DVD's I watched warned me of this.

Maybe it was a problem in SQL Server 2000, as my SSMS 2005 works the same way as that shown

above. Oh! well, one less thing to worry about.

PHH - July 21st. 2010.


Chapter 17 - Transactions & Rollback

'BoL' has this to say about Transactions (Transact-SQL) :-

A transaction is a single unit of work.

If a transaction is successful, all of the data modifications made during the transaction are committed

and become a permanent part of the database.

If a transaction encounters errors and must be cancelled, or rolled back, then all of the data modifications

are erased.

========================================================================================

Before we get into detail on Transactions and Rollback, let us take a minute to view the

error log for the AdventureWorks DB.

======================================================================================================================

It would appear that for my system using the AW database, the log is currently empty.

We can soon change that ;-) Lets make a few errors ! Well, one at least, just as a start :-

===================================================================================================================================================================

And after running the transaction a second time, just a few minutes later, we see below

that we now have two entries in the error log.

=================================================================================================================================================================

I will leave it to you to work out how long there was between my first error and the second one ;O)

Also, you will notice that there is an 'ErrorLog' table in the 'dbo' schema table list as seen below :-

=====================================================

Summary :

To complete this first section on Transactions, Rollback and error logs, we simply need to say that if

we have two error rows in our table then there must have been two Rollbacks processors completed, as

when an error is caught both a Rollback and an entry into the error log table take place.

Obviously it is the SPROC called 'dbo.uspLogError which stores the error details to the relevant table.

Below is shown the line which does this :-

========================================================================================

We can try and see some of the relevant code from this SP (Stored Procedure) if we get SSMS to script it

for us to a new query tab - see below :-

========================================================================================================================================

Study the script in the image below, and see in particular the section which inserts the error

values - from system functions - into the new row placed into the 'ErrolLog' data table :-

=========================================================================================================================

PHH - July 21st. 2010.


What are Transactions ?

Once again let us see what 'BoL' has to say :-

A transaction is a sequence of operations performed as a single logical unit of work.

A logical unit of work must exhibit four properties, called the atomicity, consistency, isolation,

and durability (ACID) properties, to qualify as a transaction.

Atomicity :

A transaction must be an atomic unit of work; either all of its data modifications are performed, or

none of them is performed.

Consistency :

When completed, a transaction must leave all data in a consistent state. In a relational database, all

rules must be applied to the transaction's modifications to maintain all data integrity. All internal data

structures, such as B-tree indexes or doubly-linked lists, must be correct at the end of the transaction.

Isolation :

Modifications made by concurrent transactions must be isolated from the modifications made by any

other concurrent transactions. A transaction either recognizes data in the state it was in before another

concurrent transaction modified it, or it recognizes the data after the second transaction has completed,

but it does not recognize an intermediate state. This is referred to as serializability because it results in

the ability to reload the starting data and replay a series of transactions to end up with the data in the

same state it was in after the original transactions were performed.

Durability :

After a transaction has completed, its effects are permanently in place in the system. The modifications

persist even in the event of a system failure.

PHH - August 4th. 2010.


How do they happen ?

Specifying :

It is up to the programmer / developer to be responsible for writing script which ensures logical consistency of data.

The 'scripter' writes script which starts and ends transactions to ensure this consistency.

In the defined script is a sequence of data modifications which after completion leave the data in a consistent state

with regard to the business rules being applied or adhered to.

The modification statements are included in single transaction so that the database engine itself, can enforce the physical

(or real) integrity of the transaction as a whole.

So the programmer must get the script right but it is then up to the database engine (software application) to make it 'stick' !

Enforcing :

It is the responsibility of the database engine to provide ways of ensuring the integrity of each transaction. So that 'real' or

physical integrity is maintained.

These are provided by the DB 'engine' (remember this is software) in the form of :-

- Locking facilities - which preserve isolation

- Logging facilities - which ensure durability

[ Even after hardware / software system failure, the instance of the 'engine' uses the transaction logs at restart to roll back all

uncompleted transactions to the point just before they were started - before system failure. ]

- Transaction management features that enforce atomicity and consistency. Once a transaction has started it must be

successfully completed - if not then the 'engine' undoes any data modifications made since the start of the transaction.

To put this more correctly, the transactions are all placed into a duplicate set of tables, and only once the transaction is

completed do the changes get 'committed' as such. So it is more of a matter of 'NOT committing' data changes if a

transaction fails, rather than a matter of 'undoing' them.

PHH - August 4th. 2010.


Modes

SQL Server operates in the following transaction modes :-

Autocommit transactions

- Each individual statement is a transaction.

Explicit transactions

- Each transaction is explicitly started with the BEGIN TRANSACTION statement and explicitly ended

with a COMMIT or ROLLBACK statement.

Implicit transactions

- A new transaction is implicitly started when the prior transaction completes, but each transaction is

explicitly completed with a COMMIT or ROLLBACK statement.

Batch-scoped transactions

- Applicable only to multiple active result sets (MARS), a Transact-SQL explicit or implicit transaction

that starts under a MARS session becomes a batch-scoped transaction. A batch-scoped transaction

that is not committed or rolled back when a batch completes is automatically rolled back by

SQL Server.

PHH - August 4th. 2010.


a Simple Test

===============================================================================================

End of Person table :-

End of Family table :-

==================================================================

==============================================================================================

=====================================================================

PHH - August 8th 2010.


a Clear Example

Let us try and make all this theory make more sense by picking a good example.

Once we have a suitable example we can script it and follow it through.

Let us look at the AWLT database, and specifically the Customer and related tables.

There are three tables which seem to go 'hand in hand'. These are 'Customer', Address,

and 'CustomerAddress'.

Let us say that it is company policy that any new customer to the software system, has to have

an address entered at the point of entry. A new customer can't be taken without an address.

The software GUI will be written to accept suitable data and check that entry details are valid.

However, it is then up to the programmer / scripter to make a suitable script 'batch' so that each

of the three tables has a suitable (and related) row added to it.

Writing a Pseudo Script :

Let us assume that at the point this script is to be run the full customer details have been entered

at the GUI screen - address, contact, personal, ..... the lot !

We now need to 'roll together' the three necessary actions and check for errors as we go - the script

'flow' will be something like this :-

BEGIN Transaction;

INSERT into Customer( ... << values >> ... )

if an error

BEGIN

ROLLBACK Transaction;

RAISERROR( ' Customer transaction error ' .... );

RETURN;

END;

INSERT into CustomerAddress( ... << values >> ... )

if an error

BEGIN

ROLLBACK Transaction;

RAISERROR( ' CustomerAddress transaction error ' .... );

RETURN;

END;

INSERT into Address( ... << values >> ... )

if an error

BEGIN

ROLLBACK Transaction;

RAISERROR( ' Address transaction error ' .... );

RETURN;

END;

COMMIT Transaction;

As you can see above in the pseudo script, there are three separate and distinct sections to this.

After each attempted row insert into a data table we check for an error. If an error has occurred the

whole transaction is stopped and the 'engine' will not put the changes in place in the main tables.

In the script the only way the execution of the script gets to the COMMIT stage is if there are no

errors raised at all in the data inserting process.

Obviously, as soon as an error does take place - the first one - the overall process is stopped and

a message about the error entered into the log.

Recording Errors :

Just as it is the responsibility of the programmer to structure the script correctly for the whole single

unit of work, it is also his/her responsibility to record any errors into a suitable log, for reference later.

The detail of the error message will depend on the business requirements for the particular application.

Programmer Responsibility :

It would appear that we can use and access a database without using Triggers, if that is how our

design works out.

However, if we are doing a reasonable amount of normalisation of the business data then we are not

likely to be able to get around one or more situations like that shown above in the pseudo script.

So I suppose I am saying that there is absolutely no way around the necessity to use and script

transactions, unless of course you are simply designing reports - which generally only read existing

data.

For these reasons we MUST look more closely at some good examples of working scripts with genuine

data tables, so here goes :-

The 'Family Tree' :

This sample database has been primarily setup to allows us to explore the SELF JOIN in many different

ways. However, since it is simple in concept and both table and column structure, it should prove ideal for

the purposes of looking at Transactions and Rollbacks.

At the moment of writing this section the 'Family Tree' is actually just two tables added into the AWLT

database, but I will make a new and complete DB for just the FT itself. This will be used to give suitable

information in other sections, like those on moving tables across databases, and a few more too.

For now lets look at both the structure of the two tables, and also their relationship. Lets start with a DB

diagram :-

==================================================================================================================================

The diagram in the image above shows that three fields in the 'Family' table are Foreign keys, and related to

the Primary key of the 'Person' table.

Let us now look at the Person table as seen in the OE pane :-

===============================================

The image of this main table in our database, shows we can only have two fields (MiddleNames and

MaindenNames) which may be let as NULL. the 'PersonID' field has to be unique too.

Now the related 'Family' table looks like this :-

===========================================

This shows that with the relationships of the three Foreign key fields, that 'PersonID' has to be a value,

but the other two may be left NULL. However, when a value is specified it must be already in the 'Person'

table.

So to look at that another way, 'Mother' and 'Father' fields may be left NULL but if they are given a value

the value must be one of the unique values existing in the 'Person' table.

This may look pretty tight to you as validation entry, BUT, as yet there is no check in place to say that

an existing unique identifier can't be used which although a 'Father' entry is pointing at a person record

which is a female, or where the date of birth is not suitable to be a parent of the new 'Person' entry.

[ This looks to be a good candidate for a 'Trigger' example in Chapter 18. We could add these extra constraints

at the point of entry, so that unsuitable persons could not be entered as a Father or Mother identity. ]

Let us get back to 'Transactions', since that is the purpose of this section.

Before we can script a 'Transaction and Rollback' batch we need to fully explore the ways to successfully

place data into the two tables we will use. We need to know what will work (without making errors) and

what will not work.

Here is a TSQL statement to insert the minimum amount of information for a new Person entry :-

======================================================================

Notice that we can insert an identity because the IDENTITY_INSERT is set to OFF.

It will be useful to leave it that way for now - just realise that when set to ON then the SQL Server will

allocate the unique numbers for us. Below is the result of our inserting attempt :-

===========================================================================

Although the IDENTITY_INSERT is set to OFF, we can't violate the 'Primary Key' constraint by entering

the same ID twice - see below :-

=====================================================================

Exploring data insert for the Family table :

Moving on to the 'Family' table, let us try an INSERT but choose a value for Mother which does not

exist in the DB yet.

=================================================================================================================

Obviously a unique person identifier of 98 does not exist, and so we got an error.

Changing this to an existing female person identifier of 6 we get no errors, and the data is entered into

the table as we require - see below :-

======================================================================

Checking the data in the table with a select statement we see the values shown below :-

=====================================================================

Moving on to the full Transaction script :

Okay then, enough of pseudo code and the likes - let us get into making TSQL script which actually does

the business. Look at this below after if has been executed - study carefully :-

=====================================================================================================================================

After running the pair or INSERT statements above, let us see what we have in the data tables :-

=============================================================================

First above we have the Person table as it now stands, and below the Family table :-

====================================================================

If you find you have some trial goes - for testing - where things go wrong then try these few script lines

to put things to rights :-

===================================================================

=========================================================================================

====================================================================================================

Now that we know we are getting errors thrown for the rights things, we can stop them being output

and instead have them saved into an error table. One already exists in the AW and AWLT databases.

If you wish to do the same for your own DB then you will have to copy the structure of this table to your

own DB. See more on this in 'ClickStartDB'.

Anyway, let us see some suitably stored - or captured - error messages. Here below is what I have just

made when I ran similar test as those with the RAISE_ERROR() script lines active.

==================================================================================================================================================================

You can see the start of some meaningful error messages in the last of the visible columns.

You can also see that I have had quite a few tries in getting all and every bit of my script to work correctly.

It is a bit tricky doing this stuff with little or no experience to call on ;-)

Study the script below to see just what was used to generate these rows in the error log table :-

==================================================================================================================================

[ *** note *** if you don't have the stored procedure 'uspLogError in AWLT, then go into AW and find it, and then

do a DROP and CREATE which will make the script for you - then all you need do is change the name of the DB

being used in about two or three places ]

PHH - August 4th. 2010.


a More Complex Example

Example 1 : (sticking with customers) :

If we look at the column specifications in the OE pane for each of our three tables we will be able to see just

which fields we can leave out, and which ones we must specify - see below :-

===================================================================================================================================================================

It looks as though our task is going to be made a little easier in that a few fields can be left as NULL.

Let us now try a test from SSMS for inserting a row into each of the tables in turn.

======================================================================================

===================================================================================

======================================================================================================================================================================================

PHH - August 8th. 2010.


Chapter 18 -

Let us start as we quite often do, by looking at what 'BoL' has to say :-

Microsoft SQL Server provides two primary mechanisms for enforcing business rules and data integrity: constraints

and triggers.

A trigger is a special type of stored procedure that automatically takes effect when a 'language event' executes. SQL

Server includes three general types of triggers: DML triggers, DDL triggers, and logon triggers.

It would seem to me that application program developers from the Windows environment will be quite at home here !

We are used to having event handler sections of code which are used by the system when certain GUI events happen,

or are triggered. Well, this is a very similar parallel situation.

The stored procedure is run as a direct response to a 'language event' firing, or taking place.

Let us look at the three types of Triggers in a little more detail :-

DDL Triggers :

DDL Triggers are a special kind of trigger that fire in response to Data Definition Language (DDL) statements. They

can be used to perform administrative tasks in the database such as auditing and regulating database operations.

DDL triggers, like regular triggers, fire stored procedures in response to an event. However, unlike DML triggers, they

do not fire in response to UPDATE, INSERT, or DELETE statements on a table or view. Instead, they fire in response

to a variety of Data Definition Language (DDL) events. These events primarily correspond to Transact-SQL statements

that start with the keywords CREATE, ALTER, and DROP. Certain system stored procedures that perform DDL-like

operations can also fire DDL triggers.

Logon Triggers :

Logon triggers fire stored procedures in response to a LOGON event. This event is raised when a user session is

established with an instance of SQL Server.

Logon triggers fire after the authentication phase of logging in finishes, but before the user session is actually

established. Therefore, all messages originating inside the trigger that would typically reach the user, such as error

messages and messages from the PRINT statement, are diverted to the SQL Server error log.

Logon triggers do not fire if authentication fails.

DML Triggers :

DML triggers are invoked when a data manipulation language (DML) event takes place in the database. DML events

include INSERT, UPDATE, or DELETE statements that modify data in a specified table or view.

DML triggers are useful in these ways:

- They can cascade changes through related tables in the database; however, these changes can be executed

more efficiently using cascading referential integrity constraints.

- They can guard against malicious or incorrect INSERT, UPDATE, and DELETE operations and enforce other

restrictions that are more complex than those defined with CHECK constraints.

- Unlike CHECK constraints, DML triggers can reference columns in other tables. For example, a trigger can

use a SELECT from another table to compare to the inserted or updated data and to perform additional actions, such

as modify the data or display a user-defined error message.

- They can evaluate the state of a table before and after a data modification and take actions based on that

difference.

- Multiple DML triggers of the same type (INSERT, UPDATE, or DELETE) on a table allow multiple, different

actions to take place in response to the same modification statement.

My General Comment on Triggers :

It would appear to me that many of us coming to SQL can do an awful lot before we ever see a need for using a trigger.

To me this is almost paralleled with GUI design and use. The more experienced the developer becomes the more

extensively they use events on GUI controls.

After surviving with few, they go looking for more, and for interesting an varied events too ;-)

I strongly suspect that Triggers are like this, once you get into using them they become routine.

My suggestion is to 'walk before you try to run' ! Wait till you have a definite need for a Trigger before trying to use one.

Like events in GUI, I suspect there are many unforeseen consequences of trying to use something you don't fully

understand - so tread carefully. Particularly if it is your clients DB.

And when you do start with Triggers, start simply at first, then become more ambitious with time.

Another 'feeling' of mine is that Triggers were provided way back when, and now things have moved on and certainly in

the world of business object designed applications, the need for Triggers has lessened. In fact some of use application

developers may never need to use them at all. I for one would prefer to have my business logic in my classes and

methods, not residing in the database.

On the other hand, web services may still be a different ball game, as the client PC and the database engine are separated

by the internet connection and its related performance. We may need to think carefully on this one.

See you in the next subsection where we will see what Triggers may do for us, and how to create and use them.

PHH - July 24th. 2010.


DDL

[ DDL - 'Data Definition Language' ]

DDL triggers can be used for administrative tasks such as auditing and regulating database operations.

Use DDL triggers when you want to do the following:

- You want to prevent certain changes to your database schema.

- You want something to occur in the database in response to a change in your database schema.

- You want to record changes or events in the database schema.

DDL triggers fire only after the DDL statements that trigger them are run. DDL triggers cannot be used as INSTEAD OF triggers.

PHH - July 24th. 2010.


DML

[ DML - 'Data Manipulation Language' ]

A DML trigger is an action programmed to execute when a data manipulation language (DML) event occurs in the

database server. DML events include UPDATE, INSERT, or DELETE statements issued against a table or view.

DML triggers are used to enforce business rules when data is modified and to extend the integrity checking logic

of Microsoft SQL Server constraints, defaults, and rules.

*** Note *** In today's parlance, the Trigger should be the DML 'event', and the procedure which is run in response

to the Trigger firing should be called the 'Trigger handler'.

Explaining AFTER triggers and INSTEAD OF triggers :

- AFTER triggers are executed after the action of the INSERT, UPDATE, or DELETE statement is performed.

Specifying AFTER is the same as specifying FOR.

AFTER triggers can be specified only on tables.

- INSTEAD OF triggers are executed in place of the usual triggering action. INSTEAD OF triggers can also be

defined on views with one or more base tables, where they can extend the types of updates a view can support.

It would appear to me that once again we may be being squeezed by the change from old to new here.

Cascaded Referential Integrity constraints seem to be the more modern (and approved?) way of doing what was done

by 'INSTEAD OF' triggers.

If you are a new guy and want to check out Triggers I would start with limiting yourself to the AFTER variety !

PHH - July 24th. 2010.


Logon

PHH - July 24th. 2010.


Chapter 19 - Schemas

Let us start this chapter by seeing what Paul Nielsen's 'Bible' has to say :-

"A schema is an object that exists purely to won database objects, most likely to segment a large

database into manageable modules, or to implement a segmented security strategy.

Typically, and by default, objects are owned by the 'dbo' schema. The schema name is the third part

of the four-part name :

' Server.database.schema.object '

If you use custom schemas, other than 'dbo', then every query will have to specify the schema. That is

not a bad thing, because using a two-part name improves performance, but always typing a long schema is no fun.

To list all the schema of the current database using script, query the 'sys.schemas' catalog view. "

Paul actually wrote this a number of years back, and now from my experience with my current clients, and their

real life DB situation, I even include the server as well as the schema in the object naming.

Let us try out this method of listing our schemas :-

[ *** Whoops ! - there is an error in his text, he has missed out the ' . ' (point). ]

====================================================================================================================

In the image above we are using a view from the 'System Views' folder in the OE pane.

And since we are now into longer names for our objects we ought to do this :-

==========================================================

It may also interest you to know that the context menu is different for system views to that shown

for user views. The image shown below has the 'scripting' option missing ! - shame, was wishing to see

what might be there ;0)

=====================================================

And now by seeing what 'BoL' has to say :-

A schema is a container that holds tables, views, procedures, and so on. It is inside a database,

which is inside a server. These entities fit together like nested boxes.

The server is the outermost box, and the schema is the innermost box.

It (the schema) holds all of the securables listed below. But it cannot hold another box.

Type - TYPE

XML Schema Collection - XML SCHEMA COLLECTION

Table - OBJECT

View - OBJECT

Procedure - OBJECT

Function - OBJECT

Aggregate - OBJECT

Constraint - OBJECT

Synonym - OBJECT

Queue - OBJECT

Statistic - OBJECT

Every securable in a specific schema must have a unique name.

The fully-specified name of a securable contained by a schema includes the name of the schema that contains it.

Thus, a schema is also a namespace.

In SQL Server 2005 and SQL Server 2008, a schema is both a container and a namespace.

Each schema is now (after SQL 2000) a distinct namespace that exists independently of the database user who created it.

In other words, a schema is simply a container of objects.

A schema can be owned by any user, and its ownership is transferable.

PHH - July 28th. 2010.


Security - a strategy !?

In the list of schema for the AdventureWorks database we see the following handful of schemas

within the list :-

======================================================

It is quite conceivable that personnel working in the Sates department need only access to DB objects

in that department, and not those in Purchasing, Production, and certainly 'Human Resources' (HR).

Often confidential information is handled by HR and this needs to be kept well away from guys in

Production and Purchasing etc.

Views, functions and procedures required by 'any and all' can be placed in 'dbo' for general use.

Obviously, once the database has been modularized, or segmented in this way, we then need to

apply access rights to users - otherwise everyone would still have access to all schemas ;-)

Remember our remit here is not about security and limiting access, but learning T-SQL, so we must

not go too far with this 'stuff'.

Lets just say there are ways to GRANT and DENY permissions on a schema for User, and this can be

mapped to a Windows login or group.

Quite honestly, the responsibility of security is that for my client, his DBA, and their third party support

company to worry over, I have to concern myself with getting my T-SQL correct, and designing and testing

the reports they require.

So it would appear that we have probably learned enough about schemas as far as accessibility is concerned.

We just need to design the DB with schemas in a modular fashion.

Now lets go and look at our next section.

PHH - July 29th. 2010.


Adding a new schema

==========================================================================================================

=================================================================================================

=============================================================================================================================================

==========================================================================================================================================================

==================================================

=====================================================================================================================================================================

======================================================

PHH - July 29th. 2010.


Chapter 20 -

Chapter 20 - different kinds of

Once again, let us start by seeing what 'BoL' has to say about 'Special Table Types'.

Special Table Types :

Besides the standard role of basic 'user-defined' tables, SQL Server provides the following types

of tables that serve special purposes in a database :

- Temporary tables

- Partitioned tables

- System tables

- Wide tables

It would appear that we need to know a little more about each of these, just so we know what is

available to us.

Temporary Tables :

There are two types of temporary tables: local and global.

Local temporary tables are visible only to their creators during the same connection to an instance of

SQL Server as when the tables were first created or referenced.

Local temporary tables are deleted after the user disconnects from the instance of SQL Server.

Global temporary tables are visible to any user and any connection after they are created, and are deleted

when all users that are referencing the table disconnect from the instance of SQL Server.

Partitioned Tables :

'Partitioned Tables' are tables whose data is horizontally divided into units which may be spread across more than

one filegroup in a database.

Partitioning makes large tables or indexes more manageable by letting you access or manage subsets of data

quickly and efficiently, while maintaining the integrity of the overall collection.

Under a partitioning scenario, operations such as loading data from an OLTP to an OLAP system, for example,

can take seconds instead of minutes or hours in previous releases.

Maintenance operations that are performed on subsets of data are also performed more efficiently because they

target only the data that is required, instead of the whole table.

Partitioning a table makes sense if the table is very large or is expected to become very large, and

either of the following is true :

- The table contains, or is expected to contain, lots of data that is used in different ways.

- Queries or updates against the table are not performing as you expect, or maintenance costs exceed predefined

maintenance periods.

Partitioned tables support all the properties and features associated with designing and querying standard tables,

including constraints, defaults, identity and timestamp values, triggers, and indexes. Therefore, if you want to

implement a partitioned view that is local to one server, you should implement a partitioned table instead.

For information to help you understand, design, and implement partitioned tables, see 'Partitioned Tables and Indexes'.

System Tables :

See the subsection on this ... for most of us coming to SQL Server and TSQL scripting it is beyond our current needs

to be delving into such tables. Another time and place perhaps ?!

Wide Tables :

A wide table is a table that has a defined column set. Wide tables use sparse columns to increase the total of

columns that a table can have to 30,000.

The number of indexes and statistics is also increased to 1,000 and 30,000, respectively.

The maximum size of a wide table row is 8,019 bytes.

Therefore, most of the data in any particular row should be NULL. To create or change a table into a wide table,

you add a column set to the table definition.

The maximum number of non-sparse columns plus computed columns in a wide table remains 1,024.

By using wide tables, you can create flexible schemas within an application. You can add or drop columns whenever

you want.

Keep in mind that using wide tables has unique performance considerations, such as increased run-time and

compile-time memory requirements.

Once again, I think many of us guys will never get to Wide Tables, simply because we will have no need.

PHH - August 7th. 2010.


User Defined

================================================================================

=========================================================

==========================================================================================================================================

PHH - August 11th. 2010.


Renaming Tables

Although this section is entitled 'Renaming Tables', it is really about renaming DB objects. A table

is just one of the types of DB object which exist, and which may be renamed if the user so wishes it.

Caution :

One BIG word of caution however, renaming DB objects can easily break things within the database, so

you MUST know both exactly what you are doing and also do it VERY sparingly.

So far I have only renamed tables, and don't expect to need to rename other types of DB object.

Also, the DB in which I am doing the renaming is really just a receptacle for copies of datasets which are

being changed in a 'Data Cleansing' exercise - so it is not really an operational DB as such.

So to summarise - BE VERY CAREFUL with renaming - that is why Microsoft have made it difficult to

do easily. It has to be done through a System Stored Procedure ('Sys Sproc') which need administrator

rights to run.

Below are shown a couple of images of script I have used to make a working script to rename a table

'dumped' into a safety DB which needs to be 'datetime' stamped so that the process could be repeated if

necessary.

The first part of the process is covered in the section two on from here, where a table is copied form the

source DB into a 'safety' DB so that data is not lost during the Data Cleansing process.

The first image is without the 'datetime' stamp.

=============================================================================================================================================================================================

And now the case where the 'datetime' stamp is added to the table name :-

===========================================================================================================================================================================================

There is only one drawback from doing things this way - there could be a few seconds where the

datetime portion of the table name stays the same. You would need to be quick however ;O))

The reason we do things this way at my client site is so we never delete, remove, or destroy data. We

just move it sideways - and yes, accessing and re-using it may be another possible nightmare, but at

least it is still there somewhere !

See you in the next section.

PHH - August 11th./29th. 2010.


Making from Excel data

We can do this in a couple of ways, one with TSQL scripts and the other, possibly more flexible

way is by using the features of SSMS version 2008 - which also works when using DBs attached

instances of the MSSS 2005 type.

So if you are a regular user of SQL Server 2005 and don't have SSMS version 2008, then I strongly

suggest you download a free copy from Microsoft.

Let us now look in more detail at the two different approaches to importing Excel data.

Using the SSMS context menu import feature :

To explore this feature we will use the 'SafetyWarehouse' database as our target, so below I have

right clicked the item line in the OE pane :-

===========================================================================================

Clicking the Import Data .... option brings up the following dialog screen :-

Clicking the 'Next' button takes us to a screen where we start to get at some 'action' :-

=================================================================================================

The drop list to allow us to select a data source has the Microsoft Excel option available to us as shown

in the image above. Once this is selected we get the following :-

=================================================================================================

So selecting to browse we look for the Excel file required :-

After we select the highlighted file using 'Open' we get the following :-

=================================================================================================

The system has obviously picked up that the file is of a 2007 Version of Excel, as the lower drop list

changed without my intervention.

We will also leave the check box checked so that existing column names in the first data row will be

used as field 'titles', or names.

Let us see what we get in the following dialog :-

=================================================================================================

Notice that the Database is already selected (although it could be changed) since we had first right clicked

this item in the OE pane.

So, what do we get when we select the 'Next' button ? Below is what I get :-

=================================================================================================

This seems a little 'odd' for the purpose we are intending, so just accept the default and see what happens

next as we progress .... :-

=================================================================================================

This is when you need to know the Excel work book ! I happen to know I only wish to make sheet 1 into

a new table, so I have checked only the first option.

At the bottom of this dialog we see that the 'mappings' option is NOT available to us :-

And clicking the 'Next' button brings up this dialog :-

==================================================================================================

This is another screen which does not mean a great deal to me at the moment, so I leave the default

and choose to run immediately. And at this point we get a summary screen, and a last chance to

return to earlier screens to change any required details - here is what I get :-

=================================================================================================

And selecting to click the 'Finish' button does this below :-

=================================================================================================

The blue link is not very informative since no errors took place !

So I clicked the 'Close' button, and all was done - completed.

REMEMBER - to refresh the table list in the OE pane by right clicking the 'Tables' item as

shown below :-

==========================================================

Now, before we look at the original Excel work sheet, let us see the results in the OE pane :-

=========================================================

And scripting a query we see this :-

============================================================================================================================

So finally, let us look back at where we extracted the data - the original Excel sheet :-

============================================================================================================================================

Looks as though we have been totally successful in our data copying to a new table.

Good luck with your efforts ;0)

Additional Information :

There is one place we can opt to change the table name, and when this is done we also get a chance

to change the 'mappings' or column header names. See below to another alternative to what we saw

above in the default choices :-

=================================================================================================================================

Hope this extra information is useful to you - the best thing you can do now is go and

try out a few combinations and possibilities with your own data and Excel sheets.

Using TSQL scripts with CVS files :

Here we will look at how to make a table from a standard CSV file which may have been created

from a third party application, or Excel itself.

The first thing we need to do is remove the first row if it has the column headers, and then re-save the

file with a secondary filename of '.TXT' so that the suggested method below will work.

[ *** note *** it will work by leaving the '.csv' as it is, but I prefer to make the change as I have edited the

first line of the standard CSV file that was sent to me originally - so '.txt' it is ]

Here is the start of my raw data shown in Notepad :-

The data shown above is 'friendly' to commas within the data fields, BUT, in other circumstances we

may be better to use something more like a double pipe character - such as '||' for our separation.

We will also need to create our own table structure which will correctly reflect the data types for

each of our fields, or columns.

Let us now look at the script to make this all happen :-

================================================================

And in our refreshed 'Tables' section of the Object Explorer pane we see this :-

=================================================================

And after a quick query we see new data like that below :-

===========================================================================================================================

This method gets particularly useful when we have data in dotNet applications and we wish to make

a text file in CSV 'general' format, and by using a different separator we can allow our data to have

embedded commas and quotes, both single and double.

See the next subsection for this approach.

Good luck converting :O))

PHH - August 29th. 2010.


Copy a Table to an Excel file

This exporting of table data can be done directly from the SSMS 'shell' of version 2008.

Either a Table, a View, or a query dataset can be exported to an Excel worksheet using the context menu

and a set of dialog screens.

Let us use the 'SafetyWarehouse' DB as our data source. Below we see an image of bringing up the

context menu after right clicking the DB item in the OE pane.

====================================================================================

Selecting the 'Export Data ...' option we get the wizard front screen as shown below :-

When the next button brings up the 'Choose a data Source' dialog shown below, we need to make

sure the correct DB is selected - here it is "SafetyWarehouse" :-

Choose 'Next' to bring up the 'Choose a Destination' dialog screen and look through the options

shown in the drop list - see below :-

=================================================================================================

Once you have selected the MS Excel option use the browse button to select a folder and filename for

the output data when the exporting takes place.

This is what mine looks like here :-

From experience I would suggest you leave the Excel version as it is - showing "97-2003".

Certainly try this for your first attempt ;-)

I now see this below :-

=================================================================================================

Click the 'Next' button at the foot of the dialog screen. You get the following screen but

I am not sure how appropriate this is to our particular case - so just leave the default option

as Copy and press 'Next'.

This is what I now see on my monitor :-

================================================================================================

Okay ! I see, the previous dialog was to allow us to make a dataset from a query, here we can select

any of the Views and Tables in the selected DB. Above we see onl tables, as that is all there is in this

sample DB used for our testing.

Before we get too ambitious, let us get one table to export to Excel before we 'push the boat out' !!

Having selected the 'ProdsCodesLU' table we can click 'Edit Mappings ... ' and change the names

of a few columns just for experience - here is what I did :-

*** Notice *** that I have change some characters in the 'Destination' column - removed some underscores

as well as making a few upper case characters too.

Close the screen with the 'OK' button and then press 'next'. This is what I now see :-

For this trial we can leave the On Error and On Truncation settings as they are, but another time

we may need to change these, and do a few trials - for experience sake.

For now lets keep going and press 'Next'. This is what I see on my monitor :-

=================================================================================================

For our experiment we need to run things immediately, so leave the default as it is and click 'Next'.

Finally, we get a summary screen so we can check out what is to be done - AND - we still have an

option to be able to use 'Back' to return to make changes.

For now click on 'Finish'. See below for my results, as well as a warning :-

Well, having read the warning information I am not sure what the fuss is about - BUT - you will

find issues like this when you try yourself to export data - it is a life skill ;-)

We seem to have a file in the target folder, and it has the size we may expect for 70 rows of data :-

======================================================================================================================================

Now we need to double click the file name in the WE screen to see what Excel makes of it :-

Well, looking at the image above, we seem to have everything that I expected - 69 data rows and the

first row extra with the column headers as edited in the mapping dialog screen.

I now think you can go and try your own tests, to see just what you can achieve ;O))

Good luck, see you in the next section.

PHH - August 29th. 2010.


Making from a dotNet application

Here are some lines of Vulcan code from a recent working application of mine.

It reads some text input file and converts the data into suitable fields segments for forming into a

single record (or row), which is suitably interspersed with a double pipe separator, as seen on line 171.

Although the code shown below does not tell the whole story, it has some useful parts to guide us when trying

to do something similar.

=====================================================================================================

=====================================================================================================================================

===========================================================================================================

Now let us try and convert the text file made by my Vulcan code, see below for the results :-

========================================================================================

As we can see in the image above, although we have followed a similar method to the CSV one shown a

little earlier, we do have a couple of warnings - about row 546 and row 3427, and both these warnings were

caused by an entry for the 'basic_price' field item.

If we look at the data in the new table we have created (yes, everything continues as it should unless

the error count get over ten, I believe the default limit is 10 ) we can see from the image directly below this

that the row with the error appears to have been dropped or ignored.

So you will need to test and check your data and the code you use to generate your 'general' form of CSV

format file.

==================================================================================================================================

Of course we could try and handle the errors ourselves, but this section is about making a

table from our own dotNet and Vulcan generated file.

Good luck with any conversions etc.

See you in the next section.

PHH - August 29th. 2010.


Copying from one DB to another

Using the Context menu option in SSMS 2008 :

New to the SSMS shell in version 2008 (it was absent in the 2005 version) is a useful feature to allow

the user to import and export data. See below to what I get when I right click on my 'GBproductCodes'

database :-

=================================================================================================

For this example we will use the Export feature to copy a table form this DB into another, and for the

target DB I will choose the 'SafetWarehouse' one which is just off the bottom of the above image.

In the image below we can see just which tables ate present before we attempt the 'table export'

procedure :-

===========================================================

So let us start the export process by right clicking the 'GBproductCodes' DB item in the OE pane

and clicking the 'export' option - here is what we get :-

Move on to the next stage after the wizard front screen, with 'next' :-

In the image shown above, notice that I have selected the required database from the set offered in the

drop list - which is shown below :-

=================================================================================================

After selecting the source DB we move on to selecting a target, or destination for the table data.

Notice in the images below that we have a lot of options, and later we will see how to use Excel as

both a source and a destination.

For now we will stick with the default of "SQL Server Native Client 10.0" which is offered most probably

because this is the type of SQL Server I have open in my SSMS.

Remember, we can quite happily process SQL Server 2005 DBs with SSMS 2008. It is just another

application - which sits on top of the DB. I use it all the time with my client's 2005 'flavour' databases

while designing and testing reports with Reporting Services.

Now lets study all the options :-

=================================================================================================

So we will stick with the default and chose our target DB as shown below :-

=================================================================================================

Click next to move on the wizard to see this below :-

=================================================================================================

For this example we are going to choose a table as our source, but we may wish another time to

script a query to get a dataset - that would be quite useful, or 'handy' !

Let us see what this option generates in the subsequent dialog screen :-

=================================================================================================

Notice in the image above I have selected three of the possible five tables on offer. So we ought to

get these three copied over to our target DB.

Before we move on to the final stages and make the data transfer 'happen', let us see what these

mappings are about.

If we highlight one of the selected tables, and then choose 'Edit Mappings' we see this sort of screen

display :-

==============================================================================================================================================

We see each field (or column) in the source table, and could if we so wish change the name that

will be used in the new table to be created.

For our test export, let us change three column header values, to 'Form Type', 'Prescriber Code', and 'Qty'.

See below for my changes :-

Notice that once a column 'title' has been selected the original name is shown in the bottom of the dialog

as the user changes it to something more suitable - we always know what we started with !

Okay then, time to make it all happen and press the 'Next' button - maybe ! :-

=================================================================================================

Well, not quite - I am not sure of the significance of this screen - I always at the moment use the

default of 'Run Immediately'. This brings up this dialog :-

This is positively your last change to check what you are doing, and go back if you so wish.

Otherwise it all starts to happen and you get the following after some processing time passes :-

Take it from me that the messages available from the blue links are not very helpful or informative in

these circumstances. It may be more use to us when things go wrong !!

Anyway, let us see what we now have in the OE pane for the target DB. Study carefully the image

below and all its detail :-

============================================================================================================================================

As you can see when you study the image, the three new tables appear to be there, and the one which

has been queried shows that the new column names have been applied successfully.

We name have 'Qty', 'Prescriber Code', and 'Form Type'.

Since these notes are about TSQl we will look in the next section at how we may do something similar

with TSQL script. Not quite as fancy however.

before we move on, let us look at what we see in the source dialog when we have 'Views' on offer as well

as tables :-

=================================================================================================

In the image shown above notice the different icons for the two views at the bottom of the list.

If we went ahead with this export we would have the views turned into tables, much like if we used

a query in the dialog options above in place of copying a table.

This works nicely, as I did my first real tests yesterday, and used the results for real work.

Using TSQL script :

Before we start to use script to transfer tables, let us clean up the target DB, and we will use

'SafetyWarehouse' once again.

Here is how it looks now :-

=====================================================

Setting our task :

Before we focus on any details, let us set out our task - so we know what we are trying to do.

From the 'GBproductCodes' DB we will copy the table entitled ' [dbo].[ProdCodesLU] into the

[SafetyWarehouse].[dbo] schema and keep the same table name and also the same column headers.

Here is a script which seems to work first time :-

================================================================================

And after remembering to Refresh the table list in the OE pane we can see this test query run :-

======================================================================================================================================================================

Now obviously three lines of script is convenient - BUT - it is hardly robust ! So more lines would

be needed to check for the existence of the table before attempting to make a new one.

Also it is also more tricky to change column names in the table structure - possible but fiddly, and

anyway we can use the SSMS shell, so why worry too much ;-)

Anyway, here is a few lines which check for a table and as scripted DROP it before copying.

Of course this can be easily adapted to ONLY warn and stop the operation - and not DROP the table.

=================================================================================================================

Also remember that you can easily rename the table using the script shown in a section two previous

to this one.

See you in the next section.

PHH - August 29th. 2010.


Bulk Copy Operations (ADO.NET)

Although these notes are mainly focused on TQSL scripting, we need to cover the modern way of getting larger

quantities of data into Tables within out SQL database.

The SDK documentation has this to say about Bulk Copying :-

Microsoft SQL Server includes a popular command-line utility named 'bcp' for quickly bulk copying large files into

tables or views in SQL Server databases. The SqlBulkCopy class allows you to write managed code solutions that

provide similar functionality. There are other ways to load data into a SQL Server table (INSERT statements, for

example) but SqlBulkCopy offers a significant performance advantage over them.

The SqlBulkCopy class can be used to write data only to SQL Server tables. But the data source is not limited to

SQL Server; any data source can be used, as long as the data can be loaded to a DataTable instance or read with

a IDataReader instance.

So the dotNet class called 'SqlBulkCopy' has a lot to offer the application programmer when it comes to placing

larger amounts of data into existing tables.

This class is a small one and there are few methods and properties. Let us look at what is in the SDK

documentation :-

===============================================================================================================================

Explaining the 'SqlBulkCopy' class :

Quite simply, this class allows the developer to take a DataTable object, and write it directly into an existing SQL

table, by just supplying a connection, the name of the existing table, and an instance of a DataTable object.

The method 'WriteToServer' is what presses the button - as far as making things happen.

Obviously there are variations on the overall process, like a mapping of columns may be provided if the default is

not suitable.

However, I will just discuss the case where the target SQL table has the same structure as the data being supplied,

and that includes having the same column names - down to correct 'casing' too (upper and lower case set properly).

If we set up things correctly before hand, the 'SqlBulkCopy' (SBC) class is easy to use, and VERY effective.

What is required for use :

Before we use the 'WriteToServer' method of an instance of the SBC class, we need to have a DataTable (DT) object

available to us.

Quite often the DT has been instantiated from a pair of XML/XSD files on disk - where the XSD file specifies the

schema of the DT.

If you do not have an existing SQL table, then one may be made from the details in the XSD file. This is how I did things

in the GreatBearTransform app I made to convert a bunch of DBF files into an SQL database - where files and tables were

on a one-to-one relationship.

If you do not have XML/XSD files as a starting place for your coding, then the DataTable object itself has a schema which

you can easily access in order to create an SQL table with a suitable structure. All the data types will be in the existing

DT schema, but you may need to take some care with mapping dotNet data types with your SQL column types.

For the rest of this section I will follow through with what I have gotten to work successfully with the Transform tool

mentioned above.

Structure of a Typical XSD file :

Let us look at a real example I have used in my development :-

=============================================================================================================================

Notice that the data types of all columns are covered by :-

"xs:string"

"xs:int"

"xs:double"

"xs:dateTime"

"xs:boolean"

We can now start to make sense of the code I wrote years ago now, to convert my DBFs to SQL tables.

Below are my Vulcan methods to take a script line (already identified as a column specification line) and

change it into the most suitable SQL data type from the XSD details, along with the column name :-

==============================================================================================

===============================================================================================================

As you may expect, a boolean becomes a 'bit'.

So basically, I have a method which puts together a full query statement - this is called 'DoCreateSQLScript' and

is shown below - and the full 'Create Table' script is assembled from the necessary parts, not just the column

specifications.

Study this method shown below :-

===============================================================================================================

Notice first, the lines 1686 to 1691, where the top, body, and bottom (tail) of the SQL statement are made, and

then assembled in line 1690 to make the full and correct statement, which is then saved in a file with the method

used in line 1691, called 'DoSaveAsTextFile'.

The top and bottom of the 'create table' code are supplied by the methods 'MakeTop' and 'MakeTail', and obviously

the body of the script statement is provided by the 'FieldsString' class variable.

So if you follow this code through with this XSD example, you will end up with a script like that shown below :-

====================================================================

[ Notice that by placing the RECID field last we can simplify the column specification method by adding

a comma to the end of each and every string representing a field in the table. ]

And this looks like the following in the OE pane of SSMS :-

=========================================================================================================================================================

Unfortunately, our simple example is not like my DBF names and the fifth column here has extra characters

inserted by the o/s when an underscore was coded into the XSD. I suppose we could adjust our code if

necessary to take account of this - for now we will just live with it.

Finally, the Bulk Copy part :

Now that we have seen how to take the XSD file and create a new table in our SQL database, we need to

focus on the XML file where the data is stored. From this we need to make a data table object.

If we have done sufficient dotNet coding we will know that it is very easy to make a DataTable object from an

XML file pair - see below :-

=================================================================================================

The third line of code is simply to display the data in the data grid view control on the Form.

So let us look at the major parts of the method called 'DoBulkCopyXML' :-

=========================================================================================================================================

*** note *** looking at my code form over a year back (and more), I am not sure that line 1274 is correct, I may

have changed the wrong 'CATCH' line. However the code does work fine for what it was used, so all is not bad ;-)

I think the thing for me to do now that I have documented this reasonably well - or good enough to copy and adapt

in the future - is to use the new VS2010 and Vulcan 2.0 to see if I can make a simplified Forms app which will

take a sample XML / XSD file pair and add a new table with data to a test DB.

Hope to report back some more success !?

See you in the next section.

PHH - August 29th. 2010.


VS2010 working test app

This Bulk Copy operation seems like a good excuse for me to really try my hand ay Vulcan 2.0 and the

present VOPS pre-release of the VS2010 (VS10) integration from Grafxsoft.

I have so far followed up on the previous code done in VS2008 and made a simpler test application to both

get me familiar with VS10 as well as to provide a clear example of how to Bulk Copy XML data into a table.

Just to get us on the right foot, and provide a useful start to this subsection, below is shown the first

results of my efforts :-

===========================================================================================================================================================================

Notice that I have used the empty table structure we previously created, at the moment my VS10 Vulcan code

only does the Bulk Copying of the XML data and does not yet make the SQL table structure itself.

In fact to be truthful, I made the empty table from a test method in my new little app - here

it is shown in detail below :-

=====================================================================================================================================

Benefits of a Simplified application :

One very good reason to make a new and simplified application of this Bulk Copy process is to make

things as clear as possible to guys new to SQL and using ADO.net from Vulcan.

In more complex apps, where GUI input of parts of the connection string make interactive use of the

application easier, and better for the user, often the important parts of the code are then made obscure,

or at least difficult to find - and see just which bits need to go together.

So hopefully this example will help make things as easy as they are really.

Time to see more success - here is the application running :-

========================================================================================================================================================

In the image above you will notice a data grid view control displaying data which has been read

from the XML file into a Data Table object. This DT object has been provided as the data source to

the grid view, before the same object was used in the Bulk Copy method.

Let us now look at the 'crunchy' bit - that part of the code which carries out the bulk copying of the

data into the SQL table, see below :-

================================================================================================================================

You may even say that it is only one line - numbered 118 - which does the useful job of the data copying !

Only lines 74 to 76 were needed to set things up before line 118, the rest is just good Vulcan dotNet coding etc.

Lets move on to the code lines which set and make both connection string and the connection itself :-

=====================================================================================================================================

I make no apologies for hard wiring the details, this is the whole point of the exercise - to make it as

simple, and as clear as possible to follow by a new user to these things ;-)

My thought is that if you can get it to work once, with hard wiring, then you can easily adapt it to become

more general and allow GUI selections etc.

PHH - 31st. August 2010.


Full working code

Enter topic text here.


Temporary Tables

It is a good idea to read 'BoL' on this topic, AND, to make special note of the final sentence !

It would appear to me that the multi-statement table-valued UDF does away with the need for Temporary

Tables for many developers / scripters.

Temporary tables are similar to permanent tables, except temporary tables are stored in tempdb and are

deleted automatically when they are no longer used.

There are two types of temporary tables: local and global. They differ from each other in their names, their

visibility, and their availability.

Local temporary tables have a single number sign (#) as the first character of their names; they are visible

only to the current connection for the user, and they are deleted when the user disconnects from the instance

of SQL Server.

Global temporary tables have two number signs (##) as the first characters of their names; they are visible to

any user after they are created, and they are deleted when all users referencing the table disconnect from the

instance of SQL Server.

For example, if you create the table employees, the table can be used by any person who has the security

permissions in the database to use it, until the table is deleted. If a database session creates the local

temporary table #employees, only the session can work with the table, and it is deleted when the session

disconnects.

If you create the global temporary table ##employees, any user in the database can work with this table. If no

other user works with this table after you create it, the table is deleted when you disconnect. If another user works

with the table after you create it, SQL Server deletes it after you disconnect and after all other sessions are

no longer actively using it.

Many uses of temporary tables can be replaced with variables that have the table data type.

[ For more information about using table variables, see table (Transact-SQL). ]

Table - data type :

Is a special data type that can be used to store a result set for processing at a later time. Table's primary use

is for temporary storage of a set of rows returned as the result set of a table-valued function.

Table variables can be referenced by name in the FROM clause of a batch, as shown the following example :

SELECT Employee_ID, Department_ID FROM @MyTableVar

Outside a FROM clause, table variables must be referenced by using an alias, as shown in the following example :

SELECT EmployeeID, DepartmentID

FROM @MyTableVar m

JOIN Employee on (m.EmployeeID =Employee.EmployeeID AND

  m.DepartmentID = Employee.DepartmentID)

Assignment operation between table variables is not supported.

Also, because table variables have limited scope and are not part of the persistent database, they are not

affected by transaction rollbacks.

PHH - August 7th. 2010.


Useful Examples

Example 1 :

Let us start simply, and try something from BoL - see below :-

=============================================================================

==============================================================================================================================================

=======================================================================================================================================

PHH - August 7th. 2010.


Partitioned Tables

The data in partitioned tables and indexes is horizontally divided into units that can be spread across

more than one filegroup in a database.

Partitioning can make large tables and indexes more manageable and scalable.

Partitioned tables and indexes are available only on the Enterprise, Developer, and

Evaluation editions of SQL Server.

This section provides the information that is required to understand, design, implement, and work with

partitioned tables and indexes.

It is good that we know of the existence of Partitioned Tables, but since they are obviously an advanced

feature to most of us, and don't come with the Express version of SQL Server, we are best to leave it here

for the moment.

PHH - August 7th. 2010.


System Tables

SQL Server stores the data that defines the configuration of the server and all its tables in a special set of tables

known as system tables.

Users cannot directly query or update the system tables except through a dedicated administrator connection (DAC)

that should be used only under the direction of Microsoft Customer Services.

For more information, see "Using a Dedicated Administrator Connection".

The system tables typically change in each new version of SQL Server.

Applications that directly reference system tables may have to be rewritten before they can be upgraded to a newer

version of SQL Server with a different version of the system tables.

The information in the system tables is made available through the catalog views.

==========================================================================================================================================================

==================================================================================================================

PHH - August 7th. 2010.


Chapter 21 - Backup, Restore, Copying

PHH - July 24th. 2010.


DB Backups

Command Line :

http://stackoverflow.com/questions/122690/what-is-a-simple-command-line-program-or-script-to-backup-sql-server-databases

To backup a single database from the command line, use osql or sqlcmd.

"C:\Program Files\Microsoft SQL Server\90\Tools\Binn\osql.exe"

   -E -Q "BACKUP DATABASE mydatabase TO DISK='C:\tmp\db.bak' WITH FORMAT"

There is a free command line tool "SQLS*Plus" (on http://www.memfix.com ) which is like Oracle SQL*Plus but for Microsoft SQL Server 2008, 2005 and 2000.

SQLS*Plus is very flexible with data formatting (set lines size, pagesize, etc), variables (var, &, &&), spool, HTML output, etc - seems like a very good alternative to standard command line tools, such as sqlcmd, osql or isql

GUI tools in Management Studio :

=================================================================================================

========================================================================================================================

=============================================================================================================================

=============================================================================

=============================================================================

============================================================================

==============================================================================

==========================================================================================================================

=======================================================================================================================================================

Script a Query :

==============================================================================================================

=========================================================================================================================================

ADO.NET code :

cScript := "BACKUP DATABASE [" + SQL_my_funcs.DBfilePath + "] "

cScript += "TO DISK = '"+cName2Use+"' "

cScript += "WITH FORMAT, "

cScript += "NAME = 'Full Backup of SmallTrader' "

=====================================================================

//========================= [ Backup Database ] ===========================

static method DoBasicBackup() as Boolean

local lResult := true as Boolean

local oF as DirectoryInfo

oF := Directory.CreateDirectory("C:\BackupSQL")

local cName2Use as string

cName2Use := oF:FullName+"\StaticBasic.bak"

MessageBox.Show(cName2Use,"Will be Used :")

local cScript as string

cScript := "BACKUP DATABASE [" + SQL_my_funcs.DBfilePath + "] "

cScript += "TO DISK = '"+cName2Use+"' "

cScript += "WITH FORMAT, "

cScript += "NAME = 'Full Backup of SmallTrader' "

MessageBox.Show(cScript, "Back-Up :")

try

SQL_my_funcs.KeepOpenSendNonQueryScript(cScript)

MessageBox.Show("Query has been sent .... ","After BU :")

catch oEx as SystemException

MessageBox.Show(oEx:Message, "ERROR :")

end try

local nError as int

nError := SQL_my_funcs.GetErrorCodeFromServer()

if nError = 0

MessageBox.Show("No apparent error","Error ?")

else

MessageBox.Show("Error number :- "+nError:ToString(),"Error ?")

endif

//MessageBox.Show("Just before force disconnect ....","TEST :")

SQL_my_funcs.DoForceDisconnect()

RETURN lResult

//========================= [ End Backup] ===========================

PHH - 21st. December 2011.


Transferring Databases

PHH - August 4th. 2010.


Transferring Tables - Moving & Copying

PHH - August 10th. 2010.


Copying SPs, UDFs & VIEWs

Enter topic text here.


Third Party Tools

In order to be noticeably better than Microsoft's offering within the SQL Server software itself, any

third party tool is likely to be expensive - BUT - it is a very important part of the survival strategy of

any real live company network and database system, so think on this carefully.

Redgate seem a reasonable source so try out this 14 days free trial offer.

=========================================================================================================================================================================

PHH - August 5th. 2010.


Chapter 22 - Errors and Catching

BoL - Functions - errors to see information ....

PHH - July 29th. 2010.


Chapter 23 - Excel, CSV and Bulk copying

This chapter is really about giving the developer a bunch of options and techniques for moving data

into and out of an existing SQL database.

Very often a client will have data in a format which is available through Excel, or has come to them

in a file of CSV type structure.

Sometimes I have written Vulcan dotNet applets which process text and PDF originating output, and

through code produce a CSV type output file. However, when I do this I usually use a different delimiter

to commas, my choice is a double pipe, ' || ', as it allows me to have commas in an address string for

example.

Short Summary :

So to summarise, this chapter is all about bringing data into an SQL table format from an external source.

Example Result :

Look at the image below and see what has been achieved :-

=======================================================

In the image above the table called 'Sheet1$' has come from an Excel workbook where I have allowed the

process to provide the default name. In our exploration of all these matters related to Excel and CSV files

etc., we will usually name our tables with a more appropriate name.

One further thing to notice is that the columns which cam across in the transfer process have been suitably

named, and where appropriate the data type has been provided as well. Notice the 'float' types for reference.

PHH - August 7th. 2010.


Method 1 - TSQL bulk copy

PHH - August 10th. 2010.


Bulk Copy - CSV

PHH - August 10th. 2010.


Bulk Copy - XML

PHH - August 10th. 2010.


Method 2 - from Excel to Table

PHH - August 10th. 2010.


Method 3 - using SSMS Context Menu

PHH - August 10th. 2010.


Chapter 26 - The Family Tree

PHH - August 5th. 2010.


Some useful functions

There is a whole host (bunch) of possible functions which could be of use with this Family Tree

problem, and they are all based on accepting a parameter which is of type 'PersonID'.

Let us start with an easy one :-

Person Details :

Below is shown a working script to provide details for any person specified by their ID :-

============================================================================================

Mother Details :

Below is another useful 'building block' of a function, one to supply mother's details :-

===================================================================================================================================

Siblings Details of :

Let us start with making a list of siblings with sufficient information for identification and checking.

We can then move on to just making an ID list for use in other queries.

Let us call this function 'SiblingsDetails' :-

==============================================================================================================================

It would appear that we need to remove the person themselves, and this is done below for the simple

list of IDs for siblings - see the next function.

Siblings - List of :

And now here is the function to make a list of IDs for any persons siblings - and they themsleves are not included :-

==============================================================================================================================

So asking for the siblings of Peter Williams we get correctly, Dominic and Susie.

Brothers Details of :

The Brothers Of function is just a filtered (restricted) version of the Siblings Details function done previously.

Study it carefully :-

==============================================================================================================================

Yep! Dominic is Peter's brother - lets see if this works the other way round in that case :-

==========================================================================================

Looks right to me ! 18 & 19 which ever way round you put it ......

Brothers of - List :

If we now design, make, and test a function to list the brothers of any specified family member (person)

we can then use this to build the 'Those with no Brothers' function later.

Lets see what we can do :-

==============================================================================================================================

Yep! Dominic is Peter's brother - lets see if this works the other way round in that case :-

================================================================================

Yep! Brothers will be brothers - as they say ;-)

Sisters Details of :

Children Details of :

Here is another useful function, that to find all the children of a family person :-

=================================================================================================================

Notice that this will work for fathers and mothers.

Children of - List :

===================================================================================================================

So our function supplies a list or NULL (empty list) of all children of the specified family person.

Parents Details of :

Grandparent Details of :

Grandchildren Details of :

Okay then, we have been successful up to now with a few of these functions, BUT, this is a slightly more difficult

problem and also one which can be solved in more than one way. Some solutions need more SQL skills than

others, but since we have been using SELF JOINs then lets stick with that approach for now.

When we get really good we may be able to write one function which does Parent, Grandparent, Great Grandparent .....

all in one script, and all that is required is a limit of how deep the recursion goes. We shall see.

For now, study this set of results - which to my knowledge are correct - as well as the script following the results set

which shows the function itself :-

============================================================================================

It would be a nice refinement to list 'grandson or granddaughter' instead of grandchild, so I will leave it

to you to check out the CASE function.

For now I will just insert it into the column specification part of the SELECT statement - as see below :-

=========================================================================================

========================================================================================================================

Here is the necessary part of the script, with the CASE insertions made :-

==========================================================================================

Well, we seem to have some useful results shown in the image above, so we must be making progress ;-)

Find Twins with Details :

Now here is an interesting problem, that of finding twins - not sure how it would handle triplets, as our family

does not have any as yet !

You will notice even after a quick glance over the script that we are using one self join, and a couple of others

which involve the same table (Person) but it is not a self join because the ' ON ' clause does not have the same

table on both sides of the operator.

Study this script in detail and get your head around what it is doing :-

================================================================================================================

Well, as you can see the family twins have been identified - correctly !

Mothers with Twins, Details of :

=====================================================================================================================

Well, all I can say is that in my knowledge this is correct.

We could now change this so that only a unique 'PersonID' value was output for each mother.

=========================================================================================================================

So now this should make a complete list of all mothers in the database with twins, triplets, and more ....

But each mother should be present only once.

If you want 2,3, etc. as the number in the multiple births, then you need to write a function which groups the

information for previously in the first attempt at the twins function - shown above.

Fathers with Twins, Details of :

Those with no Brothers :

Those with no Sisters :

Patriarchs Details of :

Here when I use the word 'Patriarch' I mean some male person with NULL 'entries' as both there father and mother.

A NULL 'expression' is probably more correct.

=======================================================================================================================

Yes, for the state of my database at the moment, these are correct.

Things will change however after I add granddad Ben Hepburn.

Matriarchs Details of :

So all we need to do for this is change what we have above so that the filter condition is for females :-

======================================================================================================================

And rightly, we see the only matriarch entry at the present moment, grandma Nancy.

Last Generation - Details of :

Biggest Family :

PHH - August 5th. 2010.


Using CTE recursion from Chpt. 12

Having just added a few useful sections to chapter 12 I have realised that we can make a couple of

very useful, and interesting, queries and related functions (UDFs) for the Family Tree solution / problem.

To get us off to a good and successful start let us look at the following results set from a short query

utilising a UDF called 'GenerationsAfter'.

Study the image below :-

===========================================================================================

In this image (that above) we can see that a few persons have a number of family, or generation, levels

beneath themselves.

Generally, the ones with the most levels 'deep' will be the "grand old folk" of the family tree. I think the

terms are Matriarch and Patriarch. Obviously the matriarch is a female ancestor from the past, and a

Patriarch a male version of the same. Mater and Pater are the Latin root words.

Now let us look at the script which makes up this UDF, and take special note of the fact that it uses

the "DescendantsOf" UDF which was based on CTE recursion (see chapter 12.) :-

======================================================================================

Basically, the 'DescendantsOf' UDF is the same as the 'GrandChildrenOf' UDF but without a filter on the level depth.

So all levels are included.

We can now restrict the results dataset in a variation of the query using the 'GenerationsAfter' UDF as follows :-

==============================================================================================================

Remember that to be able to use the 'TOP x WITH TIES' clause we need to have set an order to the output.

And shown below are two variations on this same query, where we are looking at females and males only :-

============================================================================================================

The image above does illustrate that we get suitable matriarch and patriarch results.

Having got the script to work we can now wrap it up into a function as shown below :-

================================================================================================================

I suppose we could have named this UDF 'FamilyMatriarchs' and this may well be a better name. But that is

all part of the tidying task, we are just looking at ideas and possibilities here in these sections.

And now we have the 'PatriarchsOf' UDF :-

===============================================================================================================

Once again, we must test and check out everything we do, even if we think we have just copied some script

we have already tested in an 'open query' within SSMS. This is all part of the skill and technique of working

in the SSMS environment.

I hope you are as impressed as I, at the simplicity of these more recent queries and UDFs - showing that it is

VERY important to start with the right approach, and the right table or view on which to base CTE use and

building recursive queries.

See you in the next section, it is useful and instructive ;-)

PHH - September 5th. 2010.


Chapter 27 - The curse of the cursor ?!

This is a thorny subject, and one which should be 'fully faced' by those coming new to SQL

and anyone just converting from a row based iterative style of data retrieval !

I strongly suggest you purchase and study Paul Nielsen's 'Red Bible' on SQL Server2008, as well as

his DVD video training material - both are VERY inexpensive for the amount of useful material you get

for your 'buck' !

Paul has a phrase "Kill the Cursor ! " and explains his dislike most clearly.

It would seem that we need to know how to setup and use a cursor, even though there are few times

which seem to be acceptable, if we are to use SQL Server correctly.

One such time is covered in the section after next, when we look at using system SPROCs to gain

access to system information on tables and their related details.

In the 'Bible' there are four "Paul Approved" specific situations when using a cursor is the correct

solution, and the first of these is covered by our System SProc example.

Before we go to working details, and 'allowed' examples, we need to find out more about what cursors

are and why they are considered poor generally, as far as their impact on the SQL Server performance

is concerned.

Where are we coming from ? :

It seems to be generally agreed that the current database world grew out of (or came from) the ISAM

files structures of 'Big Blue'.

Here is what Wikipedia has to say :-

'ISAM' stands for "Indexed Sequential Access Method", a method for indexing data for fast retrieval. ISAM

was originally developed by IBM for mainframe computers.

It is a database system where an application developer directly uses an API to search indexes in order to

locate records in data files.

In contrast, a relational database uses a query optimizer which automatically selects indexes.

Let Paul Nielsen have the last word here, as he feels so strongly again the old ways with new technology.

"The vestige of looping through data one row at a time remains in the form of the painfully slow SQL cursor.

Where are we headed ? :

Well, we already have a relational database engine which is highly tuned to providing best performance to

all it's clients when set-based relational algebra code is created and used.

I predict that in the future the necessity to have to use a cursor at all will be removed by the SQL Server

designers, just as they will need to move us guys to a more object oriented database engine - or at least

it's supporting software which may be a 'shell' around it to appear as a more object solution for us.

Cursors on Server and Client :

The SQL Server cursors we have been talking about above are those that are within the database engine itself.

These 'server-side' cursors are what typically slows down the server, and degrades its performance.

However, there are client-side cursors which typically ADO frequently uses to scroll through rows in a record

set used by an application. Client application cursors are OK, and not what we need to worry about.

Not only does the use of cursors typically result in a slower performance for the query script designer, but of

course the rest of the users of the shared resource - which a database engine is - are also affected. Performance

is impacted for all users.

AND, another less discussed issue is that fact that network traffic is GREATLY increased as many, many times

more query requests are made using cursor scripts. A query is sent for every row required, not just the one

batch query as usual with set-based stuff.

And on top of that, the longer the DBA or programmer stay away from coming to terms with set-based query

solutions, the longer this state of affairs will continue.

Paul Nielsen says that the second tier of "Smart database Design" is developing set-based code rather than

an iterative design solution.

Unfortunately, there are a few legitimate reasons for using a cursor, otherwise it would be easy to say to one

and all - "don't use them" !

The example we shall show in the section after next is a good example of such a case of necessity over

convenience.

The Comfort of the Cursor in place of the Knowledge of the Set-based Solution :

Probably the most common reason for people to use cursors is that they are most comfortable with a procedural

approach, and thinking in terms of loops and pointers. The iterative approach !

Often DBAs and programmers are thrown into the world of relational databases, requiring set-based relational

algebra, and not enough time and effort is expended in educating them to this new world, with its quite different

requirements.

This can be easily remedied - education, tuition and learning.

"Learn something new today. " !!!

Having a look at all this :

I suggest we stop all the words and the talking, and go and have a look at a simple example running in SSMS,

and see just what can be done to grind the Server into the dust !!

See you in the next section.

PHH - September 2nd. 2010.


Making it work for us

Let us see a working example which I have borrowed from the 'BoL' notes, and suitably adapted to

do a couple more simple (and helpful) things for us.

Before we start to discuss any details, it is probably best to see the working T-SQL script, as well as the

results from the AW database - study this below :-

==========================================================================================================

The code shown in the image above works fine, and 54 lines printed out are the result.

Of course this is a very simple example of a cursor in use, and the job could be more easily be done using

set-based script - see later for this.

For each line of output a query request was sent across the network to the SQL Server, and one way to prove this

is to ask the SSMS shell to include the actual execution plan. See the image below for the place to switch this

on in the SSMS shell :-

========================================================================================================

The result for this small set of rows is that the execution time goes up to 4 seconds and the Execution plan

windows gets populated with images like those shown below :-

===================================================================================================================

Taking this further :

If we now remove the WHERE clause by commenting it, we are trying to get almost twenty thousand rows

returned to us.

If I leave the 'include execution plan' switched on I get an error generated after over an hours processing

when in fact there was only just over three thousand rows returned.

The images shown below illustrate this, the first shows the process before the error when heavy use is being

made of processor cores and memory :-

====================================================================================================================================================

And then 40 minutes further into the query we get this error situation :-

=======================================================================================================================

Yes, I realise that we may be being a little unfair by adding the extra burden of including the execution

plan for each single query - BUT - it does illustrate the point that although the cursor code looks like one query,

it is in fact as many as the number of rows requested, and that network traffic is increased, as well as hitting

performance.

Very often it is difficult to see the affects of poor scripting style, or approach, as the hardware today is of such a

high performance, and the Server is so fast generally.

Before we move on we ought to see the simple set-based solution which even with an execution plan included is

so fast that the time registered in the yellow results bar is still zero. Below are a few 'telling' images ;-)

=============================================================================

Summary :

Although our test may be unfair for actual execution time, it does demonstrate the extra work required for cursor use.

And even on my fast machine, it was noticeable (without the execution plan being included) that the server seemed

to work a second or two more than for the set-based query. Testing over a busy network is the only real test, and that

is not possible at the moment.

Avoid cursors if you can, and only use them when you must - system stored procedures are forgiven, if used like in

the next section.

Spend your time learning more set-based techniques and not writing cursor code !

See you in the next (and interesting) section.

PHH - September 2nd. 2010.


System SProc use

When are cursors required ? When are they the best solution ? :

Paul Nielsen (in his Bible 2008) has been brave enough to stick his neck out and say :-

"Here are the four 'Paul Approved' specific situations when to using a cursor is the right solution.

I will show you his headings, and then discuss the first one. You can look at the others yourself

when you grab hold of a copy of his red 'bible' - an excellent reference book!

1.) Iterating over a stored procedure.

2.) Iterating over DDL code.

3.) Cumulative Totals / Running Sums.

4.) Time-Sensitive Data.

We will now take this first case which he expands upon in the lines below as :-

When a stored procedure must be executed several times , once for each row or value, and the

stored procedure can't be refactored into a set-based solution, or it is a system stored procedure,

then using a cursor is the right way to iteratively call the stored procedure.

Our Example :

In our example I think we demonstrate perfectly when it is permissible to use a cursor.

We need to call a system SPROC called 'sp_spaceused' for each name of a table we have in a dataset

provided by a second system object.

The system object which is a source of all the table names is in the 'System Views' portion of the OE pane

and is called 'sys.sysobjects'.

If we simply query this view we get the following :-

===============================================================================================================================================

But if we filter by asking to see just the user defined objects we get this :-

==================================================================================================================================================

As you can see our returned set falls from about 42 thousand down to just under 4 thousand.

What is our aim ? :

We are simply trying to produce a list of named tables which have zero rows in them at run time.

This example is from a real life legacy situation at a client site. They appear to have growing

numbers of tables with no content, and we need to monitor and report back.

So the first part of out set of objectives, is to produce a list from a cursor based query.

Once we have our list we can then move on to making a report using Reporting Services.

Before we move on I would like to show you that we can use a few alternative ways of invoking the

system view called 'sysobjects'. See below :-

=====================================================================================================

However, because I am always working with more than one database in my queries and joins, I

prefer to use a full and correct naming syntax, and so I use that shown below :-

=============================================================================================

Starting our cursor solution :

First of all we need a temporary table which simply holds a list of all our table names.

We are after only those in the schema called "scheme" and within the database called 'pelican'.

Here is a script which works well in SSMS :-

===========================================================================================

By creating this temporary table first, we can then check that any table we subsequently have in

focus is one of those we require.

Let us now study the main body of the script to see what is being done :-

=====================================================================================================================

Study the details in the above image, and see that they are not as complex as they first look ;-)

The result we are after are eventually all held in the temporary table called '#TempTable' and we filter

out all those with numbers of rows greater than zero. below are some results obtained like this :-

=========================================================================================

In fact I have an awful lot - 1948 to be precise and my DB is an older backup copy, so there are more by now :((

I don't think there are any big deals as far as the code is concerned, most of it is reasonably straight forward

and looks long and complex simply because we get used to seeing set-based script which is generally much

shorter and neater.

Anyway, like all good queries, this has been built up in SSMS and tested in many ways as I went along with

building the solution. You always find small things to change and get right on the way.

However, once we have a working and suitable query solution, we can wrap it into a stored procedure as we should

remember that a SPROC is a stored 'batch'.

So having just done this using the shell in SSMS I can now run it as follows :-

=======================================================================================

And once again a LOT of tables are there but not shown ;0)

Here is what our Object Explorer pane now looks like :-

=============================================

Go and refresh you mind about stored procedures, and making them with the SSMS shell tools.

Chapter 11 on SPROCs will help joggle your memory.

PHH - September 2nd. 2010.


*******************************

<< intentionally left blank !>>


Appendix A - functions

PHH - September 6th. 2010.


Appendix B - Global variables

PHH - July 24th. 2010.


Appendix C - Reporting Services

PHH - July 21st. 2010.


Appendix D - SSMS - Management Studio

PHH - July 21st. 2010.


Appendix E - -

Appendix E - impacts on ADO.NET use

PHH - July 21st. 2010.


Appendix F -

Appendix F - Scripting with SSMS

PHH - July 31st. 2010.


Appendix G - Gotchas !?

Care :

'Go' being within a commented section of text in a statement - SSMS so a 'go' on its own at the start of a line

even in a comment, as an instruction to the preprocessor belonging to SSMS.

Order By :

This clause 'usually' has to be the last one in the statement, and is applied once there is a return data set ready

to be sent back from the Server.

So the upshot of this is that Views and UDFs, and a couple of other things, can't have 'Order By' clauses.

Like may things there is a situation where this does not apply, and it has to do with using the predicate 'TOP'.

Apparently, the concept of 'TOP' does not make any sense unless the chosen records (rows) are not in some

determined order. So when using 'Select TOP ...' or 'Select TOP with Ties ....

======================================================================================================================================================================================================

PHH - July 21st. 2010.


Appendix H - Bits 'n' Bobs

The meaning or 'N' or even 'n' :

[N] 'tsql_string' - Is a constant string. tsql_string can be any nvarchar or varchar data type.

If the N is included, the string is interpreted as nvarchar data type.

To put this another way - simpler - the 'N' means multi byte characters or Unicode as we know it.

So any string data type beginning with 'n' is a Unicode type, and 'N' in a constant string environment means

'cast' the constant string to a Unicode type.

There 'simple' !

PHH - August 3rd. 2010.


Appendix I - functions

PHH - September 6th. 2010.


Appendix J - Installing SQL Server

Installing MS SQL Server can be a bit tricky, I have to say !

Over the last three years plus I have done it many times and on desktops as well as laptops.

It would seem to me that the best advice is try to download and install the latest version from Microsoft

which is on general release to the public. Do not try an early release version if you are new to SQL - it may

prove difficult to remove later, and any final public version generally will not install on your PC until you have

completely removed the pre-release version.

State of your PC / laptop :

Another thing which is likely to cause you grief when you try to install SQL Server is the state of your software

on your PC.

I found that a newish and pretty 'clean' laptop / PC would allow the download and install of the latest version of

SQL Server without too much hassle, or issues being thrown up.

The hardest case I had was with a development desktop system where I had accumulated older versions of Visual

Studio and other stuff - over years - I could not remove this because it was still needed to serve clients needs.

So if you have to do an install on an older and possibly 'messy' system - try out a download and install first on

an easier and cleaner PC.

The whole process is long and can takes days and days for you to get something suitable which works for you.

Also the 'state' of your operating system may also cause a problem or two to you while attempting an install.

Make sure you have a recent version of Windows and that it is right up to date with service patches and regular

updates. It is best to make sure this is done first, before you try an install of SQL Server.

Financial Cost, and the Express version :

These days the problem of a single developer getting hold of a genuine and fully functional version of MS

SQL Server is not a problem at all.

Microsoft provide very good downloads of the Express version of 2008 R2, with promises for 'Denali' to come.

The Express version now has quite a large database limit - far too large to worry any Vulcan guys wishing to

download and get started immediately.

Where to get your download :

I am afraid that this seems to vary with time, MS sites seem to come and go, and so you will need to be flexible

in your approach.

http://www.microsoft.com/sqlserver/en/us/get-sql-server/try-it.aspx

===================================================================================================================================================================

What else do we need ? :

Well, what I say now may be a little out of date - but you do need to be aware of it !

In previous downloads of MS SQL Server we the user had to select to include the 'Advanced Tools' or was the

phrase 'Advanced Services'.

Whatever it was it meant that we the user also got management Studio (a 'must have') and also Business Intelligence

Studio (BIDS), which allows Reporting Services to be used with your chosen version of SQL Server.

I have a feeling that these days it is all included in the one download - but try and check this as you certainly need

Management Studio (MS).

Oh! - I seem to have answered my own questions - see below :-

========================================================================================================

However, I have just had a bit of a play around with this and like all this stuff on SQL it seems to be not quite

as you may expect it to be. So from now on you are on your own - download the versions you think you need,

and hope you get ;-) and 'Save' them to your disk rather than 'Run' them. This way you can compare size of file etc.

before you commit yourself.

Pressing the Panic Button ! :

If you start an install then do not worry too much, or even 'panic' !

There is usually ways of leaving the install process once it is in progress, in fact the older install software actually

took you through screens and screens to gather data before it actually started to do anything 'serious'.

Using ClickStart for guidance :

I went through the painful process of writing detailed notes some time ago - but these are probably still very helpful

today. Most of my recent installs have seemed similar in nature to what I have previously written here :-

http://www.clickstartvulcan.net/WebNotes/SQL/ClickStartSQL.html

Try www.ClickStartVulcan2.net and find your way from there to "ClickStartSQL".

=====================================================================================================================================================================

What to look out for in the Installation :

One thing for which you need to look out is the place in the install dialog screens where extra users and usernames

may be specified.

I say this because as most of us all know by now the modern Windows O/S is tied down by security, and security issues.

I have found that by including extra entries for users - and certainly add 'Administrator' as well as the logged in user - then

there are less problems with security.

I have to admit I also set Management Studio (MS) to open and run in Administrator mode. This too helps to stop the sorts

of issues which frustrated me in early days with working with SQL Server.

Okay then you guys, having warned about how difficult and tricky this can be, we will attempt to go through the full process,

from start to finish, in the following subsection.

Hold on to your hats and we will have one hell of a ride ;-0))

See you in the next subsection ...........................

PHH - 14th. December 2011.


Twelve points about install .........

- locate and download a suitable version of MS SQL Server - get the latest public release if possible

[ .. do not use CTP version as it will cause problems later .. ]

- make sure your PC has it Windows O/S, and stuff like Visual Studio, right up to date with latest patches and SPs

- run the EXE file and follow the dialog screens - use the guides provided here to help you respond correctly

- use Management Studio (MS) to browse for all instances of MS SQL Server, and find the one you just added

- in MS open the instance of the new SQL server

- in WE locate a suitable database (in a folder which you have R/W access) and attach it to the server instance in MS

- point to a suitable SQL table in the DB and use the context menu to list the TOP 1000 rows.

- ** just as a precaution, detach the DB and close MS. OPen MS again and re-attach the same DB

- when coding ADO.NET stuff from Vulcan the first time access after attachment can seem to be an issue

[ .. by trial and error I found the above may solve any problems ..]

- in VN2 code use ADO.NET to make a connection and try to open and close as a test - use Message Boxes to show results

- once a connection can be established, write a very simple method or two to query the server and get a returned data set.

- now write a simple method to allow you to write data into the DB.

[ .. once you can do all above you are ready for some serious code writing and project design .. ]

I can assure you that once you get through all these and you are successful, you will feel like a million dollars ;-0))

PHH - 14th. December 2011.


a Full Live Install ....

Okay then, having said how this can be somewhat of a problem - installing a new instance of SQL Server onto

a desktop or laptop, we will now attempt to go through the complete process on my other PC behind me.

I choose this PC because it is more modern and the O/S is cleaner I feel. If we can't do it there then we have

little chance on this old development machine of mine.

Although one or two of my captured images show CTP in the detail at places, in fact I eventually did a full install

of a Public release of Server 2008 R2, one which needed a serial number ( I am a member of MSDN).

I suggest you keep away from CTP (Community Technology preview) versions as this can cause problems when

the public release is made. Generally the two versions can't coexist on the same PC platform.

How to use these install images :

I suggest you have a look through the full set of images shown below, before you try your own install.

Depending on the type and version of your download file, you may (will?) get some dialog screens in a slightly

different order, so you need to be flexible on top of being well prepared.

So having clicked my install or setup .EXE file we see the following :-

=========================================================================================================================================

Choose the 'New Installation' - you should then see the following :-

Notice that we get a list of all the other instances of SQL Server on the system. I show you how to do the same thing in

Vulcan code in Section I of these notes.

Depending on your version of install you may see this next screen at this stage of the installation :-

If you get issues other than the Firewall you may have to go fix then depending on what is suggested in the status 'blurb'.

The clean result above is one of the main reasons I used the PC that I did to try out the new install.

Now for Product Key you may have a free Express version - those with MSDN like me will have a key - mine is hidden beneath the

drop list shown below :-

==============================================================================================================================================

The following dialog screen is simple, but necessary to get correct - see below :-

===========================================================================================================================================

At a subsequent screen you may well have to select the features you require. For safety (meaning we want things to work for

this first install of yours) we can select 'All' - or everything.

After you select and click 'Next' you get an information screen - after a minute or two (few seconds?) :-

============================================================================================================================================

Then after this feed-back we are expected to do some real work to help our installation along.

Study the following image in great detail :-

I have manually typed the 'Named Instance' to be "SQL2008_VODCtest".

We need to remember this when we get a lot further through the installation, as well we be looking for it in list of

existing instances.

When we click the next button we see the following :-

===========================================================================================================================================

And then once again we go to a screen which needs far more input by us.

Checkout the drop list input boxes in the following image :-

============================================================================================================================================================

For this first trial install of your own, leave the 'network' option alone and just select 'system' from each of the drop lists.

And that includes the 'Agent one - don't do what I did - shown below :-

==================================================================================================================

Note that the error message does not get removed until after you click 'Next' having selected a suitable input for 'Agent.

Once again we get a screen needing careful attention.

In fact the following screen is rather 'crucial' to the success of the install - check this out :-

I have to say that the input dialog screen are sadly lacking in terms of them communicating exactly what it is that

they require :(

What we are about here, is making an entry (or a few) into the white empty input text box.

The first thing to do is click the 'Current User' button - this is what I got - see below :-

=======================================================================================================================================

Now we need a add a bunch more, and so we need to use the 'Add' button - see below :-

I have to admit then next part of this is NOT at all intuitive - BUT - what do you expect from these young guys at Microsoft ;-0))

For you PC or laptop just click 'Advanced' to bring up another dialog, just as strange, or 'odd' - see below :-

We are now needing to see the list of possibilities on the system, so click 'Find Now'.

You should see something like the image shown below :-

After selecting 'Administrator' we get the result shown beneath :-

And clicking 'OK' does this for us :-

So all we need do now is repeat the whole process to add many more items to the list as I have done.

See below to find out what I have added :-

===============================================================================================

Before we move off the "Database Engine Configuration" screen let us see what the tab called "Data Directories" has on show for us :-

================================================================================================================================================================

The next screen for the "Analysis Services Configuration" is similar in nature to what we have just completed,

so we need say little about it except make sure you add a couple of meaningful users - see below for what I did :-

====================================================================================================================================================================

And now we get the choice screen for Reporting Services :-

You need to keep going as we are almost there - YES - it is a bit tedious I have to admit ;-0))

But if you want to "SQL it" and "ADO.NET from your code" then you have to suffer just a little bit.

You need to be pressing the appropriate buttons at each stage of this install, depending on what is

on offer at the bottom of the screens.

============================================================================================================================================

And, after all this we see a dialog screen which basically says we have not done anything yet - the install system has just

gathered input information from us.

So before we click to start the install itself we see this - see below :-

After clicking 'Install' we have a period when things whir away and hard disk click and clatter etc..

Then we see the following if we are lucky :-

It is now time for us to move on to the next section where we make our first connection and then see data and the likes ;-0)

However, before we do just let me say that it took me almost three weeks to get to this stage when I did it first, some three

years plus ago now. Needles to say I did NOT have some helpful notes to help me along.

See you in the next subsection .........................

PHH - 14th. December 2011.


Making our first connection

For this next subsection we are going to use management Studio - the software tool which comes as part of the

Advanced Services download.

Below is my desktop and how MS looks as an icon :-

When we launch MS by a double click on the icon on the desktop we see the following at startup :-

===================================================================================================

You will notice that I am actually using the version of MS which comes with "Denali" - or SQL Server 2012.

You may not be aware however that we have set the Privilege Level as shown below :-

Now for the BIG shock ! :

After struggling through an installation (or many more as I did in months gone by) I was expecting to see our new

instance in the drop list of the opening dialog screen!

But no this is what we get today - see below :-

It seems silly now, in the cold light of day, but in my haste and desperation to find my "SQL2008_VODCtest"

in the list on offer, I forgot that the last entry of '<Browse for more...> ' is in fact what we need to select.

When we do this it can take a while for the system to search through what is available.

Also we need to uncheck the collapse icon so we actually see the list on offer in the "Browse for Servers" dialog.

Hers is genuinely what I got this morning when at first I did not see it - check this out :-

===================================================================================

Obviously we can now select what we want, now that we can see it - so choose as necessary :-

================================================================================================

Click the 'Connect' button and this is what happens - hey presto !

======================================================================================

The new MS software (which apparently was written in WPF) now shows we have an open server in the Object Explorer (OE)

pane at the left of the screen.

So all we have to do now is attach a database.

For this part of the exercise we will move on to the next subsection.

See you there for more success ...................

PHH - 14th. December 2011.


Attaching a Database

 Top Previous

*** notice *** when using Management Studio, you only see databases in the Object Explorer that you have previously

chosen and attached yourself. The MS software shell does not go looking for your SQL databases !!!

The process of making a database name appear in the Object Explorer (OE) under the 'Databases' section

is called 'Attachment' or attaching.

We save we attach a new database to the server instance in focus.

Study the following image :-

==========================================================================

Usually when we do this the default dialog file selection screen looks as follows :-

However, very often we are moving database files from another folder, or usually PC / laptop.

*** WARNING *** before you can copy an existing database (DB) from one place to another, the DB in question

must NOT be attached within a MS shell - even if the shell is closed. So you have to detach first - more on that later.

Here in the following image is a couple of files which make up a database :-

=======================================================================================================================================================

These files were copied from my other PC - the development one - and have come from a folder where me

as a user can have access to reading and writing.

It is important that you have DB files in folders which are NOT restricted for access. This can cause a LOT of grief to you

and make you think that there are all sorts of other problems - when in fact you do not have access.

I have placed my files under the username folder that I am logged on as - i.e.Philp (yes, I know, I left out an 'i' when I was

installing Windows - have just lived with it ever since I realised as by then it was written in other code and script etc..

Let us see what we have then - see below :-

So if we click on 'SmallTrader.mdf' we get the following :-

And simply by clicking the 'OK' button we get some positive results - see below :-

===========================================================================

In the above image I have expanded some regions by clicking with the mouse.

I will now show some data from the Customer table. I right click in the highlighted region to get

the context menu as shown below :-

===========================================================================

Now when we select the option to display 1000 rows we see the following appear :-

==================================================================================================================================================================

Note that a Query tab is automatically opened for us, and the query script and returned data set is displayed

in quite a nice and clear manner.

I will try and show you more about using the MS shell at Stuttgart.

For now we need to see that we can have a second instance of a SQL server open in the OE pane, and

in this case we have a 'Denali' version underneath the Server 2008 instance.

We can also have instances of versions for 2005 Servers too.

What we can't do is have these in an older version of MS software, or attach a 2005 DB to say a 2008 Server, or similar

mismatches.

MS is backwards compatible, but Databases need to be converted. It is not our worry at the moment, we need to focus

on ADO.NET and Data Binding.

Well, "that is that" ... we have come a long way today, BUT, I wish you to realise that everything I have shown you was

done live by me as I captured the screen images and carried out a 'live install'.

The DB files were genuinely moved and attached, and the connection made and ata found - all live stuff.

So if I can do this the so can you ;-0)) Good Luck !

Time to move on, see you in the next section ................

PHH - 14th. December 2011.