SQL 2005 setup on 2k3EE R2 cluster

Here are a few notes from my initial setup of SQL Server 2005, Standard Edition on our new IBM LS20 blades.

Our blades are equiped with 2x dual core AMD Opteron processors, two Qlogic FC HBAs, two Broadcom 57xx Ethernet ports, and 8Gb of RAM. We thusly are using Server 2003 64-bit edition for the install.

Server install and setup:

  • Installed OS from the BladeCenter Management Module, mounting Server 2003 ISO file from a neighboring blade’s web browser. It is slow and ugly, but works.
  • On first boot, we have no functioning network or FC HBA. I downloaded the latest Broadcom application ISO from IBM, mounted via managment module, and ran the installer. Also ran the “Broadcom Advanced Services” installer, then configured an Active/Active NIC team with VLANs for public networking (720) and the cluster heartbeat (4000).
  • On the heartbeat network, we disable File and Print services, as well as Client for Microsoft Networks. Be sure to disable NetBIOS over TCP as well. LAN speed duplex does not appear to be configurable with this driver, but the switch ports have been set to negotiate for 1Gbps speed only, and PortFast is on to reduce port blocking time on initial NIC connect.
  • Once networking is functional, I am able to transfer Qlogic HBA drivers to the server, and install them in the Device Manager control panel.
  • We also transfer the DS4000 Storage Manager for Windows installer to the server (obtained from IBM.com), and then perform a client install of the software (this installs the RDAC multi-path I/O driver onto the server).
  • Configure LUNs on the DS4800 storage server, configure LUN masking, and configure Zoning on the Brocade switches. SAN volumes are now available on the server.

Cluster Configuration:

  • Setup Microsoft Distributed Transaction Coordinator as a cluster resource as well. Strictly follow instructions in MS KB301600. Created 1Gb lun for this task – drive letter “T”
  • Setup a cluster group for MSSQL server, add physical disk resources for MDF and LDF files (database and transaction logs) (Drives M: and L:)

SQL Setup:

  • Ensure that you are logged on to only one node of the cluster – setup will fail if there are active RDP sessions on both cluster nodes.
  • From http://support.microsoft.com/?kbid=916760, create “Servers” and “Tools” directories for CD1 and CD2 contents, respectively.
  • Run SQL Setup from the primary node in the cluster, source files extracted to our primary file server using the base path \\\software\sql2005server.
  • Start with an installation of Workstation Components only… need to run this install on each node separately
  • Run SP1 patch on both nodes to update SQL Setup Support files.
  • Install all remaining SQL components (Database, Analysis services, Reporting, Notification, Integration). Select option to create failover cluster for Database and Analysis services, select drive M: as the destination for data files.
  • Configure service accounts for every service
  • Set server to “mixed” authentication mode for support of some of our apps
  • Keep default collation settings (this uses SQL Collations for the Database service and Windows Collations (Latin_CS_AS) for Analysis Services).
  • NOTE:Reporting Services will have to be configured after SQL install
  • enabled automatic error reporting but NOT usage statistics.

Next Steps:

  • http://msdn2.microsoft.com/en-us/library/ms171338.aspx – Review use of clustering with Notification Services
  • Review Reporting Services deployment options:
    • Separate Reporting Services Instance for each application that uses it?
    • Central Reporting Services web cluster? (Would require 2x additional Windows 2003 EE installs!)

VMWare Virtual Infrastructure 3 and IBM Director

Here are some links and notes regarding configuring VI 3 for use with IBM Director 5.10:

Good IBM Redbook on VI 3 and Director 5.10:
http://www.redbooks.ibm.com/redbooks/SG247190/wwhelp/wwhimpl/java/html/wwhelp.htm

More official guide to using ESX 2.5 with Director:
ftp://ftp.software.ibm.com/pc/pccbbs/pc_servers_pdf/managingvmware.pdf

Several management methods are available for ESX hosts. The Redbook focuses on using the Director Agent for Linux. The apparent reason for this are:

  • Support for hardware alerting though the agent is becomming available with ESX3. This was not the case with ESX2.5, so the agent did not do much for you.
  • “Level 0″ agents are managed though SSH. Under ESX 3, root login is not allowed via SSH, The linux agent relies on root access, so this agentless management method no longer works.
  • SNMP is the only other management method, and who wants to go there?

WDS – booting from UFD (USB Flash Drive)

Booting to WDS can be a bit pokey over lower-bandwidth connections. Even over GbE, WDS boot images take significantly longer to load than the original RIS menu system. Bummer!

I expedite loading of the “WDS Discover” boot image (this is the image that allows loading of WDS server images) I decided to try creating a bootable “WDS Discover” image on UFD. The Vista Beta 2 BDD (and included AIK) include instructions on generating WinPE-bootable UFDs. However, there are two problems:

  • The instructions do not work… the resuling UFD is not bootable!
  • The image generated results only in a standard WinPE environment, not a WDS deployment GUI

Here are the missing bits fo informaiton:
When formatting a UFD for WinPE, you must set the formatted partition as “active”. To do this, plug your UFD into a Vista OS:
diskpart
select disk 1
select partition 1
format fs=ntfs override quick
active
assign

Failing to make the UFD partition “active” will result in an unbootable UFD. Note that the Vista build 5472 AIK includes these updated instructions.

In order to make a UFD-bootable WDS Discover image, you need to use the “WDSUtil” CLI (with /new-discoverimage or /new-captureimage options) or WDS MMC GUI.
Here are the steps:

  1. Use ImageX to extract the Windows Vista SETUP image from the boot.wim that ships with the Vista DVD (e.g. “imagex /apply [path to boot.wim>] 2 [path to apply image]”)
  2. Apply custom drivers (e.g. “peimg /inf: [path to driver] [path to applied image “windows” directory]”)
  3. Capture the custom image (e.g. “imagex /boot /compress maximum /capture [path to applied image] [wim file name] [image name]”)
  4. Import the image into WDS using the GUI or CLI
  5. Use WDSUtil with /new-CaptureImage or /new-DiscoverImage and /Image:[image Name] to create a new capture or discovery image based on the custom WIM generated above.

You can now write your image to CD (using “oscdimg”) or to UFD (using the BDD/AIK instructions).

NOTE! If you expect to be able to capture using the WDS Capture image, you must first run “sysprep” on the OS in question. The WDS Image Capture Wizard will not work without sysprep!

Resetting corrupted performance counters

Performance counters on SYSIMG1 became corrupted recently. The MS KB is full of strange and dated advice on the subject. Here is the most recent article I could find:
http://support.microsoft.com/kb/300956/en-us

It is full of tedious and laborious steps to manually repair the performance counter object mappings. Towards the bottom of the article, we learn that on Server 2003, the following console command will restore corrupted counters, thus obviating the need for any manual config steps:
LODCTR /R

This will re-read all of the perfXXX.ini files in the system32 directory and rebuild the counter data. If you have any apps running that do not house their .ini files in “system32″ you will need to load those manually.

In any event, the above command fixed the problem on SYSIMG1 quite nicely. Thanks, MS!

RIS – adding "text mode" storage drivers

We had some fun adding Intel SATA drivers for the new Intel chipsets onto our RIS server. These drivers are required in order to deploy RIPrep-generated images onto the new Gateway M285 Tablet PCs.

Here were the key steps, as outlined in this TechNet Article:
http://technet2.microsoft.com/WindowsServer/en/library/96969653-bd0f-44d7-af4f-f95c3016d2be1033.mspx?mfr=true”

  • In the “flat” image for the platform to be imaged (in our case, the flat “Tablet PC 2005″ image) create a $OEM$ directory with a “textmode” subdirectory.
  • Copy the SATA storage driver into the textmode folder. Make sure that the entry in the [disk] section of the TXTSETUP.OEM file in this directory does not contain a “relative path” on the end of entry. (i.e. the line should end with a simple “\”).
  • Copy the exact text of the driver name from the [scsi] section into the [MassStorageDrivers] section of the Flat image’s i386\templates\ntstndrd.sif file. Also add the following lines:
    #Added this
    OemPnPDriversPath="\\%SERVERNAME%\RemInst\%INSTALLPATH\$OEM$\textmode";\Drivers\Nic
    DUDisable=no
    DriverSigningPolicy=ignore
    [MassStorageDrivers]
    "Intel(R) 82801GBM SATA AHCI Controller (Mobile  ICH7M)"="OEM"
    
  • Also in the [unattended] section, set the OemPreinstall option to “yes”

Some interesting things that we discovered in making this work:

  1. PNP NIC drivers that are accessed during the initial NDIS driver load can be added to ANY flat image on the server. A Tablet PC will access drivers from the “XP SP2″ Flat image. If you have more than one version of the same driver in different folders, only the first alphabetically listed driver will be made avaialble to RIS net boot clients.
  2. Text-mode setup drivers (which generally are mass storage drivers) are loaded when setup.exe starts in earnest, after the NIC load process. These drivers must be available in the $OEM$ directory of the platform for which you are loading an image (RIPrep or Flat). If you are loading an RIPrep image for a Tablet PC, you must have appropriate drivers available in the Tablet PC flat image.

WDS testing – firewall requirements

I have installed an instance of WDS on our current production RIS server. No worries, RIS functionality is still present, fully unadulterated.

However, I seem to be having problems getting a PXE boot client in my office to load. PXE boot works, and the WIM image gets loaded onto the workstation. However, I get a “unable to communicate with the WDS server” error after the GUI loads.

Some Ethereal packet captures show that the client is sending a port map request to the server, and the server is telling the client to connect to port “5040”. This value is not in our range of pre-allocated RPC ports, so I assumed that this port is hard-coded into the WDS service.

Sure enough, a quick registry search reveals that this is a parameter of the WDSServer service:
\\HKLM\Software\System\CurrentControlSet\Services\WDSServer\Parameters\RpcPort
The default value was “5040”. I changed it to be within our range of excepted RPC ports, then ran:
net stop WDSServer
net start WDSServer

Lo and behold, I can now boot to WDS and install Vista. Cool. Now they just need to get the bugs out and give us the RTM WIM files. We will be ready!

Note: Net booting the install WIM is kinda pokey. I think I will try making a bootable USB drive next. I think this might be our best option for mass distribution.

WSS v3 upgrade testing

I am attempting to model the upgrade of our existing WSS 2.0 site to 3.0 using the current “beta 2″ release. Fun so far:

  • The installer for WSS 3 refuses to run claiming that I need Workflow Foundation v2.2 installed and ASP.NET 2.0 enabled on my site. Workflow foundation can be downloaded from www.microsoft.com/downloads. The ASP.NET issue was a bit more puzzling, since I throught I already had .NET 2.0 installed on the test server. A quick investigation revealed that it was installed, but not configured for or activated in IIS. I just ran the .NET framework installer again and now the installer runs.
  • all of the pre-release documentation and the current “readme” file are available at http://www.microsoft.com/downloads. Irritating that they did not come with the beta 2 download… I had to hunt them down.
  • After install, configuration failed. I had to run the “prescan” tool manually. The report that was generated showed that two sites were “broken”. Interestingly, I could not find any evidence that these sites actially exist when using either “stsadm -o enumsites” or the “Sharepoint Explorer” utility. Some googling revealed the concept of “site oprhans”:
    http://blogs.msdn.com/krichie/archive/2005/10/25/484889.aspx
    It is claimed that there is a tool available to fix this:
    http://support.microsoft.com/kb/918743/
    But as luck would have it our MS Essential Support has lapsed and Procuremetn has not processed the service renewal request. Dang! I will just fix the problem manually on the test server, but we will need that hotfix for Prod! Manual fix entails removing the content database from the virtual server, then adding it back in. This forces the Config database to remove all current entries, then recreate them.
  • next problem is vexing… I get this error during the upgrade process:
    An exception of type System.Runtime.InteropServices.COMException was thrown. Additional exception information: The Indexer property in the MSSConfiguration table is not set to the name of this machine.
    System.Runtime.InteropServices.COMException (0xC0041229): The Indexer property in the MSSConfiguration table is not set to the name of this machine.
    I just can’t figure this one yet. Google is worthless.

WinPE2 with BDD 3.0 beta

MS has released new builds of the Vista deployment tools with the “Business Desktop Deployment Solutions Accelerator 3.0 Beta” (AKA BDD 3.0 beta. This pack includes the “Windows AIK” that we have looked at previously. Good! The last version needed a lot of improvement.

Here are the steps that I have followed to build a new ISO, usable in VMWare Server Beta and VMWare Workstation 5.5…

  1. Install the BDD 3.0 pack
  2. from c:\program files\bdd vista\waik, install the WAIK using “startcd.exe”.
  3. Using the included CHM files, follow the “Walkthough: Create a bootable Windows PE RAM Disk on CD-ROM”:
    1. from c:\program files\windows aik\tools\petools, run:
      copype x86 c:\winpe_x86
    2. step 2 in this procedure claims that you can copy EXE files (such as ImageX.exe) to c:\winpe_x86\iso, and they will then be included in the image…
      NOTE: These files show up on the “D:” drive after booting into WinPE. The reason for putting additional tools here is to reduce the memory footprint of WinPE. Everything in the X: drive is loaded into RAM at boot time!
    3. before actually making the iso, apply some customizations:
      1. Mount the WinPE WIM file:
        imagex /apply c:\winpe_x86\winpe.wim 1 c:\winpe_x86\mount
      2. Install drivers:
        peimg /inf=c:\drivers\vmware\vmxnet\win2k\vmware-nic.inf c:\winpe_x86\mount\Windows
        C:\winpe_x86_2>peimg /inf=c:\drivers\vmware\vmxnet\win2k\vmxnet.inf c:\winpe_x86\mount\Windows
      3. Install optional components
        peimg /install=*srt* c:\winpe_x86\mount\windows
        peimg /list c:\winpe_x86\mount\windows
        (shows currently installed optional packages
      4. If you want ImageX (or any other utilities) in the RAM disk, put them there now:
        Copy files from c:\program files\windows aik\tools\x86 to c:\winpe_x86\mount\windows\system32
        (This includes the imagex.exe utility. I suppose these really should go in the “programs” directory, to separate them from the actual OS files, but this way they are in the path so the idiot end user does not have to know where the ImageX utility is, just how to use it).
      5. Prep the PE image (don’t ask me why, it is in the directions!):
        peimg /prep c:\winpe_x86\mount\Windows
      6. capture the mounted PE instace to a WIM:
        imagex /append c:\winpe_x86\mount c:\winpe_x86\winpe.wim "MyWinPE" /verify
        (Actually appends this instance to an existing WIM)
      7. Export the appended image to a bootable WIM file:
        imagex /boot /export c:\winpe_x86\winpe.wim 2 c:\winpe_x86\ISO\sources\boot.wim
        (NOTE: the instructions on “custom images” fails to mention that you must use the “/boot” flag here!)
      8. Create the ISO:
        oscdimg -n –bc:\winpe_x86\etfsboot.com c:\winpe_x86\ISO c:\winpe_x86\winpe_x86.iso