Posts Tagged ‘Sharepoint’

Migrating Windows-auth users to Claims users in SharePoint

A short time back I published an article on upgrading a Windows-authenticated based SharePoint environment to an ADFS/Shibboleth-based claims-based environment. At that time I said I would post the script that I plan to use for the production migration when it was done. Well… here it is.

This script is based heavily on the one found here:
blog.sharepoint-voodoo.net/?p=68‎
Unfortunately, “SharePoint-Voodoo” appears to be down at the time of this writing, so I cannot make appropriate attribution to the original author. This script helped speed along this process for me… Thanks, anonymous SharePoint Guru!

My version of the script adds the following:

  • Adds stronger typing to prevent script errors.
  • Adds path checking for the generated CSV file (so that the script does not exit abruptly after running for 30 minutes).
  • Introduces options to specify different provider prefixes for windows group and user objects.
  • Introduces an option to add a UPN suffix to the new user identity
  • Collects all user input before doing any processing to speed along the process.
  • Adds several “-Limit All” parameters to the “Get-SP*” cmdlets to prevent omission of users from the migration process.

There are still some minor problems. When run in “convert” mode, the script generates an error for every migrated user, even when the user is migrated successfully. I expect this is owing to a bug in “Move-SPUser”, and there probably is not much to be done about it. Because I want to migrate some accounts from windows-auth to claims-with-windows-auth, there is some manual massaging of the output file that needs to be done before running the actual migration, but I think this is about as close as I can get to perfecting a generic migration script without making the end-product entirely site-specific.

I will need to run the script at least twice… once for my primary “CAMPUS” domain, and once to capture “GUEST” domain users. I may also want to do a pass to convert admin and service account entries to claims-with-windows-auth users.

Set-PSDebug -Strict
add-pssnapin microsoft.sharepoint.powershell -erroraction 0

# Select Options
Write-Host -ForegroundColor Yellow "'Document' will create a CSV dump of users to convert. 'Convert' will use the data in the CSV to perform the migrations."
Write-Host -ForegroundColor Cyan "1. Document"
Write-Host -ForegroundColor Cyan "2. Convert"
Write-Host -ForegroundColor Cyan " "
[int]$Choice = Read-Host "Select an option 1-2: "

switch($Choice)
{
    1 {[bool]$convert = $false}
    2 {[bool]$convert = $true}
    default {Write-Host "Invalid selection! Exiting... "; exit}
}
Write-Host ""

$objCSV = @()
[string]$csvPath = Read-Host "Please enter the path to save the .csv file to. (Ex. C:\migration)"
if ((Test-Path -LiteralPath $csvPath) -eq $false) {
	Write-Host "Invalid path specified! Exiting..."; exit
}

if($convert-eq $true)
{
	$objCSV = Import-CSV "$csvPath\MigrateUsers.csv"

    foreach ($object in $objCSV)
    {
        $user = Get-SPUser -identity $object.OldLogin -web $object.SiteCollection 
        write-host "Moving user:" $user "to:" $object.NewLogin "in site:" $object.SiteCollection 
        move-spuser -identity $user -newalias $object.NewLogin -ignoresid -Confirm:$false
    }
}
else
{
	[string]$oldprovider = Read-Host "Enter the Old Provider Name (Example -> Domain\ or i:0#.f|MembershipProvider|) "
    [string]$newprovider = Read-Host "Enter the New User Provider Name (Example -> Domain\ or i:0e.t|MembershipProvider|) "
	[string]$newsuffix = Read-Host "Enter the UPN suffix for the new provider, if desired (Example -> @domain.com) "
	[string]$newGroupProvider = Read-Host "Enter the New Group Provider Name (Example -> Domain\ or c:0-.t|MembershipProvider|domain.com\) "


    # Select Options
    Write-Host -ForegroundColor Yellow "Choose the scope of the migration - Farm, Web App, or Site Collection"
    Write-Host -ForegroundColor Cyan "1. Entire Farm"
    Write-Host -ForegroundColor Cyan "2. Web Application"
    Write-Host -ForegroundColor Cyan "3. Site Collection"
    Write-Host -ForegroundColor Cyan " "
    [int]$scopeChoice = Read-Host "Select an option 1-3: "

    switch($scopeChoice)
    {
        1 {[string]$scope = "Farm"}
        2 {[string]$scope = "WebApp"}
        3 {[string]$scope = "SiteColl"}
        default {Write-Host "Invalid selection! Exiting... "; exit}
    }
    Write-Host ""
    if($scope -eq "Farm")
    {
        $sites = @()
        $sites = get-spsite -Limit All
    }
    elseif($scope -eq "WebApp")
    {
        $url = Read-Host "Enter the Url of the Web Application: "
        $sites = @()
        $sites = get-spsite -WebApplication $url -Limit All
    }
    elseif($scope -eq "SiteColl")
    {
        $url = Read-Host "Enter the Url of the Site Collection: "
        $sites = @()
        $sites = get-spsite $url
    }

    foreach($site in $sites)
    {
		$webs = @() #needed to prevent the next foreach from attempting to loop a non-array variable
        $webs = $site.AllWebs

        foreach($web in $webs)
        {
            # Get all of the users in a site
			$users = @()
            $users = get-spuser -web $web -Limit All #added "-limit" since some webs may have large user lists.

            # Loop through each of the users in the site
            foreach($user in $users)
            {
                # Create an array that will be used to split the user name from the domain/membership provider
                $a=@()
                $displayname = $user.DisplayName
                $userlogin = $user.UserLogin

                if(($userlogin -like "$oldprovider*") -and ($objCSV.OldLogin -notcontains $userlogin))
                {
                    # Separate the user name from the domain/membership provider
                    if($userlogin.Contains('|'))
                    {
                        $a = $userlogin.split("|")
                        $username = $a[1]

                        if($username.Contains('\'))
                        {
                            $a = $username.split("\")
                            $username = $a[1]
                        }
                    }
                    elseif($userlogin.Contains('\'))
                    {
                        $a = $userlogin.split("\")
                        $username = $a[1]
                    }
    
                    # Create the new username based on the given input
					if ($user.IsDomainGroup) {
						[string]$newalias = $newGroupProvider + $username
					} else {
						[string]$newalias = $newprovider + $username + $newsuffix
					}
                    

                    $objUser = "" | select OldLogin,NewLogin,SiteCollection
	                $objUser.OldLogin = $userLogin
                    $objUser.NewLogin = $newAlias
	                $objUser.SiteCollection = $site.Url

	                $objCSV += $objUser
                }   
            }
        }
        $site.Dispose()
    }

    $objCSV | Export-Csv "$csvPath\MigrateUsers.csv" -NoTypeInformation -Force
}

Integrating SharePoint 2013 with ADFS and Shibboleth… The Motion Picture

Time again to attempt to implement that exciting technology, Federation Services (Web Single Sign On, SAML, WS-Federation, or whatever you want to call it) with SharePoint. Last time we tried this, SharePoint 2010 was a baby product, MS was just testing the waters with SAML 2.0 support in ADFS 2.0, and Shibboleth 2 was pretty new stuff here at UVM. The whole experience was unsatisfying. SharePoint STS configuration was full of arcane PowerShell commands, ADFS setup was complicated by poor farm setup documentation, and interop of Shibboleth 2 with ADFS 2 was not documented at all.  After wading though all of that mess, we ended up with user names being displayed as “i:05.t|adfsServiceName|userPrincipalName” (bleagh!), and with many applications that could not deal with Web SSO authenticators.  My general conclusion was that SharePoint really was not ready for federated login.

Four years later, things have changed a bit.  STS configuration still requires dense PowerShell commands, but at least it is better documented.  ADFS and Shibboleth interoperability also are excellently documented at this point.  Microsoft has most Office apps working with passive SAML authenticators, and has pledged to get the rest working this year.  While I would not judge the use of ADFS (with or without Shibboleth) to be the easy route to take to SharePoint 2013 deployment, it at least looks functional at this point.  So let’s kick the tires and see how it works…

Part 1: ADFS Setup

First, we need to setup ADFS.  We chose to deploy “ADFS 3″ using Windows Server 2012 R2 as the OS platform.  ADFS 3 is required to support the new “workplace join” feature of Server 2012 R2.  Since we want to test this, we would need ADFS 3 anyway.  Unfortunately, ADFS on Server 2012 R2 is pretty virgin territory, and does not have the same troubleshooting resources available for it as do earlier releases.

Most of the configuration steps followed the TechNet documentation without variation.  We did find that we needed to modify the ACL on the service account used to run ADFS… we added the “Authenticated Users” principal to the ACL, and assigned the “Read all properties” right.

For us, the most complicating factor in ADFS 3 deployment is the replacement of IIS with the Windows Kernel-mode HTTP server “http.sys”.  When we started experiencing connectivity problems with various clients to ADFS, we had no experience with HTTP.sys to assist in troubleshooting.  Most articles on HTTP.sys relate to remote desktop services, and with Server 2003.  Our problems with HTTP.sys were rooted in an undocumented requirement for clients to submit SNI (Service Name Indicator) information in their TLS “CLIENT HELLO” sequence.  I had to open a support case with Microsoft to resolve this problem, and only afterword was I able to find any Internet discussions that reflected MS advice:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/d514b5a0-c01c-4ce4-b589-bca890e5921d/how-to-properly-setup-lb-probe-for-adfs-30-servers?forum=Geneva

It seems that if http.sys is bound using the hostname:port format, then TLS will require SNI.  If the binding is instead specified using ipAddr:port, SNI will not be required.  To fix our problem, we just needed to add a second HTTPS certificate binding using an IP address.  In this case, we just used “0.0.0.0:443″.  Here is the procedure:

  1. On each ADFS server and proxy, open an elevated command prompt
  2. run: netsh http show sslcert
  3. Record the certificate hash and application ID for the certificate used by ADFS
  4. run: netsh http add sslcert ipport=0.0.0.0:443 certhash= appid={}

Part 2: Configuring SharePoint to use ADFS

I started with Microsoft’s guide to configuring SAML authentication for SharePoint 2013 using ADFS:
http://technet.microsoft.com/en-us/library/hh305235(v=office.15).aspx
This is a well written guide, and fairly easy to follow.  The only issue I take with the article is the recommendation to use the “emailAddress” claim as the “identifier claim” in SharePoint.  In many federated login scenarios, a foreign Idp may not want to release the email address attribute.  However, some variation of UPN likely will be released.  In the case of the InCommon federation, the “ePPN” value (eduPersonPrincipalName) generally is available to federation partners.  For this reason I chose to implement “UPN” as the “identifier claim” in the last command of phase 3 of the document:

$ap = New-SPTrustedIdentityTokenIssuer -Name  -Description  -realm $realm -ImportTrustCertificate $cert -ClaimsMappings $emailClaimMap,$upnClaimMap,$roleClaimMap,$sidClaimMap -SignInUrl $signInURL -IdentifierClaim $roleClaimmap.InputClaimType

Part 3: Migrating SharePoint Users From Windows to Claims

Since we are planning an upgrade, not a new deployment, we need migrate existing Windows account references in SharePoint to federated account references. To make this happen, we need to establish the federated account identity format. I simply log in to an open-access SharePoint site as an ADFS user, and record the account information. In this case, out accounts look like this:

i:0e.t|adfs.uvm.edu|user@campus.ad.uvm.edu

and groups:

c:0-.t|adfs.uvm.edu|samAccountName

We then can use PowerShell to find all account entries in SharePoint, and use the “Move-SPUser” PowerShell cmdlet to convert them.  I am still working on a final migration script for the production cutover, and I will try to post it here when it is ready.

Of some concern is keeping AD group permissions functional.  It turns out that SharePoint will respect AD group permissions for ADFS principals, but there are a few requirements:

  1. The incoming login token needs to contain the claim type “http://schemas.microsoft.com/ws/2008/06/identity/claims/role“,
    and that this claim type needs to contain the SamAccountID (or CN) of the AD
    Group that was granted access to a site.  (In ADFS, this means that you need to release the AD LDAP Attribute “Token-Groups – Unqualified Names” as the outgoing claim type “Role”.
  2. When adding AD Groups as permissions in SharePoint, we need to use the “samAccountName” LDAP attribute as the identifier claim.  The LDAPCP (see Part 4) utility makes this easy as it will do this for us automatically when configured to search AD.

Requirement 1 could be a problem when using Shibboleth as the authentication provider.  Our Shibboleth deployment does not authenticate against AD, so a Shibboleth ticket will not contain AD LDAP “tokenGroup” data in the “role” claim.  I am working with the Shibboleth guys to see if there is any way to augment Shibboleth tokens with data pulled from AD.

Part 4:  The SharePoint PeoplePicker and ADFS

Experienced SharePoint users all know (and mostly love) the “people picker” that searches Active Directory to validate user and group names that are to be added to the access list for a SharePoint site.  One of the core problems with federation services is that they are authentication systems only.  ADFS and Shibboleth do not implement a directory service.  You cannot do a lookup on an ADFS principal that a user adds to a SharePoint site.  This is particularly irksome, since all of our ADFS users actually have matching accounts in Active Directory.

Fortunately, there is a solution… you can add a “Custom Claims Provider” into SharePoint which will augment incoming ADFS claims with matching user data pulled from Active Directory.  This provider also integrates with the PeoplePicker to allow querying of AD to validate Claims users that are being added to a SharePoint site.  A good write-up can be found here:
http://blogs.msdn.com/b/kaevans/archive/2013/05/26/fixing-people-picker-for-saml-claims-users-using-ldap.aspx

“But I don’t want to compile a SharePoint solution using Visual Studio,” I hear you (and me) whine.  No problem… there is a very good pre-build solution available from CodePlex:
http://ldapcp.codeplex.com/discussions/361947

Normally I do not like using third-part add-ons in SharePoint.  I will make an exception for LDAPCP because:

  1. It works.
  2. It saves me hours of Visual Studio work.
  3. It is a very popular project and thus likely to survive on CodePlex until it is no longer needed.
  4. If the project dies, we can implement our own Claims Provider using templates provided elsewhere with (hopefully) minimal fuss.

My only outstanding problem with LDAPCP is that it will not query principals in our Guest AD forest.  However, there are some suggestions from the developer along these lines:
http://ldapcp.codeplex.com/discussions/478092

To summarize, the developer recommends compiling our own version of LDAPCP from the provided source code.   We would use the method “SetLDAPConnections” found in the “LDAPCP_Custom.cs” source file to add an additional LDAP query source to the solution.  I will try this as time permits.

Part 5: Transforming Shibboleth tokens to ADFS

So far we have not strayed too far from well-trodden paths on the Internet.  Now we get to the fun part… configuring ADFS as a relying party to our Shibboleth Idp, then transforming the incoming Shibboleth SAML token into an ADFS token that can be consumed by SharePoint.

Microsoft published a rather useful guide on ADFS/Shibboleth/InCommon integration:
http://technet.microsoft.com/en-us/library/gg317734(WS.10).aspx
Using this guide we were able to set up ADFS as a relying party to our existing Shibboleth Idp with minimal fuss.  Since we already have an Idp, we skipped most of Step 1, and then jumped to Step 4 as we did not need to configure Shibboleth as an SP to ADFS.

I had the local “Shibboleth Guy” add our ADFS server to the relying parties configuration file on the Shib server, and release “uvm-common” attributes for this provider. This allows SharePoint/ADFS users to get their “eduPersonPrincipalName” (ePPN) released to ADFS/SharePoint from Shibboleth. However, SharePoint (and ADFS) do not natively understand this attribute, so we configure a “Claim Rule” on the “Claims Provider Trust” with Shibboleth.  The rule is an “Acceptance Transformation Rule” that we title “Transform ePPN to UPN”, and it has the following syntax:

c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.6"] 
=> issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", 
Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, 
ValueType = c.ValueType);

The “urn:oid:1.3.6.1.4.1.5923.1.1.1.6″ bit is the SAML 2 identifier for the ePPN type.

After configuring a basic Relying Party trust with SharePoint 2013, I need to configure Claims Rules that will release Shibboleth User attributes/claims to SharePoint.  You could use a simple “passthough” rule for this.  However, I want incoming Shibboleth tokens that have a “@uvm.edu” UPN suffix to be treated as though they are Active Directory users.  To accomplish this, I need to do a claims transformation. In AD, the user UPN has the “@campus.ad.uvm.edu” suffix, so let’s transform the Shibboleth UPN using a Claim Rule on the SharePoint Relying Party Trust:

c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", 
Value =~ "@uvm\.edu$"]
=> issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", 
Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, 
Value = regexreplace(c.Value, "^(?[^@]+)@(.+)$", "${user}@campus.ad.uvm.edu"), 
ValueType = c.ValueType);

This appears to work… the first RegEx “@uvm\.edu$” should match an incoming UPN that ends with “@uvm.edu”. In the second set of regExps, we create a capture group for the user portion of the UPN (which is everything from the start of the value up to (but not including) the “@” character), and then replace everything trailing the user portion with “@campus.ad.uvm.edu”.

However, as noted above, the incoming Shibboleth SAML token does not contain AD group data in the “Role” attribute, so users authenticating from Shibboleth cannot get access to sites where they have been granted access using AD groups.  Boo!

Part 6:  Where are you from?  Notes on Home Realm Discovery

When an ADFS server has multiple “Claims Provider Trusts” defined, the ADFS login page automatically will create a “WAYF”, or “Where are you from?” page to allow the user to select from multiple authentication providers.  In our case, the user would see “Active Directory” and “UVM Shibboleth”.  Since I would not want to confuse people with unnecessary choices, we can disable the display of one of these choices using PowerShell:

Set-AdfsRelyingPartyTrust -TargetName "SharePoint 2013" -ClaimsProviderName "UVM Shibboleth"

In this sample, “SharePoint 2013″ is the name of the relying party defined in ADFS for which you want to set WAYF options.  “UVM Shibboleth” is the Claims Provider Trust that you want used.  This value can be provided as an array, but in this case we are going to provide only one value… the one authenticator that we want to use.  After configuring this change, ADFS logins coming from SharePoint get sent straight to Shibboleth for authentication.

Part 7: The Exciting Results

Only a sysadmin could call this exciting…

Given how heavily MS invested in implementing WS-Federation and WS-Trust into their products (MS Office support for federation services was, to the best of my knowledge, focused entirely on the WS-* protocols implemented in ADFS 1.0), I was not expecting any client beyond a web browser to work with Shibboleth.  Imagine my surprise…

Browsers:

IE 11 and Chrome both login using Shib with no problems.  Firefox works, but not without a glitch… upon being redirected back to SharePoint from our webauth page, we get a page full of un-interpreted html code.  Pressing “f5″ to refresh clears the problem.

Office 2013 Clients:

All core Office 2013 applications appear to support opening of SharePoint documents from links in the browser.  Interestingly, it appears that Office is able to share ADFS tokens obtained by Internet Explorer, and vice versa.  The ADFS token outlives the browser session, too, so you actually have to log off of ADFS prevent token re-use.  I tested the “Export to Excel” and “Add to Outlook” options in the SharePoint ribbon, and both worked without a fuss.

Getting Office apps to open content in SharePoint directly also works, although its a bit buggy.  Sometimes our webauth login dialog does not clear cleanly after authentication.

SkyDrive Pro (the desktop version included with Office 2013) (soon to be “OneDrive for Business) also works with Shibboleth login, amazingly.  The app-store version does not work with on-premises solutions at all, so I could not test it.

Mobile Clients:

I was able to access a OneNote Notebook that I stored in SharePoint using OneNote for Android.  However, it was not easy.  OneNote for Android does not have a dialog that allow for the adding of notebooks from arbitrary network locations.  I first had to add the notebook from a copy of OneNote 2013 for Windows that was linked to my Microsoft account.  The MS account then recorded the existence of the notebook.  When I logged in to OneNote on the Android, it picked up on the SharePoint-backed notebook and I was able to connect.

The OneNote “metro” app does not appear to have the same capability as the Android app.  I cannot get it to connect to anything other than Office 365 or CIFS-backed files.

I was unable to test Office for iOS or Android because I do not own a device on which those apps are supported.

I still need to look at the “Office Document Connection” that comes with Office for the Mac, and at WebDAV clients, and perhaps some other third-party SharePoint apps to see if they work.

Coping with Renamed user Accounts in sharepoint

Yesterday I received a strange error report from a person trying to create a new SharePoint site collection.  Our front line guy went to investigate and found that she was getting a “User cannot be found” error out of SharePoint when attempting to complete the self-service site creation process.  This person reported that her last name changed recently, along with her user ID, yet SharePoint will still showing her as logged in under her old name.

Linking the “Correlation ID” up to the diagnostic logs was of no great help.  The diagnostic logs simply reported “User cannot be found” when executing the method “Microsoft.SharePoint.SPSite.SelfServiceCreateSite”.  We are able to see that “ownerLogin”, “ownerEmail”, and “ownerName” strings were being passed to this function, but not what the values of those strings were.  I guessed that the web form was passing the person’s old account login name to the function, and that since this data was no longer valid, an error was getting displayed.  But how to fix this?

SharePoint 2010 (and WSS 3.0 before it) keeps a list of Site Users that can be accessed using the SharePoint Web “SiteUsers” property. This list is updated every time a new user logs in to the site.  The list entries contain username, login identity, email address, and security ID (SID) data.  It also appears that Site User data is not updated when user data changes in Active Directory (as long as the SID stays the same, that is).  Additional user account data is stored in XML data in the SharePoint databases, and can be accessed using the SharePoint Web “SiteUserInfoList” property.  All of this data needs to be purged from the root web site so that our hapless user can once again pass valid data to the SelfServiceCreateSite method.

Presumably the Site Management tools could be forced to get the job done, but the default views under SharePoint 2010 are hiding all site users from me, even when I log in as a site administrator.  Let’s try PowerShell instead:

add-pssnapin microsoft.sharepoint.powershell 
$root = get-spweb -identity "https://sharepoint.uvm.edu/" 

# "Old ID" below should be all or part of the user's original login name: 
$oldAcc = $root.SiteUsers | ? {$_.userLogin -match "oldID"} 
#Let's see if we found something: 
$oldAcc.LoginName 

#Remove the user from the web's SiteUsers list: 
$root.SiteUsers.Remove($oldAcc.LoginName) 
$root.Update() 
#Let's see if it worked: 
$id = $oldAcc.ID 
$root = get-spweb -identity "https://sharepoint.uvm.edu/" 
$root.SiteUsers.GetByID($id) 
# (This should return a "User cannot be found" error.) 

#Now to see what is in SiteUserInfoList: 
$root.SiteUserInfoList.GetItemById($id) 
# (This data can be cleaned up in the browser by visiting:
# " /_catalogs/users/simple.aspx" 
# from your site collection page.)

SharePoint 2010 – Email Alerts to Site Administrators

We are in the final stages of preparation for the long-overdue upgrade to SharePoint 2010.  I have set up a preview site with a copy of the production SharePoint content database, and I want to notify all site owners that they should check out their sites for major problems.  How to do?  PowerShell?  Absolutely!


Set-PSDebug -Strict
Add-PSSnapin -Name microsoft.SharePoint.PowerShell

[string] $waUrl = "https://sharepoint2010.uvm.edu"
[string] $SmtpServer = "smtp.uvm.edu"
[string] $From = "saa-ad@uvm.edu"

$allAdmins = @()

[string] $subjTemplate = 'Pending Upgrade for your site -siteURL-'
[string] $bodyTemplate = @"
Message Body Goes Here.
Use the string -siteURL- in the body where you want the user's site address to appear.
"@

$wa = Get-SPWebApplication -Identity $waUrl

foreach ($site in $wa.sites) {
	#Write-Host "Working with site: " + $site.url
	$siteAdmins = @()
	$siteAdmins = $site.RootWeb.SiteAdministrators
	ForEach ($admin in $siteAdmins) {
		#Write-Host "Adding Admin: " + $admin.UserLogin
		[string]$a = $($admin.UserLogin).Replace("CAMPUS\","")
		[string]$a = $a.replace(".adm","")
		[string]$a = $a.replace("-admin","")
		[string]$a = $a.replace("admin-","")
		if ($a -notmatch "sa_|\\system") { $allAdmins += , @($a; [string]$site.Url) }
	}
	$site.Dispose()
}

$allAdmins = $allAdmins | Sort-Object -Unique
#$allAdmins = $allAdmins | ? {$_[0] -match "jgm"} | Select-Object -Last 4

foreach ($admin in $allAdmins) {
	[string] $to = $admin[0] + "@uvm.edu"
	[string] $siteUrl = $admin[1]
	[string] $subj = $subjTemplate.Replace("-siteURL-",$siteUrl)
	[string] $body = $bodyTemplate.Replace("-siteURL-",$siteUrl)
	Send-MailMessage -To $to -From $From -SmtpServer $SmtpServer -Subject $subj -BodyAsHtml $body
}

SharePoint 2010 – Authentication and Browser Support Planning

We have gone back to the drawing board with SharePoint 2010 planning, and now are challenging some ideas about how authentication must be configured for SharePoint to work with our clients.  Previously, we felt the need to provide multiple supported authenticaiton types (Windows, Basic, and Forms) hosted on different IIS web sites, with unique URL’s (sharepoint, sharepointlite, partnerpoint).  We also felt the need to have a version of sharepoint with “Client Integration” features enabled, and one with these features disabled (sharepointlite).  With changes in SharePoint 2010, is this really necessary?

First, let’s look at the Windows vs. Basic assumption.  Are there any browsers out there that do not support Windows/NTLM authentication?  In fact, there are… the Android mobile browser and, um…, well, there do not appear to be any others.  However, it is not necessary to make a separate web site available to allow basic authentication.  If we enable basic auth on an exsiting Windows-auth web site, browsers that do not support Windows auth suddenly start working. (Update, 2013-02-36:  Chrome for Android now supports NTLM authentication, so there appears to be no need to support Basic authentication at all at this point in time.)

Now, let’s look at the client integrated vs. dumbed-down assumption.  Previously, we wanted to ensure the clients handled links to MS Office documents stored in SharePoint in a predictable fashion.  Users of Firefox and Safari frequently complained about “strange” document handling behavior in SharePoint links.  For people experiencing confusion caused by failed attempts to launch Office applications from SharePoint browser links, we exposed a version of SharePoint that had the client integration features disabled.  HOWEVER, under SharePoint 2010, the client integration features are much more reliable.  On Windows, I am able to make use of Office 2010 client integration in both IE9 and Firefox.  On the Mac, client integration works with Office 2011 Mac and Safari 5 or FireFox 8.  Additionally, SharePoint will detect if a browser cannot support client integration, and will disable Office inegration links automatically.  For example, the “Open in Word” link is greyed-out in my Chrome browser, while the “Download file” link is active.

Add to this the new mobile web version of SharePoint 2010.  If you connect from a browser listed in the “compat.browser” file on the SharePoint web server, you get directed to a light-weight mobile version of SharePoint instead.  See: http://blogs.technet.com/b/office2010/archive/2010/03/09/configure-sharepoint-server-2010-for-mobile-device-access.aspx for more details on how “compat.browser” works.  This version will use Office Web Apps to render Office documents, but editing will not be possible.  This grants mobile users a functional (if somewhat limited) access method for SharePoint content, while at the same time sidestepping the issue of client integration.  It also means that we do not have to deal with the hassle of attempting to ensure that Office Web Apps will work in a myriad of underpowered mobile browsers.

All things considered, things are looking bright for SharePoint 2010.  It seems we no longer will need a “lite” version of SharePoint, and we will not need two URLs to support the varying authentication needs of legacy browsers.

SharePoint site use confirmation URL

We have been using the SharePoint automatic site usage confirmation and deletion of Windows SharePoint Services 3.0. An annoying aspect of this feature (and there are many) is that if a site that was use in use gets deleted, and we subsequently restore the site, the site still is not confirmed as in use and will get deleted on the next site deletion timer job.

To combat this, we need to acknowledge the site as in use manually. Unfortunately, MS does not document the site usage confirmation URL anywhere. Fortunately, I was able to dig it out eventually:

Just use the following URL as a template, and substitute the name of the site to confirm in place of [sitename]:

https://sharepoint.uvm.edu/sites/[sitename]/_layouts/useconfirmation.aspx

NTLMv2 – Troubleshooting Notes – NTLM Negotiation is a lie!

We are now in the process, several years late, of trying to disallow use LM and NTLM authentication on our campus. First attempt? Train Wreck!

For stage 1 of enforcement we chose to restrict our domain controllers to accept only NTLMv2 authentication, and reject LM/NTLM. Our thinking was that clients are supposed to negotiage for the the best auth security available, so by forcing NTLMv2 at the DC, we would be able to quickly flush out non-compliant clients. Futher, we assumed that there would not be a lot of non-compliant clients. Guess what? We were WRONG WRONG WRONG

You know which clients are non-compliant? Windows 7, IE8, IE8, FireFox 4b10, Chrome 9. What? Really? It’s true… when accessing IIS web servers configured to use Windows authentication, with NTLMv2 enforced on the authenticating server, even Windows 7 will negotiate down to the weakest possible NTLM level.   So… access to SharePoint 2007 from IE 9 beta on Win 7 falls back to welcome-to-the-80s LM authentication.  This is very shocking, and annoying. When inspecting authentication negotiation traffic to the web server, we see that both client and server have indicated that NTLMv2 is supported, and that LM should not be used. However, when the client sends its authentication challenge response, it sends NTLM and LM responses only and not NTLMv2. Only by setting local security policy on the Windows client to “NTLMv2 only” are we able to prevent the sending of LM/NTLM. Negotiation is a lie!

So, we are going to take this us with MS Support. Some resources that may be helpful going forward:

http://technet.microsoft.com/en-us/magazine/2006.08.securitywatch.aspx
A good explanation of how the “LMCompatibilityLevel” security setting works.  Contains one major factual error… it claims that clients and servers will always negotiate the highest mutually-support LM auth level.  This may be true for SMB/CIFS.  It certainly is not true for web auth.

http://kb.iu.edu/data/bamu.html
Over at Indiana University, LM/NTLM has been disabled for some time.  They have thorough documentation on configuring workstations to function properly in their more restrictive environment.  This page concerns the use of SharePoint at IU, and the required configuration settings for various browsers and operating systems for their network.  There is nothing specific to our problem here, but there is the implication that they experienced problems similar to our own.  All of their docs state that you must configure your client to perform NTLMv2 only, or authentication will fail.  Also included are notes on FireFox and Mac platforms, where client-side configuration apparently is required.
http://kb.iu.edu/data/atpq.html
An additional page that suggests that network clients will not be able to authenticate to IU resources unless they disabled NTLMv2 on the client.

Concerning FireFox on Windows:
http://forums.mozillazine.org/viewtopic.php?f=23&t=1648755
A developer notes that as of FF 3.6, FF uses Microsoft APIs for NTLM authentication.  So, if FF is not performing NTLM auth as you expect, you need to look to your OS for the source of the problem.  It’s true… we confirmed via packet capture that FF 4.0b10 and IE9RC both perform NTLM authentication in exactly the same way.

Concerning Macintosh client settings:
http://discussions.apple.com/thread.jspa?threadID=2369451
Recommended smb.conf file settings to enforce use of NTLMv2 on the Mac.  References are made here to an “nsmb.conf” file, which is not a typographic error.  More on this file here:
http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man5/nsmb.conf.5.html

Office Web Apps and SharePoint 2010

Office web apps are here!

Where do I get them? From Microsoft Volume Licensing of course:
http://licensing.microsoft.com
You need to look in an unexpected place, though. Under the “Office” section, select “Office Professional Plus 2010″, then select the 32-bit edition (not 64-bit… weird!). Take note of the Web Apps product key in the product selection page.

Installation instructions can be found here:
http://technet.microsoft.com/en-us/library/ff431687.aspx#
And they are pretty good, with the exception of one PowerShell syntax error…

When you get to the stage of installing Service Application Proxies, if you choose to use the PowerShell method, you are given a small script that starts with the following command:

$appPool = Get-SPServiceApplicationPool -Name "SharePoint Web Services Default"

This command will not work as posted. “-Name” should be “-Identity”. A more reliable command might read:

$appPool = Get-SPServiceApplicationPool | where-object {$_.Name -match "SharePoint Web Services Default"}

Preparing the SharePoint 2007/WSS3 Database for Upgrade

  1. Run pre-upgrade check:
    stsadm –o preupgradecheck
    (to be run on the WSS 3 server… requires WSS 3.0 SP2 and October 2008 CU).
    Now, repair any problems that were reported…
  2. Delete orphaned sites:
    1. run: stsadm -o enumallwebs
      – find sites with “InSiteMap” set to “false”.
    2. take note of the SiteID GUID, then run:
      stsadm -o deletesite -force -siteid [siteid] -databaseserver [dbserver] -databasename [dbname]
    3. Delete references to missing web parts:
  3. The upgrade check report tells us:
    “ The following web part(s) are referenced by the content, but they are not installed on the web server Id = dcdbbbd0-8dd6-1ecb-a3b2-12d30061d482, Type = Unknown, Reference = 7, Status = Missing …”
    but there is no particular advice on how to fix the problem. I wasted a few hours trying to enumerate web parts in use by a site using PowerShell, and ended up digging around in the Content database using TransactSQL… here is what works:

    1. Run the following against the content database:
      use [contentDatabaseName]
      select DirName,LeafName from dbo.AllDocs where id in
      (select tp_PageUrlID from dbo.WebParts where
      (tp_WebPartTypeID='dcdbbbd0-8dd6-1ecb-a3b2-12d30061d482')
      OR (tp_WebPartTypeID='d5101cfe-e315-c578-cd06-1966f283e3ed')
      OR (tp_WebPartTypeID='602e7431-ac3e-75b9-c8e0-57533bdab161'))
    2. Access the page returned by the above query, appending “?Contents=1″.  Delete any web parts reporting errors.
  4. We are informed that various features are referenced by content, but that the features are not available on the server.  We are given only feature IDs.  Back to SQL:
    • use STSContentDBPrime
      select FullUrl from [Webs] where Id in (
      select WebId from [Features] where (FeatureId = 'bbe9def7-2fe9-a0b1-d712-aa128c837ebe')
      OR (FeatureId='bbe9def7-2fe9-a0b1-d7bb-aa128c837ebe')
      )
    • This returns the sites that are using these “bad features”.  But what to do about it?
    • http://www.gilham.org/Blog/Lists/Posts/Post.aspx?ID=229 suggests the use of “WSSAnalyzeFeatures” and “WSSRemoveFeatureFromSite“, both of which work to purge the evil old CKS:Enhanced Blog edition feature from the one site that was using it… sheesh!
  5. We are warned that “The following site definition(s) are referenced by the content, but they are not installed on the web server”. The name of the missing site definition is “Unknown”. We are given only a “template id” (11003) and a Language Code (1033). Finding where this definition is being used can be accomplished with a very small sql query:
    use SharePoint_Content_1 
    SELECT Title, MasterURL, WebTemplate FROM dbo.Webs where WebTemplate='11003'
    
  6. We are warned that “setup files” are referenced in the database that are not present on the server. We are given the names and paths of the files that are missing, but not the web sites that reference them. The offending site can be tracked down quickly using “stsadm”. Run this command:
    stsadm -o enumAllWebs -includesetupfiles > allWebs.txt
    Then search the resultant file for one or more of the filenames listed in the pre-upgrade check report. In this case, I was able to locate a site that used “SharePoint Learning Kit” components. This site was completely broken, since we removed the Learning Kit over a year ago. I got approval from the site owner, and deleted the offending site.
  7. We are warned that the feature “f374a3ca-f4a7-11db-827c-8dd056d89593″ is referenced by over 500 sites, but that the feature is no longer present on the server.  This feature is the “RadEditor for MOSS for IE”, which in fact has been phased out.  Lets use a handy “for” loop.
    1. First we export the SQL query we used above as a CSV file (this time searching for the “f374a…” feature instead).
    2. We use that csv as fuel for our “for” loop:
      for /f %i in (H:\BadFeatureSites_2010.csv) do WssRemoveFeatureFromSite.exe -scope site -url https://sharepoint.uvm.edu/%i -featureid f374a3ca-f4a7-11db-827c-8dd056d89593 -force
    3. Oh but look… WssRemoveFeatureFromSite can’t deal with sites that have spaces in the URL (why would you have a space in your site name, right?).  dang!  Let’s try “SharePoint Feature Administration and Clean Up Tool“… it works!
  8. We are told that we are missing some additional template files such as:
    Path = [C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Template\1033\MPS\..\sts\lists\wplib\dwp\MSContentEditor.dwp], Reference = [2], Status = [Missing]

    • Some searching discovered KB839877, which informs us that an earlier version of the “Workspace Meeting” template contained an accidental double period when a single period was intended.  But how do we find the site that is using this bad template?
    • The excellent tool “SharePoint Manager” from Carsten Keutman allows us to explore the SharePoint object model to more quickly find Site Template data that we need in a site.
    • The really really excellent tool “PowerGUI Script Editor” was used to quickly develop this script:
      # Discovers Meeting Workspaces from all "Webs" in the SharePoint web application defined in the "webAppUrl" variable.
      # Outputs discovered data to file "sitelist.csv
      # Greg Mackinnon, 2010-02-05
      [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") | Where-Object { $_.GetType().FullName -ne "System.Reflection.Assembly" }
      $webAppUrl = "https://sharepoint.uvm.edu"
      $wa = [Microsoft.Sharepoint.Administration.SPWebApplication]::Lookup($webAppURL)
      
      #[string]$out = "List of sites using the Meeting Workspace template"
      Remove-Item -Path .\sitelist.csv -Force
      
      foreach ($site in $wa.sites) {
      foreach ($web in $site.AllWebs) {
      Write-Output "Checking web: " + $web.Url
      if ($web.WebTemplate -match "MPS") {
      Out-File .\sitelist.csv -InputObject $web.ID.Guid -NoClobber -Append -Encoding Unicode
      }
      }
      $web.Dispose()
      }
      $site.Dispose()
      #Out-File .\sitelist.csv $out -Force
    • The script returns a list of “web” object GUIDs.  We feed this into SQL to discover where in the content database the sites that are using these web parts are implemented:
      use STSContentDBPrime
      --select DirName,LeafName,SetupPath from dbo.AllDocs where WebId='abee2623-56d2-43d6-8a9c-7b362fbd323b'
      select DirName,LeafName,SetupPath
      from dbo.AllDocs
      where ((WebId in ('siteGuid1','siteGuid2','...')) AND (SetupPath LIKE '%MSContentEditor.dwp%'))
    • Finally, the SQL query finds the “Meeting Workspace” sites that are using this corrupted web part.  We probably could just delete the web parts in question from the web part gallery, but I chose instead to use the stsadm.exe “export”, “deletesite”, “createsite”, and “import” commands to re-create the sites.  The export utility filtered out the corrupt content for us.  Once restored, the discovered sites no longer contained corrupted web parts.   Phew!
  9. The WSS3 instance has been prepped… Clone existing content database, and proceed with upgrade testing!

ADFS 2 and SharePoint 2010 Integration

Here is a quick entry on ADFS2 and SharePoint 2010 integration. It is not an implementation guide or end-to-end walkthough… that comes later, if we decide to implement this thing.

At present, I am most interested in the model of SharePoint->ADFS2->Shibboleth, where the SP-STS trusts tokens from ADFS2.  ADFS2 is part of a chained federation with our Shib service.  ADFS will consume Shib tokens, then transform them for the benefit of SharePoint.  However, I have no idea how to implement this solution at this time.

There are a few-too-many blog entires out there detailing how to configure ADFS2 and SharePoint 2010 for integration.  Trouble is, many of the step-by-step guides present contradictory configuration steps.  I guess there is no substitute for a deep, working knowledge of ADFS 2, SAML, and other Federation topics.

Here are some of the claims setup guides I have been working with:

Here are additional configuration posts on the process of upgrading an existing SharePoint Web Application from “Windows” authentication to “Claims” authentication.  The common denominators?

  1. You must add a valid new user to your claims-aware web app before migrating existing users, or the web application will be inaccessible after migration (or indeed, even before migration!)
  2. To trigger migration of users, you must invoke the “Migrate Users” method on your web app, E.g.:
    $wa = get-SpWebApplication "https://webappurl"
    $wa.MigrateUsers($true)

The things here that seem very unclear to me are:  What exactly is being done when you invoke the “MigrateUsers” method on the Web Application object?  How does SharePoint map legacy “Windows” users to new “Claims” users?  Anyway, here are the links:

Pages containing information that I have found useful while contemplating how to pull off SharePoint 2010:

Many of these links, as it turns out, were already discovered by members at dirteam.com.  Doh…

http://blogs.dirteam.com/blogs/jorge/archive/2010/07/06/configuring-sharepoint-2010-to-use-adfs-v2-as-an-authentication-provider.aspx