Tag Archives: DirSync

Replacing DirSync with Microsoft Azure Active Directory Sync Services

Back in October, MS announced that the new “MS Azure AD Sync Services” (AAD Sync) had gone GA, which means that DirSync is now officially deprecated.

Ironically, I got wind that “Azure AD Connect” would be replacing Azure AD Sync Services, not but a few weeks before:

http://blogs.technet.com/b/ad/archive/2014/08/04/connecting-ad-and-azure-ad-only-4-clicks-with-azure-ad-connect.aspx

Deprecated on release?  Not quite… it appears that Azure AD Sync Services will be at the core of Azure AD Connect, so time spent migrating to Azure AD Sync Services will not be a complete waste of time.

I decided to make this migration happen now, because we will be ramping up to provision faculty and staff with access to Office 365 Pro Plus software in the near future, and I would like to be working with a tool that does not require me to run it in an unsupported configuration (we removed the “Replicating Directory Changes” right from our DirSync service account, which at least at one time was considered a Bad Thing To Do.).

The Good News:

  • The out-of-box configuration does not require a service account with the “Replicating Directory Changes” right, making filtering of sensitive user attributes much easier.
  • The initial setup wizard allows you to de-select attributes that you don’t want to sync to Azure.  No more wading though dense miisclient.exe dialogs and walking off-support to do basic filtering!  The require attributes are listed in grey, so that you don’t risk filtering an attribute that Azure absolutely must have.
  • AAD Sync actually has some documentation available on its installation and use over at MSDN:
    http://msdn.microsoft.com/en-us/library/azure/dn790204.aspx
    …And there actually are quite a few important details in this seemingly cursory documentation.
  • AAD Sync entirely replaces the UI-heavy, un-maintainable attribute filtering tools in DirSync with a new “Declarative Provisioning” engine, that allow robust attribute filtering and transformation rules to be defined by the sys admin.
  • Custom “Inbound” attribute filters will be preserved during product upgrades, according to the documentation.
  • The documentation has been updated with instructions on how to make domain-based sync filters actually work in a multi-domain forest.  This was a key detail that was missing in the DirSync documentation.

The Bad News:

  • The Declarative Provisioning language, while based on VB for Applications (The “best known language by systems administrators”, according to the docs.  Whaaaat?!?!), is not actually VBA.  Debugging methods for dealing with this new VB-like thing are not at all documented, and examples of its usage are few and far between.

So what did I learn during this deployment process that was Blog-worthy?  How to debug Declarative Provisioning!

Debugging Declarative Provisioning (Import Expression Filters):

Let’s take an example.  We want to sync only students, and users who are to be provisioned for use of Lync in Office 365 into Azure.  To accomplish this, we have populated students with the text value “Student” in extensionAttribute1.  Lync users have had extensionAttribute2 populated with the text value “lync”.

When running the Synchronization Rule Editor, we create a new “inbound” rule, assign it to object type “inetOrgPerson”, link type is “join”, and give it a precedence under 100.  We skip the Scoping and Join Rules options, and go to “Transformations”.

The flowType gets set to “Expression”, the Target Attribute to “cloudFiltered”, and we define a VBA-like expression in the “Source” field:

EditSyncRule

My expression follows:

IIF(CBool(InStr([extensionAttribute1], "Student", 1)), NULL, IIF(CBool(InStr([extensionAttribute2], "lync", 1)), NULL, True)

So what is the intent here? “cloudFiltered” is a metaverse attribute that can be set to suppress synchronization of an account upon export to Azure AD.  If set to “True”, the account should not get synced to Azure.
(Note: The expression that we use was updated on 2014-12-22 as follows:

 
IIF(CBool(InStr([extensionAttribute1], "Student", 1)), NULL, IIF(CBool(InStr([extensionAttribute1], "Faculty", 1)), NULL, IIF(CBool(InStr([extensionAttribute1], "Staff", 1)), NULL, True)))

This change was necessary to allow provisioning of faculty and staff accounts with the new Office 365 ProPlus Faculty/Staff benefit.)

Our expression uses the “IIF” fuction to check to see if “extensionAttrubute1” contains “Student”.  If so, then “IIF” returns “NULL”, and NULL is fed to the “cloudFiltered” attribute (which, according to the docs, will cause the attribute to be cleared if it is already set).  However, if extensionAttribute1 does not contain “Student”, we perform a second test to see if “extensionAttribute2” contains “lync”.  If so, cloudFiltered again gets set to NULL.  If “extensionAttribute2” does not contain “lync”, then “cloudFiltered” gets set to the boolean value “True”.

Syntax quirks:

  • Items enclosed in square braces [] refer to metaverse attributes, and serve as a sort of automatic variable for provisioning.
  • Note that the InStr() function used here is not the same one documented in the VB for Applications language reference:
    http://msdn.microsoft.com/en-us/library/office/gg264811(v=office.15).aspx
    The “start position” parameter in the docs is not supported here, although it does appear that the “compare” parameter is supported (represented by the “1” in my example, which means perform a case-insensitive comparison).
  • Everything is case sensitive… even the function names!  “IIF()” is all caps.  CBool(), CStr(), InStr(), and Error() are all “Caml Cased”.
  • IIF() is a bit of a strange beast by itself.  You might need this syntax reference:
    http://msdn.microsoft.com/en-us/library/office/gg264412(v=office.15).aspx

Now that we have a filter, how to we test it?  The docs say “Run a full import followed by a full sync” on the AD management agent.  While this will work, it also is a bit time consuming.  If there are syntax errors in your expression, you will get a lot of errors in your sync logs.  Is there something better?

As it turns out, yes there is.  Special thanks to Brian Desmond (co-author of the o’Riley press “Active Directory Cookbook”) for putting me on the right track:

  1. In the MIISClient.exe, right click your AD management agent, and select “Search Connector Space”:
    SearchConnectorSpace
  2. Specify the DN of a user that you want to test your expression on in the search box.  You can quickly get a DN using PowerShell with the ActiveDirectory module:
    Get-ADUser -Identity [samAccountName]
    Click “Search”, click the returned object, and click “Preview”:
    PreviewUser
  3. In the “Preview” dialog, select either a Full or Delta sync, then click “Generate Preview”, and select the “Import Attribute Flow” option on the left.  Review the list of Sync Rules on the bottom pane to make sure your rule was processed.  Any rule that generates a change will be displayed in the top pane:
    ImportFlow
    In this case, the expression did not generate a change in the existing metaverse object.
  4. If an expression that you have authored generates an error, a full stack trace will appear in the Error Details section of the Preview pane.  Scroll down to the bottom of the stack, where it is likely that the actual problem with your expression will be identified:
    PreviewError
    In this example, I inserted an “Error()” function with a null argument.  The language engine did not like this.
  5. Back in the “Start Preview” tab, you could “Commit Preview” to apply the filter rules for this specific object, and then view the results in the Metaverse Search component of miisclient.exe.

Using the preview tool, I was able to get my filter expressions working correctly with minimal fuss and delay.  The final hurdle to getting filtered attributes set correctly was understanding what the various “run profiles” in AAD Sync actually do.

  • AD Agent – Full Import:  Appears to sync data from AD to the AAD Sync Metaverse without applying Declarative Provisioning Filters.  I think.  Not 100% on this one.
  • AD Agent – Full Synchronization:  Appears to apply Declarative Provisioning filters on all metaverse objects.
  • Azure AD Agent – Full Sync: Appears to apply export filtering logic to metaverse objects entering the Azure connector space.
  • Azure AD Agent – Export:  Appears to sync metaverse objects to Azure without applying filtering logic.  My impression is that you must do a “sync” before an export, if you want logic applied to Azure AD objects as you intended.

My understanding of the run profiles may be flawed, but I will note that if I perform syncs in the following sequence, I get the expected results:

AD Agent:Full Import -> AD Agent:Full Sync -> Azure Agent:Full Sync -> Azure Agent:Export

I am not sure what the “authoritative” way to perform synchronizations might be, but this works for me. When trying other sequences, I have seen strange results such as metaverse objects not getting updated with the intended filter results, or objects getting declared as “disconnectors” by the Azure AD Agent, and thus getting deleted from Azure.  Ugh!

That’s all for today… hopefully this info will help keep someone else out of trouble, too.

Provisioning students with Office 365 ProPlus licenses

NOTE:  This post was updated on 2014-12-23 to reflect refinements in the PowerShell provisioning script. The script now has support for provisioning O365 ProPlus for faculty/staff under the new Office 365 Faculty/Staff Benefit.

Interesting… I still seem to be working at UVM. There must be a story there, but you won’t read about it here.

Anyway, after getting back from my brief hiatus at Stanford University, I got back on the job of setting up Azure DirSync with Federated login to our in-house Web SSO platforms. I’ll need to post about the security changes required to make that work with UVM’s FERPA interpretation. To summarize, we got it working.

However, once students can log in to Office 365, we need to provision them with licenses. DirSync can’t do this, so I needed to script a task that will grant an office 365 ProPlus license to any un-provisioned active student. You will find the script, mostly unaltered, below. I just set it up as a scheduled task that runs sometime after the nightly in-house Active Directory update process.

To be useful outside of UVM, the code would need to be customized to handle the logic used in your organization to determine who is a student. We have extended the AD schema to include the eduPerson schema, and have populated the “eduPersonPrimaryAffiliation” attribute with “Student” for currently active students. If you do something different, have a look at the “Get-ADUser” line, and use a different LDAP query to fetch your student objects.

Enjoy.

# Provision-MSOLUsers.ps1 script, by J. Greg Mackinnon, 2014-07-30
# Updated 2014-11-20, new license SKU, corrections to error capture commands, and stronger typing of variables.
# Updated 2014-11-21, added "license options" package to the add license command, for granular service provisioning.
# Updated 2014-12-22, Now provisions student, faculty, and staff Office 365 Pro Plus with different SKUs.
#
# Provisions all active student accounts in Active Directory with an Office 365 ProPlus license.
#
# Requires:
# - PowerShell Module "MSOnline"
# - PowerShell Module "ActiveDirectory"
# - Azure AD account with rights to read account information and set license status
# - Credentials for this account, with password saved in a file, as detailed below.
# - Script runs as a user with rights to read the eduPersonAffiliation property of all accounts in Active Directory.
#
#    Create a credential file using the following procedure:
#    1. Log in as the user that will execute the script.
#    2. Execute the following line of code in PowerShell:
#    ConvertTo-SecureString -String 'password' -AsPlainText -Force | ConvertFrom-SecureString | out-file "c:\pathToCreds\msolCreds.txt" -Force
#

Set-PSDebug -Strict

#### Local Variables, modify for your environment: ####
#
[string] $to = 'saa-ad@uvm.edu'
[string] $from = 'Provisioning@myserver.mydomain.edu'
[string] $smtp = 'smtp.mydomain.edu'
[string] $msolUser = 'dirsync@myTennant.onmicrosoft.com'
[string] $searchBase = 'ou=people,dc=mydomain,dc=edu'
[string] $ea1Filter = '(&(ObjectClass=inetOrgPerson)(|(extensionAttribute1=*Student*)(extensionAttribute1=*Staff*)(extensionAttribute1=*Faculty*)))'
### Example Filters:
# Filter for all fac/staff/students: 
#   (&(ObjectClass=inetOrgPerson)(|(extensionAttribute1=*Student*)(extensionAttribute1=*Staff*)(extensionAttribute1=*Faculty*)))
# Filter for just students:
#   (&(ObjectClass=inetOrgPerson)(extensionAttribute1=*Student*))
#
#### End Local Variables ##############################


#### Initialize logging and script variables: #########
#initialize log and counter:
[string[]] $log = @()
[long] $pCount = 0

#initialize logging:
[string] $logFQPath = "c:\local\temp\provision-MSOLUsers.log"
New-Item -Path $logFQPath -ItemType file -Force

[DateTime] $sTime = get-date
$log += "Provisioning report for Office 365/Azure AD for: " + ($sTime.ToString()) + "`r`n"


#### Define Functions:  ###############################
#
function errLogMail ($err,$msg) {
    # Write error to log and e-mail function
    # Writes out the error object in $err to the global $log object.
    # Flushes the contents of the $log array to file, 
    # E-mails the log contents to the mail address specified in $to.
    [string] $except = $err.exception;
    [string] $invoke = $err.invocationInfo.Line;
    [string] $posMsg = $err.InvocationInfo.PositionMessage;
    $log +=  $msg + "`r`n`r`nException: `r`n$except `r`n`r`nError Position: `r`n$posMsg";
    $log | Out-File -FilePath $logFQPath -Append;
    [string] $subj = 'Office 365 Provisioning Script:  ERROR'
    [string] $body = $log | % {$_ + "`r`n"}
    Send-MailMessage -To $to -From $from -Subject $subj -Body $body -SmtpServer $smtp
}

function licReport ($sku) {  
    $script:log += 'License report for: ' + $sku.AccountSkuId 
    $total = $sku.ActiveUnits
    $consumed = $sku.ConsumedUnits
    $script:log += 'Total licenses: ' + $total  
    $script:log += 'Consumed licenses: ' + $consumed  
    [int32] $alCount = $total - $consumed
    $script:log += 'Remaining licenses: ' + $alCount.toString() + "`r`n"
}
#
#### End Functions  ###################################

#Import PS Modules used by this script:
try {
    Import-Module MSOnline -ErrorAction Stop ;
} catch {
    $myError = $_
    [string] $myMsg = "Error encountered loading Azure AD (MSOnline) PowerShell module."
    errLogMail $myError $myMsg
    exit 101
}

try {
    Import-Module ActiveDirectory -ErrorAction Stop ;
} catch {
    $myError = $_
    [string] $myMsg = "Error encountered loading ActiveDirectory PowerShell module."
    errLogMail $myError $myMsg
    exit 102
}

#Get credentials for use with MS Online Services:
try {
    $msolPwd = get-content C:\pathToCreds\msolCreds.txt | convertto-securestring -ErrorAction Stop ;
} catch {
    $myError = $_
    [string] $myMsg = "Error encountered getting creds from file."
    errLogMail $myError $myMsg
    exit 110
}

try {
    $msolCreds = New-Object System.Management.Automation.PSCredential ($msolUser, $msolPwd) -ErrorAction Stop ;
} catch {
    $myError = $_
    [string] $myMsg = "Error encountered in generating credential object."
    errLogMail $myError $myMsg
    exit 120
}
#Use the following credential command instead of the block above if running this script interactively:
#$msolCreds = get-credential

#Connect to MS Online Services:
try {
    #ErrorAction set to "Stop" for force any errors to be terminating errors.
    # default behavior for connection errors is non-terminating, so the "catch" block will not be processed.
    Connect-MsolService -Credential $msolCreds -ErrorAction Stop
} catch {
    $myError = $_
    [string] $myMsg = "Error encountered in connecting to MSOL Services."
    errLogMail $myError $myMsg
    exit 130
}
$log += "Connected to MS Online Services.`r`n"

#Generate license report:
$studAdv = Get-MsolAccountSku | ? {$_.accountSkuId -match 'STANDARDWOFFPACK_IW_STUDENT'}  
$facStaffBene = Get-MsolAccountSku | ? {$_.accountSkuId -match 'OFFICESUBSCRIPTION_FACULTY'} 
licReport($studAdv)
licReport($facStaffBene)

#Set license options for student and faculty/staff SKUs: 
# NOTE: License options for a SKU can be enumerated by examining the "ServiceStatus" property of the MsolAccountSku objects fetched above using "Get-MsolAccountSku".
$stuLicOpts = New-MsolLicenseOptions -AccountSkuId $studAdv.AccountSkuId -DisabledPlans YAMMER_EDU,SHAREPOINTWAC_EDU,SHAREPOINTSTANDARD_EDU,EXCHANGE_S_STANDARD,MCOSTANDARD
$facStaffLicOpts = New-MsolLicenseOptions -AccountSkuId $facStaffBene.AccountSkuId -DisabledPlans ONEDRIVESTANDARD

#Retrieve active fac/staff/student accounts into a hashtable:
[hashtable] $adAccounts = @{}
try {
    #$NOTE:  The filter used for collecting students needed to be modified to fetch admitted students that are not yet active
    #   This is a 'hack' implemented by FCS to address the tendency for the registrar not to change student status until the first day of class.
    #   (Actual student count should be lower, but we have no way to know what the final count will be until the first day of classes.)
    #
    #get-aduser -LdapFilter '(&(ObjectClass=inetOrgPerson)(eduPersonAffiliation=Student))' -SearchBase 'ou=people,dc=campus,dc=ad,dc=uvm,dc=edu' -SearchScope Subtree -ErrorAction Stop | % {$students.Add($_.userPrincipalName,$_.Enabled)}
    get-aduser -LdapFilter $ea1Filter -Properties extensionAttribute1 -SearchBase $searchBase -SearchScope Subtree -ErrorAction Stop | % {$adAccounts.Add($_.userPrincipalName,$_.extensionAttribute1)}
} catch {
    $myError = $_
    $myMsg = "Error encountered in reading accounts from Active Directory."
    errLogMail $myError $myMsg
    exit 200
}
$log += "Retrieved accounts from Active Directory."
$log += "Current account count: " + $adAccounts.count


#Retrieve unprovisioned accounts from Azure AD:
[array] $ulUsers = @()
try {
    #Note use of "Synchronized" to suppress processing of cloud-only accounts.
    $ulUsers += Get-MsolUser -UnlicensedUsersOnly -Synchronized -All -errorAction Stop
} catch {
    $myError = $_
    $myMsg = "Error encountered in reading accounts from Azure AD. "
    errLogMail $myError $myMsg
    exit 300
}
$log += "Retrieved unlicensed MSOL users."
$log += "Unlicensed user count: " + $ulUsers.Count + "`r`n"

#Provision any account in $ulUsers that also is in the $adAccounts array:
foreach ($u in $ulUsers) {
    #Lookup current unlicensed user in the AD account hashtable:
    $acct = $adAccounts.item($u.UserPrincipalName)
    if ($acct -ne $null) {
        #Uncomment to enable verbose logging of user to be processed.
        #$log += $u.UserPrincipalName + " is an active student."
        try {
            if ($u.UsageLocation -notmatch 'US') {
                #Set the usage location to the US... this is a prerequisite to assigning licenses.
                $u | Set-MsolUser -UsageLocation 'US' -ErrorAction Stop ;
                #Uncomment to enable verbose logging of usage location assignments.
                #$log += 'Successfully set them usage location for the user. '
            }
        } catch {
            $myError = $_
            $myMsg = "Error encountered in setting Office 365 usage location to user. "
            errLogMail $myError $myMsg
            exit 410
        }
        try {
            # Note: Order of if/elseif logic determines which SKU "wins" for people with multiple affiliations (both student and faculty/staff)
            # At present student affiliation "wins", which is preferable because we have 1 million student licenses and only 6 thousand fac/staff.
            # If we change licensing options in the future, we will want to revisit this logic.
            if ($acct -match 'Student') {
                #Uncomment to enable verbose logging of Student license assignments.
                #$log += 'Setting Student license options for user...'
                $u | Set-MsolUserLicense -AddLicenses $studAdv.AccountSkuId -LicenseOptions $stuLicOpts -ErrorAction Stop ;
            } elseif ($acct -match 'Faculty|Staff') {
                #Uncomment to enable verbose logging of Fac/Staff license assignments.
                #$log += 'Setting Fac/Staff license options for user...'
                #Assign the student advantage license to the user, with desired license options
                $u | Set-MsolUserLicense -AddLicenses $facStaffBene.AccountSkuId -LicenseOptions $facStaffLicOpts -ErrorAction Stop ;
            }
            #Uncomment to enable verbose logging of license assignments.
            #$log += 'Successfully set the Office license for user: ' + $u.UserPrincipalName
            $pCount += 1
        } catch {
            $myError = $_
            $myMsg = "Error encountered in assigning Office 365 license to user. "
            errLogMail $myError $myMsg
            exit 420
        }
    } else {
        $log += $u.UserPrincipalName + " does not have an active AD account.  Skipped Provisioning."
    }
    Remove-Variable acct
}

#Add reporting details to the log:
$eTime = Get-Date
$log += "`r`nProvisioning successfully completed at: " + ($eTime.ToString())
$log += "Provisioned $pCount accounts."
$tTime = new-timespan -Start $stime -End $etime
$log += 'Elapsed Time (hh:mm:ss): ' + $tTime.Hours + ':' + $tTime.Minutes + ':' + $tTime.Seconds

#Flush out the log and mail it:
$log | Out-File -FilePath $logFQPath -Append;
[string] $subj = 'Office 365 Provisioning Script:  SUCCESS'
[string] $body = $log | % {$_ + "`r`n"}
Send-MailMessage -To $to -From $from -Subject $subj -Body $body -SmtpServer $smtp