PowerShell: Synchronizing a Folder (and Sub-Folders) Part 3

I’ve used the Sync-Folder script quite a bit and I thought it might be time to revisit it, clean it up and add some more functionality.

Here’s what I’m adding in this first part;

1. Load all the configuration from an XML file.  Useful to allow more complex configurations and a lot easier to edit that parameters or local variables.
2. Proper documentation (so that Get-Help returns useful information).
3. Allow multiple Source / Target pairs instead of just the one source folder.   Multiple copies of the same source is good for extra safe backups.

The script and explanation follows after the line.

Update : I’ve revisited this script a few times with new additions and modifications.The latest full version of the script is here.  That post also includes links covering the other revisions to the script.

I’ll go over the changes below;

<Configuration>
	<SyncPair>
		<Source>c:\temp\test</Source>
		<Target>d:\test</Target>
		<ExceptionList>
			<Exception>*.jpg</Exception>
		</ExceptionList>
	</SyncPair>
	<SyncPair>
		<Source>c:\temp\test</Source>
		<Target>d:\test2</Target>
		<ExceptionList>
			<Exception>*.jpg</Exception>
		</ExceptionList>
	</SyncPair>
</Configuration>

This is the format for the XML file I’m going to use to hold the configuration.  Under Configuration you can have multiple SyncPairs each with a Source and Target folder.  Additionally, each SyncPair also has its own ExceptionList;  you could have the same folder synced to two different locations and with one you skip the jpgs and the other you skip the gifs.

Now here’s the full, updated script.  I’ll go through the changes afterwards;

<# .SYNOPSIS Synchronises folders (and their contents) to target folders. Uses a configuration XML file. .DESCRIPTION Reads in the Configuration xml file (passed as a parameter or defaults to Sync-FolderConfiguration.xml in the script folder. .PARAMETER ConfigurationFile Holds the configuration the script uses to run. Required. .NOTES Version: 1.0 Author: HerringsFishBait.com Creation Date: 17/11/2015 .EXAMPLE Sync-Folder -d:\temp\Config.xml #>
[CmdletBinding()]
param
(
    [ValidateScript({Test-Path $_ -PathType leaf})]
    [string]$ConfigurationFile=$PSScriptRoot+"\Sync-FolderConfiguration.xml"
)
<# .SYNOPSIS Checks a file doesn't match any of a passed array of exceptions. .PARAMETER TestPath The full path to the file to compare to the exceptions list. .PARAMETER $PassedExceptions An array of all the exceptions passed to be checked. #>
function Check-Exceptions
{
    param
    (
        [parameter(Mandatory=$True)]
        [string]$TestPath,
        [string[]]$PassedExceptions
    )
    Write-Verbose "Checking $TestPath"
    $PassedExceptions | % {if($TestPath -like $_) {$Result=$True;$MatchingException=$_}}
    If ($Result) {Write-Verbose "Matched Exception : $MatchingException, skipping."}
    $Result
}
<# .SYNOPSIS Synchronises the contents of one folder to another. It recursively calls itself to do the same for sub-folders. Each file and folder is checked to make sure it doesn't match any of the entries in the passed exception list. if it does, the item is skipped. .PARAMETER SourceFolder The full path to the folder to be synchronised. .PARAMETER SourceFolder The full path to the target folder that the source should be synched to. .PARAMETER $PassedExceptions An array of all the exceptions passed to be checked. #>
function Sync-OneFolder
{
    param
    (
        [parameter(Mandatory=$True)]
        [string]$SourceFolder,
        [parameter(Mandatory=$True)]
        [string]$TargetFolder,
        [string[]]$PassedExceptions

    )
    Write-Verbose "Source Folder : $SourceFolder"
    Write-Verbose "Target Folder : $TargetFolder"
    if (!(Test-Path -Path $TargetFolder -PathType Container))
    {
        Write-Verbose "Creating Folder : $($TargetFolder)"
        New-Item $TargetFolder -ItemType "Directory" > $null
    }
    $SourceList=gci $SourceFolder
    $TargetList=gci $TargetFolder
    $SourceFiles=$TargetFiles=@()
    $SourceFolders=$TargetFolders=@()

    $SourceFiles+=$SourceList | ? {$_.PSIsContainer -eq $False -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $TargetFiles+=$TargetList | ? {$_.PSIsContainer -eq $False -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $SourceFolders+=$SourceList | ? {$_.PSIsContainer -eq $True -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $TargetFolders+=$TargetList | ? {$_.PSIsContainer -eq $True -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $MissingFiles=Compare-Object $SourceFiles $TargetFiles -Property Name
    $MissingFolders=Compare-Object $SourceFolders $TargetFolders -Property Name
    foreach ($MissingFile in $MissingFiles)
    {
        if($MissingFile.SideIndicator -eq "<=") { Write-Verbose "Copying missing file : $($TargetFolder+"\"+$MissingFile.Name)" Copy-Item ($SourceFolder+"\"+$MissingFile.Name) ($TargetFolder+"\"+$MissingFile.Name) }elseif ($MissingFile.SideIndicator="=>")
        {
            Write-Verbose "Removing file missing from Source : $($TargetFolder+"\"+$MissingFile.Name)"
            Remove-Item ($TargetFolder+"\"+$MissingFile.Name)
        }
    }
    foreach ($MissingFolder in $MissingFolders)
    {
        if ($MissingFolder.SideIndicator -eq "=>")
        {
            Write-Verbose "Removing folder  missing from Source : $($TargetFolder+"\"+$MissingFolder.Name)"
            Remove-Item ($TargetFolder+"\"+$MissingFolder.Name) -recurse -confirm:$false
        }
    }
    foreach ($SourceFile in $SourceFiles)
    {
        if ($SourceFile.LastWriteTime -gt (gci ($TargetFolder+"\"+$SourceFile.Name)).LastWriteTime)
        {
            Write-Verbose "Copying updated file : $($TargetFolder+"\"+$SourceFile.Name)"
            Copy-Item ($SourceFolder+"\"+$SourceFile.Name) ($TargetFolder+"\"+$SourceFile.Name) -Force
        }
    }
    foreach($SingleFolder in $SourceFolders)
    {
        Sync-OneFolder $SingleFolder.FullName ($TargetFolder+"\"+$SingleFolder.Name) $PassedExceptions
    }
}
Write-Verbose "Running Sync-Folder Script"
$Config=[xml](Get-Content $ConfigurationFile)
foreach ($Pair in $Config.Configuration.SyncPair)
{
    Write-verbose "Processing Pair"
    Sync-OneFolder $Pair.Source $Pair.Target $Pair.ExceptionList.Exception
}

As the script is now getting all its configuration from a file, the parameters for Source and Target folder are a bit redundant.  Instead I just want to pass the location of the configuration xml file.  While I’m at it I put some parameter validation in there to run a small script to ensure what is passed is really a file (Test-Path with PathType leaf).  If nothing is passed at all it defaults to Sync-FolderConfiguration.xml in the scripts’ local folder (given by the special $PSScriptRoot variable).

I’ve modified both the Check-Exceptions and Sync-OneFolder functions to accept an additional parameter of $PassedExceptions.  These functions don’t need to be able to read the xml file, they just need to process the exceptions.  So for the sake of simplicity I just pass an array of exceptions when they run.

One note;  the Sync-OneFolder function is called twice in the script;  once in the main body and once again by itself to process any sub-folders recursively.  The additional passed parameter (for the exceptions) needs to be added to both.  I missed the recursive call and I scratched my head for quite a while wondering why PowerShell was reporting a missing parameter when it was clearly there :).

The last major change was to the main body of the script;

Write-Verbose "Running Sync-Folder Script"
$Config=[xml](Get-Content $ConfigurationFile)
foreach ($Pair in $Config.Configuration.SyncPair)
{
    Write-verbose "Processing Pair"
    Sync-OneFolder $Pair.Source $Pair.Target $Pair.ExceptionList.Exception
}

I read the xml in via Get-Content (cast to [xml] to convert it).  Then you can treat the xml entries as objects and object properties.  I used this to enumerate all the SyncPairs in the xml and kick of the processing of Sync-OneFolder using the xml properties as the parameters.