PowerShell: Synchronizing a Folder (and Sub-Folders) Part 3

I’ve used the Sync-Folder script quite a bit and I thought it might be time to revisit it, clean it up and add some more functionality.

Here’s what I’m adding in this first part;

  1. Load all the configuration from an XML file.  Useful to allow more complex configurations and a lot easier to edit that parameters or local variables.
  2. Proper documentation (so that Get-Help returns useful information).
  3. Allow multiple Source / Target pairs instead of just the one source folder.   Multiple copies of the same source is good for extra safe backups.

The script and explanation follows after the line.

Update : I’ve revisited this script a few times with new additions and modifications.The latest full version of the script is here.  That post also includes links covering the other revisions to the script.

I’ll go over the changes below;

<Configuration>
	<SyncPair>
		<Source>c:\temp\test</Source>
		<Target>d:\test</Target>
		<ExceptionList>
			<Exception>*.jpg</Exception>
		</ExceptionList>
	</SyncPair>
	<SyncPair>
		<Source>c:\temp\test</Source>
		<Target>d:\test2</Target>
		<ExceptionList>
			<Exception>*.jpg</Exception>
		</ExceptionList>
	</SyncPair>
</Configuration>

This is the format for the XML file I’m going to use to hold the configuration.  Under Configuration you can have multiple SyncPairs each with a Source and Target folder.  Additionally, each SyncPair also has its own ExceptionList;  you could have the same folder synced to two different locations and with one you skip the jpgs and the other you skip the gifs.

Now here’s the full, updated script.  I’ll go through the changes afterwards;

<# .SYNOPSIS Synchronises folders (and their contents) to target folders. Uses a configuration XML file. .DESCRIPTION Reads in the Configuration xml file (passed as a parameter or defaults to Sync-FolderConfiguration.xml in the script folder. .PARAMETER ConfigurationFile Holds the configuration the script uses to run. Required. .NOTES Version: 1.0 Author: HerringsFishBait.com Creation Date: 17/11/2015 .EXAMPLE Sync-Folder -d:\temp\Config.xml #>
[CmdletBinding()]
param
(
    [ValidateScript({Test-Path $_ -PathType leaf})]
    [string]$ConfigurationFile=$PSScriptRoot+"\Sync-FolderConfiguration.xml"
)
<# .SYNOPSIS Checks a file doesn't match any of a passed array of exceptions. .PARAMETER TestPath The full path to the file to compare to the exceptions list. .PARAMETER $PassedExceptions An array of all the exceptions passed to be checked. #>
function Check-Exceptions
{
    param
    (
        [parameter(Mandatory=$True)]
        [string]$TestPath,
        [string[]]$PassedExceptions
    )
    Write-Verbose "Checking $TestPath"
    $PassedExceptions | % {if($TestPath -like $_) {$Result=$True;$MatchingException=$_}}
    If ($Result) {Write-Verbose "Matched Exception : $MatchingException, skipping."}
    $Result
}
<# .SYNOPSIS Synchronises the contents of one folder to another. It recursively calls itself to do the same for sub-folders. Each file and folder is checked to make sure it doesn't match any of the entries in the passed exception list. if it does, the item is skipped. .PARAMETER SourceFolder The full path to the folder to be synchronised. .PARAMETER SourceFolder The full path to the target folder that the source should be synched to. .PARAMETER $PassedExceptions An array of all the exceptions passed to be checked. #>
function Sync-OneFolder
{
    param
    (
        [parameter(Mandatory=$True)]
        [string]$SourceFolder,
        [parameter(Mandatory=$True)]
        [string]$TargetFolder,
        [string[]]$PassedExceptions

    )
    Write-Verbose "Source Folder : $SourceFolder"
    Write-Verbose "Target Folder : $TargetFolder"
    if (!(Test-Path -Path $TargetFolder -PathType Container))
    {
        Write-Verbose "Creating Folder : $($TargetFolder)"
        New-Item $TargetFolder -ItemType "Directory" > $null
    }
    $SourceList=gci $SourceFolder
    $TargetList=gci $TargetFolder
    $SourceFiles=$TargetFiles=@()
    $SourceFolders=$TargetFolders=@()

    $SourceFiles+=$SourceList | ? {$_.PSIsContainer -eq $False -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $TargetFiles+=$TargetList | ? {$_.PSIsContainer -eq $False -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $SourceFolders+=$SourceList | ? {$_.PSIsContainer -eq $True -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $TargetFolders+=$TargetList | ? {$_.PSIsContainer -eq $True -and !(Check-Exceptions $_.FullName $PassedExceptions)}
    $MissingFiles=Compare-Object $SourceFiles $TargetFiles -Property Name
    $MissingFolders=Compare-Object $SourceFolders $TargetFolders -Property Name
    foreach ($MissingFile in $MissingFiles)
    {
        if($MissingFile.SideIndicator -eq "<=") { Write-Verbose "Copying missing file : $($TargetFolder+"\"+$MissingFile.Name)" Copy-Item ($SourceFolder+"\"+$MissingFile.Name) ($TargetFolder+"\"+$MissingFile.Name) }elseif ($MissingFile.SideIndicator="=>")
        {
            Write-Verbose "Removing file missing from Source : $($TargetFolder+"\"+$MissingFile.Name)"
            Remove-Item ($TargetFolder+"\"+$MissingFile.Name)
        }
    }
    foreach ($MissingFolder in $MissingFolders)
    {
        if ($MissingFolder.SideIndicator -eq "=>")
        {
            Write-Verbose "Removing folder  missing from Source : $($TargetFolder+"\"+$MissingFolder.Name)"
            Remove-Item ($TargetFolder+"\"+$MissingFolder.Name) -recurse -confirm:$false
        }
    }
    foreach ($SourceFile in $SourceFiles)
    {
        if ($SourceFile.LastWriteTime -gt (gci ($TargetFolder+"\"+$SourceFile.Name)).LastWriteTime)
        {
            Write-Verbose "Copying updated file : $($TargetFolder+"\"+$SourceFile.Name)"
            Copy-Item ($SourceFolder+"\"+$SourceFile.Name) ($TargetFolder+"\"+$SourceFile.Name) -Force
        }
    }
    foreach($SingleFolder in $SourceFolders)
    {
        Sync-OneFolder $SingleFolder.FullName ($TargetFolder+"\"+$SingleFolder.Name) $PassedExceptions
    }
}
Write-Verbose "Running Sync-Folder Script"
$Config=[xml](Get-Content $ConfigurationFile)
foreach ($Pair in $Config.Configuration.SyncPair)
{
    Write-verbose "Processing Pair"
    Sync-OneFolder $Pair.Source $Pair.Target $Pair.ExceptionList.Exception
}

As the script is now getting all its configuration from a file, the parameters for Source and Target folder are a bit redundant.  Instead I just want to pass the location of the configuration xml file.  While I’m at it I put some parameter validation in there to run a small script to ensure what is passed is really a file (Test-Path with PathType leaf).  If nothing is passed at all it defaults to Sync-FolderConfiguration.xml in the scripts’ local folder (given by the special $PSScriptRoot variable).

I’ve modified both the Check-Exceptions and Sync-OneFolder functions to accept an additional parameter of $PassedExceptions.  These functions don’t need to be able to read the xml file, they just need to process the exceptions.  So for the sake of simplicity I just pass an array of exceptions when they run.

One note;  the Sync-OneFolder function is called twice in the script;  once in the main body and once again by itself to process any sub-folders recursively.  The additional passed parameter (for the exceptions) needs to be added to both.  I missed the recursive call and I scratched my head for quite a while wondering why PowerShell was reporting a missing parameter when it was clearly there :).

The last major change was to the main body of the script;

Write-Verbose "Running Sync-Folder Script"
$Config=[xml](Get-Content $ConfigurationFile)
foreach ($Pair in $Config.Configuration.SyncPair)
{
    Write-verbose "Processing Pair"
    Sync-OneFolder $Pair.Source $Pair.Target $Pair.ExceptionList.Exception
}

I read the xml in via Get-Content (cast to [xml] to convert it).  Then you can treat the xml entries as objects and object properties.  I used this to enumerate all the SyncPairs in the xml and kick of the processing of Sync-OneFolder using the xml properties as the parameters.

11 thoughts on “PowerShell: Synchronizing a Folder (and Sub-Folders) Part 3”

  1. Remove-Item ($TargetFolder+”\”+$MissingFolder.Name)
    needs to have “-recurse -confirm:$false” added to it or it fails when a deleted folder had files in it.

  2. Love this script! Only have one comment on the statistics. When you run the script in the same session window and there are no changes at all, the stats will be from the last time there was a change. I am just starting to learn Powershell so I don’t know the steps to do reset the stats before it runs, however, it may be a nice addition.

  3. A really nice job, thanks !
    New to Powershell, I’d like to modify some features :
    – Is there a way to Include some files instead of the Exclude feature ?
    – How to use the environments vars ? Like the userprofile or appdata ones.
    Thanks

      1. Yes, that’s it for the filter ! Is it possible ?
        I’ll try tomorrow for the environment variables, thank you.

        Thanks

      2. Hello,

        I tried to use the environment variables but I think the variables are interpreted like text and not variables. I mean if I put “Gci Env:USERPROFILE” in the source field of the xml file and I write $Pair.Source, I get “Gci Env:USERPROFILE” instead of the path of the user profile.

        Thanks

      3. That’s the case; it won’t try and see it as a ‘command’ normally. You can modify the script to detect a command and use “Invoke-Command” instead.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s