This is the latest version of my PowerShell folder synchronisation script.
I’ve revisited this script a few times with new additions and modifications. If you want explanations about how the script was developed over time or how parts of the script work, have a look at the following posts;
Part 1 covers the basic script to sync two folders.
Part 2 adds the ability to specify exceptions that the sync process will skip.
In part 3 I added an XML configuration file so more complicated processes can be run.
I customised and validated parameter input (parameter sets) and added some proper error checking in part 4.
The script was updated to deal with non-standard paths and to output objects reporting on the changes it made in part 5.
In part 6 I add more use of literal paths (to prevent path errors) and generate statistics.
I add a filter option, turn strict-mode on and optimise the statistics generation in part 7 (plus fix a bug!).
In part 8 I’ve added some validity checks on the XML configuration file.
WhatIf functionality was added in Part 9 plus some corrections to make sure Filter and Exceptions worked correctly.
In Part 10 I added text file logging.
In Part 11 I fixed an odd bug where $Nulls were being added to arrays and added a -PassThru switch.
Part 12 covers some fixes for behaviour and adding the ability to support multiple target folders (so the source folders can be synced to multiple locations).
Part 13 added more control over what gets logged and how, added a timestamp to the log filename, creating a new log on each run, code cleanup and some bug- fixes.
A zipped version of the script is here;
As above you can pass a configuration XML file to the script instead of just Source and Target parameters;
<Configuration> <SyncPair> <Source>C:\synctest\source1</Source> <Target>C:\synctest\dest1</Target> <Target>C:\synctest\dest2</Target> <Filter>*.txt</Filter> <ExceptionList> <Exception>*p234*.txt</Exception> </ExceptionList> </SyncPair> <SyncPair> <Source>C:\synctest\source2</Source> <Target>C:\synctest\dest2</Target> <Filter>*.txt</Filter> </SyncPair> </Configuration>
Note that you need to use WildCards of some description in the Filter and Exception parts. So if you want it to skip all paths with “Old” in it add “Old” to the exceptions list.
Note that the config file supports multiple target folders but you can also sync to multiple folders from the cmd line as well;
Sync-Folder.ps1 -SourceFolder d:\tempa -TargetFolders "d:\temp2","d:\temp3"
Here’s the script itself;
<#
.SYNOPSIS
Synchronises folders (and their contents) to target folders. Uses a configuration XML file (default) or a pair of
folders passed as parameters.
.DESCRIPTION
Reads in the Configuration xml file (passed as a parameter or defaults to
Sync-FolderConfiguration.xml in the script folder.
.PARAMETER ConfigurationFile
Holds the configuration the script uses to run.
.PARAMETER SourceFolder
Which folder to synchronise.
.PARAMETER TargetFolder
Where to sync the source folder to.
.PARAMETER Exceptions
An array of path names to skip from synchronisation. Accepts wild-cards (*.jpg, c:\temp\*.jpg etc).
.PARAMETER Filter
An array of path names to only process via synchronisation. Accepts wild-cards (*.jpg, c:\temp\*.jpg etc).
.PARAMETER LogFile
A logfile to write to. Defaults to LogFile.txt in the script's folder.
.PARAMETER LoggingLevel
0=Only Errors and Warnings, 1=Changes made, 2=All Items
.PARAMETER LogToScreen
If specified show the desired logs on the screen
.NOTES
1.0
HerringsFishBait.com
17/11/2015
1.1
Fixed path check to use LiteralPath
Added returning status object throughout
1.2 4/Aug/2016
Added LiteralPath to the Get-ChildItem commands
Added totals to report on what was done
1.3 6/10/2016
Added StrictMode
Set $Changes to an empty collection on script run to reset statistics
Rewrote Statistics
Added $Filter option
1.4 4/11/2016
Added Get-PropertyExists function to make sure parts of the config XML are not missing.
1.5 13/01/2017
Fixed Type in Tee-Object that was preventing statistics showing correctly
1.6 20/01/2017
Fixed Filters not working if not specified in config file
Fixed Exceptions not working in some cases in Exception file
Added Write-Verbose on all the passed parameters to Sync-OneFolder
Added first pass at WhatIf
1.7 03/03/2017
Added Write-Log function to write output to file
1.8
Fixed bug in copying matched files
"$MatchingSourceFile= $SourceFiles | Where-Object {$_.Name -eq $TargetFile.Name}"
Made most logs not write to the file (for performance)
Fixed a bug where not all the statistics were recorded when a configuration XML was used.
1.9 09/05/2017
Corrected bug where an error was generated if there were no Changes
Added -PassThru switch to return objects
1.10 22/10/2017
Added "-Attributes !ReparsePoint" to Get-ChildItem lines to avoid traversing Symbolic Links
(as per Scotts suggestion in the comments!)
Added LiteralPath lines to the Test-Path commands to stop errors where the path has odd characters
in it (as per Roberts suggestion in the comments)
Added the ability to have multiple targets in the XML file or passing an array to $TargetFolders which
allows the sync to happen to multiple targets
1.11 17/04/2018
Changed defaults so that only changes or errors are written to the screen or logs
1.12 06/01/2020
Changed verbs to recommended ones and removed comparisons that flag warnings (although they run correctly)
Added a timestamp to the text log and has the script generate a new log each time it's run
Made MissingFiles and MissingFolders start as an array no matter the result; this will stop a single entry making them hold a single object and
not a collection (making For-Each results odd)
Cleaned up logging
Added "LoggingLevel" parameter to control how much goes to the logs; 0= errors only, 1=errors and changes and 2 is everything
Added LogToScreen
Added additional error-handling when the source and target folders were returned
.EXAMPLE
Sync-Folder -configurationfile:"d:\temp\Config.xml"
.EXAMPLE
Sync-Folder -SourceFolder:c:\temp -TargetFolder:d:\temp -Exceptions:"*.jpg"
#>
[CmdletBinding(DefaultParameterSetName="XMLFile")]
param
(
[parameter(
ParameterSetName="XMLFile")]
[ValidateScript({Test-Path -LiteralPath $_ -PathType leaf})]
[string]$ConfigurationFile=$PSScriptRoot+"\Sync-FolderConfiguration.xml",
[parameter(
Mandatory=$True,
ValueFromPipelineByPropertyName=$True,
ParameterSetName="FolderPair")]
[string]$SourceFolder,
[parameter(
Mandatory=$True,
ValueFromPipelineByPropertyName=$True,
ParameterSetName="FolderPair")]
[string[]]$TargetFolders,
[parameter(
ParameterSetName="FolderPair")]
[string[]]$Exceptions=$Null,
[parameter(
ParameterSetName="FolderPair")]
[string]$Filter="*",
[ValidateScript({Test-Path -LiteralPath $_ -PathType leaf -IsValid})]
[string]$LogFile=$PSScriptRoot+"\SyncFolderLog.txt",
[int]$LoggingLevel=1,
[switch]$LogToScreen=$false,
[switch]$PassThru=$False,
[switch]$Whatif=$False
)
set-strictmode -version Latest
<#
.SYNOPSIS
This writes verbose or error output while also logging to a text file.
.DESCRIPTION
This writes verbose or error output while also logging to a text file.
.PARAMETER Output
The string to write to the log file and Error / Verbose streams.
.PARAMETER IsError
If this switch is specified the $Output string is written to the Error stream instead
of Verbose.
.PARAMETER Heading
Makes the passed string a heading (gives it a border)
.PARAMETER Emphasis
Puts an emphasis character on either side of the string to output.
.PARAMETER WriteHost
Writes the output to the host instead of the verbose stream
.PARAMETER NoFileWrite
Does not write this output to the files
#>
function Write-Log
{
[CmdletBinding()]
param
(
[Parameter(
ValueFromPipeline=$true)]
[String]$Output="",
[switch]$IsError=$False,
[switch]$IsWarning=$False,
[switch]$Heading=$False,
[switch]$Emphasis=$False,
[switch]$WriteHost=$False,
[switch]$NoFileWrite=$False,
[switch]$IsInfo=$False
)
BEGIN
{
$TitleChar="*"
}
PROCESS
{
if(($IsInfo -and $LoggingLevel -gt 0) -or $IsError -or $IsWarning)
{
$FormattedOutput=@()
if ($Heading)
{
$TitleBar=""
#Builds a line for use in a banner
for ($i=0;$i -lt ($Output.Length)+2; $i++)
{
$TitleBar+=$TitleChar
}
$FormattedOutput=@($TitleBar,"$TitleChar$Output$TitleChar",$TitleBar,"")
}elseif ($Emphasis)
{
$FormattedOutput+="","$TitleChar$Output$TitleChar",""
}else
{
$FormattedOutput+=$Output
}
if ($IsError)
{
$PreviousFunction=(Get-PSCallStack)[1]
$FormattedOutput+="Calling Function: $($PreviousFunction.Command) at line $($PreviousFunction.ScriptLineNumber)"
$FormattedOutput=@($FormattedOutput | ForEach-Object {(Get-Date -Format HH:mm:ss.fff)+" : ERROR " + $_})
$FormattedOutput | Write-Error
}elseif ($IsWarning)
{
$FormattedOutput=@($FormattedOutput | ForEach-Object {(Get-Date -Format HH:mm:ss.fff)+" : WARNING " + $_})
$FormattedOutput | Write-Warning
}else
{
$FormattedOutput=$FormattedOutput | ForEach-Object {(Get-Date -Format HH:mm:ss.fff)+" : " + $_}
if ($WriteHost)
{
$FormattedOutput | Write-Host
}else
{
$FormattedOutput | Write-Verbose
}
}
if (!$NoFileWrite)
{
if (($Null -ne $Script:LogFileName) -and ($Script:LogFileName -ne ""))
{
$FormattedOutput | Out-File -Append $Script:LogFileName
}
}
}
}
END
{
}
}
<#
.SYNOPSIS
Checks a file doesn't match any of a passed array of exceptions.
.PARAMETER TestPath
The full path to the file to compare to the exceptions list.
.PARAMETER PassedExceptions
An array of all the exceptions passed to be checked.
#>
function Test-Exceptions
{
param
(
[parameter(Mandatory=$True)]
[ValidateScript({Test-Path -LiteralPath $_ -IsValid })]
[string]$TestPath,
[string[]]$PassedExceptions
)
$Result=$False
$MatchingException=""
if ($Null -eq $PassedExceptions)
{
Return $False
}
Write-Log "Checking $TestPath against exceptions" -IsInfo:($LoggingLevel -gt 1)
foreach ($EnumeratedException in $PassedExceptions)
{
if($TestPath -like $EnumeratedException)
{
$Result=$True
$MatchingException=$_
}
}
If ($Result)
{
Write-Log "Matched Exception : $MatchingException, skipping." -IsInfo -WriteHost:$LogToScreen
}
$Result
}
<#
.SYNOPSIS
Creates an object to be used to report on the success of an action
#>
function New-ReportObject
{
New-Object -typename PSObject| Add-Member NoteProperty "Successful" $False -PassThru |
Add-Member NoteProperty "Process" "" -PassThru |
Add-Member NoteProperty "Message" "" -PassThru
}
<#
.SYNOPSIS
Returns if a property of an object exists.
.PARAMETER Queryobject
The object to check the property on.
.PARAMETER PropertyName
The name of the property to check the existance of.
#>
function Get-PropertyExists
{
param
(
[PSObject]$Queryobject,
[string]$PropertyName
)
Return (($Queryobject | Get-Member -MemberType Property | Select-Object -ExpandProperty Name) -contains $PropertyName)
}
<#
.SYNOPSIS
Synchronises the contents of one folder to another. It recursively calls itself
to do the same for sub-folders. Each file and folder is checked to make sure
it doesn't match any of the entries in the passed exception list. if it does,
the item is skipped.
.PARAMETER SourceFolder
The full path to the folder to be synchronised.
.PARAMETER SourceFolder
The full path to the target folder that the source should be synched to.
.PARAMETER PassedExceptions
An array of all the exceptions passed to be checked.
.PARAMETER Filter
Only files matching this parameter will be synced.
#>
function Sync-OneFolder
{
param
(
[parameter(Mandatory=$True)]
[ValidateScript({Test-Path -LiteralPath $_ -PathType Container})]
[string]$SourceFolder,
[parameter(Mandatory=$True)]
[ValidateScript({Test-Path -LiteralPath $_ -IsValid })]
[string[]]$TargetFolders,
[string[]]$PassedExceptions,
[string]$Filter="*",
[switch]$WhatIf=$False
)
Write-Log "Source Folder : $SourceFolder" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
Write-Log "Target Folder : $TargetFolders" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
Write-Log "Filter : $Filter" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
if ($null -ne $PassedExceptions)
{
Write-Log "Exceptions:" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
$PassedExceptions | ForEach-Object{Write-Log $_ -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen}
}
Foreach ($TargetFolder in $TargetFolders)
{
Write-Log "Checking For Folders to Create" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
if (!(Test-Path -LiteralPath $TargetFolder -PathType Container))
{
$Output=New-ReportObject
Write-Log "Creating Folder : $($TargetFolder)" -IsInfo -WriteHost:$LogToScreen
$Output.Process="Create Folder"
try
{
$Output.Message="Adding folder missing from Target : $TargetFolder"
Write-Log $Output.Message -IsInfo -WriteHost:$LogToScreen
New-Item $TargetFolder -ItemType "Directory" -WhatIf:$WhatIf > $null
$Output.Successful=$True
}
catch
{
$Output.Message="Error adding folder $TargetFolder)"
Write-Log $Output.Message -IsError -WriteHost:$LogToScreen
Write-Log $_ -IsError
}
$Output
}
Write-Log "Getting File Lists" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
$FilteredSourceFiles=$FilteredTargetFiles=$TargetList=@()
$FilteredSourceFolders=$FilteredTargetFolders=@()
try
{
$SourceList=Get-ChildItem -LiteralPath $SourceFolder -Attributes !ReparsePoint
}
catch
{
Write-Log "Error accessing $SourceFolder" -IsError
Write-Log $_ -IsError
$SourceList=@()
}
try
{
$TargetList=Get-ChildItem -LiteralPath $TargetFolder -Attributes !ReparsePoint
}
catch
{
Write-Log "Error accessing $TargetFolder" -IsError
Write-Log $_ -IsError
$SourceList=@()
}
$FilteredSourceFiles+=$SourceList | Where-Object {$_.PSIsContainer -eq $False -and $_.FullName -like $Filter -and
!(Test-Exceptions $_.FullName $PassedExceptions)}
$FilteredTargetFiles+=$TargetList | Where-Object {$_.PSIsContainer -eq $False -and $_.FullName -like $Filter -and
!(Test-Exceptions $_.FullName $PassedExceptions)}
$FilteredSourceFolders+=$SourceList | Where-Object {$_.PSIsContainer -eq $True -and !(Test-Exceptions $_.FullName $PassedExceptions)}
$FilteredTargetFolders+=$TargetList | Where-Object {$_.PSIsContainer -eq $True -and !(Test-Exceptions $_.FullName $PassedExceptions)}
$MissingFiles=@(Compare-Object $FilteredSourceFiles $FilteredTargetFiles -Property Name)
$MissingFolders=@(Compare-Object $FilteredSourceFolders $FilteredTargetFolders -Property Name)
Write-Log "Comparing Missing File Lists" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
foreach ($MissingFile in $MissingFiles)
{
$Output=New-ReportObject
if($MissingFile.SideIndicator -eq "<=")
{
$Output.Process="Copy File"
try
{
$Output.Message="Copying missing file : $($TargetFolder+"\"+$MissingFile.Name)"
Write-Log $Output.Message -IsInfo -WriteHost:$LogToScreen
Copy-Item -LiteralPath ($SourceFolder+"\"+$MissingFile.Name) -Destination ($TargetFolder+"\"+$MissingFile.Name) -WhatIf:$WhatIf
$Output.Successful=$True
}
catch
{
$Output.Message="Error copying missing file $($TargetFolder+"\"+$MissingFile.Name)"
Write-Log $Output.Message -IsError -WriteHost:$LogToScreen
Write-Log $_ -IsError -WriteHost:$LogToScreen
}
}elseif ($MissingFile.SideIndicator="=>")
{
$Output.Process="Remove File"
try
{
$Output.Message="Removing file missing from Source : $($TargetFolder+"\"+$MissingFile.Name)"
Write-Log $Output.Message -IsInfo -WriteHost:$LogToScreen
Remove-Item -LiteralPath ($TargetFolder+"\"+$MissingFile.Name) -WhatIf:$WhatIf
$Output.Successful=$True
}
catch
{
$Output.Message="Error removing file $($TargetFolder+"\"+$MissingFile.Name)"
Write-Log $Output.Message -IsError -WriteHost:$LogToScreen
Write-Log $_ -IsError -WriteHost:$LogToScreen
}
}
$Output
}
Write-Log "Comparing Missing Folder Lists" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
foreach ($MissingFolder in $MissingFolders)
{
if ($MissingFolder.SideIndicator -eq "=>")
{
$Output=New-ReportObject
$Output.Process="Remove Folder"
try
{
$Output.Message="Removing folder missing from Source : $($TargetFolder+"\"+$MissingFolder.Name)"
Write-Log $Output.Message -IsInfo -WriteHost:$LogToScreen
Remove-Item -LiteralPath ($TargetFolder+"\"+$MissingFolder.Name) -Recurse -WhatIf:$WhatIf
$Output.Successful=$True
}
catch
{
$Output.Message="Error removing folder $($TargetFolder+"\"+$MissingFolder.Name)"
Write-Log $Output.Message -IsError -WriteHost:$LogToScreen
Write-Log $_ -IsError -WriteHost:$LogToScreen
}
$Output
}
}
Write-Log "Copying Changed Files : $($FilteredTargetFiles.Count) to check" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
ForEach ($TargetFile in $FilteredTargetFiles)
{
Write-Log "Getting Matching Files for $($TargetFile.Name)" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
$MatchingSourceFile= $FilteredSourceFiles | Where-Object {$_.Name -eq $TargetFile.Name}
If ($null -ne $MatchingSourceFile)
{
If ($MatchingSourceFile.LastWriteTime -gt $TargetFile.LastWriteTime)
{
$Output=New-ReportObject
$Output.Process="Update File"
try
{
$Output.Message="Copying updated file : $($TargetFolder+"\"+$MatchingSourceFile.Name)"
Write-Log $Output.Message -IsInfo -WriteHost:$LogToScreen
Copy-Item -LiteralPath ($SourceFolder+"\"+$MatchingSourceFile.Name) -Destination ($TargetFolder+"\"+$MatchingSourceFile.Name) -Force -WhatIf:$WhatIf
$Output.Successful=$True
}
catch
{
$Output.Message="Error copying updated file $($TargetFolder+"\"+$MatchingSourceFile.Name)"
Write-Log $Output.Message -IsError -WriteHost:$LogToScreen
Write-Log $_ -IsError -WriteHost:$LogToScreen
}
$Output
}
}
}
Write-Log "Comparing Sub-Folders" -IsInfo:($LoggingLevel -gt 1) -WriteHost:$LogToScreen
foreach($SingleFolder in $FilteredSourceFolders)
{
Sync-OneFolder -SourceFolder $SingleFolder.FullName -TargetFolder ($TargetFolder+"\"+$SingleFolder.Name) -PassedExceptions $PassedExceptions -Filter $Filter -WhatIf:$WhatIf #
}
}
}
<#Main Program Loop#>
#Add a timestamp to the logfile
#Split off the suffix first
[string[]]$FileNameSplit=$LogFile.Split(".")
$Suffix=".txt"
if($FileNameSplit.Count -gt 1)
{
$Suffix="."+$FileNameSplit[$FileNameSplit.Count-1]
}
$Script:LogFileName=$LogFile.Split(".")[0] + "-"+[string](Get-Date -Format "yyyy-MM-dd-HH-mm")
$LogFileNameCount=0
#Check the logfilename is unique, if not add a number from 1 to 9 to it
While($LogFileNameCount -lt 8 -and (Test-Path -LiteralPath ($Script:LogFileName+$Suffix) -PathType Leaf))
{
$LogFileNameCount+=1;
$Script:LogFileName=$LogFile.Split(".")[0] + "-"+[string](Get-Date -Format "yyyy-MM-dd-HH-mm")+"-"+[string]$LogFileNameCount
}
$Script:LogFileName+=$Suffix
#If the LogFileName is STILL not unique throw and error
if(Test-Path -LiteralPath $Script:LogFileName -PathType Leaf)
{
Write-Log -IsError "Unable to create a unique LogFileName"
}else
{
Write-Log ("LogFile: " + $Script:LogFileName) -NoFileWrite -WriteHost -IsInfo
}
$ResultObjects=$Changes=$CurrentExceptions=@()
$CurrentFilter="*"
Write-Log "Running Sync-Folder Script" -NoFileWrite -IsInfo
If ($WhatIf)
{
Write-Host "WhatIf Switch specified; no changes will be made."
}
if ($PSBoundParameters.ContainsKey("SourceFolder"))
{
Write-Log "Syncing folder pair passed as parameters." -IsInfo -WriteHost:$LogToScreen
foreach ($TargetFolder in $TargetFolders)
{
$ResultObjects=Sync-OneFolder -SourceFolder $SourceFolder -TargetFolder $TargetFolder -PassedExceptions $Exceptions -Filter $Filter -WhatIf:$WhatIf |
Tee-Object -Variable Changes
}
}else
{
Write-Log "Running with Configuration File : $ConfigurationFile" -IsInfo
$Config=[xml](Get-Content $ConfigurationFile)
$FolderChanges=$Null
foreach ($Pair in $Config.Configuration.SyncPair)
{
Write-Log "Processing Pair $($Pair.Source) $($Pair.Target)" -IsInfo -WriteHost:$LogToScreen
$CurrentExceptions=$Null
If(Get-PropertyExists -Queryobject $Pair -PropertyName ExceptionList)
{
$CurrentExceptions=@($Pair.ExceptionList.Exception)
}
If(Get-PropertyExists -Queryobject $Pair -PropertyName Filter)
{
if (($null -ne $Pair.Filter) -and ($Pair.Filter -ne ""))
{
$CurrentFilter=$Pair.Filter
}
}
foreach($Target in $Pair.Target)
{
$ResultObjects=Sync-OneFolder -SourceFolder $Pair.Source -TargetFolder $Target -PassedExceptions $CurrentExceptions -Filter $CurrentFilter -WhatIf:$WhatIf |
Tee-Object -Variable FolderChanges
}
if($FolderChanges -ne $Null)
{
$Changes+=$FolderChanges
}
}
}
$FolderCreations=$FileCopies=$FileRemovals=$FolderRemovals=$FileUpdates=0
Foreach ($Change in $Changes)
{
switch ($Change.Process)
{
"Create Folder"
{
$FolderCreations+=1
}
"Copy File"
{
$FileCopies+=1
}
"Remove File"
{
$FileRemovals+=1
}
"Remove Folder"
{
$FolderRemovals+=1
}
"Update File"
{
$FileUpdates+=1
}
}
}
Write-Log "" -WriteHost -IsInfo
Write-Log "Statistics" -WriteHost -IsInfo
Write-Log "" -WriteHost -IsInfo
Write-Log "Folder Creations: `t$FolderCreations" -WriteHost -IsInfo
Write-Log "Folder Removals: `t$FolderRemovals" -WriteHost -IsInfo
Write-Log "File Copies: `t`t$FileCopies" -WriteHost -IsInfo
Write-Log "File Removals: `t`t$FileRemovals" -WriteHost -IsInfo
Write-Log "File Updates: `t`t$FileUpdates`n" -WriteHost -IsInfo
If ($PassThru)
{
$ResultObjects
}
Herring's Fishbait 
like the look of this and looks like what I’ve been looking for……. Previously used robocopy.
Would you set this as a scheduled task to keep the files/folders uptodate?
Hi. I can’t remember if I scheduled this or just ran it when I needed to update a backup. Think this was what I used to schedule PowerShell scripts;
http://blogs.technet.com/b/heyscriptingguy/archive/2012/08/11/weekend-scripter-use-the-windows-task-scheduler-to-run-a-windows-powershell-script.aspx
Cheers
Yep that’s cool, id like to see if I can get this working to sync every hour our data offsite. I’ll give it a go and let you know how I get on.
Thanks, and good luck!
ok I’m probably being a bit thick here, well that and the heat is frying my brain. But where do I put the paths for the $SourceFolder and $TargetFolder
You can do it in a couple of ways. The script is for a function, so it needs to be included in another script and called or . sourced into your current shell. So if you save that script to c:\temp\TestScript.ps1 you’d run;
. c:\temp\TestScript.ps1
Then in your PowerShell window you’d have a new function available, Sync-Folder. You could then run;
Sync-Folder -SourceFolder:”c:\temp” -TargetFolder:”d:\temp” -Verbose (this last if you want to see what it’s doing).
If you include it within an existing script just call the script with the previous line.
You could modify it by removing the initial function statement (and enclosing braces) and run the entire thing as a script;
c:\Temp\ScriptName -SourceFolder:”C:\temp” -TargetFolder:”D:\temp”
Hope this helps!
Line 16 should be: [parameter(Mandatory=$True)]
Thanks! Will update the post.
Thanks! I will have to test this.
But line 16 should be: [parameter(Mandatory=$True)]
I think 🙂
Thanks for the proof-reading! 🙂 Have corrected it.
The link at the top of the page to “Part 3” doesn’t work. I was interested in seeing your xml sample but it redirects to a wordpress login page.
Thanks. Should be fixed now.
Hi, I modified line 149 to “Remove-Item ($TargetFolder+”\”+$MissingFolder.Name) -Recurse”. I needed the -Recurse to delete removed items from subfolders without a prompt to run it as a scheduled task.
It works perfect! So thanks a lot!
You’re welcome! Glad it’s helped you out! 😀
Hi There,
I’m testing this out and I’ve noticed line 46 has an “=” instead of the powershell “-eq” comparision. I think the elseif should look like this –
elseif ($MissingFile.SideIndicator -eq “=>”)
instead of
elseif ($MissingFile.SideIndicator=”=>”)
Thanks; have corrected it. I struggle to keep the code formatting in shape going from the ISE to WordPress so I sometimes miss things like that. Thanks again!
I should have thanked you for the function also! Very useful. I’m using it in a script I’m building. I had to translate some of the other sections of code due to wordpress, nothing that find and replace and a bit of testing can’t fix! Cheers again
You’re very welcome! It’s cool to know people are finding them helpful 🙂
I’ve had another pass at correcting the WordPress code-munging too; fingers crossed they stay correct!
Running the script. I got some problems with [ or ] in the filename.
Mostly it’s not syncing those files.
Hi, thanks for the info. I’ve added some more -LiteralPath changes so that special characters should be ignored.
The latest version of the script is at;
Hi ,nice script.Unfortunately I have this issue:
Compare-Object : Cannot bind argument to parameter ‘ReferenceObject’ because it is null.
+ $MissingFiles=Compare-Object <<<< $SourceFiles $TargetFiles -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObjectCommand
Any idea? Thanks
Guys any idea about this error:
Compare-Object : Cannot bind argument to parameter ‘ReferenceObject’ because it is null.
At C:\script\script.ps1:144 char:35
+ $MissingFolders=Compare-Object <<<< $SourceFolders $TargetFolders -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObjectCommand
Hiya! Sorry there’s a problem. I’m not near a machine with PowerShell for a while but looking at the error it’s probably because $Sourcefolders is $null which is probably because there are no subfolders in the root folder you want to sync; maybe it’s just got files in it?
To fix it you’ll need to check if Sourcefolders is null before running the command OR maybe forcing Sourcefolders to be an array; you could wrap the assignment with @().
I really need to make catch the cases where there are no files or folders but one of those might stop the error!
Maybe like;
$SourceFolders+=@($SourceList | ? {$_.PSIsContainer -eq $True})
Nice script!
Thanks!
Compare-Object : Cannot bind argument to parameter ‘ReferenceObject’ because it is null.
At C:\script\script.ps1:143 char:33
+ $MissingFiles=Compare-Object <<<< $SourceFiles $TargetFiles -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObjectCommand
Hi! Similar to my earlier reply; it’s when $Sourcefiles is empty.
Excellent and inspiring work. I have spent the last few hours learning about the Compare-object command and I stumbled across this blog post. I really appreciate how you took the time to break down the script and explain what it does.
As I speak it is syncing my documents and music to a backup drive, good work and thanks again for sharing.
You’re very welcome! It’s great that so many people have found it useful, plus I find it far easier to learn about things myself if I have a project to apply it to.
Looks cool! Will this grab files and folders from the target directory? Or just folder contents recursively? Will it get the files in the root (target directory)?
Hi! No it won’t copy files from the Target to the Source; it there’s a file like that it’ll assume it’s been deleted from the Source and delete it from the Target.
I might see how difficult it would be to add that as an option though…
Yes this would be very nice. Thank you for this much btw!
What about keeping a filelist from the previous sync and comparing to a current one (for both sides). The missing files must be the deletions.
Hi. Yes, that or write an attribute to the file itself somewhere. I think a manifest file would be better but I’m trying to think of an efficient way to use them (as it could get quite large / slow with a lot of files + directories).
Thanks for the comment!
Take a look here: http://stackoverflow.com/questions/6526441/comparing-folders-and-content-with-powershell
One of the answers is a MD5 hashing code to compare folders. Should speed up the sync drastically in case not much has changed. Maybe it’s also helping with the manifest file.
Sounds like a good update. I did some work with hash comparisons here so maybe I can recycle a bit of it…
Looks cool. Will this sync the files located in the target folder?
Hi! No, it won’t “sync back”; if there’s a file in the target folder that isn’t there in the source it’ll remove it (so that it syncs deletions).
It might be an interesting option to add though…
Hi! tested your Script.
Syncing works awesome!
Only the statistics aren’t working. Showing = 0 every time on every stat.
Even if the script has devinitly copied files and created folders.
Statistics
Folder Creations: 0
Folder Removals: 0
File Copies: 0
File Removals: 0
File Updates: 0
Successful Process Message
———- ——- ——-
True Remove Folder Removing folder missing from Source : E:\Sync Target\6
True Create Folder Adding folder missing from Target : E:\Sync Target\5
True Create Folder Adding folder missing from Target : E:\Sync Target\5\e1
True Copy File Copying missing file : E:\Sync Target\5\e1\78a3e8c5314e0dae41c531cb19366085.gif
True Copy File Copying missing file : E:\Sync Target\5\e1\876aadb57de1d589c196ee1842c6ea0c.gif
True Copy File Copying missing file : E:\Sync Target\5\e1\a982a30cc67bb6934834395239572800.gif
True Create Folder Adding folder missing from Target : E:\Sync Target\5\e2
True Copy File Copying missing file : E:\Sync Target\5\e2\78a3e8c5314e0dae41c531cb19366085.gif
True Copy File Copying missing file : E:\Sync Target\5\e2\876aadb57de1d589c196ee1842c6ea0c.gif
True Copy File Copying missing file : E:\Sync Target\5\e2\a982a30cc67bb6934834395239572800.gif
True Create Folder Adding folder missing from Target : E:\Sync Target\5\e3
True Copy File Copying missing file : E:\Sync Target\5\e3\78a3e8c5314e0dae41c531cb19366085.gif
True Copy File Copying missing file : E:\Sync Target\5\e3\876aadb57de1d589c196ee1842c6ea0c.gif
True Copy File Copying missing file : E:\Sync Target\5\e3\a982a30cc67bb6934834395239572800.gif
Glad its (mostly) working 😀 I’ll have a look at the stats and hit with the scripting hammer…
Fixed the statistics not showing! There was a typo;
Sync-OneFolder -SourceFolder $Pair.Source -TargetFolder $Pair.Target -PassedExceptions $CurrentExceptions -Filter $CurrentFilter |
Tee-Object -Variable Changess
should be;
Sync-OneFolder -SourceFolder $Pair.Source -TargetFolder $Pair.Target -PassedExceptions $CurrentExceptions -Filter $CurrentFilter |
Tee-Object -Variable Changes
I’ve updated the script above.
you are the man 🙂
works perfectly now!
Awesome script, but I have 2 issues…
1. I could never get the script to work with mapped network drives or UNC paths when using the Sync-FolderConfiguration.xml file, is this possible?
2. and more importantly I cannot get the -Exceptions to work. I simply want to exclude a directory from the source. my command is… .\Sync-Folders.ps1 -SourceFolder:”\\192.168.1.102\music” -TargetFolder:”e:\” -Exceptions:”.\excludedfolder”
Thanks
Thanks! I’ll have a look at the two examples and see what’s going on 🙂
I had a look and I tested UNC names in the Config file and passed as a parameter and they worked when I tried them. That said, there were some bugs with Filter and Exceptions I found that meant they wouldn’t work in some situations so it may have been that (fixed in the latest version). 🙂
Also, Exceptions and Filters need to be wild-carded. The script works by checking the filepath of the file to be processed against each entry in Exceptions with a -Like command. So the Exception needs to have wild-cards in it. So if you have a file d:\temp\old\test.txt that you don’t want synced add *.txt, *test.txt or *old* to the exceptions list.
I’ve added that to the docs, cheers!
Line 118 has some issues with misplaced “try”. Also, is there a download link for the script, the copy/paste from the website is lacking.
Thanks! I’ve re-pasted it. I’ll have a look at trying to set up a download location too.
Hi,
Can you update the script with email functionality (including the output.message info)?
For now I have the following but I’m not able to catch the output.message info.
$to = “bla@bla.com”
$subject = “backup mail”
$body = “`nBackup Statistics`nFolder Creations: `t$FolderCreations `nFolder Removals: `t$FolderRemovals `nFile Copies: `t`t$FileCopies `nFile Removals: `t`t$FileRemovals `nFile Updates: `t`t$FileUpdates `n”
$smtpServer = “smtp.bla.bla”
$smtpFrom = “info@bla.com”
$smtpTo = $to
$messageSubject = $subject
$messageBody = $body
$smtp = New-Object Net.Mail.SmtpClient($smtpServer)
$smtp.Send($smtpFrom,$smtpTo,$messagesubject,$messagebody)
And, if possible, with an option to log the output in a logfile?
Hi! LogFile is definitely something I’m going to add. Not considered auto-emailing, though that would be useful too!
Keep getting the below error on a simple sync test and the second sync-pair never get’s synced…
The property Exception cannot be found on this object. Verify that the property exists.
At C:\Source\Repos\PowerShell Scripts\Sync-Folder.ps1:435 char:34
$CurrentExceptions=@($Pair.ExceptionList.Exception)
Hi! Can you post an example of the XML file you’re using (if you are)? It’s a problem in that bit of code so I’ll see if I can duplicate and fix it!
Cheers
It was a pretty simple file like the below:
C:\synctest\source1
C:\synctest\dest1
*.txt
p234*
C:\synctest\source2
C:\synctest\dest2
Hi. I ran it with the following XML file which seems to be close to what you want;
C:\synctest\source1
C:\synctest\dest1
*.txt
*p234*.txt
C:\synctest\source2
C:\synctest\dest2
I did find a slightly related bug which I’ve fixed though! I’ll upload a new version (1.9) as well.
I’ve just noticed WordPress is stripping all the XML from the XML configuration file! I’ve updated the example with the one I posted above.
Excellent script. Adding a two way sync where the most recent “date modified” wins would be an awesome addition.
On line 308 the Test-Path fails if there is are square brackets in the path. I had to add -LiteralPath to that command too.
Thanks!
Thank you will try!
Sent from my iPhone
>
Thanks for this awesome script! I had a situation where I was syncing a folder with a symlink. That symlink pointed to a folder that was above my target folder, causing the sync to run recursively forever. I found that adding “-Attributes !ReparsePoint” to the Get-ChildItem lines stopped the recursive syncing.
Thanks! That’s a helpful tip; I’m going to put a few updates in it shortly and I’ll make sure I add that. Glad you’re finding it useful!
A bit of a wishlist item, as I’m tweaking this more and more. It would be cool to be able to specify multiple targets for one source without duplicating the syncpair entry in the config file. It might be more trouble than it’s worth, but I thought I’d mention it.
Ta! I’ll have a look next time I do an update.
I’m getting the following error, although it looks to be working correctly.
The variable ‘$TargetFolder’ cannot be retrieved because it has not been set.
At C:\Users\Administrator\Documents\PowerShellScripts\Sync-Folder.ps1:283 char:32
+ Write-Log “Target Folder : $TargetFolder”
+ ~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (TargetFolder:String) [], RuntimeException
+ FullyQualifiedErrorId : VariableIsUndefined
Ok thanks, I’ll have a look! What was the command you used to run the script?
I was just running it from ISE
Looks there was a small typo in the script; on line 283 the line;
Write-Log “Target Folder : $TargetFolder”
should be;
Write-Log “Target Folder : $TargetFolders”
I’ve uploaded a fixed version and changed the text!
That fixed it, thanks for your help.
404 on the sync.zip download link
Thanks again, hopefully works now.
line 333: $CurrentFilter=$Pair.Filtaer <—typo
Thanks! Have fixed it.
Thanks was just what i was after.
Great! Glad it’s useful!
Thanks for the script, but I can’t seem to understand, how do I pass -IsError, -IsWarning, or -NoFileWrite parameters?
Hi! You shouldn’t need to yourself; those are parameters for the logging function. They just make it display the passed message in different ways to different output streams. If you’re taking that bit of code out and running it on its own you’d use;
Write-Log “This is a warning!” -IsWarning
Thanks.
Okay, then how do I disable or otherwise limit logging to file just for warnings/errors? Right now log file after every sync takes dozens of MBs which could clutter the server. I scheduled to delete it everyday, but thought I could limit logging to errors and delete/rename every month or so.
Hi. Probably the easiest / quickest way is to change the line;
if (!$NoFileWrite)
to
if (!$NoFileWrite -and ($IsError -or $IsWarning))
Not the cleanest (as there’s redundant code in there) but it should work with the least changes.
Cheers
Thanks Man, great post!
Thanks! Glad it was useful!
I have a error. What version of PowerShell need to use?
error:
Get-ChildItem : Не удается найти параметр, соответствующий имени параметра “Attributes”.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:316 знак:73
+ $SourceList=Get-ChildItem -LiteralPath $SourceFolder -Attributes <<<< !ReparsePoint
+ CategoryInfo : InvalidArgument: (:) [Get-ChildItem], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand
Get-ChildItem : Не удается найти параметр, соответствующий имени параметра "Attributes".
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:319 знак:77
+ $TargetList=Get-ChildItem -LiteralPath $TargetFolder -Attributes <<<< !ReparsePoint
+ CategoryInfo : InvalidArgument: (:) [Get-ChildItem], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand
Не удалось получить переменную "$SourceList", так как она не установлена.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:321 знак:42
+ $FilteredSourceFiles+=$SourceList <<<< | Where-Object {$_.PSIsContainer -eq $False -and $_.FullName -like $F
ilter -and
+ CategoryInfo : InvalidOperation: (SourceList:Token) [], RuntimeException
+ FullyQualifiedErrorId : VariableIsUndefined
Не удалось получить переменную "$SourceList", так как она не установлена.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:325 знак:44
+ $FilteredSourceFolders+=$SourceList <<<< | Where-Object {$_.PSIsContainer -eq $True -and !(Check-Exceptions
$_.FullName $PassedExceptions)}
+ CategoryInfo : InvalidOperation: (SourceList:Token) [], RuntimeException
+ FullyQualifiedErrorId : VariableIsUndefined
Compare-Object : Не удается привязать аргумента к параметру "DifferenceObject", так как он имеет значение NULL.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:327 знак:37
+ $MissingFiles=Compare-Object <<<< $FilteredSourceFiles $FilteredTargetFiles -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObje
ctCommand
Compare-Object : Не удается привязать аргумента к параметру "DifferenceObject", так как он имеет значение NULL.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:328 знак:39
+ $MissingFolders=Compare-Object <<<< $FilteredSourceFolders $FilteredTargetFolders -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObje
ctCommand
Не удалось получить переменную "$MissingFiles", так как она не установлена.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:330 знак:47
+ foreach ($MissingFile in $MissingFiles <<<< )
+ CategoryInfo : InvalidOperation: (MissingFiles:Token) [], RuntimeException
+ FullyQualifiedErrorId : VariableIsUndefined
Не удалось получить переменную "$MissingFolders", так как она не установлена.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:370 знак:51
+ foreach ($MissingFolder in $MissingFolders <<<< )
+ CategoryInfo : InvalidOperation: (MissingFolders:Token) [], RuntimeException
+ FullyQualifiedErrorId : VariableIsUndefined
Невозможно обнаружить свойство "Name" у данного объекта. Проверьте, существует ли это свойство.
C:\Users\ANTON\Documents\PowerShellScripts\Sync-Folder.ps1:395 знак:13
+ $TargetFile. <<<< Name
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Hi. I think the -Attributes parameter for Get-ChildItem was added in PS3. If you’re not using ReparsePoints (symbolic links) you can remove -Attributes !ReparsePoint from the script or do something like;
Get-ChildItem -LiteralPath $SourceFolder | Where-Object {$_.Attributes -notmatch ‘ReparsePoint’}
Hello, I don’t mean to be a burden but I’m fairly new at powershell. I have been watching videos on MVA to get a better understanding. I am trying to run the script but I receive errors. When I run it in Powershell ISE debugger I receive this:
At line:221 char:83
+ … ile.SideIndicator -eq “<=") { $Output.Process="Copy File" try { $Outp …
+ ~~~
Unexpected token 'try' in expression or statement.
At line:221 char:169
+ … sing file : $($TargetFolder+"\"+$MissingFile.Name)" Write-Log $Output …
+ ~~~~~~~~~
Unexpected token 'Write-Log' in expression or statement.
At line:221 char:442
+ … issing file $($TargetFolder+"\"+$MissingFile.Name)" Write-Log $Output …
+ ~~~~~~~~~
Unexpected token 'Write-Log' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : UnexpectedToken
Is there something that I am not doing correctly? Your assistance would be greatly appreciated. Thanks.
Hi! Did you download the zip file or copy and paste? It looks like the lines have been jumbled up (merged). That first error it’s reporting all those commands on the same line where they _should_ look like this;
if($MissingFile.SideIndicator -eq “<=")
{
$Output.Process="Copy File"
try
{
In theory you don't need PS commands to be properly new-lined (assuming their brackets are correct) but it might indicate the script has been an error with the paste / download?
I found your script very useful and I added a feature to it. If you are interested in the modification I could send it to you but basically I made added a -NoRemove parameter so that I can prevent it from deleting anything in the target folder. I also needed the target folder to be on a remote computer so I made this wrapper to handle that and thought I’d share it here it helps anyone else.. I haven’t tested it too much but it seems to be working. Note that the “-NoRemove” parameter is something I added to my customized version of Sync-Folder so anyone using this function would need to remove that.
Function Copy-FolderTo {
Param([string]$sourceFolder, [string]$target
, [switch]$NoRemove
, [string[]]$Exceptions=$Null
)
$toolkitdrive=’ToolKitDrive’
$out=new-item -Path $target -ItemType Directory -Force -ea Stop
try {
New-PSDrive -name toolkitdrive -PSProvider FileSystem -root $target -ea Stop >$null
& “$env:TOOLS_HOME/Lib/Sync-Folder” -SourceFolder $sourceFolder -TargetFolders “${toolkitdrive}:” -NoRemove -Exceptions $Exceptions -ea Stop
}finally{
remove-psdrive $toolkitdrive
}
}
Thanks for posting this, looks like a cool addition!
Hello,
First of all thanks for the script it works quite well. I have a question, however. Why does this take so much longer to churn through a directory than, say, robocopy /mir?
Hi. Not sure exactly but a few guesses;
1) It’s because robocopy is a low-level, compiled program optimised for it’s one task. PowerShell has to run through some additional layers of abstraction / access-control.
2) The script performs filtering checks to exclude specific filetypes, locations etc
3) The script supports multiple syncing pairs (and checks these as it goes, so even in the case of a 1 to 1 map the checks are performed)
4) Logging; Writing to a log slows things down quite a bit.
So a combination of extra functionality, logging and the fact that it’s a script rather than focused, compiled code.
Dedicated programs can be more efficient (I mentioned Alway Sync, which I used previously), but the script allows a more bespoke solution if you need it (or hooking into another script easily).
Cheers,
Neil
h3rring author the attached above one is the final script?
Hi! It should be, yes!
Great script! I am considering using this script to keep User Profile Disks (UPDs) in sync between different RDSH Hosts in two distinct RDS Farms in two different Azure Regions as part of a DR solution. I notice that if a file is ‘open\locked’, no action is taken until all handles are cleared. That is expected with opportunistic locking. Have you considered adding some code that would report in the stats if file(s) were open and hence not copied?
Hi! Thanks for the kind words! I’ve recently modified a different script to check for and log locked files (a PST Import/Export script) so it should be easier to add it to this! Just need a bit of time to give it a revamp 🙂 Thanks again!
Hi, is there anyway to check if file exist in target but is different from source (e.g. creation date, size) then sync from source?
Hi! Yes, that’s perfectly viable. I did a series of posts about a script to check properties of files in folders to find duplicates (using size or binary checks) here;
https://herringsfishbait.com/tag/duplicate-files/
Something similar should work to add to this.
Hi Expert,
Is this script works on over ftp site synchronization between ftp server / client?
Hi! I’ve not tested it with an FTP site, but if you map one to a drive letter I don’t see why it wouldn’t work;
https://www.thewindowsclub.com/map-an-ftp-drive-windows
Many thanks! I try
Hi Author Expert !
I know the your super script works great and useful !
I am getting error not sure how to fix it.
Executed in PS
PS E:\> Powershell.exe -executionpolicy remotesigned -File E:\SyncFolders\Sync-Folder.ps1
Script area changed the location files =
E:\SyncFolders\sourceServer\Sync-FolderConfiguration.xml
E:\SyncFolders\sourceServer\SyncFolderLog.txt
[CmdletBinding(DefaultParameterSetName=”XMLFile”)]
param
(
[parameter(
ParameterSetName=”XMLFile”)]
[ValidateScript({Test-Path -LiteralPath $_ -PathType leaf})]
[string]$ConfigurationFile=$PSScriptRoot+”E:\SyncFolders\sourceServer\Sync-FolderConfiguration.xml”,
[parameter(
Mandatory=$True,
ValueFromPipelineByPropertyName=$True,
ParameterSetName=”FolderPair”)]
[string]$SourceFolder,
[parameter(
Mandatory=$True,
ValueFromPipelineByPropertyName=$True,
ParameterSetName=”FolderPair”)]
[string[]]$TargetFolders,
[parameter(
ParameterSetName=”FolderPair”)]
[string[]]$Exceptions=$Null,
[parameter(
ParameterSetName=”FolderPair”)]
[string]$Filter=”*”,
[ValidateScript({Test-Path -LiteralPath $_ -PathType leaf -IsValid})]
[string]$LogFile=$PSScriptRoot+”E:\SyncFolders\sourceServer\SyncFolderLog.txt”,
[switch]$PassThru=$False,
[switch]$Whatif=$False
)
set-strictmode -version Latest
$Script:LogFileName=$LogFile
Xml File change:
E:\SyncFolders\sourceServer
E:\SyncFolders\TargetServer
E:\SyncFolders\TargetServer1
*.txt
*.log
E:\SyncFolders\sourceServer
E:\SyncFolders\TargetServer
*.txt
Hi. Maybe the lines where you change the locations of the Logfile and ConfigurationFile?
[string]$ConfigurationFile=$PSScriptRoot+”E:\SyncFolders\sourceServer\Sync-FolderConfiguration.xml”,
$PSScriptRoot always holds the location of the script being run; so if you ran the script in “E:\Syncfolders\sourceserver” the above line would set ConfigurationFile to ”E:\SyncFolders\sourceServerE:\SyncFolders\sourceServer\Sync-FolderConfiguration.xml” which would be invalid.
Hi H3rring,
I requesting you to resolve the issue , I tried much but failed I know i am doing some wrong please help me out.
PS E:\synctest> E:\synctest\Sync-Folder.ps1
Property ‘SideIndicator’ cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:336 char:29
+ if($MissingFile. <<<< SideIndicator -eq "<=")
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property 'SideIndicator' cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:375 char:32
+ if ($MissingFolder. <<<“)
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property ‘PSIsContainer’ cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:324 char:62
+ $FilteredTargetFiles+=$TargetList | Where-Object {$_. <<<< PSIsContainer -eq $False -and $_.FullName -like $Filter -and
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property 'PSIsContainer' cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:327 char:64
+ $FilteredTargetFolders+=$TargetList | Where-Object {$_. <<<< PSIsContainer -eq $True -and !(Check-Exceptions $_.FullName $PassedExceptions)}
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Compare-Object : Cannot bind argument to parameter 'DifferenceObject' because it is null.
At E:\synctest\Sync-Folder.ps1:329 char:37
+ $MissingFiles=Compare-Object <<<< $FilteredSourceFiles $FilteredTargetFiles -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObjectCommand
Compare-Object : Cannot bind argument to parameter 'ReferenceObject' because it is null.
At E:\synctest\Sync-Folder.ps1:330 char:39
+ $MissingFolders=Compare-Object <<<< $FilteredSourceFolders $FilteredTargetFolders -Property Name
+ CategoryInfo : InvalidData: (:) [Compare-Object], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.CompareObjectCommand
Property 'SideIndicator' cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:336 char:29
+ if($MissingFile. <<<< SideIndicator -eq "<=")
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property 'SideIndicator' cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:375 char:32
+ if ($MissingFolder. <<<“)
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property ‘Name’ cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:398 char:13
+ $TargetFile. <<<< Name
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property 'Name' cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:399 char:95
+ $MatchingSourceFile= $FilteredSourceFiles | Where-Object {$_.Name -eq $TargetFile. <<<< Name}
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
Property 'FullName' cannot be found on this object. Make sure that it exists.
At E:\synctest\Sync-Folder.ps1:427 char:56
+ Sync-OneFolder -SourceFolder $SingleFolder. <<<< FullName -TargetFolder ($TargetFolder+"\"+$SingleFolder.Name) -PassedExceptions $PassedExceptions -Filter $Filter -WhatIf:$WhatIf #
+ CategoryInfo : InvalidOperation: (.:OperatorToken) [], RuntimeException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
07:58:12.587 : Statistics
07:58:12.610 :
07:58:12.614 : Folder Creations: 0
07:58:12.618 : Folder Removals: 0
07:58:12.626 : File Copies: 0
07:58:12.631 : File Removals: 0
07:58:12.634 : File Updates: 0
Hi. Not sure what your speciific problem is, but I just released a new version with some bug fixes and better logging that might help!
Hello h3rring, I’m having problems with the script, it’s not deleting files on the target folder that are not in the source folder and its always coping files instead of checking if they are the same or have been updated. Also, the script that is here and the one that’s linked is not the updated one because from what I can tell, it still has elseif ($MissingFile.SideIndicator=”=>”) error pointed out by nebule. Fixing that error still didn’t help though, so I’m not sure what I’m doing wrong. Thank you for the script by the way, I’m a beginner to this and your script really is awesome.
Something is broken with the filter. As an example I tried with 2019-*zip as my filter pattern and it didn’t work so I tried to reverse engineer how the list was generated. I found if I used $_.Name instead of $_.FullName then my filter worked.
On further research as to the difference between $_.Name and $_.FullName I realised that there is the path name included so I modified my filter to *2019-*.zip and this worked. So I guess this by design but just wanted to confirm.
I forgot to say great script and thank you.
I’ve been loving this script and have been using it to sync a folder across 100+ servers using a looping script that I built on top of it. The script I built creates a psdrive during each loop for the server it is syncing with and then removes the psdrive at the end. The servers have been running 2008R2 for a long time and we have been upgrading to 2016 recently. Since upgrading the server where the script runs, it had slowed down a lot. If I run the script in a 2008R2 environment it blazes through the server list, but on a 2016 server it drags. Any thoughts on what is slowing it down?
So I realized that the slow issue was actually due to the log file. It was slow at times before, but I noticed a significant slowdown with 2016. I blew away the log file and it is running much smoother now. Can the logging be set to limit itself so that the log file doesn’t continue to grow so large?
Ah! That teaches me to reply before reading all the comments 🙂 Thanks for the update, I’ll have a look at that over the next few days 🙂
Hi! Glad it’s working for you!
No idea why it may be slower on 2016. Some suggestions;
You could try running with -Verbose on and seeing if there’s particular bits that slow down.
Looking at Time-Stamps in the log file.
Using Measure-Command to see which loops use all the time (https://4sysops.com/archives/measure-the-run-time-of-a-powershell-command-with-measure-command/).
I’m going to have another look at the code in the next few days so I’ll see if anything crops up!
Hello, and as others have said, thank you.
I believe this script will work exactly for what I’m trying to do. I’m a bit new to powershell, but I did read through everything posted. I ultimately decided to try dot sourcing, however I could not get the backup to complete. I also tried to wrap it in a function and call that and received the same result as when dot sourcing:
PS Powershell>.\Sync-Folder.ps1 -SourceFolder:”D:\Users\b1gmasterm\Library” -TargetFolder:”F:\Backup”
22:42:33.515 : LogFile: D:\Users\b1gmasterm\Logs\Backup\SyncFolderLog-2020-10-13-22-42-1.log
22:42:33.526 :
22:42:33.529 : Statistics
22:42:33.532 :
22:42:33.535 : Folder Creations: 0
22:42:33.538 : Folder Removals: 0
22:42:33.541 : File Copies: 0
22:42:33.544 : File Removals: 0
22:42:33.547 : File Updates: 0
Would you be able to advise possibly on what I could be missing?
Hi! Glad you’ve found the script helpful!
. Sourcing might be a bit fiddly; FWIR when you . source you merge the script in with another so you can call the original scripts functions. As my script has a lot of the code outside the defined functions (as it’s a run as a script) you’d only be able to call the Sync-OneFolder part from your “parent” script. That should work to sync the folders, but you’d get no log processing, no summary etc. You’d have to write code to process the output Sync-OneFolder creates too (lots of objects showing the results of syncing each file).
My memory of . sourcing my be a bit fuzzy though as that kind of functionality has been replaced by creating PowerShell modules (which might be a better fit for what you want to do?)
Thank you very much for the advice. I looked up how to make modules and was able to get it to work; thank you! I also found an issue where I couldn’t use this on the “OneDrive” directory… “Google Drive” presented no issues when I used the script to make a backup to a USB, but OneDrive wouldn’t let me copy anything at all. I’ll do that manually, but everything else works great and I learned something new. THANK YOU!
Hi h3rring
How to change the default root path C to any drive folder like D:\Syncronization\synctest all areas of the scripts?
[string]$ConfigurationFile=$PSScriptRoot+”\Sync-FolderConfiguration.xml”,
[string]$LogFile=$PSScriptRoot+”\SyncFolderLog.txt”,
Hi,
Thank you for this script, it really is excellent.
Can i ask for a bit of advice on how i can handle an ampersand in the file path for folders please.
Thanks again