Hi. I wrote and ran a quick script to crawl a web-site and archive all it’s web-pages. It worked but I noticed some serious performance issues; its memory usage would occasionally balloon up to 19 GB (!); it would slow down occasionally and it just generally seemed inefficient.
So I had a think and came up with a better script with a few simple changes; it stays on about 500MB usage now and runs much better 🙂 Continue reading “PowerShell: Optimising a Scripts Performance (Archiving Web-Pages Script)”