A few months ago, I helped one of my clients get up and running with an offsite backup. We wanted to mirror (keep a working copy) of all files in one folder to another computer at a different location. We also wanted to keep this mirror up-to-date when anything changes by either transferring new/changed files or deleting removed files – and all of this needs to be done via the Internet. We implemented a free, easy-to-use solution that is secure and works every time. For your benefit, I’m sharing the high-level details of this solution, not only to provide technical help, but to show that it can be done.
For this how-to, we’ll call our files to be backed up the “source” and the backed up copy the “destination.”
What Do We Need?
Simply stated – create an off-site mirror.
To elaborate a bit, that means that there is a collection of files on a server (the source) that we want replicated somewhere else (the destination) – in this case, to another server at a different physical location (and on a different network). The mirroring doesn’t have to be in real time, but it does need to happen at least once per day, and automatically.
Breaking this down a bit further into technical terms, we need to implement software that will monitor the source, detect changes (such as new files, changed files, or deleted files), and “mirror” those files to the destination (transfer new and changed files, but remove the deleted files). Further, we’ll need the two source and destination to communicate over the open internet, and so we need to use a secure (encrypted) protocol to do this, such as SFTP.
We decided to use WinSCP, a free and open-source file transfer client. WinSCP supports the ability to use SFTP, which ensures that our communication is secure.
WinSCP also has a command-line interface, which means that we can create a script that will “mirror” our files. We need this feature so that we can create a script that can automatically run each day. We can configure cron (in Linux) or Task Scheduler (in Windows) to run our script at regular intervals.
Seems easy enough, right? Let’s roll it out…
Our client already had a Network-Attached Storage (NAS) device configured at the destination, so we didn’t have much to do there. Using a NAS is probably the quickest way to configure the remote end, but if not, there’s a lot of information out there to help you set up your own SFTP server.
The rest of this article talks about how to configure WinSCP at the source to send files to the destination.
First, let’s set up a command-line script which tells WinSCP what to do. Create a folder in your C: drive called scripts and create a script in that folder. You can use notepad to edit the script; I’ve named mine sftp-mirror.txt and it contains the following:
open offsite-mirror option batch continue synchronize remote -delete -mirror -criteria=time -transfer=automatic <source_dir> <destination_dir> option batch off
Let’s run through what the script does:
- This script tells WinSCP to connect to a saved connection called “offsite-mirror” (documentation on how to create saved connections available here).
- Use the option batch continue command to tell WinSCP that if a prompt or error is returned in the script, the script should continue processing the next command. Since this is an automatic script, there will be no human present to answer the prompt, so the command ensures that the program will not “hang” while waiting for input.
- The synchronize command is the actual command that is going to do most of the work for us. Replace <source_dir> with the source folder path and <destination_dir> with the folder path on the destination server. This command is also set to delete any files from the destination that are no longer present at the source.
- The criteria switch tells WinSCP to determine if the files were changed based on the last time they were modified; if there is a newer modified time on the source file than the destination file, the local file is re-copied to the destination.
- The transfer switch tells WinSCP to scan files to determine if they should be transferred via ASCII mode or binary mode.
You should be able to run this script inside WinSCP whenever you’d like, and the synchronization will occur. We want to do better than that – the script should run automatically, every night. To do that, we’ll need to create a Windows Command Line wrapper and schedule the wrapper to run each night.
I’ve created a wrapper called backup-run.cmd and placed it in the c:\scripts folder. Again, you can edit this file with notepad. My wrapper contains the following commands:
cd c:\scripts echo Mirror Started > mirror_log_%time:~0,2%%time:~3,2&&time:~6,2&_&date:~-10,2&&date:~-7,2&&date:~-4,4%.txt winscp.com /log=”mirror_log_%time:~0,2%%time:~3,2%%time:~6,2%_%date:~-10,2%%date:~-7,2%%date:~-4,4%.txt” /script=sftp-mirror.txt
The wrapper script tells Windows to look in c:\scripts for our sftp-mirror script. It also creates a log file with all of the output of this script so that you can look back and see all of the activity for a particular date; this is especially helpful for troubleshooting.
Once that file is created, our last step is to create a Windows Task Scheduler task to run the wrapper each night.
After that, you should be up and running! Sit back and rest assured that your files are being mirrored.
This article provides generalized information about how to start an off-site backup. We’ve withheld specific details because every network configuration is different. If you’re not familiar with how to set up an off-site mirror, please contact a professional before making any changes that could affect the integrity of your files or the security of your network. As always, 7 River Systems can help you, no matter where you are located – contact us and we’ll start working with you quickly!