An approch to find a file synchronization solution where others doesn’t fit properly.
When it comes to file sync operations between machines and version controlling the easiest catch comes to mind is to use popular tools like ‘git’, ‘rcp’, ‘scp’, ‘ftp’, ‘sftp’, and last but not least ‘rsync’. Each one of these command line tools of CLI tools has PROS and CONS 🤞. Here are some worst case scenerio with these popular tools.
For ‘git’
- Have to push and pull repo on both machines seperately.
- For large files it’s not a good choice to store files on remote and sync.
- Not everyone is comfortable using git
- Comes with manual authentication overhead
- git is focused and biased by design that is not specific for files and folder sychronization.
For ‘rcp’ ‘scp’ ‘ftp’ ‘sftp’
- Uses uses secure shell access or dedicated permission controlled ‘ftp’ channel.
- performs all operations except file synchronization
- Doesn’t keep track of modified, new created files
- Doesn’t provide flexibility for command line input
Now for ‘rsync’
‘rsync’ is a great tool for file synchronization. But when you find that the hosting server or machine you want to interact with is not priviledged enough to have the ‘rsync’ command available, you simply get doomed. Specially it happens with cheap, budget friendly server. But it is a good tool that must run on both the client and the server side to achieve the file synchronization facility.
Recently I have faced the same issue with one of my servers which doesn’t have enough priviledge to have the ‘rsync’ command available. I’ve asked the hosting providers and they’ve clearly mentioned the fact. It was fair. Then I thought,
“I need a solution where I can perform the same file/folder synchronization operations with peace in mind and not paying extra charges for that.”
Now that also sounds fair right? At first, I thought we’ll it’s easy, I can
- Trigger a Github Action
- Instead of ‘rsync’ use ‘scp’ of ‘fpt’ or ‘sftp’ to upload/copy file to the server
- Build an agent on another server to automate my tasks
- Use or trigger another service from the server while manually checking the files and head commits and everything …
But I realized, well, I can do that. But it’s not the solution what I want and although, these ways are somewhat promising but not aligned with my use case and what I want. Life would be much simpler if I had access to the following command.
The features I wanted
- 0 Dependencies
- Track file changes
- Operation mode selection
- SHA256 file checksum generation
- Async execution
- Platform independent
- Optionally delete duplicate files
- Verbose mode
- Dry run mode
- Hash gneration mode
- Sync mode
- Generate directory file’s JSON hashmap
- Flexible argment input
- Better error handling
- Detailed documentation
- Generate command for ‘scp’ tool —default
- Multi flag support
- Async multiple action support
Then I build ‘filesync’
‘filysync’ is a CLI(Command line Interface) helper tool to help you keep your files synced between source and destination while preserving directory structure.
This tool also generates file Hashmap to keep track file changes even when there is no ‘git’ installed on machine and also machines that doesn’t have ‘rsync’ access or installed on both sides. Meaning you can keep track of file’s changes and sync them between machines efficiently.
This tool comes with no official guarantee for this version. May cause errors. Update for patch.
Enough for the intro, now you can download with below command and I’ll post some images below to show it works. Don’t forget to try ‘filesync —help’ to know more.
Download and install on WSL / Mac OS / Linux
$ sudo curl -L https://shorturl.at/4hzV5 -o /usr/local/bin/filesync && sudo chmod +x /usr/local/bin/filesync
Fig1: filesync help screenshot