Obsolete:Wmsync
This page contains historical information. It may be outdated or unreliable.
wmsync is the provisional name for a replacement for scap/sync-file
Justification
Our current scap/sync-file solution is a bit long in the tooth. This would be a more robust replacement that we hope makes deploys go more smoothly. A few goals:
- No NFS dependency, thus eliminates a significant single point of failure
- Non-root sync will actually work, instead of failing in a lot of subtle ways as it does currently.
- Allows atomic multi-file sync, so avoiding the flood of fatal errors we sometimes see when running scap
- Should be faster than rsync, and should scale well with increasing file count and size.
- Eliminates the subtle differences between scap and sync-file, such as the fact that the former rebuilds message caches and the latter doesn't.
- Simplifies monitoring for servers with out-of-date copies of MediaWiki, by providing an overall content hash.
- Not written in shell script (i.e. easier to read, maintain, and extend for our staff)
- Baby step toward Heterogeneous deployment
Example usage
Recursive push of current directory to all servers:
wmsync -r .
Recursive push of whole tree:
wmsync -r /home/wikipedia/common
Push of single file:
wmsync CommonSettings.php
Planned features
- Single-file sync
- Atomic subtree incremental sync
- SSH setup, shared secret config, whole tree pull
- Local make scripts, remote make scripts
- Any member can push -- configured within tree.
- Network-aware distribution
- Compact, prioritised error display
- Add member, delete member. Node group update.
- Whole tree integrity check on subtree/file sync, to fix servers which were disconnected.
- File catalog to determine which files have changed