From 8f0d0cdf8c6f306c21d87745a0b0834ae895c3e8 Mon Sep 17 00:00:00 2001 From: deajan Date: Wed, 1 Apr 2015 23:17:40 +0200 Subject: [PATCH] Revert "Revert "Typo and documentation fixes"" This reverts commit c13e61f793a67ed30142ffd7d33e07426f321639. --- README.md | 35 ++++++++++++++++++----------------- osync-batch.sh | 22 +++++++++++----------- 2 files changed, 29 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index ebf77ef..ab1df62 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -osync 0.99RC4 -============= +osync +===== A two way filesync script with fault tolerance, resuming, deletion backup and conflict backups running on linux and virtually any system supporting bash. File synchronization is bidirectional, based on rsync, and can be run manually, by cron, or triggered via inotifytools (whenever a file changes on master, a file sync is triggered). @@ -20,27 +20,27 @@ Bitpocked inspired me to write my own implementation of a two way sync script, i - Time control - Directory monitoring - Running on schedule or as daemon +- Batch runner for multiple sync tasks with rerun option for failed sync tasks -Osync uses a master / slave sync schema. It can sync local to local or local to remote directories. By definition, master replica should always be a local directory on the system osync runs on. -Also, osync uses pidlocks to prevent multiple concurrent sync processes on/to the same master / slave replica. Be sure a sync process is finished before launching next one. +Osync uses a master / slave sync schema. It can sync local to local or local to remote directories. By definition, master replica is always a local directory on the system osync runs on. +Osync uses pidlocks to prevent multiple concurrent sync processes on/to the same master / slave replica. Be sure a sync process is finished before launching next one, or use osync-batch. You may launch concurrent sync processes on the same system but only for different master replicas. -Currently, it has been tested on CentOS 5.x, 6.x, 7.x, Debian 6.0.7, Linux Mint 14, 15 and 16, Ubuntu 12.04 and 12.10, and FreeBSD 8.3. -Some users report MacOS X to work good, but some tests are still needed. -Microsoft Windows is supported via MSYS environment. +Currently, it has been tested on CentOS 5.x, 6.x, 7.x, Debian 6.0.7, Linux Mint 14, 15 and 16, Ubuntu 12.04 and 12.10, FreeBSD 8.3 and 10.1. +Some users report osync to work on MacOS X too. Microsoft Windows is supported via MSYS environment. ## Installation -Keep in mind that Osync has been designed to not delete any data, but rather make backups or soft deletes. -Nevertheless, you should always consider making backups of your data before trying a new sync tool. +Osync has been designed to not delete any data, but rather make backups of conflictual files or soft deletes. +Nevertheless, you should always have a neat backup of your data before trying a new sync tool. You can download the latest stable release of Osync at www.netpower.fr/osync -You may also get the last development snapshot at https://github.com/deajan/osync with the following command +You may also get the last development version at https://github.com/deajan/osync with the following command $ git clone https://github.com/deajan/osync You may copy the osync.sh file to /usr/local/bin if you intend to use it on a regular basis, or just run it from the directory you downloaded it to. -There is also a very basic installation script if you plan to use osync as a daemon too. +There is also a very basic installation script that does the copies for you, and creates /etc/osync to store your osync configuration files. Osync needs to run with bash shell. Using any other shell will most probably result in errors. If bash is not your default shell, you may invoke it using @@ -50,16 +50,16 @@ If bash is not your default shell, you may invoke it using ## Usage Osync can work with in three flavors: Quick sync mode, configuration file mode, and daemon mode. -While quick sync mode is convenient to do fast sync sceanrios, a configuration file gives much more functionnality. +While quick sync mode is convenient to do fast syncs between some directories, a configuration file gives much more functionnality. Please use double as directoires delimiters. Do not use escaped characters in directory names. QuickSync example ----------------- $ ./osync.sh --master="/path/to/dir1" --slave="/path/to/remote dir2" - $ ./osync.sh --master="/path/to/another dir" --slave="ssh://user@host.com:22//path/to/dir2" --rsakey=/home/user/.ssh/id_rsa + $ ./osync.sh --master="/path/to/another dir" --slave="ssh://user@host.com:22//path/to/dir2" --rsakey=/home/user/.ssh/id_rsa_private_key_example.com -Configuration files example ---------------------------- +Running osync with a Configuration file +--------------------------------------- You'll have to customize the sync.conf file according to your needs. If you intend to sync a remote directory, osync will need a pair of private / public RSA keys to perform remote SSH connections. Also, running sync as superuser requires to configure /etc/sudoers file. @@ -74,7 +74,7 @@ If everything went well, you may run the actual configuration with one of the fo $ ./osync.sh /path/to/your.conf --verbose $ ./osync.sh /path/to/your.conf --no-maxtime -Verbose option will display which files and attrs are actually synchronized. +Verbose option will display which files and attrs are actually synchronized and which files are to be soft deleted / are in conflict. You may mix "--silent" and "--verbose" parameters to output verbose input only in the log files. No-Maxtime option will disable execution time checks, which is usefull for big initial sync tasks that might take long time. Next runs should then only propagate changes and take much less time. @@ -98,7 +98,8 @@ Having multiple conf files can then be run in a single cron command like Daemon mode ----------- Additionnaly, you may run osync in monitor mode, which means it will perform a sync upon file operations on master replica. -This can be a drawback on functionnality versus scheduled mode because this mode only launches a sync task if there are file modifications on the master replica, without being able to monitor the slave replica. Slave replica changes are then only synced when master replica changes occur. +This can be a drawback on functionnality versus scheduled mode because this mode only launches a sync task if there are file modifications on the master replica, without being able to monitor the slave replica. +Slave replica changes are only synced when master replica changes occur, or when a given amount of time (default 600 seconds) passed without any changes on master replica. File monitor mode can also be launched as a daemon with an init script. Please read the documentation for more info. Note that monitoring changes requires inotifywait command (inotify-tools package for most Linux distributions). BSD, MacOS X and Windows are not yet supported for this operation mode, unless you find a inotify-tools package on these OSes. diff --git a/osync-batch.sh b/osync-batch.sh index 18295ad..f2c9bc4 100755 --- a/osync-batch.sh +++ b/osync-batch.sh @@ -3,19 +3,19 @@ PROGRAM="Osync-batch" # Batch program to run osync instances sequentially and rerun failed ones AUTHOR="(L) 2013-2014 by Orsiris \"Ozy\" de Jong" CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr" -PROGRAM_BUILD=3003201501 +PROGRAM_BUILD=0104201501 ## Runs an osync instance for every conf file found ## If an instance fails, run it again if time permits ## Configuration file path. The path where all the osync conf files are, usually /etc/osync -CONF_FILE_PATH=. +CONF_FILE_PATH=/etc/osync ## If maximum execution time is not reached, failed instances will be rerun. Max exec time is in seconds. Example is set to 10 hours. MAX_EXECUTION_TIME=36000 -## Max retries specifies the number of reruns an instance may get -MAX_RETRIES=3 +## Specifies the number of reruns an instance may get +MAX_RERUNS=3 ## Osync executable full path can be set here if it cannot be found on the system @@ -61,13 +61,13 @@ function Batch fi done - RETRIES=0 - while [ $MAX_EXECUTION_TIME -gt $SECONDS ] && [ "$RUN" != "" ] && [ $MAX_RETRIES -gt $RETRIES ] + RERUNS=0 + while [ $MAX_EXECUTION_TIME -gt $SECONDS ] && [ "$RUN" != "" ] && [ $MAX_RERUNS -gt $RERUNS ] do Log "Osync instances will be run for: $RUN" for i in $RUN do - $OSYNC_EXECUTABLE $i $opts + $OSYNC_EXECUTABLE "$i" "$opts" if [ $? != 0 ] then Log "Run instance $(basename $i) failed" @@ -84,7 +84,7 @@ function Batch done RUN="$RUN_AGAIN" RUN_AGAIN="" - RETRIES=$(($RETRIES + 1)) + RERUNS=$(($RERUNS + 1)) done } @@ -99,7 +99,7 @@ function Usage echo "" echo "[OPTIONS]" echo "--path=/path/to/conf Path to osync conf files, defaults to /etc/osync" - echo "--max-retries=X Number of retries max for failed instances, defaults to 3" + echo "--max-reruns=X Number of runs max for failed instances, (defaults to 3)" echo "--max-exec-time=X Retry failed instances only if max execution time not reached (defaults to 36000 seconds)" echo "--dry Will run osync without actually doing anything; just testing" echo "--silent Will run osync without any output to stdout, used for cron jobs" @@ -129,8 +129,8 @@ do --path=*) CONF_FILE_PATH=${i##*=} ;; - --max-retries=*) - MAX_RETRIES=${i##*=} + --max-reruns=*) + MAX_RERUNS=${i##*=} ;; --max-exec-time=*) MAX_EXECUTION_TIME=${i##*=}