Coding style used for my bash projects (v3.0 Dec 2016) As bash is clearly an error prone script language, we'll use as much standard coding as possible, including some quick and dirty debug techniques described here. ++++++ Header Always use the following header ----BEGIN HEADER #!/usr/bin/env bash PROGRAM="program-name" # Long description AUTHOR="(C) 20XX-20YY by Orsiris de Jong" CONTACT="http://www.example.com me@example.com" PROGRAM_BUILD=YYYYMMDDVV ## Optional instructions ----END HEADER Using bind style versionning: YYYYMMDDVV (Year, Month, Day, Revision): Example: 2015012402 = 2nd revision of 24 Jan 2015 #!/usr/bin/env bash instead of #!/bin/bash Change old scripts with for i in $(grep -r '#!/bin/bash' * |cut -f1 -d':'); do sed -i 's&#!/bin/bash&#!/usr/bin/env bash&g' $i; done type instead of type -p for bash test (other shells don't know -p) ++++++ Indentation Using tabs Transform old shell scripts using unexpand command ++++++ Comments Some command # comment ## Some comment on a new line ################################################# Some separation ++++++ Work comments Whenever there is some idea to postpone, use #TODO(priority):[dev-name:] some remark Priority can be critical, high, medium, low, verylow. No release can happen if there are TODOs other than low or verylow. Example: #TODO(high):deajan: need to do something A "work in progress" marker must be left on the line a dev is working when it's work isn't finished). Marker is #WIP:dev-name: some remark dev-name is mandatory if more than one person is coding Example: #WIP:deajan: missing function something ++++++ Variables All local variables names have each first letter of the words uppercase and all others lowercase, except for the first word where all letters are lowercase Example: someLongVariable All global variables are full upercase, separated by _ Example: EXEC_TIME All environment variables (verbose, silent, debug, etc) have prefix _ and are full upercase, separated by _ Example: _PARANOIA_DEBUG Exec time variables that can take boolean values should use true and false instead of 1 and 0. ++++++ Functions All function names should begin with an uppercase letter for every word, the other letters should be lowercase Example: SomeFunctionThatRocks Bash does not provide any checks against missing function arguments. Also, missing quotes can lead to an inconsistent number of arguments. Most functions should have a first line that calls the special function __CheckArguments, which checks the number of given arguments for a function in order to find possible problems. Number of arguments are given as first argument to __CheckArguments. May be a number or a range, eg 0-2 if the function takes optional arguments. __CheckArguments will only trigger when the script is launched with _PARANOIA_DEBUG=yes. Also, it will only exist in the debug version. Use the following convention for function definition: function SomeFunction { __CheckArguments 0 $# "$@" #__WITH_PARANOIA_DEBUG ... } Use sed ':a;N;$!ba;s/\n{\n/ {\n/g' to convert functions that have opening brackets on a new line. If the function has arguments, use local variable names that are more readable than $1...$n. Explain via comments what those variables contain if needed. Declare arguments before launching __CheckArguments: function AnotherFunction { local varName="${1}" local otherVarName="${2}" # This variable contains stuff __CheckArguments 2 $# "$@" #__WITH_PARANOIA_DEBUG ... } Functions should always have return status function RandomFunction { ... return $? } ++++++ Sub functions When a function is a subroutine of another function, it is called _SomethingAsSubFunction: Example: function _ApplyLocally function _ApplyRemotely function Apply ++++++ For and While statements For and while statements will have the "do" part on the first line. Example: for i in "${var[@]}"; do ... done while [ $i -eq 1 ]; do ... done ++++++ If statements If statements will be fully written (word "if" must be used). then is written on the same line. (Use sed ':a;N;$!ba;s/]\n\t*then/]; then/g' to convert files to this format... Replace "],new line, zero or more tabs, then" by "; then") if [ something ]; then stuff else other stuff fi ++++++ Logging A logging function is available that writes both to log file and stdout/stderr. It has the following global variable modifiers: _LOGGER_SILENT=true/false: disables any output to stdout/stderr _LOGGER_VERBOSE=true/false: logs messages with log level VERBOSE _LOGGER_ERR_ONLY=true/false: disables logging to log file and stdout/stderr except for CRITICAL, ERROR, WARN and ALWAYS log levels. The following log levels exist: - PARANOIA_DEBUG: Only used by debugging functions themselves - DEBUG: Only log this when _DEBUG flag is set in program. Any command forged for eval instruction should be logged by this. - NOTICE: Standard messages - ALWAYS: Standard messages, regardless of _LOGGER_ERR_ONLY - WARN: Requires attention - ERROR: Program produced an error but continues execution - CRITICAL: Program execution is halted Can be called with: Logger "My message" "LOGLEVEL" $retval $retval is an optional parameter that passes the exit code of the command that triggered the logging message $retval, along with function stack, script pid and current pid can be found in the ERROR /WARN alert files ($RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID) ++++++ Eval Most commands should be logged to a tmp file. The basic way of doing is: cmd='"something '$somevar'" > some_file 2>&1' eval $cmd & WaitForTaskCompletion $! 0 0 1 0 true false true false retval=$? if [ $retval -ne 0 ]; then Logger "Some error message" "ERROR" $retval fi Remote commands should always invoke bash (using '"'"' to escape single quotes of 'bash -c "command"'). It is preferable to use ssh heredoc in order to use plain code. If local and remote code is identical, wrap remote code in a function so only minor modifications are needed. Remote code return code is transmitted via exit. cmd=$SSH_CMD' '"'"'bash -c "some; commands \"'$VARIABLE'\" some; other; commands" > some_file'"'"' 2>&1' Better formule $SSH_CMD remoteVar="'$localVar'" 'bash -s' << 'ENDSSH' > 2>&1 function remoteSub { some code return 0 } remoteSub exit $? ENDSSH retval=$? if [ $retval -ne 0 ]; then Logger "Some error message" "ERROR" $retval fi ++++++ File variables All eval cmd should exit their content to a file called "$RUNDIR/$PROGRAM.${FUNCNAME[0]}.$SCRIPT_PID" Dots are used instead of '_' so variables can be separated with a forbidden char in variable names, so the separtors apply as wished. ++++++ String function calls String returning functions should only be called this way in order to deal with spaces: Quoting happens outside the function call. echo "$(myStringFunction $myStringVar)" ++++++ ofunctions As obackup and osync share alot of common functions, ofunctions.sh will host all shared code. Dev programs n_osync.sh and n_obackup.sh will source ofunctions.sh Release programs will still include ofunctions.sh in order to enhance ease of use. Ofunctions are defined like: #__FUNC:FunctionName function FunctionName { } #__ENDFUNC These functions are inserted into code that has placeholders like #__FUNC:FuncName +++++++ includes ofunctions parts can be directly included in shell scripts. The parts that needs to be included must be contained within specific comments: #### MyFunction SUBSET #### function MyFunction { ... } #### MyFunction SUBSET END #### These can later be included in shell scripts with: include #### MyFunction SUBSET #### In order to have those includes parsed, we use bootstrap.sh to launch the original shell script. Original shell script will not work because include is not a bash statement. Include the following code into original shell script include #### _OFUNCTIONS_BOOTSTRAP SUBSET #### [ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0)" && exit 1 +++++++ Exit codes Normal exit code = 0 Run with errors exit code = 1 Run with warnings exit code = 2 Wrong shell exit code = 127 Usage function exit code = 128 +++++++ Detailled debugging When launching the program with 'bash -x', add SLEEP_TIME=1 so wait functions won't spam output Ex: SLEEP_TIME=1 bash -x ./program.sh ++++++ Finding code errors Before every release, shellcheck must be run Also a grep -Eri "TODO|WIP" osync/* must be run in order to find potential release blockers Use shellcheck.net now and then (ignore SC2086 in our case) Use a low tech approach to find uneven number of quotes per line tr -cd "'\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}' tr -cd "\"\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'