osync/dev/CODING_STYLE.TXT

Coding style used for my bash projects (v3.0 Dec 2016)

++++++ Header

Always use the following header

----BEGIN HEADER
#!/usr/bin/env bash

PROGRAM="program-name" # Long description
AUTHOR="(C) 20XX-20YY by Orsiris de Jong"
CONTACT="http://www.example.com me@example.com"
PROGRAM_BUILD=YYYYMMDDVV

## Optional instructions
----END HEADER

Using bind style versionning:
YYYYMMDDVV (Year, Month, Day, Revision): Example: 2015012402 = 2nd revision of 24 Jan 2015

#!/usr/bin/env bash instead of #!/bin/bash

Change old scripts with
for i in $(grep -r '#!/bin/bash' * |cut -f1 -d':'); do sed -i 's&#!/bin/bash&#!/usr/bin/env bash&g' $i; done

type instead of type -p for bash test (other shells don't know -p)

++++++ Indentation

Using tabs
Transform old shell scripts using unexpand command

++++++ Comments

Some command # comment
## Some comment on a new line
################################################# Some separation

++++++ Work comments

Whenever there is some idea to postpone, use #TODO(priority):[dev-name:] some remark
Priority can be critical, high, medium, low, verylow. No release can happen if there are TODOs other than low or verylow.
Example: #TODO(high):deajan: need to do something

A "work in progress" marker must be left on the line a dev is working when it's work isn't finished). Marker is #WIP:dev-name: some remark
dev-name is mandatory if more than one person is coding
Example: #WIP:deajan: missing function something

++++++ Variables

All local variables names have each first letter of the words uppercase and all others lowercase, except for the first word where all letters are lowercase
Example: someLongVariable
All global variables are full upercase, separated by _
Example: EXEC_TIME
All environment variables (verbose, silent, debug, etc) have prefix _ and are full upercase, separated by _
Example: _PARANOIA_DEBUG

Exec time variables that can take boolean values should use true and false instead of 1 and 0.

++++++ Functions

All function names should begin with an uppercase letter for every word, the other letters should be lowercase
Example: SomeFunctionThatRocks

Bash does not provide any checks against missing function arguments. Also, missing quotes can lead to an inconsistent number of arguments.
Most functions should have a first line that calls the special function __CheckArguments, which checks the number of given arguments for a function in order
to find possible problems. Number of arguments are given as first argument to __CheckArguments. May be a number or a range, eg 0-2 if the function takes optional arguments.
__CheckArguments will only trigger when the script is launched with _PARANOIA_DEBUG=yes. Also, it will only exist in the debug version.
Use the following convention for function definition:

function SomeFunction {
	__CheckArguments 0 $# ${FUNCNAME[0]} "$@"               #__WITH_PARANOIA_DEBUG
	...	
}

Use sed ':a;N;$!ba;s/\n{\n/ {\n/g' to convert functions that have opening brackets on a new line.

If the function has arguments, use local variable names that are more readable than $1...$n. Explain via comments what those variables contain if needed.
Declare arguments before launching __CheckArguments:

function AnotherFunction {
	local varName="${1}"
	local otherVarName="${2}" # This variable contains stuff
	__CheckArguments 2 $# ${FUNCNAME[0]} "$@"		#__WITH_PARANOIA_DEBUG
	...
}

Functions should always have return status
function RandomFunction {
	...
	return $?
}

++++++ Sub functions

When a function is a subroutine of another function, it is called _SomethingAsSubFunction:
Example:

function _ApplyLocally
function _ApplyRemotely
function Apply

++++++ For and While statements

For and while statements will have the "do" part on the first line.
Example:

for i in "${var[@]}"; do
	...
done

while [ $i -eq 1 ]; do
	...
done

++++++ If statements

If statements will be fully written (word "if" must be used). then is written on the same line.
(Use sed ':a;N;$!ba;s/]\n\t*then/]; then/g' to convert files to this format... Replace "],new line, zero or more tabs, then" by "; then")
if [ something ]; then
	stuff
else
	other stuff
fi

++++++ Logging

A logging function is available that writes both to log file and stdout/stderr.
It has the following global variable modifiers:

_LOGGER_SILENT=true/false: disables any output to stdout/stderr
_LOGGER_VERBOSE=true/false: logs messages with log level VERBOSE
_LOGGER_ERR_ONLY=true/false: disables logging to log file and stdout/stderr except for CRITICAL, ERROR, WARN and ALWAYS log levels. 

The following log levels exist:

- PARANOIA_DEBUG: Only used by debugging functions themselves
- DEBUG: Only log this when _DEBUG flag is set in program. Any command forged for eval instruction should be logged by this.
- NOTICE: Standard messages
- ALWAYS: Standard messages, regardless of _LOGGER_ERR_ONLY
- WARN: Requires attention
- ERROR: Program produced an error but continues execution
- CRITICAL: Program execution is halted

Can be called with:
Logger "My message" "LOGLEVEL" $retval

$retval is an optional parameter that passes the exit code of the command that triggered the logging message
$retval, along with function stack, script pid and current pid can be found in the ERROR /WARN alert files ($RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID)

++++++ Eval

Most commands should be logged to a tmp file.
The basic way of doing is:

cmd='"something '$somevar'" > some_file 2>&1'
eval $cmd &
WaitForTaskCompletion $! 0 0 $FUNCNAME
retval=$?
if [ $retval -ne 0 ]; then
	Logger "Some error message" "ERROR" $retval
fi

Remote commands should always invoke bash (using '"'"' to escape single quotes of 'bash -c "command"'). It is preferable to use ssh heredoc in order to use plain code.
If local and remote code is identical, wrap remote code in a function so only minor modifications are needed.
Remote code return code is transmitted via exit.

cmd=$SSH_CMD' '"'"'bash -c "some; commands \"'$VARIABLE'\" some; other; commands" > some_file'"'"' 2>&1'

Better formule

$SSH_CMD remoteVar="'$localVar'" 'bash -s' << 'ENDSSH' > 2>&1
function remoteSub {
	some code
	return 0
}

remoteSub
exit $?
ENDSSH
retval=$?
if [ $retval -ne 0 ]; then
	Logger "Some error message" "ERROR" $retval
fi

++++++ File variables

All eval cmd should exit their content to a file called "$RUNDIR/$PROGRAM.${FUNCNAME[0]}.$SCRIPT_PID"
Dots are used instead of '_' so variables can be separated with a forbidden char in variable names, so the separtors apply as wished.


++++++ String function calls

String returning functions should only be called this way in order to deal with spaces:
Quoting happens outside the function call.

echo "$(myStringFunction $myStringVar)"

++++++ Finding code errors

Use shellcheck.net now and then (ignore SC2086 in our case)

Use a low tech approach to find uneven number of quotes per line

tr -cd "'\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'
tr -cd "\"\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'

++++++ ofunctions

As obackup and osync share alot of common functions, ofunctions.sh will host all shared code.
Dev programs n_osync.sh and n_obackup.sh will source ofunctions.sh
Release programs will still include ofunctions.sh in order to enhance ease of use.

Ofunctions are defined like:

#__FUNC:FunctionName
function FunctionName {

}
#__ENDFUNC

These functions are inserted into code that has placeholders like #__FUNC:FuncName

+++++++ includes

ofunctions parts can be directly included in shell scripts.
The parts that needs to be included must be contained within specific comments:

#### MyFunction SUBSET ####
function MyFunction {
	...
}
#### MyFunction SUBSET END ####

These can later be included in shell scripts with:


include #### MyFunction SUBSET ####

In order to have those includes parsed, we use bootstrap.sh to launch the original shell script.
Original shell script will not work because include is not a bash statement.
Include the following code into original shell script

include #### _OFUNCTIONS_BOOTSTRAP SUBSET ####
[ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0)" && exit 1
+++++++ Exit codes

Normal exit code = 0
Run with errors exit code = 1
Run with warnings exit code = 2
Wrong shell exit code = 127
Usage function exit code = 128

+++++++ Detailled debugging

When launching the program with 'bash -x', add SLEEP_TIME=1 so wait functions won't spam output
Ex:

SLEEP_TIME=1 bash -x ./program.sh