OPENTECHLY
OPENTECHLY

Welcome to the Opentechly Blog! You can find personal experience, trends, news and tips in the open source world, in a way that is engaging and easy to understand. Thanks for stopping by!

Habnai
Author

Habnai web developer, IT-tech, and tech junkie! I love to learn all there is to know about tech and programming, working to expand my knowledge every day.Feel free to contact me habnai@opentechly.com

Share


Tags


OPENTECHLY

Secure Bash scripts with this...

New quotation, of an article which risks disappearing without warning. The defensive programming is to structure its code to limit to the strict minimum of attack surfaces. The JDN has an article that sums up a lot of concepts, but here we will focus on Bash scripts. Indeed, as with PHP we can quickly make crap, and it is preferable to follow certain practices so that their quality is not too catastrophic when they will be reused elsewhere. I don't follow them all, but I have found it useful to share them with you.

Global immutable variables

readonly PROGNAME=$(basename $0)
readonly PROGDIR=$(readlink -m $(dirname $0))
readonly ARGS="$@"

Everything is local

All variables should be local.

change_owner_of_file() {
  local filename=$1
  local user=$2
  local group=$3
 
  chown $user:$group $filename
}
 
 
change_owner_of_files() {
  local user=$1; shift
  local group=$1; shift
  local files=$@
  local i
 
  for i in $files
  do
    chown $user:$group $i
  done
kfir@goofy ~ $ local a
bash: local: can only be used in a function

hand()

main() {
  local files="/tmp/a /tmp/b"
  local i
 
  for i in $files
    do
      change_owner_of_file kfir users $i
    done
}
main

Everything is a function

main() {
local files=$(ls /tmp | grep pid | grep -v daemon)
}
temporary_files() {
  local dir=$1
 
  ls $dir \
    | grep pid \
    | grep -v daemon
}
 
main() {
  local files=$(temporary_files /tmp)
}
test_temporary_files() {
  local dir=/tmp
 
  touch $dir/a-pid1232.tmp
  touch $dir/a-pid1232-daemon.tmp
 
  returns "$dir/a-pid1232.tmp" temporary_files $dir
 
  touch $dir/b-pid1534.tmp
 
  returns "$dir/a-pid1232.tmp $dir/b-pid1534.tmp" temporary_files $dir
}

As you can see, this test is not about main ().

Debug functions

bash -x my_prog.sh

temporary_files() {
  local dir=$1
 
  set -x
  ls $dir \
    | grep pid \
    | grep -v daemon
  set +x
}
temporary_files() {
  local dir=$1
 
  set -x
  ls $dir \
    | grep pid \
    | grep -v daemon
  set +x
}

So, by calling the function:

temporary_files /tmp

will print on standard output:

temporary_files /tmp

Code clarity

What does this code do?

main() {
  local dir=/tmp
 
  [[ -z $dir ]] \
    && do_something...
 
  [[ -n $dir ]] \
    && do_something...
 
  [[ -f $dir ]] \
    && do_something...
 
  [[ -d $dir ]] \
    && do_something...
}
main

Let your code do the talking:

is_empty() {
  local var=$1
 
  [[ -z $var ]]
}
 
is_not_empty() {
  local var=$1
 
  [[ -n $var ]]
}
 
is_file() {
  local file=$1
 
  [[ -f $file ]]
}
 
is_dir() {
  local dir=$1
 
  [[ -d $dir ]]
}
 
main() {
  local dir=/tmp
 
  is_empty $dir \
    && do_something...
 
  is_not_empty $dir \
    && do_something...
 
  is_file $dir \
    && do_something...
 
  is_dir $dir \
    && do_something...
}
main

Each line does one thing

temporary_files() {
  local dir=$1
 
  ls $dir | grep pid | grep -v daemon
}

Can be written more clearly:

temporary_files() {
  local dir=$1
 
  ls $dir \
    | grep pid \
    | grep -v daemon
}
temporary_files() {
  local dir=$1
 
  ls $dir | \
    grep pid | \
    grep -v daemon
}

Good example where we can clearly see the connection between the lines and the symbols:

print_dir_if_not_empty() {
  local dir=$1
 
  is_empty $dir \
    && echo "dir is empty" \
    || echo "dir=$dir"
}

Show usage

Do not do that :

Shell
echo "this prog does:..."
echo "flags:"
echo "-h print help"
1
2
3
echo "this prog does:..."
echo "flags:"
echo "-h print help"

It should be a function:

usage() {
  echo "this prog does:..."
  echo "flags:"
  echo "-h print help"
}

echo is repeated on each line. Instead we have 'Here Document':

usage() {
	cat <<- EOF
	usage: $PROGNAME options
 
	Program deletes files from filesystems to release space.
	It gets config file that define fileystem paths to work on, and whitelist rules to
	keep certain files.
 
	OPTIONS:
		-c --config configuration file containing the rules. use --help-config to see the syntax.
		-n --pretend do not really delete, just how what you are going to do.
		-t --test run unit test to check the program
		-v --verbose Verbose. You can specify more then one -v to have more verbose
		-x --debug debug
		-h --help show this help
		   --help-config configuration help
 
	Examples:
		Run all tests:
		$PROGNAME --test all
 
		Run specific test:
		$PROGNAME --test test_string.sh
 
		Run:
		$PROGNAME --config /path/to/config/$PROGNAME.conf
 
		Just show what you are going to do:
		$PROGNAME -vn -c /path/to/config/$PROGNAME.conf
	EOF
}

Be careful to use '\ t' tabs for the start of each line. In vim you can use this trick if your tab consists of 4 spaces:

:s/^    /\t/

Command line arguments

Here is an example of a complement to the usage function above. This code is taken from the article bash shell script to use get opts with gnu style long positional parameters on Kirk's blog:

cmdline() {
  # got this idea from here:
  # http://kirk.webfinish.com/2009/10/bash-shell-script-to-use-getopts-with-gnu-style-long-positional-parameters/
  local arg=
  for arg
  do
    local delim=""
    case "$arg" in
      #translate --gnu-long-options to -g (short options)
      --config) args="${args}-c ";;
      --pretend) args="${args}-n ";;
      --test) args="${args}-t ";;
      --help-config) usage_config && exit 0;;
      --help) args="${args}-h ";;
      --verbose) args="${args}-v ";;
      --debug) args="${args}-x ";;
      #pass through anything else
      *) [[ "${arg:0:1}" == "-" ]] || delim="\""
         args="${args}${delim}${arg}${delim} ";;
    esac
  done
 
  #Reset the positional parameters to the short options
  eval set -- $args
 
  while getopts "nvhxt:c:" OPTION
  do
    case $OPTION in
      v)
          readonly VERBOSE=1
          ;;
      h)
          usage
          exit 0
          ;;
      x)
          readonly DEBUG='-x'
          set -x
          ;;
      t)
          RUN_TESTS=$OPTARG
          verbose VINFO "Running tests"
          ;;
      c)
          readonly CONFIG_FILE=$OPTARG
          ;;
      n)
          readonly PRETEND=1
          ;;
    esac
  done
 
  if [[ $recursive_testing || -z $RUN_TESTS ]]; then
    [[ ! -f $CONFIG_FILE ]] \
      && eexit "You must provide --config file"
  fi
  return 0
}

We then use it this way by using the immutable ARGS variable we defined at the start of the script:

main() {
cmdline $ARGS
}
main

Unit tests

test_config_line_paths() {
  local s='partition cpm-all, 80-90,'
 
  returns "/a" "config_line_paths '$s /a, '"
  returns "/a /b/c" "config_line_paths '$s /a:/b/c, '"
  returns "/a /b /c" "config_line_paths '$s /a : /b : /c, '"
}
 
config_line_paths() {
  local partition_line="$@"
 
  echo $partition_line \
    | csv_column 3 \
    | delete_spaces \
    | column 1 \
    | colons_to_spaces
}
 
source /usr/bin/shunit2

Here is another example on the df command:

DF=df
 
mock_df_with_eols() {
    cat <<- EOF
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /very/long/device/path
                         124628916  23063572 100299192  19% /
    EOF
}
 
test_disk_size() {
  returns 1000 "disk_size /dev/sda1"
 
  DF=mock_df_with_eols
  returns 124628916 "disk_size /very/long/device/path"
}
 
df_column() {
  local disk_device=$1
  local column=$2
 
  $DF $disk_device \
    | grep -v 'Use%' \
    | tr '\n' ' ' \
    | awk "{print \$$column}"
}
 
disk_size() {
  local disk_device=$1
 
  df_column $disk_device 2
}

Here there is an exception, for the tests I declare DF at the global level and not readonly. This is because shunit2 does not allow function changes at the global level.


Those more experienced in software development may find these script constructions obvious, for my part I had already read some of these tips, especially for Python. It's nice to see that we can reuse the same concepts for Bash, by default it is a particularly permissive language, like PHP. It didn't take much for the two to build up a disastrous reputation.

PS: Thank you Kefir https://kfirlavi.herokuapp.com/

Habnai
Author

Habnai

Habnai web developer, IT-tech, and tech junkie! I love to learn all there is to know about tech and programming, working to expand my knowledge every day.Feel free to contact me habnai@opentechly.com

View Comments