Week 3 Discussion - Bash Programming

Week 3 Discussion - Bash Programming#

Intro#

Exercise 66

What is your favorite text/source-code editor/s? Why?

Math#

Exercise 67

What is the problem with the following Bash expressions?

expr 24 * 7 * 52
expr 10+5/3

Exercise 68

Carry out the following operations using Bash:

24*7*52  # 8736
1+5*3    # 16
(1+5)*3  # 18
3.333*3  # 9.999
3/2      # 1.5

Exercise 69

  1. What does the -l option exactly do in bc?

  2. When is the option -l useful?

Variables#

Exercise 70

What is the problem with the following?

JOKE_LIMIT = 2
JOKE_TERM = student
curl -H "Accept: text/plain" "https://icanhazdadjoke.com/search?term=$JOKE_TERM&limit=$JOKE_LIMIT"

Exercise 71

What is the difference between these two examples?

FILE_ID=5
FILE_ID=$FILE_ID+1

FILE_ID=5
let FILE_ID=$FILE_ID+1

Exercise 72

  1. How can you store the output of a command in a variable? Complete the line beginning with TEMP_CELSIUS

  2. How is the syntax which you used called?

  3. What is wrong with the Fahrenheit conversion?

# use wttr.in API to get the temperature in Celsius using the output of
# `curl wttr.in/\?format=%f | cut -c -3`
TEMP_CELSIUS=
echo Temperature in Celsius: $TEMP_CELSIUS

# Convert to Fahrenheit
let TEMP_FAHRENHEIT=$TEMP_CELSIUS\*1.8+32
echo Temperature in Fahrenheit: $TEMP_CELSIUS

Exercise 73

Bash can do arithmetics using expr 5 + 4. But Bash supports a shorter syntax to expand arithmetic expressions. How does this syntax look like and how it is called?

You may take a look at Shell Expansions (Bash Reference Manual)

Exercise 74

What do the following variables refer to in a shell script?

You may take a peek at Shell Parameters (Bash Reference Manual)

$2
$15
${15}
$@
$#
$0

Exercise 75 

DAYS_UNTIL_BDAY=5
echo "You have $DAYS_UNTIL_BDAY days until your birthday, haven't you?"

echo Today\'s date is $(date -I).

In the former script the strings preceded by $ or enclosed by $(...) are replaced with another string. What is this concept called in Bash?

User Input#

Exercise 76

What is the problem with the following script?

echo "Which city's weather forecast do you need?"
read $CITY
echo "Ok! Showing the weather forecast for $CITY":
curl wttr.in/$CITY

Logic and If/Else#

Exercise 77

What is the output of the following lines?

false
echo $?
false && true
echo $?
false && true && true
echo $?
true || false
echo $?

Exercise 78

What is the output?

true || echo Hello!

Exercise 79

  1. You are writing code which checks if the right amount of arguments are supplied to your script. What would be the output if $# (number of arguments) is 3?

  2. Why?

[[ $# -eq 3 ]] || echo 😒 || exit 1 && echo arguments: $1 $2 $3 || echo 🤷

Exercise 80

  • x, y and z are variables. x or y and z is called a … .

  • or, and are … .

  • In Bash syntax or and and are written as … and …, respectively.

  • If the last command was successful, its … will be 0.

  • If the last command was unsuccessful, its … will be … .

Exercise 81

You want to impress your friend with your fresh shell hacking skills by writing a lyrics retriever script. Your script expects exactly two arguments, artist and title. Your script should exit with an error message if these arguments are missing. The error message could be “Usage: ./script.sh ARTIST TITLE” Complete the following script:

## YOUR CODE HERE

ARTIST=$1
TITLE=$2

# encode space chars and
# get the lyrics using lyrics.ovh API
curl -L api.lyrics.ovh/v1/${ARTIST// /%20}/${TITLE// /%20}

Exercise 82

Give two command examples which use an unary expression:

Exercise 83

The following script shortens URLs using the API of the is.gd URL shortening service.

It accepts a single parameter, the URL which should be shortened. Implement two checks for the parameter:

  1. Exactly one single parameter is allowed.

  2. The parameter should be a valid URL. Implement a simple check which tests if the URL starts with a valid domain (domain.suffix), e.g., aydos.de.

Listing 14 shorten-url.sh#
### YOUR CODE HERE
URL=$1
### YOUR CODE HERE

curl "https://is.gd/create.php?format=simple&url=$URL"

Exercise 84

Rewrite the following script using if/else.

Listing 15 shorten-url.sh#
[[ $# -ne 1 ]] && echo Usage: $0 URL && exit 1 
URL=$1
[[ ! $URL =~ .+\..+ ]] && echo The provided parameter is not a valid URL && exit 1 

curl "https://is.gd/create.php?format=simple&url=$URL"

Arrays#

Exercise 85

What is wrong with the following code?

DIRS_TO_BACKUP=home etc var
FIRST_PRIORITY=${DIRS_TO_BACKUP[0]}
LAST_PRIORITY=${DIRS_TO_BACKUP[-1]}

echo First directory to backup: $FIRST_PRIORITY
echo Last directory to backup: $LAST_PRIORITY

echo There are ${#DIRS_TO_BACKUP} directories to backup.

Exercise 86

You have some files with .intro, .body and .conclusion extensions. These contain introduction, body and conclusion paragraphs.

The following block creates some example files.

tee $(mktemp XXX.intro) <<< intro1
tee $(mktemp XXXX.intro) <<< intro2
tee $(mktemp XXXXX.intro) <<< intro3
tee $(mktemp XXX.body) <<< body1
tee $(mktemp XXXX.body) <<< body2
tee $(mktemp XXX.conclusion) <<< conclusion1
tee $(mktemp XXXX.conclusion) <<< conclusion2
tee $(mktemp XXXXX.conclusion) <<< conclusion3
tee $(mktemp XXXXXX.conclusion) <<< conclusion4

Write a program which generates all possible stories consisting of these paragraphs, where every story follow the introduction-body-conclusion structure. The filenames of the stories have the following format: NAME1-NAME2-NAME3.txt. Do not use the first intro, body and conclusion that you encounter (the first item in your array).

You can get rid of the file extension by using parameter expansion as follows:

file=filename.extension
echo ${file%.*}  # outputs: filename

Array variables vs variables that contain a sequence of strings#

Note that the following does not create an array but a normal variable which consists of a sequence of filenames:

mktemp XXX.intro
mktemp XXXX.intro
intros=*intro
echo ${intros[1]}  # outputs nothing
echo ${intros[0]}  # outputs all .intro files

In this example we see that even we can use indexing (like intros[1]) on normal variables, but they will output nothing. This behavior is partly documented in Arrays (Bash Reference Manual):

… Any reference to a variable using a valid subscript is legal, …

All the items are stored in the index 0, because everything is stored as a single item but not partitioned as in an array.

The manual additionally states:

… and bash will create an array if necessary.

Example:

var='1 2 3'
declare -p var  # outputs: declare -- var="1 2 3"

echo ${var[1]}  # outputs nothing
declare -p var  # outputs: declare -- var="1 2 3"

var[1]=new
echo ${var[1]}  # outputs: new
declare -p var  # outputs: declare -a var=([0]="1 2 3" [1]="new")

But an array is not converted to a normal variable if we access it as a normal variable. As documented in Arrays (Bash Reference Manual):

Referencing an array variable without a subscript is equivalent to referencing with a subscript of 0.

var[0]=1
var[1]=2
declare -p var  # outputs: declare -a var=([0]="1" [1]="2")
echo $var  # outputs: 1 instead of 1 2

In the previous example we converted a normal variable to an array by accessing the variable using an index, but the existing items were assigned to the index 0. If every space separated string must be assigned to a separate index, the following conversion approach is better:

unset var
# make sure that var is not defined, otherwise we may be overwriting an
# existing array variable

var='1 2 3'
declare -p var  # outputs: declare -- var="1 2 3"
echo ${var[1]}  # outputs nothing
var=($var)
declare -p var  # outputs: declare -a var=([0]="1" [1]="2" [2]="3")
echo ${var[1]}  # outputs the second item

Braces#

Exercise 87

Use brace expansion to generate following commands/strings:

0 1 2 3 4 5
22 23 24 32 33 34 42 43 44
141 142 14x 14y 151 152 15x 15y 161 162 16x 16y
AGGGA AGGGG AGGGC AGGGT GGGGA GGGGG GGGGC GGGGT CGGGA CGGGG CGGGC CGGGT TGGGA TGGGG TGGGC TGGGT
echo cp IMG_1934.jpg IMG_1934-backup.jpg

Exercise 88

If we want to use variables to generate a sequence — for example in a script — we need to use the eval command:

a=0
b=5
echo {$a..$b}
eval echo {$a..$b}

This is strange, isn’t it? What could be the reason? Take a look into Brace Expansion (Bash Reference Manual) and find the reason.

Exercise 89

  1. Write a script which accepts FILE_COUNT as a single parameter and generates FILE_COUNT empty files with the names FILE_0 ... FILE_{FILE_COUNT-1}, e.g., ./file_gen.sh 2 should generate FILE_0 and FILE_1.

  2. (Optional) Pay attention that ./file_gen.sh 11 generates FILE_00 to FILE_10, and not FILE_0 to FILE_10.

    Hint: There is a special syntax for generating terms with the same number of digits. Brace Expansion (Bash Reference Manual) may help you.

Loops#

Exercise 90

Write a script which accepts FILE_COUNT as a single parameter and generates FILE_COUNT files with the names FILE0 ... FILE_{FILE_COUNT-1} and their id as their content, e.g., ./file_gen_using_loops.sh 2 should generate FILE_0 and FILE_1 with the contents 0 and 1, respectively.

Exercise 91

Write a command which loops over all the jpg files in the working directory and prepends today’s date to their filenames.

Exercise 92

Write a command which generates new files with random names in working directory unless the number of files in the working directory has reached 100. Use a while loop.

You can use the Bash variables RANDOM or SRANDOM to generate random numbers.

Exercise 93

You have a file which contains lines in the following format: name score email, for example:

database.csv

Artayi 34 artayi@posteo.de
Kettar 87 kettar@jfa.cn
Akaca 34 email@aka.eu

Write a script which reads a file like this and generates for every line a file with the filename name.txt and the score as the content. For example ./your_script database.csv should generate the following files:

Artayi.txt
Kettar.txt
Akaca.txt

Their content should correspond to their individual scores.

Hint: use while and read. The example in read(1p) manual could help.

Exercise 94

Write a command which removes every file which only contains the string NA.

Functions#

Exercise 95

What is the role of the keyword local in functions?

Exercise 96

You have the following file which contains a function:

Listing 20 count_extensions.sh#
function countext {
	local extensions=$@
	local sum=0
	
	for extension in $extensions
	do
		let sum=sum+$(ls *.$extension | wc -l)
	done
	
	echo $sum
}

This function counts the number files with given extensions, e.g., if your working directory contains a.jpg b.jpg c.txt k.odt, then countext jpg txt would output 3.

Before using this function in your shell you have to make it available in your current shell. How would you do that? Try with bash count_extensions.sh, and then using source. What is the difference? What is the purpose of the source command in general?

Exercise 97

In a project you have downloaded numerous files in your directory, but the filenames do not have any file extensions, e.g., database instead of database.csv and you want to fix that. If the file contains at least ten lines then you decide to add the .csv extension to the filename, .txt otherwise. Additionally you want to log every rename operation in a log file.

  1. What would be the rough structure of your program? Try to use functions.

  2. What would be the advantage and disadvantage of using functions?

  3. Implement your program

Hint: if your script is in the same directory as the working directory, pay attention that you do not rename your script.

You can use the following script to create dummy files:

Listing 21 create_dummy_files.sh#
# Creates ten random CSV files in current working directory.
function random_true_or_false {
        if (( $RANDOM %2 == 0 ))
        then
                true
        else
                false
        fi
}

function generate_file {
        local number_of_lines=$1
        local filename=$(mktemp XXX)

        for i in $(seq $number_of_lines)
        do
                echo $RANDOM, $RANDOM, $RANDOM >> db-$filename.csv
        done
		echo generated the file db-$filename
}

for i in $(seq 10)
do
        if random_true_or_false
        then
                generate_file $(( $RANDOM + 10 ))
        else
                generate_file $(( $RANDOM % 10 ))
        fi
done

Exercise 98

What is the difference between return and echo in functions?

Writing Programs#

Exercise 99

What makes the Unix shell highly composable?

Exercise 100

How can we make a file executable, and what is the role of the shebang in this context?

Exercise 101

How can we run a Bash script without making it executable?

Exercise 102

Give three examples of environment variables defined on your current shell environment. What do these three variables stand for and why could they be defined at all?

Exercise 103

  1. What is the role of the environment variable PATH?

Summary and reflection#

Exercise 104

Did you reach the following learning objectives for this week?

  • Create a program using Bash

  • Use Bash to perform basic arithmetic

  • Store data in variables using Bash

  • Use Bash to collect user input

  • Create arrays in Bash

  • Use brace expansion to create strings out of sequences

  • Use loops to repeat lines of codes based on logical conditions or sequence

Contents