Nephology Exercises

Cloud Computing Basics

Exercise 22

1. Write a bash script that takes a file path as an argument and copies that file to a designated folder on your server.

2. Find a file online that changes periodically, then write a program to download that file every time it changes.

3. Try creating your own Twitter or GitHub bot with the Twitter API or the GitHub API.

Solution to

1. Listing 11 cloud-backup.sh

   [[ $# -ne 1 ]] && echo Usage: $0 FILE-OR-DIR && exit 1
   FILE_OR_DIR_SAVE_AS=$(date -I)_$(basename $1)
   # `basename` gets rid of parent directories, otherwise `scp` cannot store it as
   # `scp` cannot create many levels of directories. `rsync` is better but must be
   # installed both on server and the client
   # e.g., if the argument is `dir1/file`, then without `basename` `scp` will fail.
   SSH_SERVER=aydos.de
   BACKUP_FOLDER=backups
   scp -r $1 $SSH_SERVER:$BACKUP_FOLDER/$FILE_OR_DIR_SAVE_AS
   

2. For instance the index page of https://lsi.th-deg.de/the-unix-workbench-notes changes frequently. Using [curl -z] we can download only if a file has changed. Probably this flag makes much more sense if we are checking a very large file periodically.

3. Listing 12 download-if-modified.sh

   [[ $# -ne 1 ]] && echo Usage: $0 URL && exit 1
   URL=$1
   curl --remote-name -z $(basename $URL) $URL
   # `--remote-name` uses the remote name of the downloaded file to save the file 
   # as. For instance https://aydos.de/dir1/dir2/file.jpg will be installed as
   # `file.jpg`.
   # `basename` used for getting the remote name.
   

   To check if curl -z really works, you can check

   • the modification date of the downloaded file

   • or use curl -v to make curl more verbose.

4. A Twitter bot that posts jokes:

   Listing 13 tweet-a-joke.sh

   USER_AGENT='Joke Tweeter https://aydos.de/uni'
   JOKE=$(curl -H 'User-Agent: $USER_AGENT' -H 'Accept: text/plain' https://icanhazdadjoke.com/)
   TWEET="$JOKE Credit: icanhazdadjoke.com"
   echo Tweeting
   echo $TWEET
   echo ...
   twurl -d "status=$TWEET" /1.1/statuses/update.json
   

   Steps:

   1. Register for a Twitter developer account. Note that you have to describe what you plan to do with your developer account. Then create an app.

   2. Your app will use a Twitter account to tweet. The Twitter developer account and the Twitter user account are different from eachother. Create a new account or use your existing account which your app will use to post tweets.

   3. Install twurl which is a special version of curl tailored for the Twitter API.

   4. Authorize the app using your user account.

      twurl authorize \
      	--consumer-key=NvrPvtYAWeiJSAdGpHRD \
      	--consumer-secret=OXtcqpmahpgcjDZDuVOkHPaIFuJwROvyLybZSsWvCzPOyxnqzpolBdmmSnsz
      

   5. Try the script by running it manually.

   6. Deploy the script on a computer or server so that it runs periodically. There are multiple ways to achieve this.

      • You can use cron to run a program preiodically.

      • If you have root access on the server you can use systemctl timers.

      • If you only have user access, you can create a persistent session — in other words a shell which will not be killed even you disconnect from the server. Some programs for creating a persistent session are:

        • screen

        • tmux

        • xpra

   For example using tmux:

   1. Run tmux, a new shell will be opened.

   2. To run your script every 24h assuming that the script is stored under $HOME/.scripts

      while true; do $HOME/.scripts/tweet-a-joke.sh; sleep 1d; done
      

   3. You can exit your persistent sessions by pressing: <CTRL>B then d (stands for detach)

   4. You can reenter your session by tmux attach

   This bot works similar to @icanhazdadjoke