Lecture 2

  1. Read man ls and write an ls command that lists files in the following manner

    A sample output would look like this

    -rw-r--r--   1 user group 1.1M Jan 14 09:53 baz
drwxr-xr-x   5 user group  160 Jan 14 09:53 .
-rw-r--r--   1 user group  514 Jan 14 06:42 bar
-rw-r--r--   1 user group 106M Jan 13 12:12 foo
drwx------+ 47 user group 1.5K Jan 12 18:08 ..

    -l for the long listing format

    -a to include all files

    -h for human readable format

    -t to sort by recency

    --color=auto to color the output if going to the terminal

    ls -laht --color=auto

  2. Write bash functions marco and polo that do the following. Whenever you execute marco the current working directory should be saved in some manner, then when you execute polo, no matter what directory you are in, polo should cd you back to the directory where you executed marco. For ease of debugging you can write the code in a file marco.sh and (re)load the definitions to your shell by executing source marco.sh.

    marco_polo.sh

    #!/usr/bin/bash

function marco() {
	#Save current directory in global variable
	export MARCO_DIR=$(pwd)
}

function polo() {
	#Verify if there is a saved directory
	if [ -z "$MARCO_DIR" ]; then
		echo "No directories saved, please run 'marco' first."

	else
		#Change directory and display error message if failed
		cd $MARCO_DIR || echo "Failed to change directory to $MARCO_DIR"
	
	fi
}

    Marco assigns the current directory to a variable using export to ensure the variable has environment scope.

    Polo first verifies if there is a directory stored (-z verifies if a value is empty).

    It then changes the directory and displays a message if it fails with ||

  3. Say you have a command that fails rarely. In order to debug it you need to capture its output but it can be time consuming to get a failure run. Write a bash script that runs the following script until it fails and captures its standard output and error streams to files and prints everything at the end. Bonus points if you can also report how many runs it took for the script to fail.

    #!/usr/bin/env bash

n=$(( RANDOM % 100 ))

if [[ n -eq 42 ]]; then
	echo "Something went wrong"
	>&2 echo "The error was using magic numbers"
	exit 1
fi

echo "Everything went according to plan"

    run_until_fail

    #!/usr/bin/bash

#To create and clear the files
> output.log
> error.log

#initialise counter
count=0

#infinite while loop
while true; do

	#increment count
	count=$((count + 1))

	#execute script and put output and error in files
	./script.sh >> output.log 2>> error.log

	#break from while loop if exit code  is not 0
	if [[ $? -ne 0 ]]; then
		break
	fi
done

echo "The script failed after $count runs."

echo "Standard output:"
cat output.log

echo "Standard error:"
cat error.log
    Make both files executable:
    chmod +x script.sh run_until_fail.sh
    Then run
    ./run_until_fail

    Terminal output:

    The script failed after 173 runs.
Standard output:
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Everything went according to plan
Something went wrong
Standard error:
The error was using magic numbers

  4. As we covered in the lecture find’s -exec can be very powerful for performing operations over the files we are searching for. However, what if we want to do something with all the files, like creating a zip file? As you have seen so far commands will take input from both arguments and STDIN. When piping commands, we are connecting STDOUT to STDIN, but some commands like tar take inputs from arguments. To bridge this disconnect there’s the xargs command which will execute a command using STDIN as arguments. For example ls | xargs rm will delete the files in the current directory.

    Your task is to write a command that recursively finds all HTML files in the folder and makes a zip with them. Note that your command should work even if the files have spaces (hint: check -d flag for xargs).

    If you’re on macOS, note that the default BSD find is different from the one included in GNU coreutils. You can use -print0 on find and the -0 flag on xargs. As a macOS user, you should be aware that command-line utilities shipped with macOS may differ from the GNU counterparts; you can install the GNU versions if you like by using brew.

    1. First you need to install zip if it isn't already done.
      sudo apt install zip
    2. Then you can run
      find . -name "*.html" -print0 | xargs -0 zip html_files.zip

      find . searches the current directory

      *.html matches any file ending with .html

      -print0 and -0 arguments handle whitespace in file names by seperating files with null characters which won't interfere with whitespace in file names

  5. (Advanced) Write a command or script to recursively find the most recently modified file in a directory. More generally, can you list all files by recency?
    find . -type f -exec stat --format="%Y %n" {} + | sort -n -r | head -n1 | sed 's/^[0-9]* //'

    find . -type f -exec stat --format="%Y %n" {} +: Finds all regular files recursively and prints the modification time followed by the file name.

    sort -n -r: Sorts the files by modification time.

    head -n1: Selects the first (most recently modified) file.

    sed 's/^[0-9]* //': Uses sed to remove the number (modification time in seconds) at the beginning of the line. The ^[0-9]* matches the leading digits, and the space following removes the space between the time and the file name. You will learn more about sed in the upcomming lectures.

Previous lecture
Next lecture