Problem 2: Write a Bash script that removes all zero length ordinary files in the directory (including those in the sub-directories at all levels) passed as an optional argument. If you do not specify...

86195/bash-scripting-3.pptx
Bash Scripting
The shell of Linux
Different linux shells: Bourne shell (sh), C shell (csh), Korn shell (ksh), TC shell (tcsh), Bourne Again shell (bash).
Bash: the most popular Linux shell
It is a command line interface. We used it to type in and run commends.
It is a scripting language. It interprets and runs scripts. We will write bash scripts.
Shell scripting uses the shell's facilities and existing software tools as building blocks to automate a lot of tasks.
Shell facilities: if, for loops, arrays, some built-in commands in shell (e.g., echo).
Existing software tools: grep, tr, uniq, …., any other executable files (binary and scripts).
Do not need to type in a lot of commands repeatedly.
Do not need to build programs from scratch (e.g., instructions).
2
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
The first bash program
Create a script and save the script
The first line (Shebang) tells Linux to use the bash interpreter to run this script.
Note # also starts the comments. But first line is special.
make the file executable using chmod.
Run the script.
Revise the script if it does not run correctly.
$vi hello.sh
#!/bin/bash
echo hello
$chmod 700 hello.sh
$./hello.sh
hello
$vi hello.sh
#!/bin/bash
echo Hello
3
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Including multiple commands in a script
Using commands to create a directory and copy all files into that directory before removing them.
Instead of having to type all the commands interactively on the shell, write a script
$ mkdir trash
$ mv * trash
$ vi trash.sh
#!/bin/bash
mkdir trash
mv * trash
$ ./trash.sh
4
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Bash scripts view all the data as texts/strings.
When a text contains space, tab, newline, the text must be enclosed in either single or double quotes.
$ cat my file1.txt #print out files "my" and "file1.txt"?
$ cat "my file1.txt" #print out file "my file1.txt"
When a text contains special characters, to be safe, the text must be enclosed in either single or double quotes.
The parts with special characters may be translated and replaced (see "expansions"), and new text may contain space/tab/newline.
5
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Variables
Variable values are always stored as strings
Introduce later: How to convert variables to numbers for calculations?
No need to declare a variable
assigning a value to a variable creates it.
Value extracted using $
Use { } when necessary
$ cat variable.sh
#!/bin/bash
STR="Hello World!"
echo $STR
STR2=Hello
echo $STR2
echo ${STR}2
$ ./variable.sh
Hello World!
Hello
Hello World!2
6
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Single and double quotes
When assigning character data containing spaces or special characters, the data must be enclosed in either single or double quotes.
#!/bin/bash
var="test string"
newvar="Value of var is $var"
echo $newvar
Output: Value of var is test string
#!/bin/bash
var='test string'
newvar='Value of var is $var'
echo $newvar
Output: Value of var is $var
Using double quotes to show a string of characters will allow any variables in the quotes to be resolved.
Using single quotes to show a string will not allow variable resolution.
7
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Single and double quotes
Quotes marking the beginning and end of a string are not saved in variables
#!/bin/bash
var="test string"
#get the first character, will introduce later
echo ${var:0:1} # echo prints letter t not quote
var="\"test string\"" #escape quotes to include them
echo ${var:0:1} # echo prints double quote
Apply quotes properly when the string in a variable is retrieved and there exits space character(s) in the string.
Without quotes, space characters break one string into multiple strings.
#!/bin/bash
var="my file.txt" #a space character in file name
cat $var #cannot find the file
#cat: my: No such file or directory
#cat: file.txt: No such file or directory
cat "$var" #print file content correctly
Scope of a variable
By default, all variables are global, even if declared inside a function.
Can be accessed from anywhere in the script regardless of the scope.
Inaccessible from outside of the script
Inaccessible in other scripts run by the script defining the variable
$ cat a.sh
#!/bin/bash
a=hello
echo $a
$ ./a.sh
hello
$ echo $a
$
$ cat a.sh
#!/bin/bash
a=hello
./b.sh
$ cat ./b.sh
#!/bin/bash
echo $a
$ ./a.sh
$
nothing is printed out
nothing is printed out
What if we want to make b.sh print out “hello”
Environment variables and export command
The export command makes a variable an environment variable, so it will be accessible from ''children'' scripts.
$ cat a.sh
#!/bin/bash
export a=hello
./b.sh
$ cat ./b.sh
#!/bin/bash
echo $a
$ ./a.sh
hello
$
If a “child” script modifies an environment variable, it will NOT modify the parent’s original value.
$ cat ./a.sh
#!/bin/bash
export a=hello
./b.sh
echo $a
$ cat ./b.sh
#!/bin/bash
a=bye
$ ./a.sh
hello
$
10
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Some common environment variables
Created by the system for saving some system settings
Can be found with the env command.
Accessible in command line interface and any shell scripts.
$ echo $SHELL
/bin/bash
$ echo $PATH
/usr/X11R6/bin:/usr/local/bin:/bin:/usr/bin
$ cat a.sh
#!/bin/bash
echo $HOME
$ ./a.sh
/home/fall2020/tom
11
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Some common environment variables
?: exit status of previous command
LOGNAME, USER: contains the user name
RANDOM: random number generator
SECONDS: seconds from the beginning of the execution
PS1: sequence of characters shown before the prompt
    \t hour                    \d date        \w current directory     
\W last part of the current directory    \u user name        \$ prompt character
Example:
    $ PS1='hi \u *$'
    hi userid*$ _
12
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Read command
The read command allows you to prompt for input and store it in a variable.
#!/bin/bash
echo -n "Enter pathname of file to backup: "
read file_pathname
cp $file_pathname /home/tom/backup/
The script reads a pathname into variable file_pathname, and copies the corresponding file into the backup directory.
13
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Expansions: a few ways to operate texts
Bash may perform a few types of expansions to commands before executing them.
Replace special expressions with texts
variable expansion
brace expansion
tilde expansion
command substitution
arithmetic expansion
filename expansion
Variable expansion
${var} : string saved in var
${#var} gives the string length
${var:position} extracts sub-string from $string at $position
${var:position:length} extracts a sub-string of $length from $position
$ st=0123456789
$ echo ${#st}
    10
$ echo ${st:6}
    6789
$ echo ${st:6:2}
    67
15
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Brace expansion and tilde expansion
Brace expansion expands a sequence expression or a comma separated list of items inside curly braces "{}"
Brace expansion is performed before any other expansions, and any characters special to other expansions are preserved in the result.
"${" for variable expansion is not considered eligible for brace expansion
$ echo a{d,c,b}e
ade ace abe
$ echo a{0..3}b
a0b a1b a2b a3b
$ mkdir home_{tom,berry, jim}
$ ls
home_berry home_jim home_tom
Tilt expansion replaces an unquoted tilde character "~" at the beginning of a word with pathname of home directory
~ : home directory of current user ($HOME)
~/foo : foo subdirectory under the home
~fred/foo : the subdirectory foo of the home directory of the user fred
Command substitution: saving the output of a command into a variable
command substitution using backquotes : `command`
(use backquote "`" , not single quote "'").
$ LIST=`ls`
$ echo $LIST
hello.sh read.sh
$ PS1="`pwd`>"
/home/userid/work> _
$ LIST=$(ls)
$ echo $LIST
hello.sh read.sh
$ rm $( find / -name "*.tmp" )
$ cat > backup.sh
#!/bin/bash
BCKUP=/home/$USER/backup-$(date +%F).tgz
tar -czf $BCKUP $HOME
Command substitution using $ and (): $(command)
17
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Evaluating arithmetic expressions
Translate a string into a numerical expression
Command substitute and expr: ` expr expression` or $(expr expression)
e.g., z=`expr $z + 3`
Read manual of command expr
double parentheses: $((expression))
$ echo "$((123+20))“
143
$ echo "$((123*$VALORE))"
The let statement: let var=expression
$ let X=10+2*7
$ echo $X
24
Arithmetic expansion
18
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Arithmetic operators: +, -, /, *, %
$ cat arithmetic.sh
#!/bin/bash
echo -n "Enter the first number: "; read x
echo -n "Enter the second number: "; read y
add=$(($x + $y)); sub=$(($x - $y))
mul=$(($x * $y)); div=$(($x / $y))
mod=$(($x % $y));
echo "Sum: $add"
echo "Difference: $sub"
echo "Product: $mul"
echo "Quotient: $div"
echo "Remainder: $mod"
19
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
filename expansion
Bash scans each word for the characters ‘*’, ‘?’, and ‘[’. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of filenames matching the pattern.
* Matches any string, including the null string.
? Matches any single character.
[…] Matches any one of the enclosed characters.
$ ls *.pdf
$ ls fig?.pdf
$ ls fig[0-9].pdf
$ ls fig_[abc].pdf
$ mkdir home{1..3}
$ mkdir home{1,2}{a..c}
$ echo home*
$ echo home*
home1 home1a home1b home1c home2 home2a home2b home2c home3
$ echo home[12345]
home1 home2 home3
$ echo home?[bc]
home1b home1c home2b home2c
Spaces and word Splitting
$ echo "Hello World"
"Hello World"
$ a="Hello World"
$ echo ${a}
Hello World
$ echo ${a#}
16
$ echo "${a}"
Hello World
$b=$a
$ echo ${b#}
16
The shell scans the results of variable expansion, command substitution, and arithmetic expansion for word splitting.
Usually happens when the results are used in command lines, not in assignments
double quotes prevent world splitting
A general rule: double-quote every expansion
Conditional statements
if COMMANDS
then
    statements
elif COMMANDS
then
    statements
else
    statements
fi
if COMMANDS; then
    statements
elif COMMANDS; then
    statements
else
    statements
fi
if COMMANDS; then statements; elif COMMANDS; then statements; else statements; fi
elif (else if) and else sections are optional
Conditions are exit code ($?) of COMMAND
22
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Conditional statements
[ is a command usually used in if
[ is another implementation of the traditional test command.
[ or test is a standard POSIX utility.
Implemented in all POSIX shells.
An expression can compare numbers, strings, check files, combine multiple conditions…
Put spaces before and after each expression, and around the operators in each expression.
if [ expression ]; then
    statements
elif [ expression ]; then
    statements
else
    statements
fi
23
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Comparing numbers
-eq    compare if two numbers are equal
-ge compare if one number is greater than or equal to a number
-le     compare if one number is less than or equal to a number
-ne     compare if two numbers are not equal
-gt     compare if one number is greater than another number
-lt     compare if one number is less than another number
Examples:
[ n1 -eq n2 ]     true if n1 same as n2, else false
[ n1 -ge n2 ]     true if n1greater then or equal to n2, else false
[ n1 -le n2 ]     true if n1 less then or equal to n2, else false
[ n1 -ne n2 ]     true if n1 is not same as n2, else false
[ n1 -gt n2 ]     true if n1 greater then n2, else false
[ n1 -lt n2 ]     true if n1 less then n2, else false
24
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Examples
$ cat number.sh
#!/bin/bash
echo -n "Enter a number 1read num
if [ $num -lt 10 ]; then
if [ $num -gt 1 ]; then
echo "$num*$num=$(($num*$num))"
else
echo "Wrong number!"
fi
else
echo "Wrong number!"
fi
25
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Comparing strings
=     compare if two strings are equal
!= compare if two strings are not equal
-n evaluate if string length is greater than zero
-z     evaluate if string length is equal to zero
Examples:
[ s1 = s2 ]     true if s1 same as s2, else false
[ s1 != s2 ]     true if s1 not same as s2, else false
[ s1 ]         true if s1 is not empty, else false
[ -n s1 ]     true if s1 has a length greater then 0, else false
[ -z s2 ]     true if s2 has a length of 0, otherwise false
$ cat user.sh
#!/bin/bash
echo -n "Enter your login name: "
read name
if [ "$name" = "$USER" ];
then
    echo "Hello, $name."
else
    echo "You are not $USER"
fi
26
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Checking files/directories
-e    check if file/path name exists
-d     check if path given is a directory
-f     check if path given is a file
-r     check if read permission is set for file or directory
-s     check if a file has a length greater than 0
-w check if write permission is set for a file or directory
-x     check if execute permission is set for a file or directory
Examples:
[ -d fname ]     (true if fname is a directory, otherwise false)
[ -f fname ]     (true if fname is a file, otherwise false)
[ -e fname ]     (true if fname exists, otherwise false)
[ -s fname ]     (true if fname length is greater then 0, else false)
[ -r fname ]     (true if fname has the read permission, else false)
[ -w fname ]     (true if fname has the write permission, else false)
[ -x fname ]    (true if fname has the execute permission, else false)
#!/bin/bash
read fname
if [ -f $fname ]; then
cp $fname .
echo "Done."
else
if [ -e $fname ]; then
echo "Not a file."
else
echo "Not exist."
fi
exit 1
fi
27
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Exercise
Write a shell script which:
Allows user to type in a file name (e.g., ./myfile.txt)
checks if the file exists
if the file exists, make a copy of the file under the same directory. Append a “.bak” to the file name of the copy (e.g., ./myfile.txt.bak).
If the file does not exist, print out “file does not exist.”
28
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Logically operators: AND (-a, &&), OR (-o, ||), NOT (!)
#!/bin/bash
echo -n "Enter a number 1read num
if [ $num -lt 10 ]; then
if [ $num -gt 1 ]; then
echo "$num*$num=$(($num*$num))"
else
echo "Wrong number!"
fi
else
echo "Wrong number!"
fi
#!/bin/bash
echo -n "Enter a number 1read num
if [ $num -lt 10 -a $num -gt 1 ]; then
echo "$num*$num=$(($num*$num))"
else
echo "Wrong number!"
fi
29
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Pay attention to the forms when combining conditions
#!/bin/bash
echo -n "Enter a number 1read num
if [ $num -gt 1 ] && [ num -lt 10 ]; then
echo "$num*$num=$(($num*$num))"
else
echo "Wrong number!"
fi
if [ condition1 ] && [ condition2 ]
if [ condition1 -a condition2 ]
if [ condition1 ] || [ condition2 ]
if [ condition1 -o condition2 ]
30
6/9/2021
Prof. Ding, Xiaoning. Summer 2021. Protected content.
Prof....