 Introduction to Linux |
| Introduction to Linux |
Most computers come with a few utilities that provide the minimum capabilities required to operate the system. Unix systems, and Linux systems that followed them, typically come with an enormous variety of useful tools and programming languages. The entire list is extensive and too long to list here, however, you can generate a list of all of the executable programs in any given system by using some of the tools provided:
wg:root /root> find / -type f | while read a; do if test -x $a; then echo $a; fi; done
As you can see, the list is rather long. This particular example uses two programs we have seen already, 'find' which searches file systems for files, and the 'bash' command interpreter, also known as a 'shell'.
When you type into the xterm window or into the console window that you use when the computer starts up, you are typing into the program stored in '/bin/bash'. 'bash' is a newer version of '/bin/sh', the original Unix 'shell'. The term shell is used for this command interpreter and for most Unix and Linux command interpreters.
The shell has some very nice built-in features. One of them is built-in line editing of commands. So if you make a typing mistake and don't see it till later in the line, instead of using the [BACKSPACE] button and losing all of the subsequent things you entered, you can use the [LEFT-ARROW] and [RIGHT-ARROW] keys to move the cursor through the line, changing things where you are. When the line looks right, regardless of where the cursor is in the line, you can press [ENTER] to invoke it. Similarly, you can use [UP-ARROW] and [DOWN-ARROW] to move through the stored history of commands entered. As a good example, in order to see the details of the executable programs, use the [UP-ARROW] key once to get the previous command line, use the [RIGHT-ARROW] key to get to the end of the line, and add the characters:
| less
to the end of the line. Then press [ENTER] and you can page through the output of the previous command. The shell is a general purpose programming language specifically designed to allow the use and interconnection of other Unix/Linux programs through a mechanism called 'pipes'. Pipes allow one program's output to become the next program's input. This provides the means by which a series of programs can be used to transform any input data into a desired output for a wide range of automated processing purposes. The example we just gave included two pipes. One from the 'find' command into the rest of the first line, and the second from the earlier example into the 'less' program that allows us to see it one page at a time and move through the output.
Another simple example that is commonly used involves a program called 'grep'. Grep searches its input for regular expressions and returns any content containing (or missing) the specified expression. In this example we will look for lines in files that contain or don't contain several different sequences of characters by piping the output of 'grep' into another 'grep' and another and another. Here is an example of a search of perl scripts for particular criteria:
In this example, grep is used to look for '.pl' in the files in the '/usr/local/bin' directory. Any errors in the results are sent to the device '/dev/null' (which throws these results away) while results are then filtered to look for the word 'display' then to look for the string 'ps -' then to eliminate lines containing the string '-v'. The result is the set of lines shown from the perl script '/usr/local/bin/status.pl', all of which print different variations on the 'ps' command into the output display area. This file is the perl script run when you select System Status from the Administrators X11 menu.
Now suppose we want to something more complex. In this example, we will use a variation on the previous example to get the list of files containing strings, using a similar technique to the one above with the 'while' built-in shell command.
wg:root /root> grep '.pl' /usr/local/bin/* 2>/dev/null | sed 's/:/ /g' | while read a b; do echo $a; done | noreps.pl
This listing goes on for quite a while, but so does the explanation of how it works. In this case, we included a cameo appearance from the stream editor ('sed') which replaced all of the appearances of [COLON] (':') with [SPACE] (' '). This separates the output into things before and after the first apace, and since grep provides the filename along with the line from that file when it looks for contents in more than one file, the result can be parsed to extract the filenames from the content. This is done with the shell's 'read' command.
In this case, the built-in 'while' statement will loop doing the next command until that command fails. The command is the built-in 'read' which reads one line of text from the input and assigns each of the variable names following the 'read' to each of the items in the line where items are separated by 'whitespace' (actually specified in the shell variable IFS but usually [SPACE] or [TAB]). So in this case, since we replaced the ':' at the end of each filename with a [SPACE], the filename will be assigned to the variable 'a' and the rest of the line to the variable 'b'. If we also had a 'c' following the 'b', then 'b' would get the second word, and 'c' the rest of the line, and so forth.
For each line, the variables 'a' and 'b' are assigned, and the statements after the 'do' are then executed, up until the corresponding 'done'. In this case, all that is done is that the built-in command 'echo' is invoked to print the first argument, the filename. After the done, the result of the entire set of printouts is piped into a program called 'noreps.pl', the only part of this example that is only normally available in White Glove Linux. The noreps.pl program is a program written in the 'perl' programming language that only prints one copy of each distinct line of its input. The details can be examined by looking at the program, which is included on the White Glove CD.
The program examples above may read the user to use the 'man' command to look into the details of the programs described above. The built-in programs will be covered under the 'bash' command, while the other programs have their own manual entries (except for noreps.pl). The 'man' command, as described earlier, is vital to building up programs like the ones you have just seen in order to do all sorts of common tasks. As you practice, you get much better, and your job as a systems administrator becomes far easier, especially for tasks that are done again and again. Eventually, you build your own scripts like these and put them in files so you can run them when you want to.
An example of a simple script of this sort is the command 'l' used sometimes to generate a directory listing. Here it is in all of its glory.
ls --color=yes -C $*
The 'l' program is actually just a very short shell script that runs the 'ls' command adding some arguments to the command line so that you get color output and files are classified according to type. The '$*' indicates that whatever else was put on the command line to the 'l' command should be added to the command line for the 'ls' command it runs. All you need to do to create your own commands is to put your version in a file with the name you want to use when you run it, make the file executable by doing:
wg:root /root> chmod 755 [name]
where [name] is replaced by the filename you use, and you can then execute it as a program.
Linux comes with a lot of other programs, and White Glove comes with a particular set of the most common and useful ones. Each other version of Linux comes with a slightly different set of these programs. Here are some of the ones that are included in most Linux distributions with simple example to try:
perl: A programming language that is most often used by systems administrators.
gcc: The GNU C compiler - translates C programs into executable programs.
vi: This Visual Editor is rather primitive but exists in every Unix and Linux system since the first ones that were ever built.
wg:root /root> vi
Use 'i' to start inserting, the arrow keys to move around, [ESC] to stop inserting, ':q!' (when not in insert mode) to exit without saving changes, ':w' to write (when not in insert mode).
awk: A language for parsing and processing file contents.
wg:root /root> (ls;ls;ls) | sort | awk 'z != $1 { print ; z = $1 } '
This example uses awk to eliminate duplicates in a sorted list. In this case we make a list with 3 sets of each item by appending the output of 'ls' run 3 times. The parens around the 3 instances of 'ls' run the 3 programs within a single subprocess.
bc: A simple unlimited precision calculator.
wg:root /root> bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
2343398734983498349834989834 * 123987239487239487239487239487239487324 ^ 23
[CTRL-D] to exit. Sample problems: How many seconds are there in a year? How many seconds has it been since you were born? How many days has it been since the start of the common era? How many seconds?
cal: Show a calendar.
wg:root /root> cal
October 2002
Su Mo Tu We Th Fr Sa
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31
wg:root /root> cal 2525
Sample problems: What day of the week were you born on? What day of the week will New Years be in the year 2108?
csh: A different command interpreter.
wg:root /root> csh
[CTRL-D] to exit.
diff: Differences between files and directories.
wg:root /root> cp -a /bin /tmp/bin
wg:root /root> cp /etc/passwd /tmp/bin
wg:root /root> diff -r /tmp/bin /bin
split: Split big files into smaller files.
wg:root /root> mkdir /tmp/s;cd /tmp/s; split /etc/passwd -b 10
wg:root /root> ll
dd: Disk dump - makes bit for bit images of disks and other things.
wg:root /root> dd if=/dev/cdrom of=/tmp/file bs=2048 count=3
wg:root /root> less /tmp/file
ed: A line oriented editor.
lex: A lexical analyzer.
join: Join lines of two files on a common field.
m4: Macro pre-processor.
me: Micro emacs editor
wg:root /root> me /etc/passwd
Use the arrow keys to move around, backspace to delete before the cursor, and [CTRL-C] to exit. DO NOT SAVE CHANGES!!!
pico: A very common text editor.
wg:root /root> pico
sash: Stand-alone shell with built-in commands
wg:root /root> sash
[CTRL-D] to exit.
sort: Sort inputs into outputs.
wg:root /root> ll | sort
wg:root /root> ll | sort +1
wg:root /root> ll | sort -n +4
wg:root /root> ll | sort -n +8
wg:root /root> ll | sort +8
wg:root /root>
tar: Tape archiver
Each of these programs is very useful and built into almost every Linux and Unix system. Depending on how much time you have, please try a wide range of these programs to see what they do and how they work.
We have covered basics of the 'bash' command interpreter and demonstrated examples using pipes between programs to create complex filters and other small programs. We have also demonstrated several of the many programs that come with Linux and pointed the user to many others to explore.
It is time for the student to try out many of these commands, using the 'man' command to learn how they work, and trying to fit them together into scripts. For in-class students, the instructor will be happy to answer any questions and work with you to try to create custom programs to try out and demonstrate in class.