AOS Linux Tutorial Introduction to Linux, Part 2 Michael Havas - - PowerPoint PPT Presentation

AOS Linux Tutorial

Introduction to Linux, Part 2 Michael Havas

• Dept. of Atmospheric and Oceanic Sciences

McGill University September 21, 2010

Outline

1 Review 2 The Linux Command-Line

Working With Text Redirection and Piping Chaining Commands Jobs and Job Control Running Commands Environment Variables Printing Getting Computer Information When Things Go Bad

3 Shell Scripting with bash 4 Fun Tips 5 Next Time

Review

1 Introduction to Linux. 1

Benefits of Linux.

2

What exactly is Linux?

3

The Free-Software philosophy.

2 The graphical user interface. 1

Cross-platform applications.

2

Windows applications and their Linux counterparts.

3 The command-line. 1

The filesystem.

2

File and directory management.

3

Globbing.

4

Finding files.

5

Ownership and Permissions.

Outline

1 Review 2 The Linux Command-Line

Working With Text Redirection and Piping Chaining Commands Jobs and Job Control Running Commands Environment Variables Printing Getting Computer Information When Things Go Bad

3 Shell Scripting with bash 4 Fun Tips 5 Next Time

Working With Text

Why Text

Working with text Almost everything you use on a computer is really just text:

Files. This presentation. A webpage. Email. Source code.

Text is a form of communication between programs. More on this later.

Working With Text

Reading Text

Commands for Reading Text Files cat file Reads a file and prints in to standard out. tac file Reads a file and prints in to standard out in reverse

• rder.

more file Reads a file with pagination. less file Reads a file with better pagination. Can go back and forth and search. head file Reads the first few lines from file. tail file Reads the last few lines from file.

Working With Text

Manipulating Text

Commands For Getting Information from Text Files wc file Counts the number of lines, words and characters in a text file. grep needle file Search for the search term needle in the file. Changing Text sort file Sorts a file in lexicographical order line by line. uniq file Prints all but repeated lines. Must be sorted. sed ’s/from/to/’ file Substitutes from with to in file. Can do much more. awk ’{print $n}’ file Prints only column n in file. Can do much more.

Redirection and Piping

Redirecting Standard Output

Standard output is normally set to your screen. Can be redirected using > operator. Can be redirected and appended using >> operator. Example

[ mhavas@lappy tmp ] $ l s > contents . t x t [ mhavas@lappy tmp ] $ cat contents . t x t a .1 b .1 c .1 contents . t x t t2 . sh t e s t 1 . t x t

Redirection and Piping

Redirection Standard Input

Standard input is normally your keyboard. Can be redirected using < operator. Example

[ mhavas@lappy tmp ] $ tac < contents . t x t t e s t 1 . t x t t2 . sh contents . t x t c .1 b .1 a .1

Redirection and Piping

Redirecting Standard Input and Output [ mhavas@lappy tmp ] $ tac < contents . t x t >> contents . t x t [ mhavas@lappy tmp ] $ cat contents . t x t a .1 b .1 c .1 contents . t x t t2 . sh t e s t 1 . t x t t e s t 1 . t x t t2 . sh contents . t x t c .1 b .1 a .1

Redirection and Piping

Piping

Make the output of one program the input of the next using the | operator. Example

[ mhavas@lappy tmp ] $ cat contents . t x t | s o r t | uniq b .1 c .1 t2 . sh t e s t 1 . t x t contents . t x t a .1

Chaining Commands

Chaining Commands cmd1 ; cmd2 Run cmd1 then cmd2 cmd1 && cmd2 Run cmd2 after cmd1 only if cmd1 completes successfully. cmd1 || cmd2 Run cmd2 after cmd1 only if cmd1 does not completes successfully.

Jobs and Job Control

Linux is a multi-user, multi-process operating system. We should be able to put processes in the background while doing other things. Job Control Commands cmd & Runs command cmd in the background. jobs Get a list of jobs currently running or paused. fg jobid Brings command with jobid=jobid to the foreground. Control-z Pauses the process running in the foreground. bg Puts the paused job into the background. Jobs in the background stop running when you log out

Running Commands

Two cases:

1 Command is in your path.

Just type the command.

2 Command is not in your path.

Command requires full path. What’s a path The path is an environment variable used to tell the system where to find your executable programs.

Environment Variables

Seeing enviornment variables env Displays all environment variables. echo ${envar} Displays environment variable envar. Setting enviornment variables envar=val cmd Sets the environment variable envar to value val just for the command cmd. export envar=val Sets the environment variable envar to value val for your session. Be careful!

Printing

Commands for printing lpstat -a List all printers. lpq -P printer List queue of printer printer. lpr -P printer file Print file to printer. lpr can only print ps, pdf and txt files.

Getting Computer Information

Commands for getting computer information cat /proc/cpuinfo CPU information. cat /proc/meminfo Memory information. free Memory information. top Process information. ps Process information. df Disk space information. quota Look at your disk quota information.

When Things Go Bad

1 Get the process id: ps aux | grep NAME 2 Kill the process: kill PID. 3 Didn’t work, kill -9 PID.

Outline

1 Review 2 The Linux Command-Line

Working With Text Redirection and Piping Chaining Commands Jobs and Job Control Running Commands Environment Variables Printing Getting Computer Information When Things Go Bad

3 Shell Scripting with bash 4 Fun Tips 5 Next Time

Shell Scripting with bash

What is shell scripting and why would I use it?

What is shell scripting? A scripting language written in the language of your shell. A comfortable environment writing scripts. Why should I use it Batch jobs. Shortcuts.

Shell Scripting with bash

A shortcut example

Goal Often, I want to use ls with various options to view file format indicators, columns, all files (none omitted), and a size in blocks. The command that reflects this is ls -FCAs. I don’t want to type this every time! Code

#!/ bin / bash l s −FCas ”$@”

Instructions Save code in some directory in your path as a file called l chmod a+x l

Shell Scripting with bash

An automation example

Goal Convert a bunch of jpg files to png Code

#!/ bin / bash for jpg i n ”$@” ; do png=”${ jpg %. jpg }. png” echo c o n v e r t i n g ” $jpg ” . . . i f convert ” $jpg ” tmp . jpg . to . png ; then mv jpg . to . png ”$png” e l s e echo ’ e r r o r : f a i l e d

• utput

saved i n ”tmp . jpg . to . png” . ’ 1>&2 e x i t 1 f i done echo a l l c o n v e r s i o n s s u c c e s s f u l

Shell scripting with bash

A project

Goal We have trajectory 3-D trajectory data for a satellite in space. We would like to: Plot this trajectory. Generate a movie showing the satellite’s path in time. Tools BASH shell scripting. Gnuplot.

Shell scripting with bash

A project: Plot the trajectory

Use gnuplot

set terminal p o s t s c r i p t eps enhanced c o l o r s i z e 7 in ,5 i n set

• utput

’ run3 001 12 . eps ’ set nokey set x l a b e l ’ x a x i s ’ set y l a b e l ’ y a x i s ’ set z l a b e l ’ z a x i s ’ set l a b e l ’E ’ at −0.1 , 0 , 0 # Label f o r Earth set l a b e l ’M’ at 0.9 , 0 , 0 # Label f o r Moon set l a b e l ’S ’ at 6.025750E−01, −1.752196E−01, 2.370230E−01 # Label f o r S t a r t splot ’ run3 001 12 . dat ’ using 1 : 2 : 3 with l i n e s

Shell scripting with bash

A project: Plot the trajectory (continued)

• 1
• 0.8
• 0.6
• 0.4
• 0.2

0.2 0.4 0.6 0.8 1

• 0.4
• 0.3
• 0.2
• 0.1

0.1 0.2 0.3 0.4

• 0.4
• 0.3
• 0.2
• 0.1

0.1 0.2 0.3 z axis E M S x axis y axis z axis

Shell scripting with bash

A project: Generate a movie

Idea A movie is just a collection collection of images displayed over time. Generate an image of a plot for datapoint. Aggregate these images into a movie. Algorithm Require: N The number of datapoints in the dataset. for i = 1 to N do Plot datapoints 1 to i Save to file “plot-i”.eps end for Make movie using N eps images in order

Shell scripting with bash

A project: Generate a movie

The code

#!/ bin / bash DATFILE=’ run3 001 12 . dat ’ BASE=${DATFILE%.dat } NUMLINES=$ (wc −l ${DATFILE} | cut −d ’ ’ −f 1) for i i n $ ( seq −f %06.0 f ${NUMLINES} ) ; do gnuplot << EOF set t e r m i n a l png t r u e c o l o r nocrop enhanced s i z e 700 ,700 set

• utput

’ movies /${BASE} $ { i }. png ’ set nokey set xrange [ − 1 . 0 : 1 . 0 ] set yrange [ − 0 . 4 : 0 . 4 ] set zrange [ − 0 . 4 : 0 . 3 ] set l a b e l ’E ’ at −0.1 , 0 , 0 # Label f o r Earth set l a b e l ’M’ at 0.9 , 0 , 0 # Label f o r Moon set l a b e l ’S ’ at 6.025750E−01, −1.752196E−01, 2.370230E−01 s p l o t ’ ${DATFILE} ’ every : : 1 : : ${ i } using 1 : 2 : 3 with l i n e s EOF done

Shell scripting with bash

A project: Generate a movie

Converting images to movies

ffmpeg −i ’ run3 001 12 %6d . png ’ movie . mp4

Outline

1 Review 2 The Linux Command-Line

Working With Text Redirection and Piping Chaining Commands Jobs and Job Control Running Commands Environment Variables Printing Getting Computer Information When Things Go Bad

3 Shell Scripting with bash 4 Fun Tips 5 Next Time

Fun Tips

You can write small scripts in the terminal

Inline Shell Scripts

[ mhavas@lappy t u t o r i a l s ] $ l s

• 1. png
• 2. png
• 3. png
• 4. png

[ mhavas@lappy t u t o r i a l s ] $ for png i n ∗. png ; do > convert ${png} ${png%.png }. jpg > done [ mhavas@lappy t u t o r i a l s ] $ l s

• 1. jpg
• 2. jpg
• 3. jpg
• 4. jpg

Outline

1 Review 2 The Linux Command-Line

Working With Text Redirection and Piping Chaining Commands Jobs and Job Control Running Commands Environment Variables Printing Getting Computer Information When Things Go Bad

3 Shell Scripting with bash 4 Fun Tips 5 Next Time

Next Time

Remote access using ssh from Linux, OS X and Windows. Transferring files using scp and rsync. Running long jobs with tmux. Suggested topics.