Tools and Advanced Commands Emanuele Valea <valea@lirmm.fr> - PowerPoint PPT Presentation

Tools and Advanced Commands Emanuele Valea <valea@lirmm.fr> LIRMM CNRS / Université de Montpellier

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • sort • cut • sed

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • Sort • cut • sed

Find • find <dir> [-opt] • Some options: • -name pattern Caveat: put the pattern between ‘...’ when using regular expressions

Find • It’s possible to execute a command that operates on all the found files: -exec command \; • in command you can use \{} to select the current file • Example: find . -name core -exec rm \{} \;

Find: advanced use • The search expression can be composed by more than one condition in order to build a complex logic condition • Logic Operators: • AND: specify the conditions one after the other • OR: use -o • NOT: use !

Find: advanced use • Use parenthesis to build complex expression • Example: find . \( -name core -o -size +2000 \)

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • Sort • cut • sed

grep • In order to found string occurrences in a list of files, the following command can be used: • grep [-options] pattern files

grep • Options: • -c: prints only a count of the lines that contain the pattern • -i: ignores upper/lower case distinction during comparisons. • -l: prints only the names of files with matching lines, separated by NEWLINE characters • -n: precedes each line by its line number in the file (first line is 1) • -v: prints all lines except those that contain the pattern

Regular Expression in grep • The search pattern can be normal string or regular expression • Some characters correspond to special ones (unless those preceded by \) • . Any character • ^ begin of line • $ end of line

Regular Expression in grep (cont) • * repetition (0 or plus) • + repetition (1 or plus) • [ ] Any character inside ‘ [ ] ’ • [^ ] Any character except those inside the ‘ [ ] ’ • \< begin of the word • \> end of the word • ^ beginning of the line • $ end of the line

Recursive search in the directories • find . - name ‘*’ - exec grep ‘pattern’ \{} \; -print • rgrep

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • Sort • cut • sed

tar • tar [option] files • create a file archive (tarfile) and add or extract files

tar - Creation • Options to create a tarfile: • -c create a new tarfile • -f file specify the tarfile name • -v verbose • Example: • tar -cvf /tmp/valea.tar /home/valea

tar - Extract • Options to extract from a tarfile: • -x Extract files from the tarfile • -t Check the tarfile • -f file Specify the tarfile name • -v Verbose • Example: • tar -tvf /tmp/valea.tar • tar -xvf /tmp/valea.tar

gzip • gzip [opt] file • Compress a file • Options: • -1 fastes • -9 max compression

gunzip • gunzip file • Decompress a file

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters : • sort • cut • sed

Filter • A filter is a program that receives input data from stdin and generate output data on stdout • Filters are very useful with I/O redirection and with pipe • Examples: • more, less • head, tail

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • sort • cut • sed

Data Sort • sort [-options] [-k begin[,end]] [file ...] • Options: • -b Ignores leading blank characters • -n Considers the string as number • -d Considers only the characters • -f Ignores upper/lower case distinction during comparisons

Data Sort • sort [-options] [-k begin[,end]] [file ...] • -o Specifies the name of an output file to be used instead of the standard output • -r Reverses the sense of comparisons • -t ‘ car ’ field separator • -k “begin ” and “ end ” specify the field to be considered during sorting, in the format: • n-field[.n-character]

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • sort • cut • sed

cut • It cuts out selected portions of each line of a file • cut -f list [-d delim] [file ...] • It shows the list specifies fields, separated in the input by the field delimiter character (-d option) • Example: Extract users' login names and shells from the system passwd file: • cut -d : -f 1,7 /etc/passwd

Outline • Advanced commands: • find • grep • tar, gzip, gunzip • Filters: • sort • cut • sed

sed - Stream text EDitor • The sed utility reads the specified files, or the standard input if no files are specified, modifying the input as specified by a list of commands. The input is then written to the standard output • sed [- n] script [file …] • sed [-n][-e script][- f script_file][file …] • -n does not write to stdout

sed - Stream text EDitor • Script syntax: • [address[,address]] function [args] • address: line number or regular expression • function: command to be executed to the address • args: function arguments

sed functions • p print current line • d delete current line • q end execution • y/<orig>/<subs>/ replace characters from orig to subs

sed functions • s/<regexp>/<replace>/<flags> replace patterns that match with regexp with replace • flags: • num replace only num occurrences • g replace all the occurrences • p print the replaced

sed: examples • sed ’1,3 d’ <filename> • sed ’3,$ d’ <filename> • sed - n ’/^example/ p’ <filename> • sed -f sedfile filename

sed: examples 1,1 { s/^/Begin:/ s/$/ -- End/ } • sed ’1,3 d’ filename /\/\*.*\*\// d • sed ’3,$ d’ filename • sed - n ’/^example/ p’ filename • sed -f sedfile filename

sed: examples 1,1 { s/^/Begin:/ s/$/ -- End/ } • sed ’1,3 d’ filename /\/\*.*\*\// d • sed ’3,$ d’ filename • sed - n ’/^example/ p’ filename • sed -f sedfile filename One Two Three A B C /* Comment */

sed: examples 1,1 { s/^/Begin:/ s/$/ -- End/ } • sed ’1,3 d’ filename /\/\*.*\*\// d • sed ’3,$ d’ filename • sed - n ’/^example/ p’ filename • sed -f sedfile filename Begin: One Two Three -- End A B C One Two Three A B C /* Comment */

Sed: examples • Print all the lines of a file unless comments (comments begin with '#') cat $FILE | sed -e '/^#/d ‘ Print all the lines that begin with a number • cat $FILE | sed -n "/^[0-9]/ p “ • Print all the lines that DO NOT begin with a number cat $FILE | sed "/^[0-9]/ d “ • Remove all the empty lines cat $FILE | sed -e "/^$/d"

Sed: examples • Remove all the lines composed by spaces cat $FILE | sed -e "/^[ ][ ]*$/d “ • Remove the filename path • echo "/usr/bin/example" | sed -e "s/.*\/// “ • Print only the path echo "/usr/bin/example" | sed -e "s/\/[^\/]*$//"

Sed: examples • Print the 'n ’ th line of a file (n = 5) sed -n -e ” 5,5p" $FILE • Print lines file (from 2 to 5) sed -n -e "2,5p" $FILE • Replace examples: sed "s/\"/ /g".. Replace all the '"' with space sed "s/,/ /g".. Replace all the ',' with space sed ’ s/bin//'.. Replace the first 'bin' with nothing (delete)