Tools and Advanced Commands Emanuele Valea <valea@lirmm.fr> - - PowerPoint PPT Presentation

Tools and Advanced Commands

Emanuele Valea <valea@lirmm.fr> LIRMM CNRS / Université de Montpellier

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• sort
• cut
• sed

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• Sort
• cut
• sed

Find

• find <dir> [-opt]
• Some options:
• -name pattern

Caveat: put the pattern between ‘...’ when using regular expressions

Find

• It’s possible to execute a command that operates on

all the found files:

• exec command \;
• in command you can use \{} to select the current

file

• Example:

find . -name core -exec rm \{} \;

Find: advanced use

• The search expression can be composed by more

than one condition in order to build a complex logic condition

• Logic Operators:
• AND: specify the conditions one after the other
• OR: use -o
• NOT: use !

Find: advanced use

• Use parenthesis to build complex expression
• Example:

find . \( -name core -o -size +2000 \)

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• Sort
• cut
• sed

grep

• In order to found string occurrences in a list of files,

the following command can be used:

• grep [-options] pattern files

grep

• Options:
• -c: prints only a count of the lines that contain the

pattern

• -i: ignores upper/lower case distinction during

comparisons.

• -l: prints only the names of files with matching lines,

separated by NEWLINE characters

• -n: precedes each line by its line number in the file

(first line is 1)

• -v: prints all lines except those that contain the pattern

Regular Expression in grep

• The search pattern can be normal string or regular

expression

• Some characters correspond to special ones (unless

those preceded by \)

• .

Any character

• ^

begin of line

• $

end of line

Regular Expression in grep (cont)

• *

repetition (0 or plus)

• +

repetition (1 or plus)

• [ ]

Any character inside ‘[ ]’

• [^ ]

Any character except those inside the ‘[ ]’

• \<

begin of the word

• \>

end of the word

• ^

beginning of the line

• $

end of the line

Recursive search in the directories

• find . -name ‘*’ -exec grep ‘pattern’ \{} \; -print
• rgrep

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• Sort
• cut
• sed

tar

• tar [option] files
• create a file archive (tarfile) and add or extract files

tar - Creation

• Options to create a tarfile:
• -c

create a new tarfile

• -f file

specify the tarfile name

• -v

verbose

• Example:
• tar -cvf /tmp/valea.tar /home/valea

tar - Extract

• Options to extract from a tarfile:
• -x

Extract files from the tarfile

• -t

Check the tarfile

• -f file

Specify the tarfile name

• -v

Verbose

• Example:
• tar -tvf /tmp/valea.tar
• tar -xvf /tmp/valea.tar

gzip

• gzip [opt] file
• Compress a file
• Options:
• -1 fastes
• -9 max compression

gunzip

• gunzip file
• Decompress a file

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• sort
• cut
• sed

Filter

• A filter is a program that receives input data from

stdin and generate output data on stdout

• Filters are very useful with I/O redirection and with

pipe

• Examples:
• more, less
• head, tail

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• sort
• cut
• sed

Data Sort

• sort [-options] [-k begin[,end]] [file ...]
• Options:
• -b Ignores leading blank characters
• -n Considers the string as number
• -d Considers only the characters
• -f Ignores upper/lower case distinction during

comparisons

Data Sort

• sort [-options] [-k begin[,end]] [file ...]
• -o Specifies the name of an output file to be used

instead of the standard output

• -r Reverses the sense of comparisons
• -t‘car’

field separator

• -k “begin” and “end” specify the field to be considered

during sorting, in the format:

• n-field[.n-character]

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• sort
• cut
• sed

cut

• It cuts out selected portions of each line of a file
• cut -f list [-d delim] [file ...]
• It shows the list specifies fields, separated in the input by

the field delimiter character (-d option)

• Example: Extract users' login names and shells from

the system passwd file:

• cut -d : -f 1,7 /etc/passwd

Outline

• Advanced commands:
• find
• grep
• tar, gzip, gunzip
• Filters:
• sort
• cut
• sed

sed - Stream text EDitor

• The sed utility reads the specified files, or the

standard input if no files are specified, modifying the input as specified by a list of commands. The input is then written to the standard output

• sed [-n] script [file …]
• sed [-n][-e script][-f script_file][file …]
• -n does not write to stdout

sed - Stream text EDitor

• Script syntax:
• [address[,address]] function [args]
• address: line number or regular expression
• function: command to be executed to the address
• args: function arguments

sed functions

• p

print current line

• d

delete current line

• q

end execution

• y/<orig>/<subs>/ replace characters from orig to subs

sed functions

• s/<regexp>/<replace>/<flags>

replace patterns that match with regexp with replace

• flags:
• num

replace only num occurrences

• g

replace all the occurrences

• p

print the replaced

sed: examples

• sed ’1,3 d’ <filename>
• sed ’3,$ d’ <filename>
• sed -n ’/^example/ p’ <filename>
• sed -f sedfile filename

sed: examples

• sed ’1,3 d’ filename
• sed ’3,$ d’ filename
• sed -n ’/^example/ p’ filename
• sed -f sedfile filename

1,1 { s/^/Begin:/ s/$/ -- End/ } /\/\*.*\*\// d

sed: examples

• sed ’1,3 d’ filename
• sed ’3,$ d’ filename
• sed -n ’/^example/ p’ filename
• sed -f sedfile filename

1,1 { s/^/Begin:/ s/$/ -- End/ } /\/\*.*\*\// d One Two Three A B C /* Comment */

sed: examples

• sed ’1,3 d’ filename
• sed ’3,$ d’ filename
• sed -n ’/^example/ p’ filename
• sed -f sedfile filename

1,1 { s/^/Begin:/ s/$/ -- End/ } /\/\*.*\*\// d One Two Three A B C /* Comment */ Begin: One Two Three -- End A B C

Sed: examples

• Print all the lines of a file unless comments (comments

begin with '#') cat $FILE | sed -e '/^#/d‘

• Print all the lines that begin with a number

cat $FILE | sed -n "/^[0-9]/ p“

• Print all the lines that DO NOT begin with a number

cat $FILE | sed "/^[0-9]/ d“

• Remove all the empty lines

cat $FILE | sed -e "/^$/d"

Sed: examples

• Remove all the lines composed by spaces

cat $FILE | sed -e "/^[ ][ ]*$/d“

• Remove the filename path
• echo "/usr/bin/example" | sed -e "s/.*\///“
• Print only the path

echo "/usr/bin/example" | sed -e "s/\/[^\/]*$//"

Sed: examples

• Print the 'n’th line of a file (n = 5)

sed -n -e ”5,5p" $FILE

• Print lines file (from 2 to 5)

sed -n -e "2,5p" $FILE

• Replace examples:

sed "s/\"/ /g".. Replace all the '"' with space sed "s/,/ /g".. Replace all the ',' with space sed ’s/bin//'.. Replace the first 'bin' with nothing (delete)