Program Development Tools lex makefiles vi and gvim ctags source - - PowerPoint PPT Presentation

Program Development Tools

• lex
• makefiles
• vi and gvim
• ctags
• source level debugging
• diff and cmp
• svn
• gprof

Lexical Analyzers

• A lexical analyzer reads in a stream of characters

as input and produces a sequence of symbols called tokens as output.

• Useful for a variety of tasks.
• Tools exist to automatically create a lexical

analyzer from a specification.

Lexical Analysis Terms

• A token is a group of characters having a collective

meaning (e.g. id).

• A lexeme is an actual character sequence forming a

specific instance of a token (e.g. num).

• A pattern is the rule describing how a particular token

can be formed (e.g. [A-Za-z_][A-Za-z_0-9]*).

• Characters between tokens are called whitespace

(e.g.blanks, tabs, newlines, comments).

Attributes for Tokens

• Tokens can have attributes that can be passed

back to the function calling the lexical analyzer.

– Constants

• value of the constant

– Identifiers

• pointer to a location where information is kept about the

identifier

General Approaches to Implementing Lexical Analyzers

• Use a lexical-analyzer generator, such as Lex.
• Write the lexical analyzer in a conventional

programming language.

• Write the lexical analyzer in assembly language.

Lex - A Lexical Analyzer Generator

• Can link with a lex library to get a main routine.
• Can use as a function called yylex().
• Easy to interface with yacc.

Lex Specifications

Lex Source

{ definitions } %% { rules } %% { user subroutines }

Definitions

declarations of variables, constants, and regular definitions

Rules

regular expression action

Regular Expressions

Operators ''\ [ ] ^ -? . * + | ( ) $ / { }

Actions

C code fragments

Lex Regular Expression Operators

• “s”

string s literally

• \c

character c literally (used when c would normally be used as a lex operator)

• [s]

for defining s as a character class

• ^

to indicate the beginning of a line

• [^s] means to match characters not in the s

character class

• [a-b] used for defining a range of characters

(a to b) in a character class

• r?

means that r is optional

Lex Regular Expression Operators (cont.)

• .

means any character but a newline

• r*

means zero or more occurances of r

• r+

means one or more occurances of r

• r1|r2

r1 or r2

• (r)

r (used for grouping)

• $

means the end of the line

• r1/r2

means r1 when followed by r2

• r{m,n} means m to n occurences of r

Example Regular Expressions in Lex

a* zero or more a's a+

• ne or more a's

[abc] a, b, or c [a-z] lower case letter [a-zA-Z] any letter [^a-zA-Z] any character that is not a letter a.b a followed by any non-newline char followed by b ab|cd ab or cd a(b|c)d abd or acd ^B B at the beginning of line E$ E at the end of line

Lex Specifications (cont.)

Actions

Actions are C source fragments. If it is compound or takes more than one line, then it should be enclosed in braces.

Example Rules

[a-z]+ printf(''found word\n''); [A-Z][a-z]* { printf(''found capitalized word\n''); printf{'' %s\n'', yytext); }

Definitions

name translation

Example Definition

digits [0-9]

Example Lex Program

digits [0-9] ltr [a-zA-Z] alpha [a-zA-Z0-9] %% [-+]{digits}+ | {digits}+ printf(''number: %s\n'', yytext); {ltr}(_|{alpha})* printf(''identifier: %s\n'', yytext); "'"."'" printf(''character: %s\n'', yytext); . printf(''?: %s\n'', yytext); Prefers longest match and earlier of equals.

Another Example Lex Program

%% BEGIN { return (1); } END { return (2); } IF { return (3); } THEN { return (4); } ELSE { return (5); } letter(letter|digit)* { return (6); } digit+ { return (7); } < { return (8); } <= { return (9); } = { return (10); } <> { return (11); } > { return (12); } >= { return (13); }

Make and Makefiles

• The make utility is used to:

– Automate the execution of commands for file

generation.

– Minimize the number of commands needed for

rebuilding a target.

• A makefile describes

– a hierarchy of dependencies between individual files – commands to generate each file

Make and Makefiles (cont.)

• There can be one or more target entries in a
• makefile. Each target entry in a makefile has the

following format:

target : dependency_file ... command ...

• The target is a file. There can be one or more

dependency files on which the target depends. There can be one or more commands each preceded by a tab that comprise a rule for the

• target. These commands are used to create or

regenerate the target file.

Make and Makefiles (cont.)

• A target is remade when the target file either does

not exist or has an older date/time than one or more of the dependency files.

• The targets and dependency files comprise a

DAG structure representing the dependencies between the different components.

• The make utility recursively checks each target

against its dependencies, starting with the first target entry in the makefile.

Example Makefile

expr.exe : expr.o lex.yy.o gcc -g -o expr.exe expr.o lex.yy.o expr.o : expr.c defs.h gcc -g -c expr.c lex.yy.o : lex.yy.c gcc -g -c lex.yy.o lex.yy.c : scan.l lex scan.l

Invoking Make

• General form.

make [ -f makefilename ] [ target ]

• If no makefilename is given, then make looks for

a file called makefile or Makefile in that order. Make uses one of these files if found in the current directory.

• By default make attempts to create the first target

in the file. Alternatively, a user can specify a specific target within the makefile to make.

Example Invocations of Make

# Make the first target in makefile or Makefile. make # Make the lex.yy.o target in makefile or Makefile. make lex.yy.o # Make the first target in the makeexpr makefile. make -f makeexpr # Make the expr.o target in the makeexpr makefile. make -f makeexpr expr.o

Defining Symbols in a Makefile

• You can define a symbol in a makefile and

reference the symbol in multiple places.

• General form of the definition.

symbol = definition

• General form of the reference.

$(symbol)

Example Makefile with Symbols

CC = gcc CFLAGS = -g -c expr.exe : expr.o lex.yy.o $(CC) -g -o expr.exe expr.o lex.yy.o expr.o : expr.c defs.h $(CC) $(CFLAGS) expr.c lex.yy.o : lex.yy.c $(CC) $(CFLAGS) lex.yy.o lex.yy.c : scan.l lex scan.l

The Vi Editor (17.1)

• Vi stands for the VIsual editor.
• Why use the vi editor?

– The vi editor is standard on every Unix system. – The vi editor allows you to use ex line commands. – The vi editor has many special features that are very

useful.

– The vi editor is efficient compared to emacs.

Invoking Vi

• The vi editor is invoked by issuing the command

in the following form. The -r option is for recovering a file after the system crashed during a previous editing session. The -t option is to indicate the initial cursor position within a file where the editing should start. The use of tags will be discussed more later.

vi [-t tag] [-r ] filename

Vi Modes

• The vi editor has three main modes:

– character input mode: where text can be entered

• insert, append, replace, add lines

– window mode: where regular commands can be issued

• basic cursor motions
• screen control
• word commands
• deletions
• control commands
• miscellaneous commands

– line mode: where ex commands can be issued

Vi Character Input Mode

• After invoking vi, the user is in the window

command mode. There are a few different commands to enter character input mode. At that point a user types in the desired text. A user selects the ESC key to return back to command mode.

Vi Commands to Go into Character Input Mode

• The following commands are used to go into

character input mode. All but the r command require the user to hit the ESC key to go back into window command mode.

a append text after the cursor position A append text at the end of line i insert text before the cursor position I insert text before the first nonblank character in the line

• add text after the current line

O add text before the current line rchr replace the current character with chr R replace text starting at the cursor position

Vi Basic Cursor Motions

• The basic cursor motions allow you to move

around in the file. The letter options allow movement when the arrow keys are not defined.

h  go back one character j  go down one line k  go up one line l  go forward one character (space also works) + CR go down one line to first nonblank character  go up one line to first nonblank character go to the beginning of the line $ go to the end of the line H go to the top line on the screen L go to the last line on the screen

Vi Word Movements

• The following commands can be used to position the

cursor.

w position the cursor at the beginning of the next word b position the cursor at the beginning of the previous word e position the cursor at the end of the current word

Vi Screen Control

• The following vi commands can be used to

control the screen being displayed.

^U scroll up one half page ^D scroll down one half page ^B scroll up one page ^F scroll down one page ^L redisplay the page

Deletions in Vi

• The following vi commands can be used to delete

text.

dd delete the current line D delete text from the cursor to the end of the line x delete character at the cursor X delete character preceding the cursor dw delete characters from the cursor to the end of the word

Searching in Vi

• The following vi commands can be used to search

for text patterns, where each pattern is a regular expression.

/pattern search forward for the specified pattern / search forward for the last specified pattern ?pattern search backward for the specified pattern ? search backward for the last specified pattern n perform the last / or ? command

Miscellaneous Vi Commands

• Below are some miscellaneous vi commands.

Commands that delete text also place the deleted text into a buffer.

u undo previous command U restore entire line Y save current line into buffer p put saved buffer after cursor position P put saved buffer before cursor position J join current line with following line % position cursor over matching (, ), {, or } ZZ save file and exit (same as :wq)

Repeating Vi Commands

• You can indicate how many times a particular

command is to be performed.

• Examples:

3dd delete 3 lines 4w advance 4 words 7x delete 7 characters 5n perform last search 5 times

Vi Line Command Mode

• You can enter line command mode by typing a
• colon. At that point you can enter ex commands.
• Examples:

:150 # go to line 150 :+50 # go forward 50 lines :1,.d # delete lines from line 1 to current line :1,$ s/old/new/g # perform substitutions on the entire file :w [filename] # write file [to specified filename] :q # quit editing session :wq # write file and quit :q! # quit and don't save changes

Ctags: Create a Tags File for Use with Vi

• ctags is a Unix utility that takes in a set of source

files as input and creates a tags file as output.

• The tags file contains for each function and macro:

– Object name. – File in which the object is defined. – Pattern describing the location of the object.

• General form:

ctags list_of_files

Using the Tags File

• You can use the tags file with the vi (vim, gvim)

editors.

• Use the -t option when invoking vi. The editor will

look in the current directory for the tags file and attempt to find the location of the function or macro in the appropriate file. If it finds it, then the file is

• pened and the cursor is positioned to that location.

vi -t tag

• Examples:

vi -t main vi -t max

Vim: Vi IMproved, a Programmer's Text Editor

• Vim is a text editor that is upwards compatible

with vi.

• Enhancements over vi:

– multi-level undo – syntax highlighting – on-line help – visual selection – tag searching support

Gvim: GUI Vim

• Below is a snapshot of using gvim.

Multi-Level Undo in Vim

• Can use the “u” command to undo multiple

changes, as opposed to vi, which can only undo the last change. Each time you enter “u”, the previous change is undone.

• Can redo the last undone change by using the

CTRL-R command. Each time you enter CTRL- R, the last undone change is reperformed.

Syntax Highlighting in Vim

• Syntax highlighting allows vim to show parts of

the text in another font or color.

• The rules for syntax highlighting are stored in a
• file. So vim can always be easily extended to

support syntax highlighting in new types of files. Vim currently supports syntax highlighting for files with the following extensions:

*.c, *.cpp, *.pl, *.sh, *.csh, *.java, *.html, *.tex, *.tr

Visual Selection in Vim

• Select text portions using the left mouse button.
• Some of the commands you can perform on visually

selected portions include:

d delete the highlighted text y place in a buffer U make uppercase u make lowercase gq format lines to width of window

• Can change the position of the cursor by clicking on

a specific location in the window.

• Can use the middle mouse button to insert text

selected by the left mouse button.

Tag Searching Support in Vim

• Can position the cursor over a name, such as the

name of a function.

• Typing CTRL-] will cause vim to search in the

tags file for the name under the cursor. If found, then the file containing the function or macro name is opened and the cursor is positioned to that function.

• This command is equivalent to exiting vim and

typing

vi -t name

On-line Help in Vim

• You can type “:help” to get online help in Vim.
• There are several keywords or files that can be

selected within the on-line help to learn about specific vim features.

The .gvimrc File

• The .gvimrc file is used to initialize various settings.
• Example commands often used in a .gvimrc file:

:set incsearch # shows text where the pattern has # matched so far when typing a search # command :win width height # set the width and the height in # characters of the window :syntax on # enable syntax highlighting :set background=dark # set background of window to be # dark (default text will be white)

Source Level Debugging

• A source level debugger has a number of useful

features that facilitates debugging problems associated with executing your program.

• You have to create symbolic table information

within the executable by using the -g option when compiling with gcc or g++.

• This symbolic table information includes the

correspondances between

– statements in the source code and locations of instructions

in the executable

– variables in the source code and locations in the data

areas of the executable

GDB: Gnu DeBugger

• GDB is a line oriented debugger where actions

are issued by typing in commands.

• It can be invoked for executables compiled by

gcc or g++ with the -g option.

• General capabilities:

– Starting/exiting your program from the debugger. – Stopping and continuing the execution of your

program.

– Examining the state of your program. – Changing state in your program.

Starting/Exiting GDB

• Bring up the gdb debugger by typing:

gdb executable_name

• Initiate your executable by using the command:

run [command-line arguments]

– Command line arguments can include anything that

would normally appear after the executable name on the command line.

• run-time options, filenames, I/O redirection, etc.

– When no arguments are specified, then gdb uses the

arguments specified for the previous run command during the current gdb session.

• Exit the gdb debugger by typing:

quit

Stopping and Continuing the Execution of Your Program in GDB

• Setting and deleting breakpoints.
• Execution stepping and continuing.

Setting and Deleting Breakpoints

• Can set a breakpoint to stop:

– at a particular source line number or function – when a specified condition occurs

• General form.

break [linenum | function] [if condition]

• Specifying a break command without any

arguments causes a breakpoint to be set at the current line where execution has stopped.

• Can delete a breakpoint:

delete breakpoint_number

Examples of Setting and Deleting Breakpoints

(gdb) break # sets breakpoint at current line # number (gdb) break 74 # sets breakpoint at line 74 in the # current file (gdb) break foo # sets breakpoint when entering # function foo (gdb) break 60 if i == 10 # sets breakpoint at line 60 in the # current file if the variable i has # the value 10 (gdb) delete 3 # remove the third breakpoint (gdb) delete # remove all breakpoints

Execution Stepping in GDB

• Can step to the next statement or into a function

that is being invoked.

• General form. n indicates the number of steps to

perform or until the program encounters a breakpoint or terminates execution. By default

• nly one step is performed.

step [n]

• Can step to the next statement and across any

function that is being invoked.

• General form. n again is the number of steps.

next [n]

Continuing Execution in GDB

• You can continue execution in gdb up to

encountering the next breakpoint or program termination.

• General form. The n indicates to ignore the

current breakpoint at which the debugger has stopped n-1 times. If not specified, then n is 1 by default.

cont [n]

Continuing Execution Until Leaving a Loop

• The until command executes your program until

it reaches a source line greater than the current

• line. If the program is not executing a backward

jump, then it has the same effect as the next command.

• This command is useful for continuing execution
• n a backward jump until the program leaves a

loop.

Examining the State of Your Program in GDB

• Listing source code.
• Printing the values of expressions.
• Displaying the values of expressions.
• Printing the trace of function calls.
• Switching context within the trace.

Listing Source Code

• You can list source code at a specified line or

function.

• General form. You can optionally specify a filename

before a line number or function name. You can specify a range of line numbers. By default ten lines are listed. If no arguments are specified, then ten lines are listed from the current program location or last line that was listed.

list [[filename:]linenum[,linenum] | [filename:]functionname]

Listing Source Code Examples

(gdb) list # list 10 lines from current # location (gdb) list 72 # list lines 67..76 (10 lines # around the specified line) in # the current file (gdb) list cal.c:55 # list lines 50..59 in cal.c (gdb) list 80,95 # list lines 80..95 (gdb) list main # list starting at function main (gdb) list cal.c:jan1 # list starting at function jan1 #in file cal.c

Printing the Values of Expressions in GDB

• One can print the value of legal expressions. One

can also specify the format for the value to be printed.

• General form. The fmt character can be d (decimal),

u (unsigned), f (float), c (character), o (octal), c(character), and x (hexadecimal). When no expression is specified, it prints the value of the last expression specified in a print command. If no fmt is given, then it uses the type associated with the expression.

print[/fmt] [expression]

Example Print Commands

print i # prints the value of variable i print a[i] # prints the value of a[i] print/x a[i] # prints a[i] in hexadecimal print a[i]-b[i] # prints the value of a[i]-b[i] print a # prints the values in the array a print p # prints the address contained in pointer p print *p # prints the value pointed to by p print p->next # prints a field of the struct pointed to by p

Displaying the Values of Expressions in GDB

• The display command is similar to the print

command, but the expression is evaluated and printed after each time you step or continue in gdb.

• General form. All current expressions to be

displayed will be reprinted if you do not give an expression as an argument.

display[/fmt] [expression]

Undisplaying Expressions in GDB

• Use the undisplay command to remove expressions

from being displayed.

• General form. Undisplay with no argument removes

all display expressions.

undisplay [display_number]

Printing the Trace of Function Calls in GDB

• You can print the trace of function calls

corresponding to the activation records that were made to reach the current routine.

• This trace shows the names of the routines, the

values of the arguments passed to each routine, and the line number last executed in each routine.

• One has the option to only print information for the

last n calls. A negative n will print the first n calls.

• General form.

where [n]

Switching Context within the Trace

• One can switch contexts between functions up or

down the call trace. This feature is handy for viewing the values of local variables not in the current function.

• General forms. By default n is 1.

up [n] # go back n caller contexts down [n] # go forward n caller contexts

Changing State in Your Program

• You can assign values to variables during a

debugging session. This can allow a user to test a potential fix without changing the source code and recompiling.

• General form.

set variable = expression

• Examples

set i = 5 set a[i] = 10

Making Calls from GDB

• You can make calls from the gdb prompt. It is

very useful to write diagnostic routines that will print the values in dynamically allocated data structures since these structures are often difficult to access using the print command directly. Calls to functions can also be used to change the state

• f the program.
• General form.

call function_name(args)

Other Useful Features

• Can run a program that has a segmentation fault

to easily find out exactly where the program dies.

• Can type cntrl-C within GDB when a program

does not terminate to find out where the endless loop occurs.

Command Shortcuts

• You do not have to type the complete command.

You only have to type enough characters to ensure that the command name is unique.

(gdb) r < in.dat # same as “run < in.dat” (gdb) p n # same as “print n” (gdb) l foo # same as “list foo” (gdb) c # same as “cont” (gdb) n # same as “next”

• Likewise you can apply command and identifier

name completion using the tab character, just as you do command and filename completion in tcsh.

DDD: Data Display Debugger

• DDD is a graphical front-end for GDB and other

command-line debuggers.

• From DDD you can invoke all of the commands

available in GDB.

• In addition, it has a graphical user interface that:

– Shows the source code where the program execution

has currently stopped.

– Allows you to select commonly specified options

with the mouse.

DDD Interface

• DDD displays four different windows.

– Data window to display variables. – Source window to display source code. – Machine code window to display disassembled

machine code.

– GDB console where conventional gdb commands can

be typed.

• DDD also has other panels which include

common commands that can be selected with a mouse click.

DDD Snapshot

Using the Console Window

• This console is setup based on the type of line
• riented debugger that is being used.
• For the gdb console, any gdb command can be

entered.

• Commonly used gdb commands can also be

issued by clicking on the panel.

Using the DDD Source Window

• Can set a breakpoint by using the right mouse

button and positioning the cursor to the left of a source code line.

• Can display the value of a variable by moving the

mouse cursor over a variable.

• Can highlight a variable and select to print or

display its value by using the options at the top.

Using the DDD Data Window

• To have a variable with its value appear in the

data window as a display:

– A user can highlight a variable in the source window

and then click on the display button.

– A user can double click on a variable in the source

window.

Diff (11.1)

• The diff Unix utility compares two files and

displays the differences between the two files. The differences are displayed with an ed-like notation indicating what changes are needed to modify the first file to make it similar to the second.

• Diff is very useful in shell scripts to detect

differences between expected output and actual

• utput.

Diff Output (11.1)

• Diff output consists of a list of changes.
• General form consists of a sequence of:

commands lines

• Commands are of the form (a for append, c for

change, and d for delete).

linenums [acd] linenums

• Lines from the first file are preceded by “<”.

Lines from the second file are preceded by “>”.

Diff Example

tmp1.txt tmp2.txt tmp3.txt cat cat dog dog mouse mouse mouse cow % diff tmp1.txt tmp2.txt 2d1 < dog % diff tmp1.txt tmp3.txt 1d0 < cat 3a3 > cow % diff tmp2.txt tmp3.txt 1c1 < cat

• > dog

2a3 > cow

Patch (20.9)

• Patch is a Unix utility that takes diff output and

applies the commands to make the first file have the same contents as the second.

• Updates to free software are typically

accomplished using patch. Often the differences between versions of files is much smaller than the files themselves.

Gvimdiff

• The utility (g)vimdiff allows you to edit two or

three files and visualize the differences.

• Changed and new lines are highlighted.
• Deleted lines are also depicted.
• This display is updated as changes are made to

any of the files.

• General form.

gvimdiff file1 file2 [file3]

Cmp

• The cmp Unix utility just returns a status

indicating if the files differ.

Exit Status Meaning The files were identical. 1 The files differed. >1 An error occurred.

• The cmp utility is often used when comparing

two binary files and is typically faster than diff.

Configuration Management Systems

• Configuration management systems provide:

– backup of versions of files – control access to shared files

• Best to keep only source files under configuration

management as long as the other files can be automatically regenerated.

Brief History of Popular Free Configuration Management Systems

• SCCS (Source Code Control System)

– Popular configuration management system. – Problem is that it stores the original file and builds the

new versions by applying the changes.

• RCS (Revision Control System)

– More efficient in that it stores only the changes and

the current file is the complete one.

• CVS (Concurrent Version System)

– Allows more than one person to edit the same file at

the same time and allows merges.

Brief History of Popular Free Configuration Management Systems (cont.)

• SVN (SubVersioN)

– Changes are tracked per-change rather than per-file.

• Some other software configuration management

systems support a decentralized approach rather than using a centralized repository.

Commonly Used SVN Commands

svnadmin create repository # Creates a directory using the repository # (repo) name. Use a full path as the repo # name and place it in a different directory # from the files you wish to put under SVN. # Only needs to be done once. svn checkout file://repository # Allows a user to work in the repository. # Only needs to be done once for each user. svn add filename # Adds a file to the repo. File will be added # on the next commit. svn delete filename # Deletes a file from the repo. svn commit # Updates the repo with latest files in the # cwd. You will be prompted for a comment # describing the latest commit.

Commonly Used SVN Commands (cont.)

svn status # Shows status of files in repository and cwd # by the first column of each listed file. # ' ' indicates file has no changes. # 'M' indicates file has been modified. # '?' indicates file not under version control. # 'A' indicates file has been added. svn diff [filename]# Shows differences between file in cwd and # repo. svn log filename # Shows log messages about the specified # file. svn update [revnum] # Updates current working directory (cwd) # with latest files in the repo. Can be used to # get up-to-date files when working with # other users or to go back to a previous # revision.

SVN Identification and Log Information

• Each time you perform an svn commit command,

you are prompted for a message (comment) giving an overview of that change. Each version is identified by a revision number.

• You can see messages about the history of the

revisions of a file by using the svn log command.

• You can go back to an older version by specifying a

revision number when using the svn update command.

Finding More Information about SVN

• You can get online help by typing:

svn help command_name

• You can also look at tutorials that are available on

the web.

http://www.thegeekstuff.com/2011/04/svn-command-examples/

Gprof

• The gprof Unix utility produces an execution

profile of the call graph of a program.

• The profile is useful for finding out where most
• f the time is spent during the execution of your

program.

• A developer can use this information to tune the

time-consuming portions of a long-running program.

Using Gprof

• You can have a program instrumented to collect

data that can be processed by gprof by using the

• pg option when compiling with gcc or g++.

g++ -pg filename.cpp

• A gmon.out file will be produced as a side effect
• f the execution of your instrumented program.
• You can obtain a brief profile with an annotated

source listing from the gmon.out file by running the following command:

gprof -b -A