Files and Exceptions Joan Boone jpboone@email.unc.edu Summer 2020 - - PowerPoint PPT Presentation

Slide 1

Files and Exceptions

Joan Boone

jpboone@email.unc.edu

Summer 2020

INLS 560

Programming for Information Professionals

Slide 2

Topics

Part 1

• Opening and writing to files

Part 2

• Reading data from files

Part 3

• Errors and exceptions

Slide 3

Why Use Files?

• Most applications use persistent data, i.e., data stored in files

and databases

– Read existing data from files/databases – Write (add, update, delete) data to files/databases for future use

• Types of files

– Text files (contain readable text, such as Unicode) – Binary files (only readable by specific applications)

• Ways to access files

– Direct (or random) access: similar to how video/audio players

work – they can 'jump' directly to any data in the file

– Sequential access: start at the beginning and read to the end of

the file

Slide 4

File Names and File Objects

• Each operating system (Win, Mac, Linux, etc.) has its own

rules for naming files

– In general, the convention is filename.extension – Some typical file extensions (or file types)

my_resume.pdf, screenshot.png, class_notes.txt, hello_world.py, python.exe

• File objects are used by programs to access files

– A file object is created for a specific file and is stored in a

variable that represents that object

– The file object variable is used perform any operations on the

file, e.g., to open, read, write, or close a file

Slide 5

Object-oriented View of a File Object

Source: Starting Out with Python by Tony Gaddis

my_file.readline() my_file.write('some stuff') my_file.close()

name mode encoding buffer errors ... Methods that access the file File attributes

my_file = open(filename, mode)

When a you open a file, the open function returns a file object:

Slide 6

Opening Files

• Before you can access a file, you have to open the file
• Use the open function to create a file object that is associated

with a specific file my_file = open(filename, mode)

• filename is a string with the file name
• mode specifies how the file is opened
• my_file is the variable that references the file object
• Python file modes:

Source: Python documentation for open function

Slide 7

Opening Files

Frequently used file modes:

• 'r' mode opens a file for reading only; the file cannot be changed
• r written to
• 'w' mode opens a file for writing

–

Important: if the file already exists, its contents are erased and a new file is created.

–

If the file does not exist, it is created

• 'a' mode opens a file for writing

–

Data written to the file is appended to the end of the file

–

If the file does not exist, it is created

If opening a file in the same directory as your program, you do not need to specify the path for the file name

test_file = open('test.txt', 'r')

Slide 8

Files in different directories

If opening a file in a different directory than where your program is located, you must specify the full directory path with the file name Mac and Linux

test_file = open('/Users/Joan/temp/test.txt', 'w')

Windows

• If using forward slashes

test_file = open('C:/Users/Joan/temp/test.txt', 'w')

• If using backward slashes, 2 ways

– use rawstring r prefix, so that Python interprets everything in the

string as a literal character

test_file = open(r'C:\Users\Joan\temp\test.txt', 'w')

– or, use double back slash to escape the back slash

test_file = open('C:\\Users\\Joan\\temp\\test.txt', 'w')

Slide 9

Writing Data to a File

• Once the file is opened, string data can be written to it
• Use the write function with the file object to be written to

my_file.write(string_variable)

• Example
• scars = open('oscar_movies.txt', 'w')
• scars.write('Wings') # literal string

movie_name = 'The Broadway Melody'

• scars.write(movie_name) # variable with string value
• After you have finished writing to the file, close the file:
• scars.close()
• Failure to close a file can cause a loss in data

–

Data is buffered before it is actually written. This improves performance because writing to memory is faster than writing to disk. When the buffer is full, it is written to the file (disk)

–

Closing the file ensures that any buffered data is written to the file

Slide 10

Writing Data to a File

# This program writes three lines of data to a file. def main(): # Open a file named oscar_movies.txt.

• scars = open('oscar_movies.txt', 'w')

# Write Oscar-winning movie names to the file

• scars.write('Wings')
• scars.write('The Broadway Melody')
• scars.write('All Quiet on the Western Front')

# Close the file.

• scars.close()

# Call the main function. main() Contents of file, oscar_movies.txt: WingsThe Broadway MelodyAll Quiet on the Western Front

Slide 11

Writing Data to a File

# This program writes three lines of data to a file. def main(): # Open a file named oscar_movies.txt.

• scars = open('oscar_movies.txt', 'w')

# Write Oscar-winning movie names to the file

• scars.write('Wings\n')
• scars.write('The Broadway Melody\n')
• scars.write('All Quiet on the Western Front\n')

# Close the file.

• scars.close()

# Call the main function. main()

file_write.py

File contents: Wings The Broadway Melody All Quiet on the Western Front

Use the newline escape sequence, \n , to separate items in the file:

Slide 12

Exercise: Create a simple web page

• Prompt user for name and description
• Create an HTML file
• Write the HTML with the name and description values

webpage_generator.py Enter your name: Monty Python Describe yourself: Inspired the name of the Python language.

<html> <head> <title>My Personal Web Page</title> </head> <body> <h1>Monty Python</h1> <hr> Inspired the name of the Python language. <hr> </body> </html>

my_page.html Displayed in browser

Slide 13

Topics

Part 1

• Opening and writing to files

Part 2

• Reading data from files

Part 3

• Errors and exceptions

Slide 14

Reading Data from a File

# This program reads the contents of a file def main(): # Open a file named oscar_movies.txt.

• scars = open('oscar_movies.txt', 'r')

# Read the file's contents file_contents = oscars.read()

• scars.close() # Close the file.

print(file_contents) # Print the contents # Call the main function. main() Output: Wings The Broadway Melody All Quiet on the Western Front

file_read.py

The read method returns the entire contents of the file as a string

Slide 15

Reading Data from a File (recommended)

# This program uses readline to read the file contents def main(): # Open a file named oscar_movies.txt.

• scars = open('oscar_movies.txt', 'r')

# Read three lines from the file line1 = oscars.readline() line2 = oscars.readline() line3 = oscars.readline()

• scars.close() # Close the file.

print(line1) print(line2) print(line3) main()

Output:

Wings The Broadway Melody All Quiet on the Western Front file_readline.py

The readline method reads one line at a time from the file as a string

Slide 16

Removing the Newline Character

# This program uses rstrip to remove newline def main():

• scars = open('oscar_movies.txt', 'r')

line1 = oscars.readline() line2 = oscars.readline() line3 = oscars.readline() # Strip the \n from each string. line1 = line1.rstrip('\n') line2 = line2.rstrip('\n') line3 = line3.rstrip('\n')

• scars.close()

print(line1) print(line2) print(line3) main()

Output:

Wings The Broadway Melody All Quiet on the Western Front rstrip_newline.py

Use rstrip method to remove \n from the (right) end of the string

Returns a copy of the string with trailing newline character removed

Slide 17

Writing Numeric Data to a File

# This program adds 3 user input numbers and # converts to strings before writing to a text file def main():

• utfile = open('numbers.txt', 'w')

# Get three numbers and calculate the sum num1 = int(input('Enter a number: ')) num2 = int(input('Enter another number: ')) num3 = int(input('And one more number: ')) sum = num1 + num2 + num3 # Write the numbers to the file.

• utfile.write(str(num1) + '\n')
• utfile.write(str(num2) + '\n')
• utfile.write(str(num3) + '\n')
• utfile.write(str(sum) + '\n')
• utfile.close()

print('Data written to numbers.txt') main()

write_numbers.py

• Strings can be written directly to a text file, but numbers must be

converted to strings before they can be written to a file.

• Built-in function, str, converts a value to a string.

Slide 18

Reading Numeric Data from a File

# This program reads numbers from a file, # converts them to numbers and calculates product def main(): infile = open('numbers.txt', 'r') # Read three numbers from the file. num1 = int(infile.readline()) num2 = int(infile.readline()) num3 = int(infile.readline()) infile.close() # Multiply the three numbers. product = num1 * num2 * num3 # Display the numbers and their total. print('The numbers are:', num1, num2, num3) print('Their product is:', product) main()

read_numbers.py

• When reading numbers from a file, you must convert them from strings to

numbers before you can use them in arithmetic expressions

• Built-in functions, int and float, convert strings to numbers

Slide 19

Using Loops to Write to a File

# This program prompts the user for sales amounts # and writes those amounts to the sales.txt file. def main(): # Get the number of days. num_days = int(input('For how many days do you have sales? ')) sales_file = open('sales.txt', 'w') # Get the sales amount and write to the file for count in range(1, num_days + 1): # Get the sales for a day. sales = float(input('Enter the sales for day #' + str(count) + ': ')) # Write the sales amount to the file. sales_file.write(str(sales) + '\n') sales_file.close() print('Data written to sales.txt.') main()

write_sales.py

Files generally contain a lot of data, so programs often use loops to read and write the contents.

Source: Starting Out with Python by Tony Gaddis

Slide 20

Using Loops to Read Files

• Often programs must read the contents of a file and process each line

(or record) in the file, without knowing how many lines are in the file.

• Loops are the best approach for reading files, but you need to know

when the end of the file has been reached.

• The readline method returns an empty string, ' ', when it attempts

to read beyond the end of the file. The empty string serves as a sentinel value.

• General algorithm

Open the file Use readline to read the first line from the file While the value returned by readline is not an empty string Process the line that was read from the file Use readline to read the next line from the file Close the file

Source: Starting Out with Python by Tony Gaddis

Slide 21

Using while Loop to Read a File

Source: Starting Out with Python by Tony Gaddis # This program reads all of the lines in the sales.txt file. def main(): sales_file = open('sales.txt', 'r') # Read the first line, but don't convert to a number yet. # Need to test for an empty string. line = sales_file.readline() # As long as an empty string is not returned # from readline, continue processing. while line != '': amount = float(line) # Convert line to a float print(format(amount, '.2f')) # Format and display amount line = sales_file.readline() # Read the next line sales_file.close() main()

read_sales_while_loop.py

NOTE: most programming languages provide a similar technique for detecting the end of file. This is an important technique to understand if you plan to learn other languages

Slide 22

Using for Loop to Read a File

(simpler approach)

Source: Starting Out with Python by Tony Gaddis

# This program reads all of the lines in the sales.txt file. def main(): sales_file = open('sales.txt', 'r') # Read all the lines from the file. for line in sales_file: # Convert line to a float. amount = float(line) # Format and display the amount. print(format(amount, '.2f')) sales_file.close() main() read_sales_for_loop.py

• Python also provides a simpler approach for looping through a file by

using a for loop that automatically reads a line in a file without checking any special end-of-file (EOF) conditions

• The loop iterates once for each line in the file

Slide 23

Exercise: Reading Data from a File

• Update the average_steps.py program to read

the numbers from the file, steps.txt. Each number represents the number of steps taken on each day of the year.

• Calculate the total number of steps
• Keep a counter of the number of lines (days) read
• Display the average number of steps taken per day.

Output should be:

The average number of steps taken in 365 days was 5,296.8

average_steps.py

Slide 24

Topics

Part 1

• Opening and writing to files

Part 2

• Reading data from files

Part 3

• Errors and exceptions

Slide 25

Errors and Debugging

3 kinds of errors can occur in a program

• Syntax: Language syntax is incorrect, so the

program won't run

• Runtime: Error occurs after the program

starts running

• Semantic: Program runs successfully, but

produces wrong results

Debugging

• Process of figuring out what went wrong and

fixing it

• Can be both frustrating and interesting
• One of the best ways to learn programming

Source: http://xkcd.com/568/

Slide 26

Many Types of Exceptions

A few of the more common ones are SyntaxError: invalid syntax found in program ValueError: an operation or function receives an invalid value NameError: when a local or global name is not defined TypeError: an operation on an object is not appropriate OSError: a general exception for input and output errors IndexError: a subscript or index is out of range Exception: general purpose, non-specific error Complete list: Python Built-in Exceptions, Exception Class Hierarchy

• Exceptions are errors that occur while a program is running, and

will cause a program to stop where the error occurred.

• Exceptions are not user errors, such as entering invalid input;

they are program errors that you, the developer, must fix

Slide 27

SyntaxError Exception

File "C:/Users/.../average_steps.py", line 5 steps_file = open('steps.txt', 'r) ^ SyntaxError: EOL while scanning string literal Process finished with exit code 1 An exit code > 0 usually means the program terminated with an error Type of Python exception and a descriptive message

Raised when the parser encounters a syntax error. Review the messages in the PyCharm Run window. They describe the type of error and where it occurred in your code.

Line number and statement in error

Slide 28

ValueError Exception

Traceback (most recent call last):

File "C:/Users/.../average_steps.py", line 24, in <module> main() File "C:/Users/.../average_steps.py", line 5, in main steps_file = open('steps.txt', 'z') ValueError: invalid mode: 'z' Process finished with exit code 1

Raised when an operation or function receives an argument that has the right type, but an inappropriate value

Slide 29

NameError Exception

Traceback (most recent call last):

File "C:/Users/.../average_steps.py", line 24, in <module> main() File "C:/Users/.../average_steps.py", line 18, in main print('The average number of steps taken in', line_count, 'days was', format(avrage, ',.1f')) NameError: name 'avrage' is not defined

Raised when a local or global name is not found.

Slide 30

TypeError Exception

Traceback (most recent call last):

File "C:/Users/.../average_steps.py", line 24, in <module> main() File "C:/Users/.../average_steps.py", line 13, in main total_steps = total_steps + line TypeError: unsupported operand type(s) for +: 'int' and 'str' Process finished with exit code 1

Raised when an operation or function is applied to an object of inappropriate type.

Slide 31

OSError Exception

Traceback (most recent call last):

File "C:/Users/.../average_steps.py", line 24, in <module> main() File "C:/Users/.../average_steps.py", line 5, in main steps_file = open('steps*txt', 'r') OSError: [Errno 22] Invalid argument: 'steps*txt' Process finished with exit code 1

This exception is raised when a system function returns a system- related error, including I/O failures such as “file not found” or “disk full”

Slide 32

FileNotFoundError Exception

def main(): # Open the file. steps_file = open('step.txt', 'r') # Initialize counters total_steps = 0 line_count = 0 # Read each line in the file for line in steps_file: total_steps = total_steps + int(line) line_count = line_count + 1 # Calculate the average and display average = total_steps / line_count print('The average number of steps taken in', line_count, 'days was', format(average, ',.1f')) # Close the file. steps_file.close() main()

If you specify a file name that meets the file naming conventions, but does not exist, the following exception occurs:

FileNotFoundError: [Errno 2] No such file or directory: 'step.txt'

Slide 33

Handling Exceptions

• Python, like most programming languages, allows you to handle

exceptions so that your program doesn't abruptly stop

• Define an exception handler with try/except statements
• General format:

How it works:

• If the statements in the try clause do not raise an exception specified by

ExceptionName, then the statements in the except clause are skipped and

execution resumes after the except clause

• If a statement in the try clause does raise an exception specified by

ExceptionName, then the statements in the except clause are executed

• If a statement in the try clause does raise an exception but it is not specified

by ExceptionName, then your program will stop with a traceback message try:

statement1 statement2 except ExceptionName: statement3 statement4 statement5 statement6

Add statements to handle the exception inside the

except clause.

For example, display an error message. Statements that can potentially raise an exception are placed inside the try clause.

Slide 34

FileNotFoundError Exception Handler

def main(): filename = 'step.txt' try: steps_file = open(filename, 'r') # Open the file total_steps = 0 # Initialize counters line_count = 0 for line in steps_file: # Read each ine in the file total_steps = total_steps + int(line) line_count = line_count + 1 # Calculate the average and display average = total_steps / line_count print('The average number of steps taken in', line_count, 'days was', format(average, ',.1f')) steps_file.close() # Close the file except FileNotFoundError: print('Error: cannot find file,', filename) main()

Slide 35

Handling Multiple Exceptions

• A program can raise several types of exceptions -- it depends on what

the program is doing.

• When reading the contents of a file, several types of exceptions could
• ccur

–

File does not exist, or the file name is correct but may have been moved to another directory (FileNotFoundError)

–

Invalid file name, i.e., it does not meet platform naming conventions (OSError)

–

Invalid mode for opening the file (ValueError)

–

File may contain bad data, i.e., non-numeric data (ValueError)

• It is possible that an exception can be raised that you did not
• anticipate. To handle this case, always include a “catch all” exception

that handles any exception not covered by other handlers.

except Exception:

print('An unknown error occurred')

Slide 36

Handling Multiple Exceptions

def main(): filename = 'steps.txt' try: steps_file = open(filename, 'r') # Open the file total_steps = 0 # Initialize counters line_count = 0 for line in steps_file: # Read each line total_steps = total_steps + int(line) line_count = line_count + 1 # Calculate the average and display average = total_steps / line_count print('The average number of steps taken in', line_count, 'days was', format(average, ',.1f')) steps_file.close() # Close the file except FileNotFoundError: print('Error: cannot find file,', filename) except Exception: # catch-all error handler print('An unknown error occurred') main()

Slide 37

Handling More Exceptions

Some additional exceptions that could occur in average_steps_exc.py include:

• OSError

– Occurs when a system function returns a system related

error, for example, file name is 'steps.txt:'

• ValueError

– Occurs when a variable has the right data type, but an

inappropriate value, for example, file mode is 'k'

– Occurs when reading data from a file that is expected to be

numeric, but is not. For example, suppose steps.txt contained '11o2' instead of '1102'

Slide 38

Handling Multiple Exceptions

def main(): filename = 'steps.txt' try: steps_file = open(filename, 'r') # Open the file total_steps = 0 # Initialize counters line_count = 0 for line in steps_file: # Read each record (line) in the file total_steps = total_steps + int(line) line_count = line_count + 1 # Calculate the average and display average = total_steps / line_count print('The average number of steps taken in', line_count, 'days was', format(average, ',.1f')) steps_file.close() # Close the file except FileNotFoundError: print('Error: cannot find file,', filename) except OSError: print('Error: cannot access file,', filename) except ValueError: print('Error: invalid data found in file', filename) except Exception: # catch all error handler print('An unknown error occurred') main()

• Order in which exception handlers are specified can be important
• If an exception occurs, Python will look for the first handler that can handle it

Slide 39

Displaying an Exception's Default Error Message

Exceptions are objects, and each object usually has an attribute that contains a default error message.

• The message is the same as the one displayed at the end of a traceback

when an exception has no handler

• When you write an except clause, you can optionally assign the

exception object to a variable. Pass the variable to the print function and it will display the Exception's default message.

except ValueError as err: print(err) # ok, but a context-specific message is better

# Like the message below...

print('Error: invalid data found in file', filename)

A good place to use this approach is with the 'catch all' except clause, since the type of error is unknown

except Exception as err: print(err)

Slide 40

Exceptions with Default Messages

# Read a file with number of steps taken for each day of the year. Calculate average steps taken def main(): filename = 'steps.txt' try: steps_file = open(filename, 'r') # Open the file total_steps = 0 # Initialize counters line_count = 0 for line in steps_file: # Read each record (line) in the file total_steps = total_steps + int(line) line_count = line_count + 1 # Calculate the average and display average = total_steps / line_count print('The average number of steps taken in', line_count, 'days was', format(average, ',.1f')) steps_file.close() # Close the file except FileNotFoundError as err: print('Error: cannot find file,', filename) print('Error:', err) except OSError as err: print('Error: cannot access file,', filename) print('Error:', err) except ValueError as err: print('Error: invalid data found in file', filename) print('Error:', err) except Exception as err: # catch all error handler, if the above handlers do not apply print('An unknown error occurred') print('Error:', err) main()

average_steps_exceptions.py

Slide 41

What if my program is not handling exceptions properly?

There are two ways an exception would not be handled

• No except clause specifying an exception of the right type
• Exception occurred outside of a try clause

In both cases, the exception will cause your program to stop. To avoid this situation:

• Test your program: try to 'break it' with faulty, or invalid, data.

Add exception handlers for the errors that occur.

• After the specific exception handlers, add one except clause

to catch all remaining exceptions.

• Ensure that the statement causing an exception is inside of a

try clause