Python Tutorial: File Objects - Reading and Writing to Files

PROFESSOR: Hey there.

How's it going, everybody.

In this video, we'll
be learning how

to work with file
objects in Python

and some of the
useful things that we

can do with these objects.

So whether you use Python for
desktop or web applications,

you're likely going
to be interacting

with files a good bit.

So it's definitely
a good skill to have

to know how to properly interact
with these file objects.

OK.

So let's go ahead and dive in.

So to get a file object, we can
use the built-in open command.

So I have a file here called
test.txt in the same directory

as my Python file.

Now if I open this file, we can
see that it's just a plain text

file with multiple lines.

So let's see how we
can open and read

this file from within Python.

Now the way I'm going to
open up the file right

now isn't the way that
it's normally recommended.

It's usually recommended to use
a context manager, which I'll

show you here in just a second.

But to show you why a
context manager is useful,

let me first show you this
method for opening files first.

So what we're
going to do here is

we're going to say f equals
open and we're just going

to open that test.txt file.

Now if you're working with files
from different directories,

then you're going to have to
pass the path to that file

into the open command.

But since this file is within
the same directory as my Python

file, then I can just pass
in the name of the file.

But if you want to learn
more about how paths work,

then we touch on that a
little bit in the tutorial I

did on the OS module.

OK, so the open
command here allows

us to specify whether
we want to open

this file for reading,
writing, appending,

or reading, and writing.

And now if we don't
specify anything,

then it defaults to opening
the file for reading.

But I usually like
to be explicit here.

So let's go ahead
and say that we want

to open this file for reading.

And we can do this by passing
in a second argument here.

And that's just going to be
the string of a lowercase r.

And we'll touch on
some of these later.

But if I wanted to
write to a file,

then it would just be a
lowercase w that I'd pass in.

Appending to a file
is a lowercase a.

And if I wanted to read
and write to a file,

then I could do an r plus.

But for now, we just want to
read the contents of the file.

So let's just pass
in a lowercase r.

So now the file is
actually open and we

can print the name of the file.

If I was to do a printf.name.

And also before I run this and
print the name of the file out,

there's one more thing
that we have to do here.

If we open a file
like we just did here,

then we need to
explicitly close the file

when we're done using it.

And to do this, I'm going
to do it by saying f.close.

So now that we've closed
that file, let's go ahead

and run this.

And you can see that it printed
out the name of the file

that we opened.

And so this has some
more information

that we can print out.

Also, if we wanted to print the
mode that the file is currently

opened with, I can do a f.mode.

And if I run that, you
can see it prints out

a lowercase r because we
open the file for reading.

So now even though that this
works the way that we've just

now done this, let me show
you how to instead open

the file using a context manager
and why, for most use cases,

you'll want to work
with files this way.

So if we open the
file like we did here,

then we have to remember to
explicitly close the file.

If we don't close
the file, then you

can end up with
leaks that cause you

to run over the maximum allowed
file descriptors on your system

and your applications
could throw an error.

So it's always important
to make sure that you

close the files that you open.

So in order to use
a context manager,

then it's kind of similar.

But we can do this
using the with keyword.

So we can say with and then I'm
just going to copy all of this.

So with open test.txt
in read mode.

And then here at the end,
I'm going to say as f,

and then I'm going to put in
a opening for our block here.

Now, this can be a little
confusing to people at first,

because the variable name is
actually over here on the right

using as f instead
of over on the left

when we said f equals open.

But the benefit of
these context managers

is that they allow
us to work with files

from within this block.

And after we exit
that block of code,

it'll automatically
close the file for us.

So we don't have to worry
about whether or not

we add in these closes here.

Now this will also
close the file

if there are any
exceptions that are thrown

or anything like that.

So that's why using
these context managers

are considered a best practice.

And it just automatically takes
care of all that cleanup for us.

So now I'm going to go ahead and
delete my outside open and close

statements there.

Now one thing that some
people don't realize

is that you actually have
access to this f variable.

For now, I'm just
going to say pass

within this context manager.

Now we actually have access
to this file object variable

after we exit the
context manager.

But the file will
just be closed.

So for example, if I print
the closed method on f now

and run that, you can
see that it returns true.

But even though that we have
access to this variable here,

it is closed.

So it's not like we
can read from it.

If I try to read the
contents from the file

and print that out,
then you can see

that it throws a
value error here

and it says I/O operation
on a closed file.

So for what we want, we're going
to have to work with this file

from within this
context manager.

And for the rest
of the video, I'll

be using these context
managers to work with files

since it's a good practice.

But I wanted to show you
the other way first in case

you see it in
examples or wondered

why I wasn't doing it that way.

OK, so back to our file.

So we just tried to read the
contents from the closed file

and got our error.

But let's look at
how we can read

the contents from the file from
here within our context manager.

So let's create a variable
called f_contents.

And this will just hold
the contents of our file.

Now if we do an f.read
and if I print this out,

and actually I need to actually
print out that f_contents.

So if I save that
and print that out,

then you can see that
it printed out all

of the contents of our file.

Now if you have a
small file, then this

is probably what you want.

But what if we have an extremely
large file that we want to read,

but we don't want to load all
of the contents of that file

into memory.

Well, there are a
couple of other methods

here that we have available for
reading file contents instead

of f.read.

So just to look at a couple of
those, I could say f.readlines.

And if I print
this out, then you

can see that we get a list of
all of the lines in the file.

And it looks a little weird
because we have our new line

characters in there.

But if we look
through this list,

then it actually gets
every line of the file

as a different
element of that list.

Now instead of f.readlines,
I could do f.readline.

And if I save that
and run it, then you

can see that read line grabbed
the first line of our file.

Now every time that
we run f.readline,

it gets the next
line in our file.

So if I was to copy
all of this and then

do it again and
run that, now you

can see that it got the
first and the second lines

from the file.

Now this printed out
a little weird here

because the print statement
ends with a new line by default.

But if I go up here and
pass in an empty string

to the end of our
print statement,

then it will no longer add
in that extra new line.

And now you can see that
those are the way that they

are in the file.

But we still haven't
solved our problem

of how we can read
all of the content

from an extremely large file.

If we read the entire
file in all at once,

then we could run out of memory.

And we don't want
to go through and do

F line thousands of times.

So what we're going
to do here is,

instead of using read
line or read lines,

we can simply iterate over the
lines in a file by saying for--

let me go to a new line
here, for line in f,

and then from here, we
can just print that line.

So I'm going to copy
that and save that.

So now let me go ahead and
comment out these lines

and run this iteration
over the lines.

And you can see that it
printed out all of the lines

in our file.

Now this is efficient
because it's not

reading in all of the contents
from our file all at once.

So it's not a memory issue
that we have to worry about.

What it's going
to do is it's just

going to go through and get one
line at a time from the file.

Now, this is usually good
enough for most people,

but sometimes you may want more
control over exactly what you're

reading from the file.

Now, if we go back, I'm going to
go ahead and delete this line.

If we go back to our f.readline
here and I'm going to get

rid of that one.

Now I'm going to go
back to using f.read.

And if you remember, this
read in the entire contents

of the file.

So if I run that,
you can see that we

got the exact same thing.

But with f.read, we can actually
specify the amount of data

that we want to read
at a time by passing

in the size as an argument.

So if I pass in a 100
to our read method

and then print this
out, you can see

that it printed out the first
100 characters of our file

instead of printing the
whole thing all at once.

Now, if I was to copy
this and run this again,

then you can see that it
printed out the rest of the file

because it picked
up where it left off

and read 100 more
characters of the file.

Now, when we reach
the end of the file,

then read will just
return an empty string.

So if I was to copy this for
a third time and rerun this,

then you can see
that nothing happens.

Because what happens when we
reach the end of the file,

read just returns
an empty string.

So this print statement is just
printing out an empty string.

So how are we going to use
this technique in order

to read in a large file?

So since we don't know exactly
how long the file will be,

we're going to have to use some
kind of loop that just iterates

over small chunks at a time.

So instead of hard
coding in 100 here,

I'm going to create a variable
here called size_to_read.

And for now, we'll just go
ahead and set that equal to 100.

So now instead of passing
in 100 to f.read, let's

just pass in this size_to_read.

OK, so this will grab the first
100 characters of our file.

Now remember when we hit the
end of the file, then read

will just return
an empty string.

So if we do a while loop and say
while the length of f_contents

is greater than 0, then we
will print out the contents

that we got from read.

Now don't run it like
this yet, because this

will be an infinite
loop whenever advancing

the contents of the file.

After it prints the
contents, then we

want to read in the next
chunk of characters.

So in order to do that,
then we just have to,

again, say f_contents equals
f.read of that sized chunk.

Now, what it's going
to do after this line

here is that it's going to kick
us back out to the while loop,

and it will check if we've
hit the end of the file,

because f.read will
return an empty string

and it won't meet
this condition.

So now if I go
ahead and run this,

then you can see that
it printed out all

of the contents of our file.

So to get a better idea
of what's going on here,

let's change the size to
read to 10 characters instead

of 100 characters.

And every time that we print
out f.contents here, instead

of an empty string, let's
make this an asterisk.

So now if I print
this out, then you

can see it's a little
more clear that we're

looping through 10
characters at a time,

and it's printing out these
asterisks through every loop.

So you can see that it
came through the loop here

and it printed out these.

And then the asterisk that we
know that it's just that chunk.

Then it printed out the
next 10 characters and then

the next 10 characters
and so on until we

got to the end of the file.

Now when we read
from files, you can

see that it advances
its position every time.

So we can actually see the
current position using f.tell.

So what I'm going to do is I'm
going to comment out this while

loop here.

And down here, I'm
going to say print

and we'll print out f.tell.

So if I go ahead
and run that, you

can see the f.tell returned 10.

So it's saying that we're
currently at the 10th position

in the file.

And that's because we've already
read in 10 characters here.

And we can manipulate
our current position

using the seek method.

So to show an
example of this, let

me print the first 20
characters of the file

by running f.read twice.

So I'm going to go
ahead and print out

the contents after the
first 10 characters there.

And then I'm going to
do this a second time

to get the next 10 characters.

And I'm going to go
ahead and take out

this second empty
string there so that it

pushes our finished
statement out of the way.

So now actually let
me get rid of f.tell

here and go ahead and run this.

So we can see that it
printed out the first 20

characters of our file.

Now when we read in
this second chunk here,

it picked up at the 10th
position and read in the next 10

characters like we would expect.

But what if I wanted that second
read to instead start back

at the beginning of the file?

And we can do this with f.seek.

So between these two reads,
if I was to do an f.seek of 0

and save that and
ran it, now you

can see that it's
set our position back

to the beginning of the file.

So the second time we
read in our contents,

it starts back at the beginning
instead of picking up where we

left off after the first read.

Now we used seek 0 to start
at the beginning of the file,

but you can use it to change
the position to any location

that you'd like.

So now let's take a look
at writing to files.

And a lot of this will
be similar to reading.

So first of all,
what happens if we

try to write from
within a file that we

have opened in read mode?

So let's go ahead and try that.

So I'll do an f.write and I'll
just do an f.write of test.

And I'm going to go ahead and
get rid of that while loop

also and save that.

So you see when I have
a file open in read mode

and try to write that
we get an error that

says that this is not writable.

So we have to have the
file open in write mode.

So now back up here
within our open statement,

let's create a new
file called test2.txt.

And instead of reading, we
are going to write to it.

Now in order to do that, we can
just say a lowercase w instead

of that lowercase r.

Now you can see over
here in our directory

that we don't have
a test2.txt yet.

Now, if the file
doesn't exist already,

then this will go
ahead and create it.

Now, if the file does exist,
then it will overwrite it.

So be careful if you're writing
to a file that already exists.

Now if you don't want
to overwrite a file,

then you can use a lowercase
a for appending to the file.

But we're going to go
ahead and overwrite

this file if it exists.

So first of all, instead of
writing to this file, I'm just

going to go ahead and put
in a pass statement here,

which basically says
don't do anything.

So I'm going to go
ahead and run this.

And you can see that
it created test2.txt.

So I didn't actually have to
write anything to the file

in order to create it.

Just using the open with the
write mode will create the file.

So now in order to
write to this file, then

we can just do
what we did before.

We can do an f.write test.txt.

So I'm going to go
ahead and run that.

Now if we go over
here to test2.txt,

then you can see that it
wrote test to our file.

Now if I go back here and do
another write to this file,

then it's going to pick
up where we left off,

just like the read method did.

So now if I run this
and go back to the file,

then you can see that it
wrote test twice back to back.

Now you can actually use
seek when writing files also

to set the position back to
the beginning of the file.

And we can do this.

If I go back here between
these two write statements

and I was to do an f.seek of 0.

Now if I run this, then you
can see that the second test

overwrote the first one.

So seek can get a
little confusing

for file writes because it
doesn't overwrite everything,

only what it needs to overwrite.

So for example, if instead of
writing the same thing twice,

if I was to do an
f.seek at the beginning

and write out an r as
my second one there.

And now if I run that
and go back to the file,

then you can see that the r
only overwrote the t in test.

It didn't delete the
rest of the content.

So using file seek whenever
I'm writing to files,

it can get a little confusing.

And I don't use it a whole lot.

But maybe there are
some use cases out there

that you guys will
find it useful for.

OK, so let's go ahead
and pull all of this

together and use read and
write on multiple files

at the same time.

So we're going to
use this to make

a copy of our test.txt file.

So let's go ahead and delete
our test2.txt file here.

So that we don't
confuse the two.

And I'm going to go ahead
and close that there.

So I'm going to go ahead
and get rid of these here.

So first let's open our original
test.txt file in a read mode.

And instead of f here,
I'm going to use rf.

And I'll just say rf
there for read file,

since this is the
file that we're

going to read from in
order to write to our copy.

So now within this
with statement here,

I'm going to go ahead and let's
go ahead and copy all of this

and paste another
open within here.

And I'm going to call this a
test_copy.txt and I'm going

to open this in write mode.

And I'm going to call
this wf for write file.

Now you can actually put
both of these open statements

on a single line
separated by a comma.

But I think readability
here is pretty important.

And mixing those two on
one line is sometimes

difficult to understand,
at least for me.

So this is usually how I work
with multiple files at a time

is putting them on two
different lines, one

nested within the other.

So now within here, we
have two files open--

rf for reading our original file
and wf for writing to our copy.

Now, to do this, it's just as
easy as saying for line in rf,

we want to do a
wf.write of that line.

So now let's walk over
this one more time.

So we have our original
file opened and we're

reading from that file.

And we have a file that doesn't
exist yet that's our copy.

And we're writing to that file.

And we're saying for each
line in our original file,

write that line to wf, which is
the file that we are copying to.

So if I go ahead and run that,
then you can see that it created

this test_copy.txt file.

And if I open this,
you can see that it is

an exact copy of our original.

And lastly, let's look at how
we can do something similar

and copy a large picture file.

Now this is going to
be slightly different.

So if I look in my
current directory that

has my Python script that
I'm currently running,

I also have a picture of my
dog here when he was a puppy.

And let's go ahead and try
to copy this picture file

using file objects in Python.

Now this file here
is called bronx.jpeg.

And if I just try to replace our
text files with these picture

files, and down here, I'll
call this bronx_copy.jpeg.

Now, this is exactly the
same as our previous example,

but we're trying to use a
picture instead of a text file.

Now if I try to
run this, you can

see that we got an error down
here that says utf codec can't

decode byte in position 0.

So in order to work
with images, we're

going to have to open
these files in binary mode.

And all that means
is that we're going

to be reading and writing bytes
instead of working with text.

Now I'm not going to
go into the specifics,

but if anyone is curious
about the differences,

then I'll try to leave some
resources in the description

section as to what
exactly that means.

But for this case, in order
to work with these pictures,

to use binary mode, we can just
append a B to our read r here

and our write w there.

So now with that
one simple change

if I save that and
now run it, then you

can see that we do have this
copied picture file here.

And if I go over
to finder, then you

can see that, that
file copied exactly

the way that the original is.

So last thing here.

Now I said earlier
that sometimes you

want more control over exactly
what you're reading and writing.

So instead of doing
this line by line,

let's instead do this
in specific chunk sizes.

And we saw something
like this earlier

when we were learning
how to read our files.

So to do this, let's just do a
chunk size and we'll set this

equal to 4,096.

Now you can choose
different sizes,

but this is the one
that I'm choosing here.

So now let's do an rf_chunk.

And we're just going to read in
a chunk of our read file here.

So I'll say rf.read and I'll
pass in this chunk size.

So now we're reading that much
data from our original picture.

So now let's create a loop
that will write these chunks

to our copy until
there's nothing left

to read from the original.

And if you remember
from earlier to do this,

we can do a while loop.

And while the length of this
chunk here is greater than 0,

then we want to take our copy
file and write that chunk to it.

So I'm going to write
this chunk to our copy.

And then to keep this from
being an infinite loop,

I have to read in
the next chunk size.

So I'll paste that in there
to read in the next chunk

from the original file.

So now if I come up
here and I delete

this copy that we just created.

So I'm going to delete that.

And now I'm going to go
ahead and rerun it using

the code that we just wrote.

And you can see that it
made that copy there.

And if I go back over to
finder and open up that copy,

then you can see that it made
an exact copy of our original.

So I think that's going
to do it for this video.

There's a lot more that we
could look at with files,

and I'll plan on
putting together

some tutorials in
the near future

for how to work with temporary
files, in-memory files

and things like that.

But I hope that this
was a good introduction

into working with files and
some of the useful things

that we can do with them.

Now, if you have
any questions, then

feel free to ask in the
comment section below

and I'll do my best
to answer those.

Be sure to subscribe
for future videos.

And Thank you all for watching.