Maybe this question makes no sense, but I was wondering if there was a "recommended practice" on how to pass a file to a function in Python.
Should I pass the file's path or the opened file itself ?
Should I do :
def func(file):
file.write(...)
with open(file_path, 'w') as file:
func(file)
...or :
def func(file_path):
with open(file_path, 'w') as file:
file.write(...)
func(file_path)
?
Is there some reason to use one method instead of the other ?
Both ways have their advantages and disadvantaged. When a function takes an open file object, it becomes easier to use with other file-like object such s io.StringIO. On the other hand, using a with statement inside a function is very elegant. A hybrid solution would be accepting both a path (string) and a file-like object. Several libraries do that.
Passing a file like object is recommended over passing a path. This means it will be easier to reuse your function with other types of files not just ones with a path on disk, such as BytesIO https://docs.python.org/3/library/io.html#io.BytesIO.
You can still use the with statement on the file like object, you don't have to use it only when you open it.
Related
I'm trying, just for fun, to understand if I can extract the full path of my file while using the with statement (python 3.8)
I have this simple code:
with open('tmp.txt', 'r') as file:
print(os.path.basename(file))
But I keep getting an error that it's not a suitable type format.
I've been trying also with the relpath, abspath, and so on.
It says that the input should be a string, but even after casting it into string, I'm getting something that I can't manipulate.
Perhaps there isn't an actual way to extract that full path name, but I think there is. I just can't find it, yet.
You could try:
import os
with open("tmp.txt", "r") as file_handle:
print(os.path.abspath(file_handle.name))
The functions in os.path accept strings or path-like objects. You are attempting to pass in a file instead. There are lots of reasons the types aren't interchangable.
Since you opened the file for text reading, file is an instance of io.TextIOWrapper. This class is just an interface that provides text encoding and decoding for some underlying data. It is not associated with a path in general: the underlying stream can be a file on disk, but also a pipe, a network socket, or an in-memory buffer (like io.StringIO). None of the latter are associated with a path or filename in the way that you are thinking, even though you would interface with them as through normal file objects.
If your file-like is an instance of io.FileIO, it will have a name attribute to keep track of this information for you. Other sources of data will not. Since the example in your question uses FileIO, you can do
with open('tmp.txt', 'r') as file:
print(os.path.abspath(file.name))
The full file path is given by os.path.abspath.
That being said, since file objects don't generally care about file names, it is probably better for you to keep track of that info yourself, in case one day you decide to use something else as input. Python 3.8+ allows you to do this without changing your line count using the walrus operator:
with open((filename := 'tmp.txt'), 'r') as file:
print(os.path.abspath(filename))
I have a question regarding the use of with statements in python, as given below:
with open(fname) as f:
np.save(f,MyData)
If I'm not mistaken this opens the file fname in a secure manner, such that if an exception occurs the file is closed properly. then it writes MyData to the file. But what I would do is simply:
np.save(fname,MyData)
This would result in the same, MyData gets written to fname. I'm not sure I understand correctly why the former is better. I don't understand how this one-liner could keep the file "open" after it has ran the line. Therefore I also don't see how this could create issues when my code would crash afterwards.
Maybe this is a stupid/basic question, but I always thought that a cleaner code is a nicer code, so not having the extra with-loop seems just better to me.
numpy.save() handles the opening and closing in its code, however if you supply a file descriptor, it'll leave it open because it assumes you want to do something else with the file and if it closes the file it'll break the functionality for you.
Try this:
f = open(<file>)
f.close()
f.read() # boom
See also the hasattr(file, "write") ("file" as in descriptor or "handle" from file, buffer, or other IO) check, that checks if it's an object with a write() method and judging by that Numpy only assumes it's true.
However NumPy doesn't guarantee misusing its API e.g. if you create a custom structure that's a buffer and doesn't include write(), it'll be treated as a path and thus crash in the open() call.
I have a library I need to call that takes a local file path as input and runs open(local_path, 'rb'). However, I don't have a local file--I have an in memory text string. Right now I am writing that to a temp file and passing that, but it seems wasteful. Is there a better way to do this, given that I need to be able to run open(local_path, 'rb') on it?
Current code:
text = "Some text"
temp = tempfile.TemporaryFile(delete=False)
temp.write(bytes(text, 'UTF-8'))
temp.seek(0)
temp.close()
#call external lib here, passing in temp.name as the local_path input
Later, inside the lib I need to use (I can't edit this):
with open(local_path, 'rb') as content_file:
file_content = content_file.read()
Since the function you call in turn calls open() with the passed parameter, you must give it a str or a PathLike. This means you basically need a file which exists in the file system. You won't be able to pass an in-memory object like I was originally thinking.
Original answer:
I suggest looking at the io package. Specifically, StringIO provides a file-like wrapper on an in-memory string object. If you need binary, then try BytesIO.
I'm trying to read the contents of a file in a single method call.
I don't want to have to worry about opening the file, reading from the file, and then closing the file (3 method calls).
I just want the content.
In ruby, there is File.read("/path/to/file"), which returns the contents of that file and properly closes it. Is there an equivalent in Python?
You can concatenate two instructions to get the same behaviour :/. But then the file isn't properly closed.
file = open("/path/to/file","r").read()
edit:
Best option as far as I know leaves you needing 2/3 you mention. Just use the with statement so you don't have to worry about closing said file.
with open("/path/to/file","r") as file:
text = file.read()
You can use a Context Manager in Python, which is available from Python 2.5.
with open('yourfile') as f:
contents = f.read()
It will automatically, open and close the file for you. The default mode is 'r' which stands for reading.
There is no such function included with Python. It's simple enough to define one, though.
def read_whole_file(path):
with open(path) as f:
return f.read()
So i've followed the docs on this page:
http://docs.python.org/library/ftplib.html#ftplib.FTP.retrbinary
And maybe i'm confused just as to what 'retrbinary' does...i'm thinking it retrives a binary file and from there i can open it and write out to that file.
here's the line that is giving me problems...
ftp.retrbinary('RETR temp.txt',open('temp.txt','wb').write)
what i don't understand is i'd like to write out to temp.txt, so i was trying
ftp.retrbinary('RETR temp.txt',open('temp.txt','wb').write('some new txt'))
but i was getting errors, i'm able to make a FTP connection, do pwd(), cwd(), rename(), etc.
p.s. i'm trying to google this as much as possible, thanks!
It looks like the original code should have worked, if you were trying to download a file from the server. The retrbinary command accepts a function object you specify (that is, the name of the function with no () after it); it is called whenever a piece of data (a binary file) arrives. In this case, it will call the write method of the file you opened. This is slightly different than retrlines, because retrlines will assume the data is a text file, and will convert newline characters appropriately (but corrupt, say, images).
With further reading it looks like you're trying to write to a file on the server. In that case, you'll need to pass a file object (or some other object with a read method that behaves like a file) to be called by the store function:
ftp.storbinary("STOR test.txt", open("file_on_my_computer.txt", "rb"))
ftp.retrbinary takes second argument as callback function
it can be directly write method of file object i.e.open('temp.txt','wb').write
but instead you are calling write directly
you may supply your own callback and do whatever you want to do with data
def mywriter(data):
print data
ftp.retrbinary('RETR temp.txt', mywriter)