So i've followed the docs on this page:
http://docs.python.org/library/ftplib.html#ftplib.FTP.retrbinary
And maybe i'm confused just as to what 'retrbinary' does...i'm thinking it retrives a binary file and from there i can open it and write out to that file.
here's the line that is giving me problems...
ftp.retrbinary('RETR temp.txt',open('temp.txt','wb').write)
what i don't understand is i'd like to write out to temp.txt, so i was trying
ftp.retrbinary('RETR temp.txt',open('temp.txt','wb').write('some new txt'))
but i was getting errors, i'm able to make a FTP connection, do pwd(), cwd(), rename(), etc.
p.s. i'm trying to google this as much as possible, thanks!
It looks like the original code should have worked, if you were trying to download a file from the server. The retrbinary command accepts a function object you specify (that is, the name of the function with no () after it); it is called whenever a piece of data (a binary file) arrives. In this case, it will call the write method of the file you opened. This is slightly different than retrlines, because retrlines will assume the data is a text file, and will convert newline characters appropriately (but corrupt, say, images).
With further reading it looks like you're trying to write to a file on the server. In that case, you'll need to pass a file object (or some other object with a read method that behaves like a file) to be called by the store function:
ftp.storbinary("STOR test.txt", open("file_on_my_computer.txt", "rb"))
ftp.retrbinary takes second argument as callback function
it can be directly write method of file object i.e.open('temp.txt','wb').write
but instead you are calling write directly
you may supply your own callback and do whatever you want to do with data
def mywriter(data):
print data
ftp.retrbinary('RETR temp.txt', mywriter)
Related
I have a question regarding the use of with statements in python, as given below:
with open(fname) as f:
np.save(f,MyData)
If I'm not mistaken this opens the file fname in a secure manner, such that if an exception occurs the file is closed properly. then it writes MyData to the file. But what I would do is simply:
np.save(fname,MyData)
This would result in the same, MyData gets written to fname. I'm not sure I understand correctly why the former is better. I don't understand how this one-liner could keep the file "open" after it has ran the line. Therefore I also don't see how this could create issues when my code would crash afterwards.
Maybe this is a stupid/basic question, but I always thought that a cleaner code is a nicer code, so not having the extra with-loop seems just better to me.
numpy.save() handles the opening and closing in its code, however if you supply a file descriptor, it'll leave it open because it assumes you want to do something else with the file and if it closes the file it'll break the functionality for you.
Try this:
f = open(<file>)
f.close()
f.read() # boom
See also the hasattr(file, "write") ("file" as in descriptor or "handle" from file, buffer, or other IO) check, that checks if it's an object with a write() method and judging by that Numpy only assumes it's true.
However NumPy doesn't guarantee misusing its API e.g. if you create a custom structure that's a buffer and doesn't include write(), it'll be treated as a path and thus crash in the open() call.
I have a library I need to call that takes a local file path as input and runs open(local_path, 'rb'). However, I don't have a local file--I have an in memory text string. Right now I am writing that to a temp file and passing that, but it seems wasteful. Is there a better way to do this, given that I need to be able to run open(local_path, 'rb') on it?
Current code:
text = "Some text"
temp = tempfile.TemporaryFile(delete=False)
temp.write(bytes(text, 'UTF-8'))
temp.seek(0)
temp.close()
#call external lib here, passing in temp.name as the local_path input
Later, inside the lib I need to use (I can't edit this):
with open(local_path, 'rb') as content_file:
file_content = content_file.read()
Since the function you call in turn calls open() with the passed parameter, you must give it a str or a PathLike. This means you basically need a file which exists in the file system. You won't be able to pass an in-memory object like I was originally thinking.
Original answer:
I suggest looking at the io package. Specifically, StringIO provides a file-like wrapper on an in-memory string object. If you need binary, then try BytesIO.
I am struggling to return a file content back to the user. Have a Flask code that receives a txt file from an user, then the Python function transform() is called in order to parse the infile, both codes are doing the job.
The issue is happening when I am trying to send (return) the new file (outfile) back to the user, the Flask code for that is also working OK.
But I don´t know how to have this Python transform function() "return" that file content, have tested several options already.
Following more details:
def transform(filename):
with open(os.path.join(app.config['UPLOAD_FOLDER'],filename), "r") as infile:
with open(os.path.join(app.config['UPLOAD_FOLDER'], 'file_parsed_1st.txt'), "w") as file_parsed_1st:
p = CiscoConfParse(infile)
'''
parsing the file uploaded by the user and
generating the result in a new file(file_parsed_1st.txt)
that is working OK
'''
with open (os.path.join(app.config['UPLOAD_FOLDER'], 'file_parsed_1st.txt'), "r") as file_parsed_2nd:
with open(os.path.join(app.config['UPLOAD_FOLDER'], 'file_parsed_2nd.txt'), "w") as outfile:
'''
file_parsed_1st.txt is a temp file, then it creates a new file (file_parsed_2nd.txt)
That part is also working OK, the new file (file_parsed_2nd.txt)
has the results I want after all the parsing;
Now I want this new file(file_parsed_2nd.txt) to "return" to the user
'''
#Editing -
#Here is where I was having a hard time, and that now is Working OK
#using the follwing line:
return send_file(os.path.join(app.config['UPLOAD_FOLDER'], 'file_parsed_2nd.txt'))
You do need to use the flask.send_file() callable to produce a proper response, but need to pass in a filename or a file object that isn't already closed or about to be closed. So passing in the full path would do:
return send_file(os.path.join(app.config['UPLOAD_FOLDER'], 'file_parsed_2nd.txt'))
When you pass in a file object you cannot use the with statement, as it'll close the file object the moment you return from your view; it'll only be actually read when the response object is processed as a WSGI response, outside of your view function.
You may want to pass in a attachment_filename parameter if you want to suggest a filename to the browser to save the file as; it'll also help determine the mimetype. You also may want to specify the mimetype explicitly, using the mimetype parameter.
You could also use the flask.send_from_directory() function; it does the same but takes a filename and a directory:
return send_from_directory(app.config['UPLOAD_FOLDER'], 'file_parsed_2nd.txt')
The same caveat about a mimetype applies; for .txt the default mimetype would be text/plain. The function essentially joins the directory and filename (with flask.safe_join() which applies additional safety checks to prevent breaking out of the directory using .. constructs) and passes that on to flask.send_file().
I have use the following code to read a .txt file:
f = os.open(os.path.join(self.dirname, self.filename), os.O_RDONLY)
And when I want to output the content I use this:
os.read(f, 10);
Which means that this method reads 10 bytes from the beginning of the file on. While I need to read the content as much as it is, using some values such as -1 and so. What should I do?
You have two options:
Call os.read() repeatedly.
Open the file using the open() built-in (as opposed to os.open()), and just call f.read() with no arguments.
The second approach carries certain risk, in that you might run into memory issues if the file is very large.
I am beginner in python, and I need to use some thirdparty function which basically has one input - name of a file on a hard drive. This function parses file and then proceses it.
I am generating file contents in my code (it's CSV file which I generate from a list) and want to skip actual file creation. Is there any way I can achieve this and "hack" the thirdparty function to accept my string without creating a file?
After some googling I found StringIO, and created a file object in it, now I am stuck on passing this object to a function (again, it accepts not a file object but a file name).
It looks like you'll need to write your data to a file then pass the name of that file to the 3rd party library. You might want to consider using the tempfile module to create the file in a safe and easy way.
If it requires a filename, then you're going to have to create a file. (And that's poor design on the part of the library creators.)
You should look into the python docs for I/O, seen here:
http://docs.python.org/tutorial/inputoutput.html
Python processes files by opening them, there is no extra file "created". The open file then has a few methods which can be done on them which you can use to create the output you desire; although I'm not entirely sure I understand your wording. What I do understand, you want to open a file, do some stuff with its contents and then create a string of some kind, right? If that's correct, you're in luck, as its pretty easy to do that.
Comma Seperated Values passed into python from a file is extremely easy to parse into python-friendly formats such as lists, tuples and dictionaries.
As you've said, you want a function that you input the name of a file, the file is looked up, read and some stuff is done without the creation of extra files. Alright, so to do that, your code would look like this:
def file_open(filename):
new_dictionary = {}
f = open(/directory/filename, r) ##The second param is mode, here readable
for line in f: ##iterating through each comma seperated value
key,value = line.split(',') ##set the first entry before comma to key then val
new_dictionary[key] = value ##set the new_dictionary key to value
return new_dictionary ##spit that newly assembled dictionary back to us
f.close() ##Now close the file.
As you can see, there is no other file being created in this process. We just open the file on the hard drive, do some parsing to create our dictionary, and then return the dictionary for use. To set something to the dictionary that it outputs, just set a variable to the function. Just make sure you set the directory correctly, from the root of the hard drive.
CSV_dictionary = file_open(my_file) ##This sets CSV with all the info.
I hope this was helpful, if I'm not getting your problem, just answer and I'll try to help you.
-Joseph