I'm building a software to run on a Bluehost server, and off the server it writes and reads files fine, but on the server it just builds empty files and won't read either, this is a sample of how my code works:
File = open(os.getcwd() + '/file.dat', 'wb')
File.write('data')
File.close()
It works of the server fine, but not on the server. It creates the file but won't write data to it. What could be the problem?
Are you checking the right file?
Try this and see if the print out is what you expect.
file_path = os.path.join(os.path.dirname(os.getcwd()), '/file.dat')
print file_path
Related
I'm on a Windows PC and I'm trying to download files from an FTP. The files download fine, but the only issue when I open them up in Notepad is that it's displayed with a Unix (LF). I've tried a couple of different fixes to be able to get it to be a Windows (CRLF), but nothing is working. The file is a UTF-16-LE encoded file.
Here are two sources I looked at two fix this, but nothing:
How to correctly download files using ftplib so line breaks are added for windows
https://effbot.org/librarybook/ftplib.htm
My code is currently as follows:
def downloadFiles(self, files, localFolder):
with FTP(host=self.host, user=self.username, passwd=self.password) as ftp:
ftp.cwd(self.root)
for file in files:
with open(os.path.join(localFolder, file.fileName), 'w', newline=None) as f:
ftp.retrlines(f'RETR {file.fileName}', lambda line, file=f: file.write(line+'\n'))
I've tried the line+'\r\n, but it just adds an extra line space instead.
Anyone have any ideas of how to fix this?
If anyone has the issue in the future with a utf-16 file you just need to set the ftp encoding to utf-16. I was looking for an encoding option at the file level, but apparently you need to set it at the connection level.
with FTP(host=self.host, user=self.username, passwd=self.password) as ftp:
ftp.encoding = 'utf-16'
ftp.cwd(self.root)
for file in files:
with open(os.path.join(localFolder, file.fileName), 'w', encoding='utf-16') as f:
ftp.retrlines(f'RETR {file.fileName}', lambda line: f.write(line + '\n'))
There is a simple command line utility unix2dos.
You can use use unix2dos utility on the files after ftp.
Also if you are text editing the file, use Notepad++ .
With Notepad++ you can manage the file's newline format and its encoding as well.
I have a list of dicts in Python and want to upload it as a json-file to Azure File Storage. When I print the list locally the linebreaks exist. After uploading and manually checking the file on Azure File Storage I noticed that the linebreaks were non existent.
list_of_dicts = my_json_dicts
transformed_dict_str = '\n'.join([json.dumps(x) for x in list_of_dicts])
# print(transformed_dict_str) gives me the "dicts"/lines separated by linebreaks.
service.create_file_from_text(share_name, file_path, file_name.json, transformed_dict_str, encoding='utf-8')
Can anyone tell me why the uploaded file (when i open it in notepad after downloading manually via the browser interface of Azure) does not contain any linebreaks?
Edit:
When I write the string to a local path with the following code, the linebreaks still exist. So it must happen during the create_file_from_text function?
file = open("myjson.json", "w")
file.write(transformed_dict_str)
file.close()
Please use '\r\n' instead of '\n' in your code.
I can reproduce your issue when use '\n', but works fine using '\r\n' (in notepad, there is linebreaks).
I am trying to create a script which will download a ZIP-file and extract it.
I am using Python 2.7 on Windows Server 2016.
I created a download script looking like this:
ftp = FTP()
ftp.connect("***")
ftp.login("***","***")
ftp.cwd(ftppath)
ftp.retrbinary("RETR " + filename ,open(tempfile, 'wb').write)
ftp.quit()
And a zip extraction script:
zip_ref = zipfile.ZipFile(tempfile, 'r')
zip_ref.extractall(localpath)
zip_ref.close()
These work independently. Meaning: If i run the extraction script on my test ZIP-file it will extract the file. Also if i run the FTP script from my server, it will download the file.
However! If i run the scripts together, meaning i download the file from my FTP server and then extract it, it will return an error: "file is not a Zip file".
Anyone who knows why this happens?
I have checked the following:
Correct folder
Downloading the zip-file, extracting it and recompressing it (then the script will extract it)
EDIT
I have been reading about IO bytes and the like, however without any luck on implementing it.
probably because of this bad practice one-liner:
ftp.retrbinary("RETR " + filename ,open(tempfile, 'wb').write)
open(tempfile, 'wb').write doesn't give any guarantee as to when the file is closed. You don't store the handle returned by open anywhere so you cannot decide when to close the file (and ensure full disk write).
So the last part of the file could just be not written to disk yet when trying to open it in read mode. And chaining download + unzip can trigger the bug (when 2 separate executions leave the time to flush & close the file)
Better use a context manager like this:
with open(tempfile, 'wb') as f:
ftp.retrbinary("RETR " + filename ,f.write)
so the file is flushed & closed when exiting the with block (of course, perform the file read operations outside this block).
I have a python script that makes a .xml file. I am running this script on my server. And it runs fine and at the end of the script I put the generated file on my server like this.
output_path = "/path_to_my_loication/"
ofname = "name_of_file.xml"
output_file = output_path + ofname
open(output_file, "w").write(str(BeautifulSoup(get_xml(menu_url_list[0][1]))))
and is just part of making the file, pretty sure its irrelevant to this. So run the script on my server and it outputs my file, and I can see it, but the url is not correct instead of it being.
myserver/path_to_my_loication/name_of_file.xml
it is
myserver/path_to_my_loication/%0d%0dname_of_file.xml
This is being added %0d%0d before the file name I am not sure how to fix this, and when I print my file name just before I write the file, the characters are not there? How can I get rid of this/ why does this happen.
When I just run the script on my laptop the file name does not contain theses characters. Is it a server side issues that I should contact my serve provider about?
Thanks for the help.
Okay AIG cause when I printed for my server I saw this, I though enter was just some weird thing
How can I get rid of the enters?
Here is full code
https://gist.github.com/spennyf/de3349252695cecd4e6c
I am trying to automate the download of multiple files from an ftp source. These will span multiple years, dates, and from multiple sites that collected the data. Right now, I'm trying to make the basic download work. I can download a single file, but multiple files fail. I know when doing it manually, we would get to the directory, then
$>prompt
$>mget *.*
I have the following code as a first run at this...
import ftplib, subprocess
session = ftplib.FTP(host,user,password)
session.cwd(path)
subprocess.call("prompt")
files = session.nlst()
for f in files:
print f
session.retrbinary(("RETR" + f), open(f, 'wb').write)
session.quit()
Without the subprocess.call, the code pulls the first file, then errors out saying "command not understood." My assumption is that this is the box promptingg, since it does that if being downloaded manually. That's why I'm assuming I need the subprocess.call("prompt") command in there, as I would if handling this manually. However, when I have the subprocess added, it gives me an error that "The system cannot find the file specified" so that doesn't work, either. This error comes out of the subprocess.py module.
I guess I should post this here. Thank you to Greg Hewgill in the comments for the answer. I just needed a space after "Retr" in the line
session.retrbinary(("RETR " + f), open(f, 'wb').write)