I have a list of dicts in Python and want to upload it as a json-file to Azure File Storage. When I print the list locally the linebreaks exist. After uploading and manually checking the file on Azure File Storage I noticed that the linebreaks were non existent.
list_of_dicts = my_json_dicts
transformed_dict_str = '\n'.join([json.dumps(x) for x in list_of_dicts])
# print(transformed_dict_str) gives me the "dicts"/lines separated by linebreaks.
service.create_file_from_text(share_name, file_path, file_name.json, transformed_dict_str, encoding='utf-8')
Can anyone tell me why the uploaded file (when i open it in notepad after downloading manually via the browser interface of Azure) does not contain any linebreaks?
Edit:
When I write the string to a local path with the following code, the linebreaks still exist. So it must happen during the create_file_from_text function?
file = open("myjson.json", "w")
file.write(transformed_dict_str)
file.close()
Please use '\r\n' instead of '\n' in your code.
I can reproduce your issue when use '\n', but works fine using '\r\n' (in notepad, there is linebreaks).
Related
I'm working on a script in Python 3.8.6 to load .sql files into big query. We're adding some non .sql files into our repo and I want my python script to only look at sql files, so I added an if statement in my loop and now I get an error: Invalid Character in identifier.
for filename in os.listdir(self.script_dir):
if os.path.splitext(filename)[1] == '.sql':
self.logger.info(os.path.join(self.script_dir, filename))
sql = self.read_sql(os.path.join(self.script_dir, filename))
Any idea as to why this is happening? There is actually only one file in the directory that its running for, which does not have a .sql extension. The original file was a text file saved with no extension (we use it to check in empty folders), I added a .txt extension to it as well and still get the same error.
Maybe there is a zero width space somewhere, copied from some website or pdf. Try to delete the line and retype it.
I'm on a Windows PC and I'm trying to download files from an FTP. The files download fine, but the only issue when I open them up in Notepad is that it's displayed with a Unix (LF). I've tried a couple of different fixes to be able to get it to be a Windows (CRLF), but nothing is working. The file is a UTF-16-LE encoded file.
Here are two sources I looked at two fix this, but nothing:
How to correctly download files using ftplib so line breaks are added for windows
https://effbot.org/librarybook/ftplib.htm
My code is currently as follows:
def downloadFiles(self, files, localFolder):
with FTP(host=self.host, user=self.username, passwd=self.password) as ftp:
ftp.cwd(self.root)
for file in files:
with open(os.path.join(localFolder, file.fileName), 'w', newline=None) as f:
ftp.retrlines(f'RETR {file.fileName}', lambda line, file=f: file.write(line+'\n'))
I've tried the line+'\r\n, but it just adds an extra line space instead.
Anyone have any ideas of how to fix this?
If anyone has the issue in the future with a utf-16 file you just need to set the ftp encoding to utf-16. I was looking for an encoding option at the file level, but apparently you need to set it at the connection level.
with FTP(host=self.host, user=self.username, passwd=self.password) as ftp:
ftp.encoding = 'utf-16'
ftp.cwd(self.root)
for file in files:
with open(os.path.join(localFolder, file.fileName), 'w', encoding='utf-16') as f:
ftp.retrlines(f'RETR {file.fileName}', lambda line: f.write(line + '\n'))
There is a simple command line utility unix2dos.
You can use use unix2dos utility on the files after ftp.
Also if you are text editing the file, use Notepad++ .
With Notepad++ you can manage the file's newline format and its encoding as well.
I created a simple threaded python server, and I have two parameters for format, one is JSON (return string data) and the other is zip. When a user selects the format=zip as one of the input parameters, I need the server to return a zip file back to the user. How should I return a file to a user on a do_GET() for my server? Do I just return the URL where the file can be downloaded or can I send the file back to the user directly? If option two is possible, how do I do this?
Thank you
You should send the file back to the user directly, and add a Content-Type header with the correct media type, such as application/zip.
So the header could look like this:
Content-Type: application/zip
The issue was that I hadn't closed the zipfile object before I tried to return it. It appeared there was a lock on the file.
To return a zip file from a simple http python server using GET, you need to do the following:
Set the header to 'application/zip'
self.send_header("Content-type:", "application/zip")
Create the zip file using zipfile module
Using the file path (ex: c:/temp/zipfile.zip) open the file using 'rb' method to read the binary information
openObj = open( < path > , 'rb')
return the object back to the browser
openObj.close()
del openObj
self.wfile.write(openObj.read())
That's about it. Thank you all for your help.
I was trying to write variables on an html file,
using the commands
index = open('/var/www/index.html','a')
index.write('...')
index.close()
The problems that I am facing are:
how to clear previous values from html file using python commands
how to move to next line on html using python ( '\n' is not working).
As L3viathan mentioned, 'w' is used for over-writing the file.
When you use 'a', you're appending to the existing file.
http://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files
I'm downloading files in Python using ftplib and up until recently everything seemed to be working fine. I am downloading files as such:
ftpSession = ftplib.FTP(host,username,password)
ftpSession.cwd('rlmfiles')
ftpFileList = filter(lambda x: 'PEDI' in x, ftpSession.nlst())
ftpFileList.sort()
for f in ftpFileList:
tempFile = open(os.path.join(localDirectory,f),'wb')
ftpSession.retrbinary('RETR '+f,tempFile.write)
tempFile.close()
ftpSession.quit()
sys.exit(0)
Up until recently it was downloading the files I needed just fine, as expected. Now, however, My files I'm downloading are corrupted and just contain long strings of garbage ASCII. I know that it is not the files posted onto the FTP I'm pulling them from because I also have a Perl script that does this successfully from the same FTP.
If it is any additional info, here's what the debugger puts out in the command prompt when downloading a file:
Has anyone encountered any issues with corrupted file contents using retrbinary() in Python's ftplib?
I'm really stuck/frustrated and haven't come across anything related to possible corruption here. Any help is appreciated.
I just ran into this issue yesterday when I was attempting to download text files. Not sure if that is what you were doing, but since you say it has ASCII garbage in it, I assume you opened it in a text editor because it was supposed to be text.
If this is the case, the problem is that the file is a text file and you are trying to download it in binary mode.
What you want to do instead is retrieve the file in ASCII transfer mode.
tempFile = open(os.path.join(localDirectory,f),'w') # Changed 'wb' to 'w'
ftpSession.retrlines('RETR '+f,tempFile.write) # Changed retrbinary to retrlines
Unfortunately, this strips all the new-line characters out of the file. Yuck!
So then you need to add the stripped out new-line characters again:
tempFile = open(os.path.join(localDirectory,f),'w')
textLines = []
ftpSession.retrlines('RETR '+f,textLines.append)
tempFile.write('\n'.join(textLines))
This should work, but it doesn't look as nice as it could. So a little cleanup effort would get us:
temporaryFile = open(os.path.join(localDirectory, currentFile), 'w')
textLines = []
retrieveCommand = 'RETR '
ftpSession.retrlines(retrieveCommand + currentFile, textLines.append)
temporaryFile.write('\n'.join(textLines))