I am tying to write a binary data to a zip file.
The below works but if I try to add a .zip as a file extension to "check" in the variable x nothing is written to the file. I am stuck manually adding .zip
urla = "some url"
tok = "some token"
pp = {"token": tok}
t = requests.get(urla, params=pp)
b = t.content
x = r"C:\temp" + "\check"
z = 'C:\temp\checks.zip'
with open(x, "wb") as work:
work.write(b)
In order to have the correct extension appended to the file I attempted to use the module ZipFile
with ZipFile(x, "wb") as work:
work.write(b)
but get a RuntimeError:
RuntimeError: ZipFile() requires mode "r", "w", or "a"
If I remove the b flag an empty zipfile is created and I get a TypeError:
TypeError: must be encoded string without NULL bytes, not str
I also tried but it creates a corrupted zipfile.
os.rename(x, z )
How do you write binary data to a zip file.
I converted a zip file into binary data and was able to regenerate the zip file in the following way:
bin_data=b"\x0\x12" #Whatever binary data you have store in a variable
binary_file_path = 'file.zip' #Name for new zip file you want to regenerate
with open(binary_file_path, 'wb') as f:
f.write(bin_data)
Use the writestr method.
import zipfile
z = zipfile.ZipFile(path, 'w')
z.writestr(filename, bytes)
z.close()
zipfile.ZipFile.writestr
You don't write the data directly to the zip file. You write it to a file, then you write the filepath to the zip file.
binary_file_path = '/path/to/binary/file.ext'
with open(binary_file_path, 'wb') as f:
f.write('BINARYDATA')
zip_file_path = '/path/to/zip/file.zip'
with ZipFile(zip_file_path, 'w') as zip_file:
zip_file.write(binary_file_path)
Related
I wrote a function that copies the content of one file to a new file.
the function gets 2 parameters:
the directory of the copied file
the directory of the new file.
When I try to print the content of the copied file, I get the content of the file (which is what I want), But when I try to do the same thing with the new file, I get the number of characters inside the file (14 in this case).
I don't understand why do I get 2 different outputs with the same (at list as per my understanding) lines of code.
Would be happy to get some help, thank you!
Here's my code:
# creating the file that's going to be copied:
with open(source_link, 'w') as f:
f.write('Copy this file')
# the function:
def copy_file_content(source, destination):
# getting the content of the copied file:
f1 = open(source, 'r')
copied_file = f1.read()
f1.close()
# putting the content of the copied file in the new file:
f2 = open(destination, 'w')
new_file = f2.write(copied_file)
f2.close
# print old file:
print(copied_file)
print('***')
# print new file:
print(new_file)
copy_file_content(source = source_link, destination = dest_link)
Output:
Copy this file
***
14
As referenced in Python documentation:
f.write(string) writes the contents of string to the file, returning the number of characters written.
Opposed to f.read(), which returns file contents.
If you want to read contents of copied_file, you will need to open it again in read mode:
with open(destination, 'r') as f:
new_file = f.read(copied_file)
.read() returns the file contents, which is why when copied_file is set to f1.read(), you can print the contents out. However, .write() performs the write operation on the file and then returns the number of characters written.
Therefore new_file contains the number of characters written. Rather than setting the value of new_file to f2.write(), you must open the new file again in read mode, and then perform file.read()
def copy_file_content(source, destination):
# getting the content of the copied file:
with open(source, 'r') as f1:
copied_file = f1.read()
# putting the content of the copied file in the new file:
with open(destination, 'w') as f2:
f2.write(copied_file)
with open(destination, "r") as f2_read:
new_file = f2_read.read()
# print old file:
print(copied_file)
print('***')
# print new file:
print(new_file)
copy_file_content(source = source_link, destination = dest_link)
I'm working with TSV file (here below my_file) and trying to write it down to another temp file with a random ID (here below my_temp_file)and this is what I wrote:
def temp_generator():
while True:
my_string = 'tmp' + str(random.randint(1,1000000000))
if not os.path.exists(my_string):
return my_string
randomID = temp_generator()
my_temp_file = open('mytemp_'+randomID + '.tsv', 'w')
with open(my_file, 'r+') as mf:
for line in mf:
my_temp_file.write(line)
my_temp_file.close()
mf.close()
The output is something like:
mytemp_1283189.tsv
Now I'd like to work with my_temp_file.tsv in order to modify its content and rename it but if I try to open it with:
with open (my_temp_file.tsv, 'r') as mtf:
data = mtf.read()
print(data)
This is what I obtain:
TypeError: expected str, bytes or os.PathLike object, not _io.TextIOWrapper
What can I do?
Issue
The pattern handle = open(path)
is opening a file at path from path and returns the handle assigned to handle. You can use handle to .write, .read, or .close. But you can not open it again or use it as input to open - which expects a Path-like object, e.g. a filename.
Fixed
def temp_generator():
while True:
my_string = 'tmp' + str(random.randint(1,1000000000))
if not os.path.exists(my_string):
return my_string
randomID = temp_generator()
# copy from input (my_file) to output, a random temp file (my_temp_file)
my_temp_file = 'mytemp_' + randomID + '.tsv'
with open(my_temp_file, 'w') as mtf:
with open(my_file, 'r+') as mf: # my_file is supposed to be a Path-like object
for line in mf:
mtf.write(line)
# since with..open used, no close needed (auto-close!)
# modify output (content and rename the file)
# remember: my_temp_file is holding a Path or filename
with open(my_temp_file, 'r') as mtf: # open the file again
data = mtf.read()
print(data)
See also:
[Solved] Python TypeError: expected str, bytes or os.PathLike object, not _io.TextIOWrapper
Python documentation about TextIOWrapper: io — Core tools for working with streams
Your error is here, assuming you are actually using open(my_temp_file
my_temp_file = open('mytemp_'+randomID + '.tsv', 'w')
with open(my_file, 'r+') as mf:
You've already opened the file, so you shouldn't open it again using the file handle as the parameter. You should prefer only using the with way of opening files, too
For example
my_temp_file = 'mytemp_'+randomID + '.tsv'
with open(my_temp_file, 'r+') as mf:
Even then, if you're going to eventually rename the file, just make it the name you want from the start
I have a file called usernames.py that may contain a list or does exist at all:
usernames.py
['user1', 'user2', 'user3']
In Python I now want to read this file if it exists and append to the list a new user or create a list with that user i.e. ['user3']
This is what I have tried:
with open(path + 'usernames.py', 'w+') as file:
file_string = host_file.read()
file_string.append(instance)
file.write(file_string)
This gives me an error unresolved 'append'. How can I achieve this? Python does not know it is a list and if the file does not exist even worst as I have nothing to convert to a list.
Try this:
import os
filename = 'data'
if os.path.isfile(filename):
with open(filename, 'r') as f:
l = eval(f.readline())
else:
l = []
l.append(instance)
with open(filename, 'w') as f:
f.write(str(l))
BUT this is quite unsafe if you don't know where the file is from as it could include any code to do anything!
It would be better not to use a python file for persistence -- what happens if someone slips you a usernames.py that has exploit code in it? Consider a csv file or a pickle, or just a text file with one user per line.
That said, if you don't open it as a python file, something like this should work:
from os.path import join
with open( join(path, 'usernames.py'), 'r+') as file:
file_string = file.read()
file_string = file_string.strip().strip('[').strip(']')
file_data = [ name.strip().strip('"').strip("'") for name in file_string.split(',' )]
file_data.append( instance )
file.fseek(0)
file.write(str(file_data))
If usernames contain commas or end in quotes, you have to be more careful.
I am using a script to strip exif data from uploaded JPGs in Python, before writing them to disk. I'm using Flask, and the file is brought in through requests
file = request.files['file']
strip the exif data, and then save it
f = open(file)
image = f.read()
f.close()
outputimage = stripExif(image)
f = ('output.jpg', 'w')
f.write(outputimage)
f.close()
f.save(os.path.join(app.config['IMAGE_FOLDER'], filename))
Open isn't working because it only takes a string as an argument, and if I try to just set f=file, it throws an error about tuple objects not having a write attribute. How can I pass the current file into this function before it is read?
file is a FileStorage, described in http://werkzeug.pocoo.org/docs/datastructures/#werkzeug.datastructures.FileStorage
As the doc says, stream represents the stream of data for this file, usually under the form of a pointer to a temporary file, and most function are proxied.
You probably can do something like:
file = request.files['file']
image = file.read()
outputimage = stripExif(image)
f = open(os.path.join(app.config['IMAGE_FOLDER'], 'output.jpg'), 'w')
f.write(outputimage)
f.close()
Try the io package, which has a BufferedReader(), ala:
import io
f = io.BufferedReader(request.files['file'])
...
file = request.files['file']
image = stripExif(file.read())
file.close()
filename = 'whatever' # maybe you want to use request.files['file'].filename
dest_path = os.path.join(app.config['IMAGE_FOLDER'], filename)
with open(dest_path, 'wb') as f:
f.write(image)
The computer is toying with me, I know it!
I am creating a zip folder in Python. The individual files are generated in memory and then the whole thing is zipped and saved to a file. I am allowed to add 9 files to the zip. I am allowed to add 11 files to the zip. But 10, no, not 10 files. The zip file IS saved to my computer, but I'm not allowed to open it; Windows says that the compressed zipped folder is invalid.
I use the code below, which I got from another stackoverflow question. It appends 10 files and saves the zipped folder. When I click on the folder, I cannot extract it. BUT, remove one of the appends() and it's fine. Or, add another append and it works!
What am I missing here? How can I make this work every time?
imz = InMemoryZip()
imz.append("1a.txt", "a").append("2a.txt", "a").append("3a.txt", "a").append("4a.txt", "a").append("5a.txt", "a").append("6a.txt", "a").append("7a.txt", "a").append("8a.txt", "a").append("9a.txt", "a").append("10a.txt", "a")
imz.writetofile("C:/path/test.zip")
import zipfile
import StringIO
class InMemoryZip(object):
def __init__(self):
# Create the in-memory file-like object
self.in_memory_zip = StringIO.StringIO()
def append(self, filename_in_zip, file_contents):
'''Appends a file with name filename_in_zip and contents of
file_contents to the in-memory zip.'''
# Get a handle to the in-memory zip in append mode
zf = zipfile.ZipFile(self.in_memory_zip, "a", zipfile.ZIP_DEFLATED, False)
# Write the file to the in-memory zip
zf.writestr(filename_in_zip, file_contents)
# Mark the files as having been created on Windows so that
# Unix permissions are not inferred as 0000
for zfile in zf.filelist:
zfile.create_system = 0
return self
def read(self):
'''Returns a string with the contents of the in-memory zip.'''
self.in_memory_zip.seek(0)
return self.in_memory_zip.read()
def writetofile(self, filename):
'''Writes the in-memory zip to a file.'''
f = file(filename, "w")
f.write(self.read())
f.close()
You should use the 'wb' mode when creating the file you are saving to the file system. This will ensure that the file is written in binary.
Otherwise, any time a newline (\n) character happens to be encountered in the zip file python will replace it to match the windows line ending (\r\n). The reason 10 files is a problem is that 10 happens to be the code for \n.
So your write function should look like this:
def writetofile(self, filename):
'''Writes the in-memory zip to a file.'''
f = file(filename, 'wb')
f.write(self.read())
f.close()
This should fix your problem and work for the files in your example. Although, in your case you might find it easier to write the zip file directly to the file system like this code which includes some of the comments from above:
import StringIO
import zipfile
class ZipCreator:
buffer = None
def __init__(self, fileName=None):
if fileName:
self.zipFile = zipfile.ZipFile(fileName, 'w', zipfile.ZIP_DEFLATED, False)
return
self.buffer = StringIO.StringIO()
self.zipFile = zipfile.ZipFile(self.buffer, 'w', zipfile.ZIP_DEFLATED, False)
def addToZipFromFileSystem(self, filePath, filenameInZip):
self.zipFile.write(filePath, filenameInZip)
def addToZipFromMemory(self, filenameInZip, fileContents):
self.zipFile.writestr(filenameInZip, fileContents)
for zipFile in self.zipFile.filelist:
zipFile.create_system = 0
def write(self, fileName):
if not self.buffer: # If the buffer was not initialized the file is written by the ZipFile
self.zipFile.close()
return
f = file(fileName, 'wb')
f.write(self.buffer.getvalue())
f.close()
# Use File Handle
zipCreator = ZipCreator('C:/path/test.zip')
# Use Memory Buffer
# zipCreator = ZipCreator()
for i in range(1, 10):
zipCreator.addToZipFromMemory('test/%sa.txt' % i, 'a')
zipCreator.write('C:/path/test.zip')
Ideally, you would probably use separate classes for an in-memory zip and a zip that is tied to the file system from the beginning. I have also seem some issues with the in-memory zip when folders are added which are difficult to recreate and which I am still trying to track down.