I want to read a image in binary mode so that I could save it into my database, like this:
img = open("Last_Dawn.jpg")
t = img.read()
save_to_db(t)
This is working on Mac. But on Windows, what img.read() is incorrect. It's just a little out of the whole set.
So my first question is: why code above doesn't work in Windows?
And second is: is there any other way to do this?
Thanks a lot!
You need to open in binary mode:
img = open("Last_Dawn.jpg", 'rb')
You need to tell Python to open the file in binary mode:
img = open('whatever.whatever', 'rb')
See the documentation for the open function here: http://docs.python.org/library/functions.html#open
Can't say for sure but I do know that the ISO C standard doesn't distinguish between the binary and non-binary modes when calling fopen and yet Windows does.
It's likely that the Python code just uses fopen("Last_Dawn.jpg","r") under the covers (since it's written in C) and this is being opened in Windows in non-binary mode.
This will most likely convert line end characters (LF -> CRLF) and possibly others.
If you yourself specify the mode as 'rb' on your open statement, that should fix it:
img = open("Last_Dawn.jpg", "rb")
open(filename, 'rb')
Related
I'm looking to edit a Minecraft Windows 10 level.dat file in python. I've tried using the package nbt and pyanvil but get the error OSError: Not a gzipped file. If I print open("level.dat", "rb").read() I get a lot of nonsensical data. It seems like it needs to be decoded somehow, but I don't know what decoding it needs. How can I open (and ideally edit) one of these files?
To read data just do :
from nbt import nbt
nbtfile = nbt.NBTFile("level.dat", 'rb')
print(nbtfile) # Here you should get a TAG_Compound('Data')
print(nbtfile["Data"].tag_info()) # Data came from the line above
for tag in nbtfile["Data"].tags: # This loop will show us each entry
print(tag.tag_info())
As for editing :
# Writing data (changing the difficulty value
nbtfile["Data"]["Difficulty"].value = 2
print(nbtfile["Data"]["Difficulty"].tag_info())
nbtfile.write_file("level.dat")
EDIT:
It looks like Mojang doesn't use the same formatting for Java and bedrock, as bedrock's level.dat file is stored in little endian format and uses non-compressed UTF-8.
As an alternative, Amulet-Nbt is supposed to be a Python library written in Cython for reading and editing NBT files (supposedly works with Bedrock too).
Nbtlib also seems to work, as long as you set byteorder="little when loading the file.
Let me know if u need more help...
You'll have to give the path either relative to the current working directory
path/to/file.dat
Or you can use the absolute path to the file
C:user/dir/path/to/file.dat
Read the data,replace the values and then write it
# Read in the file
with open('file.dat', 'r') as file :
filedata = file.read()
# Replace the target string
filedata = filedata.replace('yuor replacement or edit')
# Write the file out again
with open('file.dat', 'w') as file:
file.write(filedata)
I'm having problems getting images to convert out of bytes/strings/etc. I can turn an image into a string, or a byte array, or use b64encode on it, but when I try decode/revert it back to an image, it never works. I've tried a lot of things, locally converting an image and then reconverting it, saving it under a different name. However, the resulting files will never actually show anything. (black on Linux, "can't display image" on windows)
My most basic b64encoding script is as follows:
import base64
def convert(image):
f = open(image)
data = f.read()
f.close()
string = base64.b64encode(data)
convertit = base64.b64decode(string)
t = open("Puppy2.jpg", "w+")
t.write(convertit)
t.close()
if __name__ == "__main__":
convert("Puppy.jpg")
I've been stuck on this for a while. I'm sure it's a simple solution, but being new to Python, it's been a bit difficult trying to sort things out.
If it helps with any insight, the end goal here is to transfer images over a network, possibly MQTT.
Any help is much appreciated. Thanks!
Edit** This is in Python 2.7.
Edit 2** Wow, you guys move fast. What a great intro to the community - thanks a lot for the quick responses and super fast results!
For python3, you need to open and write in binary mode:
def convert(image):
f = open(image,"rb")
data = f.read()
f.close()
string = base64.b64encode(data)
convert = base64.b64decode(string)
t = open("Puppy2.jpg", "wb")
t.write(convert)
t.close()
Using python 2 on linux, simply r and w should work fine. On windows you need to do the same as above.
from the docs:
On Windows, 'b' appended to the mode opens the file in binary mode, so there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows makes a distinction between text and binary files; the end-of-line characters in text files are automatically altered slightly when data is read or written. This behind-the-scenes modification to file data is fine for ASCII text files, but it’ll corrupt binary data like that in JPEG or EXE files. Be very careful to use binary mode when reading and writing such files. On Unix, it doesn’t hurt to append a 'b' to the mode, so you can use it platform-independently for all binary files.
You can also write your code a little more succinctly by using with to open your files which will automatically close them for you:
from base64 import b64encode, b64decode
def convert(image):
with open(image, "rb") as f, open("Puppy2.jpg", "wb") as t:
conv = b64decode(b64encode(f.read()))
t.write(conv)
import base64
def convert(image):
f = open(image)
data = f.read()
f.close()
return data
if __name__ == "__main__":
data = convert("Puppy2.jpg")
string = base64.b64encode(data)
convert = base64.b64decode(string)
t = open("Puppy2.jpg", "w+")
t.write(convert)
t.close()
I am trying to grab some text written in Arabic from Youtube, writting it into a file and reading it again.
The source file to grab the text has:
#!/usr/bin/python
#encoding: utf-8
in the beginning of the file.
Writing the text are done like this:
f.write(comment + '\n' )
The file contents is readable Arabic, so I assume the previous steps were correct.
But the problem appears when trying to read the contents from the file (and writing them for example into another file) like this:
in = open('data_Pass1/EG', 'rb')
out.write(in.read())
Which results in output file like this:
\xd8\xa7\xd9\x8a\xd9\x87
What is causing this?
In python 3.x
in = open('data_Pass1/EG', 'r', encoding='utf-8')
out = open('_file_name_', 'w', encoding='utf-8')
In python 2.x.
import codecs
in = codecs.open('data_Pass1/EG', 'r', encoding='utf-8')
out = codecs.open('_file_name_', 'w', encoding='utf-8')
You're opening the input file in binary ('rb') mode. Open the file to read as text ('r'). I tend to use Python 3 so the source files are UTF-8 by default, so I don't know what effect setting the encoding for .py files inside the files has on text I/O, but if necessary you may also want to use encoding='utf8' inside the calls to open() for all your file I/O, unless that doesn't work in 2.7 in which case I'm not sure what the best way to handle that in Python 2.7 would be...
As Lee Daniel Crocker suggests, you'd probably be better off just opening both input and output files in binary mode ('rb' for the input file, 'wb' for the output) if you're passing the input directly to the output without doing any textual manipulation of it. (Though going by Andy's comment, in Python 2 it's better to open text files in binary mode and do explicit encoding/decoding anyway.)
I'm making a program using the website http://placekitten.com, but I've run into a bit of a problem. Using this:
im = urllib2.urlopen(url).read()
f = open('kitten.jpeg', 'w')
f.write(im)
f.close()
The image turns out distorted with mismatched colors, like this:
http://imgur.com/zVg64Kn.jpeg
I was wondering if there was an alternative to extracting images with urllib2. If anyone could help, that would be great!
You need to open the file in binary mode:
f = open('kitten.jpeg', 'wb')
Python will otherwise translate line endings to the native platform form, a transformation that breaks binary data, as documented for the open() function:
The default is to use text mode, which may convert '\n' characters to a platform-specific representation on writing and back on reading. Thus, when opening a binary file, you should append 'b' to the mode value to open the file in binary mode, which will improve portability.
When copying data from a URL to a file, you could use shutil.copyfileob() to handle streaming efficiently:
from shutil import copyfileobj
im = urllib2.urlopen(url)
with open('kitten.jpeg', 'wb') as out:
copyfileobj(im, out)
This will read data in chunks, avoiding filling memory with large blobs of binary data. The with statement handles closing the file object for you.
Change
f = open('kitten.jpeg', 'w')
to read
f = open('kitten.jpeg', 'wb')
See http://docs.python.org/2/library/functions.html#open for more information. What's happening is that the newlines in the jpeg are getting modified in the process of saving, and opening as a binary file will prevent this.
If you're using Windows, you have to open the file in binary mode:
f = open('kitten.jpeg', 'wb')
Or more Pythonically:
import urllib2
url = 'http://placekitten.com.s3.amazonaws.com/homepage-samples/200/140.jpg'
image = urllib2.urlopen(url).read()
with open('kitten.jpg', 'wb') as handle:
handle.write(image)
This is how the code is
with open(pickle_f, 'r') as fhand:
obj = pickle.load(fhand)
This works fine on Linux systems but not on Windows. Its showing EOFError.
I have to use rb mode to make it work on Windows.. now this isn't working on Linux.
Why this is happening, and how to fix it?
Always use b mode when reading and writing pickles (open(f, 'wb') for writing, open(f, 'rb') for reading). To "fix" the file you already have, convert its newlines using dos2unix.