Develop Reference

Simple code with PIL library in Python is not workig [duplicate]

This question already has answers here:
"Unicode Error "unicodeescape" codec can't decode bytes... Cannot open text files in Python 3 [duplicate]
(10 answers)
Closed 14 days ago.
Well, first time in writing in stackoverflow.
when I depure and run this code
from PIL import Image
import os
downloadsFolder = "\Users\fersa\Downloads"
picturesFolder = "\Users\fersa\OneDrive\Imágenes\Imagenes Descargadas"
musicFolder = "\Users\fersa\Music\Musica Descargada"
if __name__ == "__main__":
for filename in os.listdir(downloadsFolder):
name, extension = os.path.splitext(downloadsFolder + filename)
if extension in [".jpg", ".jpeg", ".png"]:
picture = Image.open(downloadsFolder + filename)
picture.save(picturesFolder + "compressed_"+filename, optimize=True, quality=60)
os.remove(downloadsFolder + filename)
print(name + ": " + extension)
if extension in [".mp3"]:
os.rename(downloadsFolder + filename, musicFolder + filename)
I get this message on terminal
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-1: truncated \UXXXXXXXX escape
PS C:\Users\fersa\OneDrive\Documentos\Automatizacion Python>
but i don't know what it means
I tried chanching the files directory many times but it doesn't work

Add an r before each string to signify that it's a raw string without escape codes. Currently, every \ character is telling python to try and interpret the next few bytes as a unicode character:
from PIL import Image
import os
downloadsFolder = r"\Users\fersa\Downloads"
picturesFolder = r"\Users\fersa\OneDrive\Imágenes\Imagenes Descargadas"
musicFolder = r"\Users\fersa\Music\Musica Descargada"
if __name__ == "__main__":
for filename in os.listdir(downloadsFolder):
name, extension = os.path.splitext(downloadsFolder + filename)
if extension in [".jpg", ".jpeg", ".png"]:
picture = Image.open(downloadsFolder + filename)
picture.save(picturesFolder + "compressed_"+filename, optimize=True, quality=60)
os.remove(downloadsFolder + filename)
print(name + ": " + extension)
if extension in [".mp3"]:
os.rename(downloadsFolder + filename, musicFolder + filename)
You can read more about string prefixes in the python docs

The cause of the SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-1: truncated \UXXXXXXXX escape is the code line:
downloadsFolder = "\Users\fersa\Downloads"
in which "\U" is telling python to interpret the next 8 characters as a hexadecimal value of an Unicode code point. And because in "\Users\fes" are characters not being 0-9,A-F,a-f there is an Error which won't occur if the string would start for example with "\Uaabbaacc\somefilename" making it then harder to find out why no files can be found.
The options for a work around fixing the problem are:
usage of forward slashes instead of backslashes: "/Users/fersa/Downloads".
definition of the string as a raw string: r"\Users\fersa\Downloads" in order to avoid interpretation of \U as an escape sequence
Check out in Python documentation page the section 2.4.1. String and Bytes literals for more about escape sequences in Python string literals.

How to extract String from a Unicoded JSONObject in Python?

I'm getting the below error when I try to parse a String with Unicodes like ' symbol and Emojis, etc :
UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f33b' in position 19: ordinal not in range(128)
Sample Object:
{"user":{"name":"\u0e2a\u0e31\u0e48\u0e07\u0e14\u0e48\u0e27\u0e19 \u0e2b\u0e21\u0e14\u0e44\u0e27 \u0e40\u0e14\u0e23\u0e2a\u0e41\u0e1f\u0e0a\u0e31\u0e48\u0e19\u0e21\u0e32\u0e43\u0e2b\u0e21\u0e48 \u0e23\u0e32\u0e04\u0e32\u0e40\u0e1a\u0e32\u0e46 \u0e2a\u0e48\u0e07\u0e17\u0e31\u0e48\u0e27\u0e44\u0e17\u0e22 \u0e44\u0e14\u0e49\u0e02\u0e2d\u0e07\u0e0a\u0e31\u0e27\u0e23\u0e4c\u0e08\u0e49\u0e32 \u0e2a\u0e19\u0e43\u0e08\u0e15\u0e34\u0e14\u0e15\u0e48\u0e2d\u0e2a\u0e2d\u0e1a\u0e16\u0e32\u0e21 Is it","tag":"XYZ"}}
I'm able to extract tag value, but I'm unable to extract name value.
Here is my code:
dict = json.loads(json_data)
print('Tag - 'dict['user']['tag'])
print('Name - 'dict['user']['name'])

You can save the data in CSV file format which could also be opened using Excel. When you open a file in this way: open(filename, "w") then you can only store ASCII characters, but if you try to store Unicode data this way, you would get UnicodeEncodeError. In order for you to store Unicode data, you need to open the file with UTF-8 encoding.
mydict = json.loads(json_data) # or whatever dictionary it is...
# Open the file with UTF-8 encoding, most important step
f = open("userdata.csv", "w", encoding='utf-8')
f.write(mydict['user']['name'] + ", " + mydict['user']['tag'] + "\n")
f.close()
Feel free to change the code based on the data you have.
That's it...

Getting "TypeError: pwd: expected bytes, got str" when extracting zip file with password

I want to unzip a specific named file located in a specific directory.
The name of the file = happy.zip.
The location = C:/Users/desktop/Downloads.
I want to extract all the files to C:/Users/desktop/Downloads(the same location)
I tried:
import zipfile
import os
in_Zip = r"C:/Users/desktop/Downloads/happy.zip"
outDir = r"C:/Users/desktop/Downloads"
z = zipfile.ZipFile(in_Zip, 'r')
z.extractall(outDir, pwd='1234!')
z.close
But I got:
"TypeError: pwd: expected bytes, got str"

In Python 2: '1234!' = byte string
In Python 3: '1234!' = unicode string
Assuming you are using Python 3, you need to either use b'1234!' or encode the string to get byte string using str.encode() this is useful if you have the password saved as a string passwd = '1234!' then you can use:
z.extractall(outDir, pwd=passwd.encode())
or use byte string directly:
z.extractall(outDir, pwd=b'1234!')

Please note that this will only work if the zip file has been encrypted using the "Zip legacy encryption" option when setting up the password.

Can't unzip a folder in Python

I tried unzipping a file through Python using zipfile.extractAll but it gave BAD zip file, hence I tried this:
zipfile cant handle some type of zip data?
As mentioned in this answer, i used the code:
def fixBadZipfile(zipFile):
f = open(zipFile, 'r+b')
data = f.read()
pos = data.find('\x50\x4b\x05\x06') # End of central directory signature
if (pos > 0):
self._log("Truncating file at location " + str(pos + 22) + ".")
f.seek(pos + 22) # size of 'ZIP end of central directory record'
f.truncate()
f.close()
else:
# raise error, file is truncated enter code here
but it gave the error
Message File Name Line Position Traceback
C:\Users\aditya1.r\Desktop\Python_pyscripter\module1.py 50
main C:\Users\aditya1.r\Desktop\Python_pyscripter\module1.py 17
fixBadZipfile C:\Users\aditya1.r\Desktop\Python_pyscripter\module1.py 37
TypeError: 'str' does not support the buffer interface
I'm using Python 3.4
How can i unzip this file?

import subprocess
subprocess.Popen('unzip ' + file_name, shell = True).wait()
Hope this help you :)

You are reading the file as a bytes object but trying to find passing a string object so just simply change this line -
pos = data.find('\x50\x4b\x05\x06')
to
pos = data.find(b'\x50\x4b\x05\x06')
Note that I have casted it to a byte object by simply prepending a b.
You don't need to do this is Python 2.X but in python 3.X you need to explicitly serialize a string object to a byte object.

UnicodeDecodeError while processing filenames

I'm using Python 2.7.3 on Ubuntu 12 x64.
I have about 200,000 files in a folder on my filesystem. The file names of some of the files contain html encoded and escaped characters because the files were originally downloaded from a website. Here are examples:
Jamaica%2008%20114.jpg
thai_trip_%E8%B0%83%E6%95%B4%E5%A4%A7%E5%B0%8F%20RAY_5313.jpg
I wrote a simple Python script that goes through the folder and renames all of the files with encoded characters in the filename. The new filename is achieved by simply decoding the string that makes up the filename.
The script works for most of the files, but, for some of the files Python chokes and spits out the following error:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 11: ordinal not in range(128)
Traceback (most recent call last):
File "./download.py", line 53, in downloadGalleries
numDownloaded = downloadGallery(opener, galleryLink)
File "./download.py", line 75, in downloadGallery
filePathPrefix = getFilePath(content)
File "./download.py", line 90, in getFilePath
return cleanupString(match.group(1).strip()) + '/' + cleanupString(match.group(2).strip())
File "/home/abc/XYZ/common.py", line 22, in cleanupString
return HTMLParser.HTMLParser().unescape(string)
File "/usr/lib/python2.7/HTMLParser.py", line 472, in unescape
return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));", replaceEntities, s)
File "/usr/lib/python2.7/re.py", line 151, in sub
return _compile(pattern, flags).sub(repl, string, count)
Here is the contents of my cleanupString function:
def cleanupString(string):
string = urllib2.unquote(string)
return HTMLParser.HTMLParser().unescape(string)
And here's the snippet of code that calls the cleanupString function (this code is not the same code in the traceback above but it produces the same error):
rootFolder = sys.argv[1]
pattern = r'.*\.jpg\s*$|.*\.jpeg\s*$'
reobj = re.compile(pattern, re.IGNORECASE)
imgs = []
for root, dirs, files in os.walk(rootFolder):
for filename in files:
foundFile = os.path.join(root, filename)
if reobj.match(foundFile):
imgs.append(foundFile)
for img in imgs :
print 'Checking file: ' + img
newImg = cleanupString(img) #Code blows up here for some files
Can anyone provide me with a way to get around this error? I've already tried adding
# -*- coding: utf-8 -*-
to the top of the script but that has no effect.
Thanks.

Your filenames are byte strings that contain UTF-8 bytes representing unicode characters. The HTML parser normally works with unicode data instead of byte strings, particularly when it encounters a ampersand escape, so Python is automatically trying to decode the value for you, but it by default uses ASCII for that decoding. This fails for UTF-8 data as it contains bytes that fall outside of the ASCII range.
You need to explicitly decode your string to a unicode object:
def cleanupString(string):
string = urllib2.unquote(string).decode('utf8')
return HTMLParser.HTMLParser().unescape(string)
Your next problem will be that you now have unicode filenames, but your filesystem will need some kind of encoding to work with these filenames. You can check what that encoding is with sys.getfilesystemencoding(); use this to re-encode your filenames:
def cleanupString(string):
string = urllib2.unquote(string).decode('utf8')
return HTMLParser.HTMLParser().unescape(string).encode(sys.getfilesystemencoding())
You can read up on how Python deals with Unicode in the Unicode HOWTO.

Looks like you're bumping into this issue. I would try reversing the order you call unescape and unquote, since unquote would be adding non-ASCII characters into your filenames, although that may not fix the problem.
What is the actual filename it is choking on?