UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte while I tried to start a Flask Server.
The following is the code
from flask import Flask
app = Flask(__name__)
app.run(debug=True, port=5000)
This generates the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/nero/.local/lib/python3.10/site-packages/flask/app.py", line 1142, in run
cli.load_dotenv()
File "/home/nero/.local/lib/python3.10/site-packages/flask/cli.py", line 709, in load_dotenv
dotenv.load_dotenv(path, encoding="utf-8")
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 332, in load_dotenv
return dotenv.set_as_environment_variables()
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 90, in set_as_environment_variables
for k, v in self.dict().items():
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 74, in dict
self._dict = OrderedDict(resolve_variables(raw_values, override=self.override))
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 222, in resolve_variables
for (name, value) in values:
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 82, in parse
for mapping in with_warn_for_invalid_lines(parse_stream(stream)):
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 24, in with_warn_for_invalid_lines
for mapping in mappings:
File "/usr/lib/python3/dist-packages/dotenv/parser.py", line 180, in parse_stream
reader = Reader(stream)
File "/usr/lib/python3/dist-packages/dotenv/parser.py", line 71, in __init__
self.string = stream.read()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte
This is a bare code and it should not generate any errors but it does,
Attaching a screenshot for reference
My Environment settings are
Python 3.10.6
Ubuntu 22.04 - Linux [Tested on a Windows machine also]
Flask 2.2.2
Thanks in Advance
NB :
This is not a platform specific issue, Tried on Linux and Windows. Tried in python shell and tried execting as python file.
This not even related to crypto issues, There could be other questions with same heading but they aren not related to Flask
I tried to run a flask server with default configurations.
Expecting to run Flask Server
If you are here with the same issue Here is the solution
I assumed it had nothing to do with the code I wrote, as it is barely anything to make errors.
Reading through the Error traceback, I saw it is related to dotenv which in turn is a python package that deals with environment variables and secrets.
Creating a bare .env file in root solved the problem
Related
I wrote a script which uses easyocr for extracting texts from some images. I exposed this scripts with an API. when I make an API request to this script, it gives an error while creating easyocr instance.
the code I am getting error for
import easyocr
ocr = easyocr.Reader(['en'])
Error
Traceback (most recent call last):
File \"./gift_card_ocr/gift_card_ocr_controller.py\", line 52, in gift_card_ocr_master
detector = OCR()
File \"./gift_card_ocr/gift_card_ocr.py\", line 84, in __init__
self.reader = easyocr.Reader(['en'])
File \"/opt/miniconda3/envs/lambda_api/lib/python3.8/site-packages/easyocr/easyocr.py\", line 90, in __init__
download_and_unzip(detection_models[detector_model]['url'], detection_models[detector_model]['filename'], self.model_storage_directory, verbose)
File \"/opt/miniconda3/envs/lambda_api/lib/python3.8/site-packages/easyocr/utils.py\", line 586, in download_and_unzip
urlretrieve(url, zip_path, reporthook=reporthook)
File \"/opt/miniconda3/envs/lambda_api/lib/python3.8/urllib/request.py\", line 283, in urlretrieve
reporthook(blocknum, bs, size)
File \"/opt/miniconda3/envs/lambda_api/lib/python3.8/site-packages/easyocr/utils.py\", line 686, in progress_hook
print(f'\\r{prefix} |{bar}| {percent}% {suffix}', end='')\nUnicodeEncodeError: 'ascii' codec can't encode character '\█' in position 12: ordinal not in range(128)\n"
But when i tried the same scripts without API request on the same environment, it works fine.
I tried researching about this issue and tried
$ export PYTHONIOENCODING=utf8
but it didn't worked.
FYI
For the API i used flask with nginx. I don't thing the error has to do anything with these.
Also the easyocr version i am using 1.5.0
and python version is 3.8
I've got a problem with the command loaddata of my Django app.
When I run this command, this UnicodeDecodeError appears because of the "é" that are in my database table.
i've try with many type of database on pgAdmin with different encode type like "UTF-8", "latin-1" or "win1252" or different "character type" or "collation" settings like "C" or "french_switzerland.1252" but every time there is this error.
Here is more information about my work space.
Let me know if you need more informations, I don't give you too much, I don't want to give useless information.
I know there are other questions which look similar to mine but I don't understand the answers. There are explanations about why there is the error but not about how to solve it.
I am on Windows 10
I'm using Pycharm 2020.3.2
Pgadmin v4
the complete error :
(venv) C:\Users\Mathias\PycharmProjects\yufindProject>python manage.py loaddata ask/dumps/ask.json
Traceback (most recent call last):
File "manage.py", line 22, in <module>
main()
File "manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\__init__.py", line 401, in execute_from_command_line
utility.execute()
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\__init__.py", line 395, in execut
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\base.py", line 330, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\base.py", line 371, in execute
output = self.handle(*args, **options)
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\commands\loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\commands\loaddata.py", line 114, in loaddata
self.load_label(fixture_label)
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\management\commands\loaddata.py", line 172, in load_label
for obj in objects:
File "C:\Users\Mathias\PycharmProjects\yufindProject\venv\lib\site-packages\django\core\serializers\json.py", line 67, in Deserializer
stream_or_string = stream_or_string.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 460: invalid continuation byte
Found the solution in Django dumpdata fails on special characters
To save json data in django the TextIOWrapper is used:
The default encoding is now locale.getpreferredencoding(False) (...)
In documentation of locale.getpreferredencoding fuction we can
read:
Return the encoding used for text data, according to user preferences. User preferences are expressed differently on different
systems, and might not be available programmatically on some systems,
so this function only returns a guess.
Here I found "hacky" but working method to overwrite these
settings:
In file settings.py of your django project add these lines:
import _locale
_locale._getdefaultlocale = (lambda *args: ['en_US', 'utf8'])
I might have stumbled into something that appears to be a bug with how rabbitmqadmin handles (or in fact, doesn't handle) unicode.
If I have a queue that has unicode characters in queue name the rabbitmqadmin command fails with UnicodeEncodeErrors.
I'm running RabbitMQ 3.6.6 on Mac, installed via Homebrew. Both the admin command and server are of same version.
Steps how to reproduce the queue name issue:
Create a queue with unicode in it: 'rabbitmqadmin declare queue name=ööö'.
List queues: 'rabbitmqadmin list queues'
Output:
Traceback (most recent call last):
File "/usr/local/sbin/rabbitmqadmin", line 1007, in <module>
main()
File "/usr/local/sbin/rabbitmqadmin", line 413, in main
method()
File "/usr/local/sbin/rabbitmqadmin", line 588, in invoke_list
format_list(self.get(uri), cols, obj_info, self.options)
File "/usr/local/sbin/rabbitmqadmin", line 705, in format_list
formatter_instance.display(json_list)
File "/usr/local/sbin/rabbitmqadmin", line 716, in display
(columns, table) = self.list_to_table(json.loads(json_list), depth)
File "/usr/local/sbin/rabbitmqadmin", line 770, in list_to_table
add('', 1, item, add_to_row)
File "/usr/local/sbin/rabbitmqadmin", line 749, in add
fun(column, subitem)
File "/usr/local/sbin/rabbitmqadmin", line 756, in add_to_row
row[column_ix[col]] = str(val)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-13: ordinal not in range(128)
I have similar issues with 'rabbitmqadmin get queue=' when the message payload contains unicode character(s).
I think you are right, you can use python3 , check this answer it explains why with python3 works.
EDIT
1 - I filed an issue
2 - Fixed to the version: 3.6.7 see the PR
I am using eyed3 to add lyrics, and I had this code for it
#-*-coding:utf-8-*-
import eyed3
mp3 = eyed3.load( r'F:\Music\=NEUTRAL=\桜華結界-Perfect Cherry Blossom-\04 - 早乙女の遊戯そして流儀（原曲：東方紅魔郷 U.N.オーエンは彼女なのか？）.mp3'.decode('utf-8'))
mp3.tag.lyrics.set('哈哈测试ひびき')
mp3.tag.save()
When I run it with my IDE(pyCharm). It return me a error
Traceback (most recent call last):
File "H:/Python/Xiami-music-lyrics/test.py", line 6, in <module>
mp3.tag.save()
File "C:\Python27\lib\site-packages\eyed3\id3\tag.py", line 773, in save
self._saveV2Tag(version, encoding, max_padding)
File "C:\Python27\lib\site-packages\eyed3\id3\tag.py", line 967, in _saveV2Tag
max_padding)
File "C:\Python27\lib\site-packages\eyed3\id3\tag.py", line 881, in _render
raw_frame = f.render()
File "C:\Python27\lib\site-packages\eyed3\id3\frames.py", line 1056, in render
self.text.encode(id3EncodingToString(self.encoding)))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 0: ordinal not in range(128)
Here is a screenshot on my python console:
How can I fix my code in my IDE in order to let this work? Sorry for the bad english.
#-*-coding:utf-8-*-
import eyed3
mp3 = eyed3.load( r'F:\Music\=NEUTRAL=\桜華結界-Perfect Cherry Blossom-\04 - 早乙女の遊戯そして流儀（原曲：東方紅魔郷 U.N.オーエンは彼女なのか？）.mp3'.decode('utf-8'))
mp3.tag.lyrics.set(u'哈哈测试ひびき')
mp3.tag.save(version=eyed3.id3.ID3_DEFAULT_VERSION,encoding='utf-8')
solved the problem :-)
Getting the error below trying to install ez_install, Windows 7 64 bit machine with fresh Python 2.7. Any ideas?
Installing Setuptools
Traceback (most recent call last):
File "setup.py", line 17, in
exec(init_file.read(), command_ns)
File "", line 8, in
File "c:\users\namar\appdata\local\temp\tmp1tanvy\setuptools-2.1\setuptools\__
init__.py", line 11, in
from setuptools.extension import Extension
File "c:\users\namar\appdata\local\temp\tmp1tanvy\setuptools-2.1\setuptools\ex
tension.py", line 5, in
from setuptools.dist import _get_unpatched
File "c:\users\namar\appdata\local\temp\tmp1tanvy\setuptools-2.1\setuptools\di
st.py", line 15, in
from setuptools.compat import numeric_types, basestring
File "c:\users\namar\appdata\local\temp\tmp1tanvy\setuptools-2.1\setuptools\co
mpat.py", line 19, in
from SimpleHTTPServer import SimpleHTTPRequestHandler
File "c:\python27\lib\SimpleHTTPServer.py", line 27, in
class SimpleHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
File "c:\python27\lib\SimpleHTTPServer.py", line 208, in SimpleHTTPRequestHand
ler
mimetypes.init() # try to read system mime.types
File "c:\python27\lib\mimetypes.py", line 358, in init
db.read_windows_registry()
File "c:\python27\lib\mimetypes.py", line 258, in read_windows_registry
for subkeyname in enum_types(hkcr):
File "c:\python27\lib\mimetypes.py", line 249, in enum_types
ctype = ctype.encode(default_encoding) # omit in 3.x!
UnicodeDecodeError: 'ascii' codec can't decode byte 0xae in position 11: ordinal
not in range(128)
Something went wrong during the installation.
See the error message above.
C:\Users\namar\Downloads>cd\
C:\>cd Python27
I met the same problem, solved it by deleting non-ASCII character in window registry "HKEY_CLASSES_ROOT\MIME\Database\Content Type" and it works.The reason is
ctype = ctype.encode(default_encoding)
will throw UnicodeDecodeError when it met non-ASCII characters and the program will stop.
Check this link.(http://blog.csdn.net/hugleecool/article/details/17996993).
In C:\Python27\Lib\mimetypes.py find next string
default_encoding = sys.getdefaultencoding()
replace it with
if sys.getdefaultencoding() != 'gbk':
reload(sys)
sys.setdefaultencoding('gbk')
default_encoding = sys.getdefaultencoding()
Notice: instead 'gbk' choose your encoding.
In C:\Python27\Lib\mimetypes.py find next string
default_encoding = sys.getdefaultencoding()
replace it with
if sys.getdefaultencoding() != 'gbk':
reload(sys)
sys.setdefaultencoding('gbk')
default_encoding = sys.getdefaultencoding()
Notice: instead 'gbk' choose your encoding.
The above answer satisfies my case of unicode decode error in using google app engine local web server for php and python .
In my case, simply comment the following four line worked,
try:
ctype = ctype.encode(default_encoding) # omit in 3.x!
except UnicodeEncodeError:
pass
And replace UnicodeEncodeError to be UnicodeError worked.
For more info, you can check this question.