I might have stumbled into something that appears to be a bug with how rabbitmqadmin handles (or in fact, doesn't handle) unicode.
If I have a queue that has unicode characters in queue name the rabbitmqadmin command fails with UnicodeEncodeErrors.
I'm running RabbitMQ 3.6.6 on Mac, installed via Homebrew. Both the admin command and server are of same version.
Steps how to reproduce the queue name issue:
Create a queue with unicode in it: 'rabbitmqadmin declare queue name=ööö'.
List queues: 'rabbitmqadmin list queues'
Output:
Traceback (most recent call last):
File "/usr/local/sbin/rabbitmqadmin", line 1007, in <module>
main()
File "/usr/local/sbin/rabbitmqadmin", line 413, in main
method()
File "/usr/local/sbin/rabbitmqadmin", line 588, in invoke_list
format_list(self.get(uri), cols, obj_info, self.options)
File "/usr/local/sbin/rabbitmqadmin", line 705, in format_list
formatter_instance.display(json_list)
File "/usr/local/sbin/rabbitmqadmin", line 716, in display
(columns, table) = self.list_to_table(json.loads(json_list), depth)
File "/usr/local/sbin/rabbitmqadmin", line 770, in list_to_table
add('', 1, item, add_to_row)
File "/usr/local/sbin/rabbitmqadmin", line 749, in add
fun(column, subitem)
File "/usr/local/sbin/rabbitmqadmin", line 756, in add_to_row
row[column_ix[col]] = str(val)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-13: ordinal not in range(128)
I have similar issues with 'rabbitmqadmin get queue=' when the message payload contains unicode character(s).
I think you are right, you can use python3 , check this answer it explains why with python3 works.
EDIT
1 - I filed an issue
2 - Fixed to the version: 3.6.7 see the PR
Related
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte while I tried to start a Flask Server.
The following is the code
from flask import Flask
app = Flask(__name__)
app.run(debug=True, port=5000)
This generates the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/nero/.local/lib/python3.10/site-packages/flask/app.py", line 1142, in run
cli.load_dotenv()
File "/home/nero/.local/lib/python3.10/site-packages/flask/cli.py", line 709, in load_dotenv
dotenv.load_dotenv(path, encoding="utf-8")
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 332, in load_dotenv
return dotenv.set_as_environment_variables()
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 90, in set_as_environment_variables
for k, v in self.dict().items():
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 74, in dict
self._dict = OrderedDict(resolve_variables(raw_values, override=self.override))
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 222, in resolve_variables
for (name, value) in values:
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 82, in parse
for mapping in with_warn_for_invalid_lines(parse_stream(stream)):
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 24, in with_warn_for_invalid_lines
for mapping in mappings:
File "/usr/lib/python3/dist-packages/dotenv/parser.py", line 180, in parse_stream
reader = Reader(stream)
File "/usr/lib/python3/dist-packages/dotenv/parser.py", line 71, in __init__
self.string = stream.read()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte
This is a bare code and it should not generate any errors but it does,
Attaching a screenshot for reference
My Environment settings are
Python 3.10.6
Ubuntu 22.04 - Linux [Tested on a Windows machine also]
Flask 2.2.2
Thanks in Advance
NB :
This is not a platform specific issue, Tried on Linux and Windows. Tried in python shell and tried execting as python file.
This not even related to crypto issues, There could be other questions with same heading but they aren not related to Flask
I tried to run a flask server with default configurations.
Expecting to run Flask Server
If you are here with the same issue Here is the solution
I assumed it had nothing to do with the code I wrote, as it is barely anything to make errors.
Reading through the Error traceback, I saw it is related to dotenv which in turn is a python package that deals with environment variables and secrets.
Creating a bare .env file in root solved the problem
The information I got as follows.
Traceback (most recent call last):
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\cocos.py", line 983, in
run_plugin(command, argv, plugins)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\cocos.py", line 875, in run_plugin
plugin.run(argv, dependencies_objects)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\plugins\plugin_new\project_new.py", line 258, in run
self.parse_args(argv)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\plugins\plugin_new\project_new.py", line 104, in parse_args
description=self.__class__.brief_description())
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\plugins\plugin_new\project_new.py", line 43, in brief_description
return MultiLanguage.get_string('NEW_BRIEF')
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\MultiLanguage.py", line 52, in get_string
fmt = cls.get_instance().get_current_string(key)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\MultiLanguage.py", line 158, in get_current_string
ret = ret.encode(self.encoding)
File "C:\CocosCreator\resources\utils\Python27\lib\encodings\cp1252.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-8: character maps to <undefined>
any one knows why this happened?
I do not know python, but the problem seems solved afterI made the modification to the line 158 of MultiLanguage.py(in the get_current_string method):
ret = ret.encode(self.encoding)
to the following line:
ret = ret.encode("utf8") #self.encoding
Hope this helps people who encounter the same problem.
Take a look at the versions of NDK and SDK, NDK. The recommended version is R17 - R19
My final purpose is to convert a running Python project to Jython interpreter because some java API's are going to be added.
Details:
The latest Jython is 2.7
The project I have is runnable with Python 3.5
So I took the following approach:
First thing was to convert my project to Python 2.7 utilizing the future module and pasteurize executable.
This step was done successfully.
Second thing is to convert the Python 2.7 project to a Jython 2.7 Project.
Switching the interpreter at Eclipse mars, the following error was indicated:
console: Failed to install '': java.nio.charset.UnsupportedCharsetException: cp0.
To workaround it, the solution from this post was utilized by passing the encoding argument -Dpython.console.encoding=UTF-8 to the java VM according to figure below:
Thumbs were up when trying to run again. But unfortunately here, the error shown below just appeared. As I don't plan to change any imported module if there's no need to do so, I decided to ask help here.
pydev debugger: starting (pid: 4216)
Traceback (most recent call last):
File "C:\Users\souzadan\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\pydevd.py", line 2364, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Users\souzadan\.p2\pool\plugins\org.python.pydev_4.4.0.201510052309\pysrc\pydevd.py", line 1784, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Users\souzadan\FirmwareDevTools\Workspaces\Eclipse\aPythonWorkspace\aPythonProject\aPythonFolder\aPythonFile.py", line 7, in <module>
standard_library.install_aliases()
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\standard_library\__init__.py", line 465, in install_aliases
from future.backports.urllib import request
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\backports\urllib\request.py", line 96, in <module>
from future.backports import email
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\backports\email\__init__.py", line 16, in <module>
from future.utils import surrogateescape
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\utils\surrogateescape.py", line 167, in <module>
FS_ENCODING = 'ascii'; fn = b('[abc\xff]'); encoded = u('[abc\udcff]')
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\utils\surrogateescape.py", line 25, in u
return text.decode('unicode_escape')
UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position 4-10: illegal Unicode character
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\threading.py", line 297, in _MainThread__exitfunc
t.join()
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\threading.py", line 128, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
Error in sys.exitfunc:
Traceback (most recent call last):
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\threading.py", line 297, in _MainThread__exitfunc
t.join()
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\threading.py", line 128, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
Running Jython through the command line results in a shorter error log:
Traceback (most recent call last):
File "somePythonCode.py", line 7, in <module>
standard_library.install_aliases()
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\standard_library\__init__.py", line 465, in install_aliases
from future.backports.urllib import request
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\backports\urllib\request.py", line 96, in <module>
from future.backports import email
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\backports\email\__init__.py", line 16, in <module>
from future.utils import surrogateescape
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\utils\surrogateescape.py", line 167, in <module>
FS_ENCODING = 'ascii'; fn = b('[abc\xff]'); encoded = u('[abc\udcff]')
File "C:\Users\souzadan\FirmwareDevTools\Compilers\Jython2.7.0\Lib\site-packages\future-0.15.2-py2.7.egg\future\utils\surrogateescape.py", line 25, in u
return text.decode('unicode_escape')
UnicodeDecodeError: 'unicodeescape' codec can't decode bytes in position 4-10: illegal Unicode character
Does anybody have a clue of how to solve this error in the most elegant manner?
A bug was created at Jython.org with a critical severity because many people are utilizing already the latest Python modules and interpreter. And they might want to add Java functionality to their code. So they would basically have to take same path as I did. Backporting the project to Python 2.7 then to Jython 2.7 subsequentely.
At python-future a feature request bug was recorded too.
Relying on py4j, as Roland Smith suggests, would be a good approach for porting your app to a JVM environment.
TL, DR: A python file encounters the UnicodeDecodeError when ran in Spyder, but works when I ran it in command line.
I downloaded a Python module that wraps a C implementation of a suffix tree. After built, it was ran in Spyder but I got the following error:
runfile('F:/src/suffix_tree-2.1/build/lib.win32-2.7/suffix_tree.py',
wdir='F:/src/suffix_tree-2.1/build/lib.win32-2.7')
Traceback (most recent call last):
File
"G:\IDE\python\Anaconda\lib\site-packages\IPython\core\interactiveshell.py",
line 3052, in run_code
self.showtraceback()
File
"G:\IDE\python\Anaconda\lib\site-packages\IPython\core\interactiveshell.py",
line 1851, in showtraceback
value, tb, tb_offset=tb_offset)
File
"G:\IDE\python\Anaconda\lib\site-packages\IPython\core\ultratb.py",
line 1240, in structured_traceback
self, etype, value, tb, tb_offset, number_of_lines_of_context)
File
"G:\IDE\python\Anaconda\lib\site-packages\IPython\core\ultratb.py",
line 1157, in structured_traceback
self, etype, value, elist, tb_offset, number_of_lines_of_context
File
"G:\IDE\python\Anaconda\lib\site-packages\IPython\core\ultratb.py",
line 511, in structured_traceback
lines = ''.join(self._format_exception_only(etype, value))
File
"G:\IDE\python\Anaconda\lib\site-packages\IPython\core\ultratb.py",
line 623, in _format_exception_only
Colors.Normal, s))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb2 in position
20: ordinal not in range(128)
This error appears right after I import _suffix_tree.pyd, even before any other operations.
However, if I run the file in command line (cmd), it works successfully without any error.
PS: I use Windows, and my user name are ASCII characters.
You are possibly trying to open a file encoded in a different format of your system, you should check it and set Spyder using the commands below.
import sys
reload(sys)
sys.setdefaultencoding('utf8')
If this does not work, check the version of the Python kernel of the program. You can get this message trying to run an Python 3 code in an Spyder 2.
I was using rdflib version 3.2.3 and everything was working fine. After upgrading to 4.0.1 I started getting the error:
RDFa parsing Error! 'ascii' codec can't decode byte 0xc3 in position 5454: ordinal not in range(128)
I tried various way to make this work but so far have not succeeded. Below are my attempts.
In each case I:
from rdflib import Graph
First attempt:
>>> lg =Graph()
>>> len(lg.parse('http://creativecommons.org/licenses/by/3.0/'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/graph.py", line 1002, in parse
parser.parse(source, self, **args)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/plugins/parsers/structureddata.py", line 268, in parse
vocab_cache=vocab_cache)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/plugins/parsers/structureddata.py", line 148, in _process
_check_error(processor_graph)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/plugins/parsers/structureddata.py", line 57, in _check_error
raise Exception("RDFa parsing Error! %s" % msg)
Exception: RDFa parsing Error! 'ascii' codec can't decode byte 0xc3 in position 4801: ordinal not in range(128)
Second attempt:
>>> lg =Graph()
>>> len(lg.parse('http://creativecommons.org/licenses/by/3.0/rdf'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/graph.py", line 1002, in parse
parser.parse(source, self, **args)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/plugins/parsers/rdfxml.py", line 570, in parse
self._parser.parse(source)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.7/xml/sax/xmlreader.py", line 123, in parse
self.feed(buffer)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 207, in feed
self._parser.Parse(data, isFinal)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 349, in end_element_ns
self._cont_handler.endElementNS(pair, None)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/plugins/parsers/rdfxml.py", line 160, in endElementNS
self.current.end(name, qname)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/plugins/parsers/rdfxml.py", line 461, in property_element_end
current.data, literalLang, current.datatype)
File "/home/alex/Projects/RDF/rdfEnv/local/lib/python2.7/site-packages/rdflib/term.py", line 541, in __new__
raise Exception("'%s' is not a valid language tag!"%lang)
Exception: 'i18n' is not a valid language tag!
Third attempt: gives no errors but also does not give any results
>>> lg =Graph()
>>> len(lg.parse('http://creativecommons.org/licenses/by/3.0/rdf', format='rdfa'))
0
So someone please tell me what I am dong wrong! :)
As Graham replied on the rdflib mailinglist, there is a html5lib problem - we will pin it correctly for python 2 for the next release, but for now just do:
pip install html5lib==0.95
The second problem is a problem in the data from creative commons, "i18n" really isn't a valid language tag according to rfc5646. I added the check, but in retrospect it seems to strict to raise an exception. I guess I'll change it to a warning.