I am having an issue with an Azure Function in Python.
The following code:
test = b'\xce\x94'
print(test)
a = test.decode('utf-8')
print(a)
prints the following if I run it in Python 3.9 on my PC running Windows 10.
b'\xce\x94'
Δ
This is correct as it should result in the Delta character.
If I run the same code in an Azure Function App (also Python 3.9) it errors if I try to print it
System.Private.CoreLib: Exception while executing function: Functions.HttpTrigger. System.Private.CoreLib: Result: Failure
Exception: UnicodeEncodeError: 'charmap' codec can't encode character '\u0394' in position 0: character maps to <undefined>
Stack: File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\dispatcher.py", line 407, in _handle__invocation_request
call_result = await self._loop.run_in_executor(
File "C:\Program Files\Python39\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\dispatcher.py", line 649, in _run_sync_func
return ExtensionManager.get_sync_invocation_wrapper(context,
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\extension.py", line 215, in _raw_invocation_wrapper
result = function(**args)
File "C:\repo\Data\XXXX\XXXXXXXX\HttpTrigger\__init__.py", line 75, in main
print(a)
File "c:\Users\XXXXX\.vscode\extensions\ms-python.python-2022.14.0\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_io.py", line 40, in write
r.write(s)
File "C:\Program Files\Python39\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
or shows a
?
if I log it using Python logging library.
Can anyone help please?
I think the issue is with the e in the test it is probably thinking that it is a latin1 character and that might be why it is giving the error.
The remedy for this would be to encode this in latin1 first and then decode this to utf-8
test.encode('latin1').decode('utf-8');
here I deployed the azure function return the decoded test string
Related
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte while I tried to start a Flask Server.
The following is the code
from flask import Flask
app = Flask(__name__)
app.run(debug=True, port=5000)
This generates the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/nero/.local/lib/python3.10/site-packages/flask/app.py", line 1142, in run
cli.load_dotenv()
File "/home/nero/.local/lib/python3.10/site-packages/flask/cli.py", line 709, in load_dotenv
dotenv.load_dotenv(path, encoding="utf-8")
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 332, in load_dotenv
return dotenv.set_as_environment_variables()
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 90, in set_as_environment_variables
for k, v in self.dict().items():
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 74, in dict
self._dict = OrderedDict(resolve_variables(raw_values, override=self.override))
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 222, in resolve_variables
for (name, value) in values:
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 82, in parse
for mapping in with_warn_for_invalid_lines(parse_stream(stream)):
File "/usr/lib/python3/dist-packages/dotenv/main.py", line 24, in with_warn_for_invalid_lines
for mapping in mappings:
File "/usr/lib/python3/dist-packages/dotenv/parser.py", line 180, in parse_stream
reader = Reader(stream)
File "/usr/lib/python3/dist-packages/dotenv/parser.py", line 71, in __init__
self.string = stream.read()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte
This is a bare code and it should not generate any errors but it does,
Attaching a screenshot for reference
My Environment settings are
Python 3.10.6
Ubuntu 22.04 - Linux [Tested on a Windows machine also]
Flask 2.2.2
Thanks in Advance
NB :
This is not a platform specific issue, Tried on Linux and Windows. Tried in python shell and tried execting as python file.
This not even related to crypto issues, There could be other questions with same heading but they aren not related to Flask
I tried to run a flask server with default configurations.
Expecting to run Flask Server
If you are here with the same issue Here is the solution
I assumed it had nothing to do with the code I wrote, as it is barely anything to make errors.
Reading through the Error traceback, I saw it is related to dotenv which in turn is a python package that deals with environment variables and secrets.
Creating a bare .env file in root solved the problem
I am trying to save control identifiers to a variable.
The reason is that: i use main_dlg.print_control_identifiers(filename="control_ids.text") but i am getting this error:
Traceback (most recent call last):
File "run_tests.py", line 7, in <module>
main_dlg.print_control_identifiers(filename="control_ids.text")
File "C:\python\lib\site-packages\pywinauto\application.py", line 696, in prin
t_control_identifiers
print_identifiers([this_ctrl, ], log_func=log_func)
File "C:\python\lib\site-packages\pywinauto\application.py", line 685, in prin
t_identifiers
print_identifiers(ctrl.children(), current_depth + 1, log_func)
File "C:\python\lib\site-packages\pywinauto\application.py", line 681, in prin
t_identifiers
log_func(output)
File "C:\python\lib\site-packages\pywinauto\application.py", line 694, in log_
func
log_file.write(str(msg) + os.linesep)
File "C:\python\lib\codecs.py", line 721, in write
return self.writer.write(data)
File "C:\python\lib\codecs.py", line 377, in write
data, consumed = self.encode(object, self.errors)
File "C:\python\lib\encodings\cp1252.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 71-76: c
haracter maps to <undefined>
because the program i am trying to control has greek characters inside.
So, i decided to save the control identifiers to a variable and then save it with the right encoding to a txt file.
A simple solution i thought is:
from pywinauto.application import Application
import os
app = Application(backend="uia").start("C:/python/Lib/site-packages/QtDesigner/designer.exe")
main_dlg = app.QtDesigner
main_dlg.wait('visible')
main_dlg.print_control_identifiers()
and then:
python script_name.py > output.txt
after that in output.txt file there are the control identifiers of the program (in this example QtDesigner).
The information I got as follows.
Traceback (most recent call last):
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\cocos.py", line 983, in
run_plugin(command, argv, plugins)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\cocos.py", line 875, in run_plugin
plugin.run(argv, dependencies_objects)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\plugins\plugin_new\project_new.py", line 258, in run
self.parse_args(argv)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\plugins\plugin_new\project_new.py", line 104, in parse_args
description=self.__class__.brief_description())
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\plugins\plugin_new\project_new.py", line 43, in brief_description
return MultiLanguage.get_string('NEW_BRIEF')
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\MultiLanguage.py", line 52, in get_string
fmt = cls.get_instance().get_current_string(key)
File "C:\CocosCreator\resources\cocos2d-x\tools\cocos2d-console\bin\MultiLanguage.py", line 158, in get_current_string
ret = ret.encode(self.encoding)
File "C:\CocosCreator\resources\utils\Python27\lib\encodings\cp1252.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-8: character maps to <undefined>
any one knows why this happened?
I do not know python, but the problem seems solved afterI made the modification to the line 158 of MultiLanguage.py(in the get_current_string method):
ret = ret.encode(self.encoding)
to the following line:
ret = ret.encode("utf8") #self.encoding
Hope this helps people who encounter the same problem.
Take a look at the versions of NDK and SDK, NDK. The recommended version is R17 - R19
Hey I'm trying to execute the following command (using psutil.Popen with python 2.7):
"C:\Program Files (x86)\Microsoft Office\root\Office16\EXCEL.EXE" "C:\docs\ת.xlsm"
Using this code:
dir = u"C:\\docs"
doc = os.listdir(dir)[0]
full_path = os.path.join(dir, doc)
command = u"\"C:\\Program Files (x86)\\Microsoft Office\\root\\Office16\\EXCEL.EXE\" \"{}\"".format(full_path)
process = psutil.Popen(command)
But I'm getting this exception:
process = psutil.Popen(command)
File "C:\Python27\lib\site-packages\psutil\__init__.py", line 1370, in __init__
self.__subproc = subprocess.Popen(*args, **kwargs)
File "C:\Python27\lib\subprocess.py", line 390, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 640, in _execute_child
startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u05ea' in position 102: ordinal not in range(128)
I have found this related question:
subprocess.Popen with a unicode path. But every given answer there doesn't work for me.
Using subprocess.Popen(command.encode(locale.getpreferredencoding())), throws the following exception:
Traceback (most recent call last):
File "scanner_watcher.py", line 53, in _execute
self.scanner.execute(path)
File "scanner_watcher.py", line 356, in execute
self._execute(file_path)
File "scanner_watcher.py", line 201, in _execute
self.open_scanner(file_path, file_package)
File "scanner_watcher.py", line 287, in open_scanner
self.scanner_process = psutil.Popen(command.encode(locale.getpreferredencoding()))
File "C:\Python27\lib\encodings\cp1252.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u05ea' in position 102: character maps to <undefined>
Using path.encode('mbcs') turns all the Unicode characters into question marks.
I don't want to use os.startfile because I need to use different commands on the program and then handle the opened process (when the os.startfile doesn't allow that).
I've found this modified Popen:
https://gist.github.com/vaab/2ad7051fc193167f15f85ef573e54eb9 But this code was not thoroughly tested.
Is there a correct way to use Popen with a Unicode command in python 2.7?
Thanks.
The issue is your use of double-quotes when defining command. The way it's written splits the statement into multiple strings in all the wrong places. Substituting single-quotes for the first and final quotes (open and close of string) fixed the issue for me.
command = u'\"C:\\Program Files (x86)\\Microsoft Office\\root\\Office16\\EXCEL.EXE\" \"{}\"'.format(full_path)
I might have stumbled into something that appears to be a bug with how rabbitmqadmin handles (or in fact, doesn't handle) unicode.
If I have a queue that has unicode characters in queue name the rabbitmqadmin command fails with UnicodeEncodeErrors.
I'm running RabbitMQ 3.6.6 on Mac, installed via Homebrew. Both the admin command and server are of same version.
Steps how to reproduce the queue name issue:
Create a queue with unicode in it: 'rabbitmqadmin declare queue name=ööö'.
List queues: 'rabbitmqadmin list queues'
Output:
Traceback (most recent call last):
File "/usr/local/sbin/rabbitmqadmin", line 1007, in <module>
main()
File "/usr/local/sbin/rabbitmqadmin", line 413, in main
method()
File "/usr/local/sbin/rabbitmqadmin", line 588, in invoke_list
format_list(self.get(uri), cols, obj_info, self.options)
File "/usr/local/sbin/rabbitmqadmin", line 705, in format_list
formatter_instance.display(json_list)
File "/usr/local/sbin/rabbitmqadmin", line 716, in display
(columns, table) = self.list_to_table(json.loads(json_list), depth)
File "/usr/local/sbin/rabbitmqadmin", line 770, in list_to_table
add('', 1, item, add_to_row)
File "/usr/local/sbin/rabbitmqadmin", line 749, in add
fun(column, subitem)
File "/usr/local/sbin/rabbitmqadmin", line 756, in add_to_row
row[column_ix[col]] = str(val)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-13: ordinal not in range(128)
I have similar issues with 'rabbitmqadmin get queue=' when the message payload contains unicode character(s).
I think you are right, you can use python3 , check this answer it explains why with python3 works.
EDIT
1 - I filed an issue
2 - Fixed to the version: 3.6.7 see the PR