Develop Reference

python bottle boddle - patch(... side-effect="Exception) not working. What am I doing wrong

With: Python 3.7, Bottle, Boddle, Requests
I have one module to be tested (server), and a second module with the test code (test).
The tested code (server.py):
#bt_app.post("/")
def server_post():
try:
bt.request.forms.get("msg", None)
except bt.BottleException:
print("BottleException")
Testing code (test.py):
def test_test():
with boddle(), \
patch(
"server.bt.request.forms.get",
side_effect=server.bt.BottleException,
), \
pytest.raises(server.bt.BottleException):
server.server_post()
I first run 'server', which successfully sets up a server.
I then run 'test', which successfully accesses and tests the server.
I expect the test to raise "BottleException" and PASS, but it does not. It DOES properly fail stating: "Failed: DID NOT RAISE <class 'bottle.BottleException'>"
Can anyone give me a pointer of what I'm doing wrong?
Thanks

Try it without the alias.
...
with pytest.raises(bottle.BottleException):
...

Python macOS builds run from Terminal but crash on Finder launch

My py2app build displays an error message with a Terminate button when the app is launched with a double-click. However, if I open it directly in Terminal then it works perfectly well. All of the following Terminal commands launch the app successfully:
# .MyApp.app/Contents/MacOS/MyApp
# open MyApp.app
# open -a MyApp.app
I've read several posts about similar py2app errors but I can't figure out what could be the issue here. The problem seems to have been around for at least four years, is not specific to py2app and seems related to a general issue with Python on macOS. Users of pyinstaller are reporting the exact same problem with no obvious solution in sight AFAIK.
My setup.py file:
from setuptools import setup
APP = ['myapp.py']
DATA_FILES = [('img', ['img/myapp-logo.png']), ('data', ['data/data.yml'])]
OPTIONS = {'iconfile': 'icon.icns'}
setup(
app=APP,
name='My App',
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)
The macOS Console shows the following message in system.log: com.apple.xpc.launchd[1] (org.pythonmac.unspecified.MyApp.4952[16783]): Service exited with abnormal code: 255
Suspect 1: file access?
This thread on the pyinstaller GitHub issues page has multiple people reporting the same error with no clear fix. This proposed solution seems related to the working directory issues described in the pyinstaller thread but doesn't solve the problem on my system.
My app is only reading a single yml file from its working directory and doesn't write anything to disk. It is nothing more than a simple file access statement:
file = os.path.realpath('data/data.yml')
with open(file) as f:
# etc
In Catalina, I've added the app to the list of apps allowed Full Disk Access in the Security Preferences but this doesn't solve the issue either (it would have been surprising if it did, though, since the open command works as mentioned above).
Suspect 2: Tkinter?
This thread on pyinstaller GitHub suggests the problem might be related to a Tkinter version. The proposed solution seems to have fixed the issue for some users. However, on my end I have a working app bundle from an earlier version, before adding the open file statement, which launches just fine when double clicked.
Suspect 1 vs 2
I have forked my code into two branches, one completely removing references to Tkinter, the other removing file access in favour of initialisation via variables in the code. The crash does not happen in the second case. This would seem to rule out Tkinter as the source of the problem, although weirdly enough a hack to Tkinter has fixed it for other users.
macOS versions
I've tested both bundle versions (working and not working) on both Catalina 10.15.6 and El Capitan 10.11.6 and the behaviour is identical.
The Firing of the Death Sentinel
Here's what the Console log looks like after launching the app normally through the Finder:
default 21:26:13.836380+0200 runningboardd Acquiring assertion targeting executable<MyPythonApplication(501)> from originator [daemon<com.apple.coreservices.launchservicesd>:165] with description <RBSAssertionDescriptor; frontmost:27280; ID: 391-165-34219; target: 27280> attributes = {
<RBSDomainAttribute: 0x7f9f1a712910; domain: com.apple.launchservicesd; name: RoleUserInteractiveFocal; sourceEnvironment: 0x0>;
}
default 21:26:13.836649+0200 runningboardd Assertion 391-165-34219 (target:executable<MyPythonApplication(501)>) will be created as active
default 21:26:13.838258+0200 runningboardd [executable<MyPythonApplication(501)>:27280] Ignoring jetsam update because this process is not memory-managed
default 21:26:13.839703+0200 runningboardd [executable<MyPythonApplication(501)>:27280] Set darwin role to: UserInteractiveFocal
default 21:26:13.840634+0200 runningboardd [executable<MyPythonApplication(501)>:27280] Ignoring GPU update because this process is not GPU managed
default 21:26:13.840912+0200 runningboardd Finished acquiring assertion 391-165-34219 (target:executable<MyPythonApplication(501)>)
default 21:26:15.166436+0200 hidd Connection removed: IOHIDEventSystemConnection uuid:B1D40AB3-FD55-455C-9E1B-2E4C4C6E4982 pid:27280 process:MyPythonApplication type:Passive entitlements:0x0 caller:HIToolbox: ___GetIOHIDEventSystemClient_block_invoke + 26 attributes:(null) state:0x1 events:0 mask:0x0
default 21:26:15.171128+0200 runningboardd [executable<MyPythonApplication(501)>:27280] Death sentinel fired!
default 21:26:15.174315+0200 runningboardd Invalidating assertion 391-165-34219 (target:executable<MyPythonApplication(501)>) from originator 165
default 21:26:15.176832+0200 loginwindow -[PersistentAppsSupport applicationQuit:] | for app:MyPythonApplication, _appTrackingState = 2
default 21:26:15.176856+0200 loginwindow -[PersistentAppsSupport applicationQuit:] | App: MyPythonApplication, quit, updating active tracking timer
default 21:26:15.179589+0200 runningboardd Invalidating assertion 391-165-34209 (target:executable<MyPythonApplication(501)>) from originator 165
default 21:26:15.281529+0200 runningboardd Removing process: [executable<MyPythonApplication(501)>:27280]
default 21:26:15.282124+0200 runningboardd Removing assertions for terminated process: [executable<MyPythonApplication(501)>:27280]
error 21:26:15.292603+0200 runningboardd RBSStateCapture remove item called for untracked item 391-165-34209 (target:executable<MyPythonApplication(501)>)
error 21:26:15.292622+0200 runningboardd RBSStateCapture remove item called for untracked item 391-165-34219 (target:executable<MyPythonApplication(501)>)
Apparently the error is mentioned in this line, in case someone can make anything out of it:
hidd Connection removed: IOHIDEventSystemConnection uuid:foo pid:bar process:MyPythonApplication type:Passive entitlements:0x0 caller:HIToolbox: ___GetIOHIDEventSystemClient_block_invoke + 26 attributes:(null) state:0x1 events:0 mask:0x0
This is immediately followed by RunningBoard firing the Death Sentinel, whatever this entity might be. The system is described in this article by Howard Oakley but it is way beyond my level of expertise.
Some more info gathered by Ulbow confirms an error with the highly informative message "MacOS error: -67062":
com.apple.runningboard 4941060 391 Received message from [daemon<com.apple.coreservices.launchservicesd>:165] (euid 0): acquireAssertionWithDescriptor:error:
com.apple.launchservices 4940599 802 OSStatus _LSLaunch(LSContext *, FSNode *, LSLaunchFlags, void *, CFArrayRef, const AppleEvent *, const AEDescList *, CFArrayRef, CFDictionaryRef, LSBundleID, const audit_token_t *, const _LSOpen2Options *, ProcessSerialNumber *, Boolean *, NSError **): launching '<private>' result=0
com.apple.securityd 9807 904 MacOS error: -67062
com.apple.runningboard 4941063 391 Received message from [daemon<com.apple.coreservices.launchservicesd>:165] (euid 0): acquireAssertionWithDescriptor:error:
com.apple.sharedfilelist 4940599 802 -[SFLGenericList _insertItem:atIndex:error:]_block_invoke com.apple.LSSharedFileList.RecentApplications
com.apple.securityd 4940933 530 UNIX error exception: 8
4941117 27642 MyPythonApplication Error
com.apple.securityd 4941064 165 UNIX error exception: 22
com.apple.securityd 4941064 165 MacOS error: -67062
com.apple.runningboard 4941002 391 Received message from [daemon<com.apple.coreservices.launchservicesd>:165] (euid 0): acquireAssertionWithDescriptor:error:
com.apple.securityd 4941161 198 MacOS error: -67062
com.apple.TCC 4941161 198 Failed to copy signing info for 27642, responsible for file:///Users/me/Files/Docs/Code/Python/MyPythonApplication/dist/MyPythonApplication.app/Contents/MacOS/Pandemic%20Deck%20Tracker: #-67062: Error Domain=NSOSStatusErrorDomain Code=-67062 "(null)"
com.apple.securityd 4941161 198 MacOS error: -67062
com.apple.securityd 4941161 198 MacOS error: -67062
com.apple.launchservices 4941064 165 CLIENT: 0x7fcec00b33b0/27642 Received XPC_ERROR on connection from our client, so invalidating it and our connection.
com.apple.appleevents 4941120 462 CONNECTION: peer=? peer-pid=27642 got event (Error "Connection invalid")
com.apple.appleevents 4941120 462 CONNECTION: Recevied XPC_ERROR on a connection from 27642, a client of ours, so unregistering any application with that pid.
com.apple.appleevents 4941120 462 CONNECTION: releasing app App:"MyPythonApplication"/"MyPythonApplication"/"org.pythonmac.unspecified.MyPythonApplication" 27642/0x0:0x308d08a ????1010 sess=100020 because we received error on its connection.
com.apple.runningboard 4941002 391 Received message from [daemon<com.apple.coreservices.launchservicesd>:165] (euid 0): acquireAssertionWithDescriptor:error:
Several exchanges on recent tests, trying to pinpoint the cause for the bug, can be found on this GitHub issues page, but it still remains a mystery (and a very bizarre one, too).
An Apple engineer should really take a look at this.

A pretty thorough investigation of this issue has been conducted on this GitHub issue page by the maintainers of pyinstaller. It turns out that if a Python script accesses resources on a macOS file system using the built-in Python os module, the app bundle crashes. The bundle does not crash if the script is run directly from a Terminal command.
The proposed solution is to check if the script is running on macOS and, in that case, use AppKit to open the file. This requires installing the pyobjc module but otherwise it is not a major hassle.
Instead of doing this:
import os
file = os.path.realpath('path/somefile.ext')
with open(file) as f:
# ...
Do this:
import os
import platform
def get_path(filename):
name = os.path.splitext(filename)[0]
ext = os.path.splitext(filename)[1]
if platform.system() == "Darwin":
from AppKit import NSBundle
file = NSBundle.mainBundle().pathForResource_ofType_(name, ext)
return file or os.path.realpath(filename)
else:
return os.path.realpath(filename)
file = get_path('path/somefile.ext')
with open(file) as f:
# ...
I can confirm this works on macOS Catalina with pyinstaller. I haven't had the chance to test this on .exe builds for Windows.

After hours of digging I finally found an answer! Even after every Step I count gain the right /Contents/Resources path.
First I had to get this working:
from AppKit import NSBundle
file = NSBundle.mainBundle().pathForResource_ofType_(name, ext)
For my Python App, finding the "AppKit" module I had to install
pip install pyobjc
But even then, the path for
NSBundle.mainBundle().pathForResource_ofType_(name, ext)
was strange. It was a path in "/private/var/" where I don't have Write Access
The final answer was to run
xattr -d com.apple.quarantine /Applications/YouPythonApp.app/
then I finally got the right path
/Applications/YouPythonApp.app/Contents/Resources/
where I also had write access and now I can save my .yaml file
with open(file, "w") as f:
yaml=YAML()
yaml.default_flow_style = False
yaml.dump(config, f)

How to fix python function in exec_python_func() autogenerated: ERROR?

When I run "bitbake core-image-sato" the following error I am facing.
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:sort_passwd(d)
0003:
File: '/home/A/poky/meta/classes/rootfs-postcommands.bbclass', lineno: 182, function: sort_passwd
0178:}
0179:
0180:python sort_passwd () {
0181: import rootfspostcommands
*** 0182: rootfspostcommands.sort_passwd(d.expand('${IMAGE_ROOTFS}${sysconfdir}'))
0183:}
0184:
0185:#
0186:# Enable postinst logging if debug-tweaks is enabled
File: '/home/A/poky/meta/lib/rootfspostcommands.py', lineno: 56, function: sort_passwd
0052: mapping = sort_file(filename, None)
0053: filename = os.path.join(sysconfdir, shadow)
0054: remove_backup(filename)
0055: if os.path.exists(filename):
*** 0056: sort_file(filename, mapping)
File: '/home/A/poky/meta/lib/rootfspostcommands.py', lineno: 22, function: sort_file
0018: name = entries[0]
0019: if mapping is None:
0020: id = int(entries[2])
0021: else:
*** 0022: id = mapping[name]
0023: new_mapping[name] = id
0024: # Sort by numeric id first, with entire line as secondary key
0025: # (just in case that there is more than one entry for the same id).
0026: lines.sort(key=lambda line: (new_mapping[line.split(b':')[0]], line))
Exception: KeyError: b'bin'
DEBUG: Python function sort_passwd finished
DEBUG: Python function do_rootfs finished
ERROR: Function failed: sort_passwd
Does any one has the idea to fix this issue.
OS:Ubuntu-16.04,Poky-Branch:SUMO

If you have changed the contents of any of the following files, which get installed to /etc, then that is likely your root cause:
passwd
shadow
group
gshadow
These files often come from the directory /files in one of your layers. It looks like one of them is ill-formatted.
There is a comment in rootfspostcommands.py, right above the point where the exception is being thrown, that reads:
# No explicit error checking for the sake of simplicity. /etc
# files are assumed to be well-formed, causing exceptions if
# not.

If nothing works for you, please do bitbake -c clean yourimagename and make the build again. This method worked for me and maybe it will help you too.
Note: do not do cleanall because it is going to delete a lot of files and hence will take more time to make the build from start.

Using Elixir, erlport with Python 2.7.9, receiving an arity error

I am trying to use Python with Elixir and I wrote the following functional code (you can find the repo I'm building here: https://github.com/arthurcolle/elixir_with_erlport)
defmodule Snake do
use Application
def start(_type, _args) do
import Supervisor.Spec, warn: false
children = [
# Define workers and child supervisors to be supervised
# worker(Snake.Worker, [arg1, arg2, arg3]),
]
opts = [strategy: :one_for_one, name: Snake.Supervisor]
Supervisor.start_link(children, opts)
end
def py do
{:ok, pp} = :python.start()
:python.call(pp, :__builtin__, :print, ["hey there"])
end
end
I can run iex -S mix run, then type in Snake.py, and I will get this output:
"hey there"
:undefined
Okay, great.
Then I try to make it print out the current version of Python by swapping out the two lines above with:
{:ok, pp} = :python.start()
:python.call(pp, :sys, :version, [])
But when I run it, it gives me this arity error
** (FunctionClauseError) no function clause matching in :erlport.call/5
src/erlport.erl:87: :erlport.call(#PID<0.108.0>, :sys, 'version.__str__', [], [])
Which doesn't make any sense to me because my call only is a :erlport.call/4, with one single list at the end (not 2 as it is saying).

{:ok, pp} = :python.start_link()
:python.call(pp, :sys, String.to_atom("version.__str__"), [])

Python Cmd Tab Completion Problems

I've got an application I'm currently working on for our company. Its currently built around Python's Cmd module, and features tab-completion for a number of tasks.
For some reason however, the Tab completion only currently works on one machine in the building - running the scripts from other machines doesn't allow the tab completion.
Here's the offending code parts:
def populate_jobs_list():
global avail_jobs
avail_jobs = os.walk(rootDir()).next()[1]
print avail_jobs
...
def complete_job(self, text, line, start_index, end_index):
global avail_jobs
populate_jobs_list()
if text:
return [
jobs for jobs in avail_jobs
if jobs.startswith(text)
]
else:
return avail_jobs
def do_job(self, args):
pass
split_args = args.rsplit()
os.environ['JOB'] = args
job_dir = os.path.join( rootDir(), os.getenv('JOB'))
os.environ['JOB_PROPS'] = (job_dir + '\\job_format.opm')
if not os.path.isdir(job_dir):
print 'Job does not exist. Try again.'
return
else:
print('Jobbed into: ' + os.getenv('JOB'))
return
populate_jobs_list()
prompt = outPrompt()
prompt.prompt = '\> '
prompt.cmdloop('Loading...')
Am I missing something obvious here? Just to clarify, on machine A, the tab completion works as intended. When its run on any other machine in the building, it fails to complete.

Check if the environment variable PYTHONSTARTUP is set properly. It should point to a script which in turn needs to do sth like this:
try:
import readline
except ImportError:
sys.stdout.write("No readline module found, no tab completion available.\n")
else:
import rlcompleter
readline.parse_and_bind('tab: complete')
Maybe (some part of) this is only done properly on the one working machine?
Maybe the readline module is available only on the one working machine?