Develop Reference

How do I use the 'keyboard' python module in a docker container without an error?

I am trying to use the python module keyboard in a docker container for testing automation that emulates keyboard presses. However, this module continually returns an error in the docker container whereas it works fine in an ubuntu VM.
I have installed different commonly used docker containers and different versions of python. I get the same error. Sadly, google hasn't been as helpful as it usually is here.
Here's an easily reproducible version of what I've done:
docker run -t -d -u root --name docker_container python:3.7
echo $(docker exec -u root docker_container bash -c 'python3 -m keyboard' )
This returns the below error message:
/usr/local/lib/python3.7/site-packages/keyboard/_nixkeyboard.py:110: UserWarning: Failed to create a device file using `uinput` module. Sending of events may be limited or unavailable depending on plugged-in devices.
device = aggregate_devices('kbd')
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/keyboard/_nixcommon.py", line 106, in start_reading
self.event_queue.put(device.read_event())
File "/usr/local/lib/python3.7/site-packages/keyboard/_nixcommon.py", line 83, in read_event
data = self.input_file.read(struct.calcsize(event_bin_format))
AttributeError: 'NoneType' object has no attribute 'read'
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/keyboard/_nixcommon.py", line 106, in start_reading
self.event_queue.put(device.read_event())
File "/usr/local/lib/python3.7/site-packages/keyboard/_nixcommon.py", line 83, in read_event
data = self.input_file.read(struct.calcsize(event_bin_format))
AttributeError: 'NoneType' object has no attribute 'read'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/site-packages/keyboard/__main__.py", line 10, in <module>
keyboard.hook(print_event_json)
File "/usr/local/lib/python3.7/site-packages/keyboard/__init__.py", line 459, in hook
append(callback)
File "/usr/local/lib/python3.7/site-packages/keyboard/_generic.py", line 67, in add_handler
self.start_if_necessary()
File "/usr/local/lib/python3.7/site-packages/keyboard/_generic.py", line 35, in start_if_necessary
self.init()
File "/usr/local/lib/python3.7/site-packages/keyboard/__init__.py", line 194, in init
_os_keyboard.init()
File "/usr/local/lib/python3.7/site-packages/keyboard/_nixkeyboard.py", line 114, in init
build_tables()
File "/usr/local/lib/python3.7/site-packages/keyboard/_nixkeyboard.py", line 73, in build_tables
dump = check_output(['dumpkeys', '--keys-only'], universal_newlines=True)
File "/usr/local/lib/python3.7/subprocess.py", line 395, in check_output
**kwargs).stdout
File "/usr/local/lib/python3.7/subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/local/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/usr/local/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'dumpkeys': 'dumpkeys'
From what I understand, it can't access a uinput device / keyboard to be able to dump the keys. Is there a way to virtually simulate a keyboard in the container to allow this to work?

try to install the following:
apt-get install -y kmod kbd
since you need this to get dumpkeys

Error when trying repo init on external drive on Ubuntu

I'm trying to init repo of Android ROM, but it returns following error on Ubuntu 16.04:
$ repo init -u git://github.com/AOSiP/platform_manifest.git -b pie
Get git://github.com/AOSiP/platform_manifest.git
Traceback (most recent call last):
File "/media/exthdd/AOSIP/.repo/repo/main.py", line 547, in <module>
_Main(sys.argv[1:])
File "/media/exthdd/AOSIP/.repo/repo/main.py", line 522, in _Main
result = repo._Run(argv) or 0
File "/media/exthdd/AOSIP/.repo/repo/main.py", line 184, in _Run
result = cmd.Execute(copts, cargs)
File "/media/exthdd/AOSIP/.repo/repo/subcmds/init.py", line 426, in Execute
self._SyncManifest(opt)
File "/media/exthdd/AOSIP/.repo/repo/subcmds/init.py", line 188, in _SyncManifest
m._InitGitDir(mirror_git=mirrored_manifest_git)
File "/media/exthdd/AOSIP/.repo/repo/project.py", line 2395, in _InitGitDir
self._UpdateHooks()
File "/media/exthdd/AOSIP/.repo/repo/project.py", line 2416, in _UpdateHooks
self._InitHooks()
File "/media/exthdd/AOSIP/.repo/repo/project.py", line 2446, in _InitHooks
os.path.relpath(stock_hook, os.path.dirname(dst)), dst)
File "/media/exthdd/AOSIP/.repo/repo/platform_utils.py", line 195, in symlink
return os.symlink(source, link_name)
OSError: [Errno 38] Function not implemented
It works when running on system drive, but I don't have enough space there.

Drive had exFAT filesystem, error gone on ext4.

Python EOFError after using os.execl to restart the script

I have a script that does the following;
Checks if a temp.txt file is present in the working directory.
If present the temp.txt file and update.pyd file is removed.
Checks whether an update.pyd module is present in the working directory.
If present it imports and runs it.
The update process creates a temp.txt file in the working directory.
After the update the script is restarted using os.execl(sys.executable, sys.executable, *sys.argv)
But I keep getting an error when os.execl(sys.executable, sys.executable, *sys.argv) is called:
Traceback (most recent call last):
File "<string>", line 73, in execInThread
File "<string>", line 44, in __call__
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\netref.py", line 196, in __call__
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\netref.py", line 71, in syncreq
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 431, in sync_request
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 379, in serve
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 337, in _recv
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\channel.py", line 50, in recv
File "C:\Program Files (x86)\PyScripter\Lib\rpyc.zip\rpyc\core\stream.py", line 166, in read
EOFError: [WinError 10054] An existing connection was forcibly closed by the remote host
I have another process in the same script that does something similar, and restarts using os.execl(sys.executable, sys.executable, *sys.argv), but restarts cleanly.
Could someone tell me why this is happening. What "connection" is the error referring to, and how do I close it before restarting? Thanks

Too Many Open Files Parallel Python Subprocess error

Questions with similar issue:
Parallel Python - too many files and Python too many open files (subprocesses)
I am using Parallel Python [V1.6.2] to run tasks. The task processes an input file and outputs a log/report. Say, there are 10 folders each with 5000 ~ 20000 files which are read in parallel, processed and logs written out. Each file is approx 50KB ~ 250KB
After ~6 Hours of running, Parallel Python fails with the following error.
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 342, in __init__
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 506, in set_ncpus
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 140, in __init__
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 146, in start
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
File "/usr/lib/python2.7/subprocess.py", line 1135, in _execute_child
File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec
OSError: [Errno 24] Too many open files
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 66, in apport_excepthook
ImportError: No module named fileutils
Original exception was:
Traceback (most recent call last):
File "PARALLEL_TEST.py", line 746, in <module>
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 342, in __init__
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 506, in set_ncpus
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 140, in __init__
File "/usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py", line 146, in start
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
File "/usr/lib/python2.7/subprocess.py", line 1135, in _execute_child
File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec
OSError: [Errno 24] Too many open files
While I understand, this could be the issue in subprocess pointed out here http://bugs.python.org/issue2320, but, seems the solution is only part of Py V3.2. I am currently tied to Py V2.7.
I would like to know if the following suggestion helps:
[1]http://www.parallelpython.com/component/option,com_smf/Itemid,1/topic,313.0
*) Adding worker.t.close() in destroy() method of /usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/pp.py
*) Increasing BROADCAST_INTERVAL in /usr/local/lib/python2.7/dist-packages/pp-1.6.2-py2.7.egg/ppauto.py
I would like to know if there is a fix available/Work Around for this issue in Python V2.7.
Thanks in Advance

My team recently stumbled upon a similar issue with the same file handle resource exhaustion issue while running celeryd task queue jobs. I believe the OP has nailed it and it's most likely the messy code in suprocess.py lib in Python 2.7 and Python 3.1.
As suggested in , Python Bug#2320, please pass in close_fds=True everywhere you call subprocess.Popen(). In fact they make that a default in Python 3.2 while also fixing the underlying race condition issue. See more details in that ticket.

I had left at some lines to destroy the job servers. job_server.destroy() fixes the issue.

DjangoAppEngine TestApp

Need some help starting up the djangoappengine testapp.I followed the instructions at http://www.allbuttonspressed.com/projects/djangoappengine, but when I run ./manage.py runserver . while in the project directory I get the following error:
WARNING 2011-03-24 12:23:21,994 datastore_file_stub.py:657] Could not read datastore
data from /media/disk/Projex/AppEngineApp/app/.gaedata/datastore
Error: option --rdbms_sqlite_path not recognized
Runs a development application server for an application.
manage.py [options] <application root>__
When I try and comment the ' 'rdbms_sqlite_path': os.path.join(DATA_ROOT, 'rdbms'),' line in app/djangoappengine/db/base.py I get the following error:
WARNING 2011-03-24 12:26:07,239 datastore_file_stub.py:657] Could not read datastore
data from /media/disk/Projex/AppEngineApp/app/.gaedata/datastore
Traceback (most recent call last):
File "./manage.py", line 11, in <module>
execute_manager(settings)
File "/media/disk/Projex/AppEngineApp/app/django/core/management/__init__.py", line 438,
in execute_manager
utility.execute()
File "/media/disk/Projex/AppEngineApp/app/django/core/management/__init__.py", line 379,
in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/media/disk/Projex/AppEngineApp/app/djangoappengine/management/commands/runserver.py",
line 94, in run_from_argv
start_dev_appserver(argv)
File
"/media/disk/Projex/AppEngineApp/app/djangoappengine/management/commands/runserver.py",
line 81, in start_dev_appserver
dev_appserver_main.main([progname] + args + [PROJECT_DIR])
File "/usr/local/gaepy/google/appengine/tools/dev_appserver_main.py", line 426, in main
static_caching=static_caching)
File "/usr/local/gaepy/google/appengine/tools/dev_appserver.py", line 3820, in
CreateServer
server = HTTPServerWithScheduler((serve_address, port), handler_class)
File "/usr/local/gaepy/google/appengine/tools/dev_appserver.py", line 3840, in __init__
request_handler_class)
File "/usr/lib/python2.6/SocketServer.py", line 400, in __init__
self.server_bind()
File "/usr/lib/python2.6/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/usr/lib/python2.6/SocketServer.py", line 411, in server_bind
self.socket.bind(self.server_address)
File "<string>", line 1, in bind
socket.gaierror: [Errno -5] No address associated with hostname
Anyone experienced this issue ?

You probably already solved this, but for future users:
The warning can have multiple causes:
data store is empty, in that case ignore
wrong permissions on the datastore files
data corruption issue
The error can be solved by updating djangoappengine

Update to the latest version of google app engine.