Is there a Pythonic way to have only one instance of a program running?
The only reasonable solution I've come up with is trying to run it as a server on some port, then second program trying to bind to same port - fails. But it's not really a great idea, maybe there's something more lightweight than this?
(Take into consideration that program is expected to fail sometimes, i.e. segfault - so things like "lock file" won't work)
The following code should do the job, it is cross-platform and runs on Python 2.4-3.2. I tested it on Windows, OS X and Linux.
from tendo import singleton
me = singleton.SingleInstance() # will sys.exit(-1) if other instance is running
The latest code version is available singleton.py. Please file bugs here.
You can install tend using one of the following methods:
easy_install tendo
pip install tendo
manually by getting it from http://pypi.python.org/pypi/tendo
Simple, cross-platform solution, found in another question by zgoda:
import fcntl
import os
import sys
def instance_already_running(label="default"):
"""
Detect if an an instance with the label is already running, globally
at the operating system level.
Using `os.open` ensures that the file pointer won't be closed
by Python's garbage collector after the function's scope is exited.
The lock will be released when the program exits, or could be
released if the file pointer were closed.
"""
lock_file_pointer = os.open(f"/tmp/instance_{label}.lock", os.O_WRONLY)
try:
fcntl.lockf(lock_file_pointer, fcntl.LOCK_EX | fcntl.LOCK_NB)
already_running = False
except IOError:
already_running = True
return already_running
A lot like S.Lott's suggestion, but with the code.
This code is Linux specific. It uses 'abstract' UNIX domain sockets, but it is simple and won't leave stale lock files around. I prefer it to the solution above because it doesn't require a specially reserved TCP port.
try:
import socket
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
## Create an abstract socket, by prefixing it with null.
s.bind( '\0postconnect_gateway_notify_lock')
except socket.error as e:
error_code = e.args[0]
error_string = e.args[1]
print "Process already running (%d:%s ). Exiting" % ( error_code, error_string)
sys.exit (0)
The unique string postconnect_gateway_notify_lock can be changed to allow multiple programs that need a single instance enforced.
I don't know if it's pythonic enough, but in the Java world listening on a defined port is a pretty widely used solution, as it works on all major platforms and doesn't have any problems with crashing programs.
Another advantage of listening to a port is that you could send a command to the running instance. For example when the users starts the program a second time, you could send the running instance a command to tell it to open another window (that's what Firefox does, for example. I don't know if they use TCP ports or named pipes or something like that, 'though).
Never written python before, but this is what I've just implemented in mycheckpoint, to prevent it being started twice or more by crond:
import os
import sys
import fcntl
fh=0
def run_once():
global fh
fh=open(os.path.realpath(__file__),'r')
try:
fcntl.flock(fh,fcntl.LOCK_EX|fcntl.LOCK_NB)
except:
os._exit(0)
run_once()
Found Slava-N's suggestion after posting this in another issue (http://stackoverflow.com/questions/2959474). This one is called as a function, locks the executing scripts file (not a pid file) and maintains the lock until the script ends (normal or error).
Use a pid file. You have some known location, "/path/to/pidfile" and at startup you do something like this (partially pseudocode because I'm pre-coffee and don't want to work all that hard):
import os, os.path
pidfilePath = """/path/to/pidfile"""
if os.path.exists(pidfilePath):
pidfile = open(pidfilePath,"r")
pidString = pidfile.read()
if <pidString is equal to os.getpid()>:
# something is real weird
Sys.exit(BADCODE)
else:
<use ps or pidof to see if the process with pid pidString is still running>
if <process with pid == 'pidString' is still running>:
Sys.exit(ALREADAYRUNNING)
else:
# the previous server must have crashed
<log server had crashed>
<reopen pidfilePath for writing>
pidfile.write(os.getpid())
else:
<open pidfilePath for writing>
pidfile.write(os.getpid())
So, in other words, you're checking if a pidfile exists; if not, write your pid to that file. If the pidfile does exist, then check to see if the pid is the pid of a running process; if so, then you've got another live process running, so just shut down. If not, then the previous process crashed, so log it, and then write your own pid to the file in place of the old one. Then continue.
The best solution for this on windows is to use mutexes as suggested by #zgoda.
import win32event
import win32api
from winerror import ERROR_ALREADY_EXISTS
mutex = win32event.CreateMutex(None, False, 'name')
last_error = win32api.GetLastError()
if last_error == ERROR_ALREADY_EXISTS:
print("App instance already running")
Some answers use fctnl (included also in #sorin tendo package) which is not available on windows and should you try to freeze your python app using a package like pyinstaller which does static imports, it throws an error.
Also, using the lock file method, creates a read-only problem with database files( experienced this with sqlite3).
Here is my eventual Windows-only solution. Put the following into a module, perhaps called 'onlyone.py', or whatever. Include that module directly into your __ main __ python script file.
import win32event, win32api, winerror, time, sys, os
main_path = os.path.abspath(sys.modules['__main__'].__file__).replace("\\", "/")
first = True
while True:
mutex = win32event.CreateMutex(None, False, main_path + "_{<paste YOUR GUID HERE>}")
if win32api.GetLastError() == 0:
break
win32api.CloseHandle(mutex)
if first:
print "Another instance of %s running, please wait for completion" % main_path
first = False
time.sleep(1)
Explanation
The code attempts to create a mutex with name derived from the full path to the script. We use forward-slashes to avoid potential confusion with the real file system.
Advantages
No configuration or 'magic' identifiers needed, use it in as many different scripts as needed.
No stale files left around, the mutex dies with you.
Prints a helpful message when waiting
This may work.
Attempt create a PID file to a known location. If you fail, someone has the file locked, you're done.
When you finish normally, close and remove the PID file, so someone else can overwrite it.
You can wrap your program in a shell script that removes the PID file even if your program crashes.
You can, also, use the PID file to kill the program if it hangs.
For anybody using wxPython for their application, you can use the function wx.SingleInstanceChecker documented here.
I personally use a subclass of wx.App which makes use of wx.SingleInstanceChecker and returns False from OnInit() if there is an existing instance of the app already executing like so:
import wx
class SingleApp(wx.App):
"""
class that extends wx.App and only permits a single running instance.
"""
def OnInit(self):
"""
wx.App init function that returns False if the app is already running.
"""
self.name = "SingleApp-%s".format(wx.GetUserId())
self.instance = wx.SingleInstanceChecker(self.name)
if self.instance.IsAnotherRunning():
wx.MessageBox(
"An instance of the application is already running",
"Error",
wx.OK | wx.ICON_WARNING
)
return False
return True
This is a simple drop-in replacement for wx.App that prohibits multiple instances. To use it simply replace wx.App with SingleApp in your code like so:
app = SingleApp(redirect=False)
frame = wx.Frame(None, wx.ID_ANY, "Hello World")
frame.Show(True)
app.MainLoop()
Using a lock-file is a quite common approach on unix. If it crashes, you have to clean up manually. You could stor the PID in the file, and on startup check if there is a process with this PID, overriding the lock-file if not. (However, you also need a lock around the read-file-check-pid-rewrite-file). You will find what you need for getting and checking pid in the os-package. The common way of checking if there exists a process with a given pid, is to send it a non-fatal signal.
Other alternatives could be combining this with flock or posix semaphores.
Opening a network socket, as saua proposed, would probably be the easiest and most portable.
I'm posting this as an answer because I'm a new user and Stack Overflow won't let me vote yet.
Sorin Sbarnea's solution works for me under OS X, Linux and Windows, and I am grateful for it.
However, tempfile.gettempdir() behaves one way under OS X and Windows and another under other some/many/all(?) *nixes (ignoring the fact that OS X is also Unix!). The difference is important to this code.
OS X and Windows have user-specific temp directories, so a tempfile created by one user isn't visible to another user. By contrast, under many versions of *nix (I tested Ubuntu 9, RHEL 5, OpenSolaris 2008 and FreeBSD 8), the temp dir is /tmp for all users.
That means that when the lockfile is created on a multi-user machine, it's created in /tmp and only the user who creates the lockfile the first time will be able to run the application.
A possible solution is to embed the current username in the name of the lock file.
It's worth noting that the OP's solution of grabbing a port will also misbehave on a multi-user machine.
Building upon Roberto Rosario's answer, I come up with the following function:
SOCKET = None
def run_single_instance(uniq_name):
try:
import socket
global SOCKET
SOCKET = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
## Create an abstract socket, by prefixing it with null.
# this relies on a feature only in linux, when current process quits, the
# socket will be deleted.
SOCKET.bind('\0' + uniq_name)
return True
except socket.error as e:
return False
We need to define global SOCKET vaiable since it will only be garbage collected when the whole process quits. If we declare a local variable in the function, it will go out of scope after the function exits, thus the socket be deleted.
All the credit should go to Roberto Rosario, since I only clarify and elaborate upon his code. And this code will work only on Linux, as the following quoted text from https://troydhanson.github.io/network/Unix_domain_sockets.html explains:
Linux has a special feature: if the pathname for a UNIX domain socket
begins with a null byte \0, its name is not mapped into the
filesystem. Thus it won’t collide with other names in the filesystem.
Also, when a server closes its UNIX domain listening socket in the
abstract namespace, its file is deleted; with regular UNIX domain
sockets, the file persists after the server closes it.
Late answer, but for windows you can use:
from win32event import CreateMutex
from win32api import CloseHandle, GetLastError
from winerror import ERROR_ALREADY_EXISTS
import sys
class singleinstance:
""" Limits application to single instance """
def __init__(self):
self.mutexname = "testmutex_{D0E858DF-985E-4907-B7FB-8D732C3FC3B9}"
self.mutex = CreateMutex(None, False, self.mutexname)
self.lasterror = GetLastError()
def alreadyrunning(self):
return (self.lasterror == ERROR_ALREADY_EXISTS)
def __del__(self):
if self.mutex:
CloseHandle(self.mutex)
Usage
# do this at beginnig of your application
myapp = singleinstance()
# check is another instance of same program running
if myapp.alreadyrunning():
print ("Another instance of this program is already running")
sys.exit(1)
Here is a cross platform example that I've tested on Windows Server 2016 and Ubuntu 20.04 using Python 3.7.9:
import os
class SingleInstanceChecker:
def __init__(self, id):
if isWin():
ensure_win32api()
self.mutexname = id
self.lock = win32event.CreateMutex(None, False, self.mutexname)
self.running = (win32api.GetLastError() == winerror.ERROR_ALREADY_EXISTS)
else:
ensure_fcntl()
self.lock = open(f"/tmp/isnstance_{id}.lock", 'wb')
try:
fcntl.lockf(self.lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
self.running = False
except IOError:
self.running = True
def already_running(self):
return self.running
def __del__(self):
if self.lock:
try:
if isWin():
win32api.CloseHandle(self.lock)
else:
os.close(self.lock)
except Exception as ex:
pass
# ---------------------------------------
# Utility Functions
# Dynamically load win32api on demand
# Install with: pip install pywin32
win32api=winerror=win32event=None
def ensure_win32api():
global win32api,winerror,win32event
if win32api is None:
import win32api
import winerror
import win32event
# Dynamically load fcntl on demand
# Install with: pip install fcntl
fcntl=None
def ensure_fcntl():
global fcntl
if fcntl is None:
import fcntl
def isWin():
return (os.name == 'nt')
# ---------------------------------------
Here is it in use:
import time, sys
def main(argv):
_timeout = 10
print("main() called. sleeping for %s seconds" % _timeout)
time.sleep(_timeout)
print("DONE")
if __name__ == '__main__':
SCR_NAME = "my_script"
sic = SingleInstanceChecker(SCR_NAME)
if sic.already_running():
print("An instance of {} is already running.".format(SCR_NAME))
sys.exit(1)
else:
main(sys.argv[1:])
I use single_process on my gentoo;
pip install single_process
example:
from single_process import single_process
#single_process
def main():
print 1
if __name__ == "__main__":
main()
refer: https://pypi.python.org/pypi/single_process/
I keep suspecting there ought to be a good POSIXy solution using process groups, without having to hit the file system, but I can't quite nail it down. Something like:
On startup, your process sends a 'kill -0' to all processes in a particular group. If any such processes exist, it exits. Then it joins the group. No other processes use that group.
However, this has a race condition - multiple processes could all do this at precisely the same time and all end up joining the group and running simultaneously. By the time you've added some sort of mutex to make it watertight, you no longer need the process groups.
This might be acceptable if your process only gets started by cron, once every minute or every hour, but it makes me a bit nervous that it would go wrong precisely on the day when you don't want it to.
I guess this isn't a very good solution after all, unless someone can improve on it?
I ran into this exact problem last week, and although I did find some good solutions, I decided to make a very simple and clean python package and uploaded it to PyPI. It differs from tendo in that it can lock any string resource name. Although you could certainly lock __file__ to achieve the same effect.
Install with: pip install quicklock
Using it is extremely simple:
[nate#Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> # Let's create a lock so that only one instance of a script will run
...
>>> singleton('hello world')
>>>
>>> # Let's try to do that again, this should fail
...
>>> singleton('hello world')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/nate/live/gallery/env/lib/python2.7/site-packages/quicklock/quicklock.py", line 47, in singleton
raise RuntimeError('Resource <{}> is currently locked by <Process {}: "{}">'.format(resource, other_process.pid, other_process.name()))
RuntimeError: Resource <hello world> is currently locked by <Process 24801: "python">
>>>
>>> # But if we quit this process, we release the lock automatically
...
>>> ^D
[nate#Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> singleton('hello world')
>>>
>>> # No exception was thrown, we own 'hello world'!
Take a look: https://pypi.python.org/pypi/quicklock
linux example
This method is based on the creation of a temporary file automatically deleted after you close the application.
the program launch we verify the existence of the file;
if the file exists ( there is a pending execution) , the program is closed ; otherwise it creates the file and continues the execution of the program.
from tempfile import *
import time
import os
import sys
f = NamedTemporaryFile( prefix='lock01_', delete=True) if not [f for f in os.listdir('/tmp') if f.find('lock01_')!=-1] else sys.exit()
YOUR CODE COMES HERE
On a Linux system one could also ask
pgrep -a for the number of instances, the script
is found in the process list (option -a reveals the
full command line string). E.g.
import os
import sys
import subprocess
procOut = subprocess.check_output( "/bin/pgrep -u $UID -a python", shell=True,
executable="/bin/bash", universal_newlines=True)
if procOut.count( os.path.basename(__file__)) > 1 :
sys.exit( ("found another instance of >{}<, quitting."
).format( os.path.basename(__file__)))
Remove -u $UID if the restriction should apply to all users.
Disclaimer: a) it is assumed that the script's (base)name is unique, b) there might be race conditions.
Here's a good example for django with contextmanager and memcached:
https://docs.celeryproject.org/en/latest/tutorials/task-cookbook.html
Can be used to protect simultaneous operation on different hosts.
Can be used to manage multiple tasks.
Can also be changed for simple python scripts.
My modification of the above code is here:
import time
from contextlib import contextmanager
from django.core.cache import cache
#contextmanager
def memcache_lock(lock_key, lock_value, lock_expire):
timeout_at = time.monotonic() + lock_expire - 3
# cache.add fails if the key already exists
status = cache.add(lock_key, lock_value, lock_expire)
try:
yield status
finally:
# memcache delete is very slow, but we have to use it to take
# advantage of using add() for atomic locking
if time.monotonic() < timeout_at and status:
# don't release the lock if we exceeded the timeout
# to lessen the chance of releasing an expired lock owned by someone else
# also don't release the lock if we didn't acquire it
cache.delete(lock_key)
LOCK_EXPIRE = 60 * 10 # Lock expires in 10 minutes
def main():
lock_name, lock_value = "lock_1", "locked"
with memcache_lock(lock_name, lock_value, LOCK_EXPIRE) as acquired:
if acquired:
# single instance code here:
pass
if __name__ == "__main__":
main()
Here is a cross-platform implementation, creating a temporary lock file using a context manager.
Can be used to manage multiple tasks.
import os
from contextlib import contextmanager
from time import sleep
class ExceptionTaskInProgress(Exception):
pass
# Context manager for suppressing exceptions
class SuppressException:
def __init__(self):
pass
def __enter__(self):
return self
def __exit__(self, *exc):
return True
# Context manager for task
class TaskSingleInstance:
def __init__(self, task_name, lock_path):
self.task_name = task_name
self.lock_path = lock_path
self.lock_filename = os.path.join(self.lock_path, self.task_name + ".lock")
if os.path.exists(self.lock_filename):
raise ExceptionTaskInProgress("Resource already in use")
def __enter__(self):
self.fl = open(self.lock_filename, "w")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.fl.close()
os.unlink(self.lock_filename)
# Here the task is silently interrupted
# if it is already running on another instance.
def main1():
task_name = "task1"
tmp_filename_path = "."
with SuppressException():
with TaskSingleInstance(task_name, tmp_filename_path):
print("The task `{}` has started.".format(task_name))
# The single task instance code is here.
sleep(5)
print("The task `{}` has completed.".format(task_name))
# Here the task is interrupted with a message
# if it is already running in another instance.
def main2():
task_name = "task1"
tmp_filename_path = "."
try:
with TaskSingleInstance(task_name, tmp_filename_path):
print("The task `{}` has started.".format(task_name))
# The single task instance code is here.
sleep(5)
print("Task `{}` completed.".format(task_name))
except ExceptionTaskInProgress as ex:
print("The task `{}` is already running.".format(task_name))
if __name__ == "__main__":
main1()
main2()
import sys,os
# start program
try: # (1)
os.unlink('lock') # (2)
fd=os.open("lock", os.O_CREAT|os.O_EXCL) # (3)
except:
try: fd=os.open("lock", os.O_CREAT|os.O_EXCL) # (4)
except:
print "Another Program running !.." # (5)
sys.exit()
# your program ...
# ...
# exit program
try: os.close(fd) # (6)
except: pass
try: os.unlink('lock')
except: pass
sys.exit()
Related
When I'm trying to read data from sqlalchemy df=pd.read_sql_table(table, con, schema) getting runtime error :
runtime/cgo: could not obtain pthread_keys
tried 0x115 0x116 0x117 0x118 0x119 0x11a 0x11b 0x11c 0x11d 0x11e 0x11f 0x120 0x121 0x122 0x123 0x124 0x125 0x126 0x127 0x128 0x129 0x12a 0x12b 0x12c 0x12d 0x12e 0x12f 0x130 0x131 0x132 0x133 0x134 0x135 0x136 0x137 0x138 0x139 0x13a 0x13b 0x13c 0x13d 0x13e 0x13f 0x140 0x141 0x142 0x143 0x144 0x145 0x146 0x147 0x148 0x149 0x14a 0x14b 0x14c 0x14d 0x14e 0x14f 0x150 0x151 0x152 0x153 0x154 0x155 0x156 0x157 0x158 0x159 0x15a 0x15b 0x15c 0x15d 0x15e 0x15f 0x160 0x161 0x162 0x163 0x164 0x165 0x166 0x167 0x168 0x169 0x16a 0x16b 0x16c 0x16d 0x16e 0x16f 0x170 0x171 0x172 0x173 0x174 0x175 0x176 0x177 0x178 0x179 0x17a 0x17b 0x17c 0x17d 0x17e 0x17f 0x180 0x181 0x182 0x183 0x184 0x185 0x186 0x187 0x188 0x189 0x18a 0x18b 0x18c 0x18d 0x18e 0x18f 0x190 0x191 0x192 0x193 0x194
Below is the code:
class TeradataWriter:
def __init__(self):
print("in init")
def read_data_from_teradata(self):
try:
print('Create main')
import pdb;pdb.set_trace()
eng = self.create_connection_engine()
df = pd.read_sql_table("table_name", eng, schema="schema")
print(df)
except Exception as ex:
print('Exception: %s', ex.with_traceback())
def create_connection_engine(self):
try:
return create_engine('teradatasql://' + constants.TERADATA_HOST + '/?user='+ constants.TERADATA_USER_NAME + '&password=' + constants.TERADATA_PWD, echo=False)
except Exception as ex:
LOGGER.error('Exception: %s', ex)
raise Exception(message_constants.ERROR_WHILE_CREATING_CONNECTION_WITH_TERADATA)
if __name__ == "__main__":
p = TeradataWriter()
p.write_dataframe_to_teradata()
Edit: This is fixed. I was finally able to get their support and engineering team to reproduce the issue. They now build the driver with a newer version of go. Upgrade to >= 17.0.3, and you shouldn't see anymore segfaults.
I think I finally figured out why this happens. According to this Go issue, it happens if "If the host process spawns threads prior to loading the shared library, the offset will have changed."
In my case, I was importing matplotlib.pyplot in IPython before calling code that loads the shared library. This starts an event loop and causes the conditions that lead to the segfault.
I changed my code to import matplotlib.pyplot after configuring the teradata driver, and it went away.
According to the Go issue, they just need to recompile the library with a newer version of Go, which I've asked them to do. We'll see what they say.
I have run in to same issue -
So to fix the problem, I moved connect statement to main and it kind of fixed. Its worth trying in your case.
I am experimenting with OS X apps written in Python and need to detect if there is already an instance of Python.app running with certain script. The script modifies CFBundleName on-the-fly from Python to MyApp to change the app title in the menubar.
bundle = NSBundle.mainBundle()
info = bundle.localizedInfoDictionary() or bundle.infoDictionary()
info['CFBundleName'] = 'MyApp'
If I start another instance and check CFBundleName of the running apps, it will only tell me the original value, i.e. Python:
for app in NSWorkspace.sharedWorkspace().runningApplications():
bundle = NSBundle.bundleWithURL_(app.bundleURL())
info = bundle.localizedInfoDictionary() or bundle.infoDictionary()
name = info.get('CFBundleName')
if name in ('Python', 'MyApp'):
print name # => prints Python
So I need to find a way to mark a Python.app instance that runs MyApp script to be able to abort launching duplicate instances.
Is there such way?
Update:
Until there is a better solution, I'll be using lockf
import fcntl
lockfile = open('/tmp/myapp.lock', 'w')
fcntl.lockf(lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
Update 2:
Well, I still need to find my application to focus. Currently, I just loop through all Python.app instances and focus them one by one. Normally, there is just one, but if there are few of them it can be messy.
from Foundation import NSWorkspace
from Cocoa import NSApplicationActivateAllWindows, NSApplicationActivateIgnoringOtherApps
try:
import fcntl
lockfile = open('/tmp/myapp.lock', 'w')
fcntl.lockf(lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError as e:
assert (e.errno, e.strerror) == (35, 'Resource temporarily unavailable')
for app in NSWorkspace.sharedWorkspace().runningApplications():
if app.bundleIdentifier() == 'org.python.python':
app.activateWithOptions_(NSApplicationActivateAllWindows | NSApplicationActivateIgnoringOtherApps)
exit()
Update 3:
I am going to use a pid file until a better solution comes up
LOCK_FILE = '/tmp/myapp.lock'
PID_FILE = '/tmp/myapp.pid'
try:
import fcntl
# NOTE: needs to be assigned to a variable for the lock to be preserved
lockfile = open(LOCK_FILE, 'w')
fcntl.lockf(lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError as e:
try:
with open(PID_FILE) as f:
pid = int(f.read())
except:
pid = None
for app in NSWorkspace.sharedWorkspace().runningApplications():
if app.bundleIdentifier() == 'org.python.python':
if not pid or pid == app.processIdentifier():
app.activateWithOptions_(NSApplicationActivateAllWindows | NSApplicationActivateIgnoringOtherApps)
exit()
from Foundation import NSProcessInfo
info = NSProcessInfo.processInfo()
pid = info.processIdentifier()
with open(PID_FILE, 'w+') as f:
f.write(str(pid))
There are several ways to solve this. Among them:
libunique - a library specifically designed for this
dbus - internal communications system
Many of them are described in the answers of this post.
A recipe for unique instances is given here.
A less subtle solution would be to call killall with the name of your application without .app from the command line. This can be done from within you Python script using os.command().
I've got an application I'm currently working on for our company. Its currently built around Python's Cmd module, and features tab-completion for a number of tasks.
For some reason however, the Tab completion only currently works on one machine in the building - running the scripts from other machines doesn't allow the tab completion.
Here's the offending code parts:
def populate_jobs_list():
global avail_jobs
avail_jobs = os.walk(rootDir()).next()[1]
print avail_jobs
...
def complete_job(self, text, line, start_index, end_index):
global avail_jobs
populate_jobs_list()
if text:
return [
jobs for jobs in avail_jobs
if jobs.startswith(text)
]
else:
return avail_jobs
def do_job(self, args):
pass
split_args = args.rsplit()
os.environ['JOB'] = args
job_dir = os.path.join( rootDir(), os.getenv('JOB'))
os.environ['JOB_PROPS'] = (job_dir + '\\job_format.opm')
if not os.path.isdir(job_dir):
print 'Job does not exist. Try again.'
return
else:
print('Jobbed into: ' + os.getenv('JOB'))
return
populate_jobs_list()
prompt = outPrompt()
prompt.prompt = '\> '
prompt.cmdloop('Loading...')
Am I missing something obvious here? Just to clarify, on machine A, the tab completion works as intended. When its run on any other machine in the building, it fails to complete.
Check if the environment variable PYTHONSTARTUP is set properly. It should point to a script which in turn needs to do sth like this:
try:
import readline
except ImportError:
sys.stdout.write("No readline module found, no tab completion available.\n")
else:
import rlcompleter
readline.parse_and_bind('tab: complete')
Maybe (some part of) this is only done properly on the one working machine?
Maybe the readline module is available only on the one working machine?
While trying to use thread in Python , with the following code
import select, sys, time, thread
TIMEOUT = 30
def listenthread(server,result):
# some code
#
#
def main(argv=None):
if argv is None:
argv = sys.argv[1:]
server = Server()
clientaddrs = []
# listen to clients in new thread
thread.start_new_thread(listenthread, (server,clientaddrs)) # that line produces a problem
I get Undefined variable from import: start_new_thread when I reach the line :
thread.start_new_thread(listenthread, (server,clientaddrs))
What might be the problem ?
Regards
This is a common PyDev bug. I see nothing wrong with your code, PyDev is simply alerting you to a non-existent error.
See the answers listed here and here.
I have a setup.py script which needs to probe the compiler for certain things like the support for TR1, the presence of windows.h (to add NOMINMAX define), etc. I do these checks by creating a simple program and trying to compile it with Distutils' Compiler class. The presence/lack of errors is my answer.
This works well, but it means that the compiler's ugly error messages get printed to the console. Is there a way to suppress error messages for when the compile function is called manually?
Here is my function which tries to compile the program, which now DOES eliminate the error messages by piping the error stream to a file (answered my own question):
def see_if_compiles(program, include_dirs, define_macros):
""" Try to compile the passed in program and report if it compiles successfully or not. """
from distutils.ccompiler import new_compiler, CompileError
from shutil import rmtree
import tempfile
import os
try:
tmpdir = tempfile.mkdtemp()
except AttributeError:
# Python 2.2 doesn't have mkdtemp().
tmpdir = "compile_check_tempdir"
try:
os.mkdir(tmpdir)
except OSError:
print "Can't create temporary directory. Aborting."
sys.exit()
old = os.getcwd()
os.chdir(tmpdir)
# Write the program
f = open('compiletest.cpp', 'w')
f.write(program)
f.close()
# redirect the error stream to keep ugly compiler error messages off the command line
devnull = open('errors.txt', 'w')
oldstderr = os.dup(sys.stderr.fileno())
os.dup2(devnull.fileno(), sys.stderr.fileno())
#
try:
c = new_compiler()
for macro in define_macros:
c.define_macro(name=macro[0], value=macro[1])
c.compile([f.name], include_dirs=include_dirs)
success = True
except CompileError:
success = False
# undo the error stream redirect
os.dup2(oldstderr, sys.stderr.fileno())
devnull.close()
os.chdir(old)
rmtree(tmpdir)
return success
Here is a function which uses the above to check for the presence of a header.
def check_for_header(header, include_dirs, define_macros):
"""Check for the existence of a header file by creating a small program which includes it and see if it compiles."""
program = "#include <%s>\n" % header
sys.stdout.write("Checking for <%s>... " % header)
success = see_if_compiles(program, include_dirs, define_macros)
if (success):
sys.stdout.write("OK\n");
else:
sys.stdout.write("Not found\n");
return success
Zac's comment spurred me to look a bit more and I found that Mercurial's setup.py script has a working method for his approach. You can't just assign the stream because the change won't get inherited by the compiler process, but apparently Python has our good friend dup2() in the form of os.dup2(). That allows the same OS-level stream shenanigans that we all know and love, which do get inherited to child processes.
Mercurial's function redirects to /dev/null, but to keep Windows compatibility I just redirect to a file then delete it.
Quoth Mercurial:
# simplified version of distutils.ccompiler.CCompiler.has_function
# that actually removes its temporary files.
def hasfunction(cc, funcname):
tmpdir = tempfile.mkdtemp(prefix='hg-install-')
devnull = oldstderr = None
try:
try:
fname = os.path.join(tmpdir, 'funcname.c')
f = open(fname, 'w')
f.write('int main(void) {\n')
f.write(' %s();\n' % funcname)
f.write('}\n')
f.close()
# Redirect stderr to /dev/null to hide any error messages
# from the compiler.
# This will have to be changed if we ever have to check
# for a function on Windows.
devnull = open('/dev/null', 'w')
oldstderr = os.dup(sys.stderr.fileno())
os.dup2(devnull.fileno(), sys.stderr.fileno())
objects = cc.compile([fname], output_dir=tmpdir)
cc.link_executable(objects, os.path.join(tmpdir, "a.out"))
except:
return False
return True
finally:
if oldstderr is not None:
os.dup2(oldstderr, sys.stderr.fileno())
if devnull is not None:
devnull.close()
shutil.rmtree(tmpdir)
Here's a context manager that I recently wrote and found useful, because I was having the same problem with distutils.ccompiler.CCompiler.has_function while working on pymssql. I was going to use your approach (nice, thanks for sharing!) but then I thought that it could be done with less code and would be more general and flexible if I used a context manager. Here's what I came up with:
import contextlib
#contextlib.contextmanager
def stdchannel_redirected(stdchannel, dest_filename):
"""
A context manager to temporarily redirect stdout or stderr
e.g.:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
"""
try:
oldstdchannel = os.dup(stdchannel.fileno())
dest_file = open(dest_filename, 'w')
os.dup2(dest_file.fileno(), stdchannel.fileno())
yield
finally:
if oldstdchannel is not None:
os.dup2(oldstdchannel, stdchannel.fileno())
if dest_file is not None:
dest_file.close()
The context for why I created this is at this blog post. Pretty much the same as yours I think.
This uses code that I borrowed from you to do the redirection (e.g.: os.dup2, etc.), but I wrapped it in a context manager so it's more general and reusable.
I use it like this in a setup.py:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
#Adam I just want to point out that there is /dev/null equivalent on Windows. It's 'NUL' but good practice is to get it from os.devnull
I'm pretty new to programming and python, so disregard this if it's a stupid suggestion, but can't you just reroute the error messages to a text file instead of the screen/interactive window/whatever?
I'm pretty sure I read somewhere you can do something like
error = open('yourerrorlog.txt','w')
sys.stderr = error
Again, sorry I'm probably repeating something you already know, but if the problem is you WANT the errors when it's called by another function (automated) and no errors when it's ran manual, can't you just add a keyword argument like compile(arg1, arg2, manual=True ) and then under your "except:" you add
if manual == False: print errors to console/interactive window
else: print to error
Then when it's called by the program and not manually you just call it with compile(arg1,arg2, manual=False) so that it redirects to the file.
Does running in quiet mode help at all? setup.py -q build
Not a direct answer to your question, but related to your use case: there is a config command in distutils that’s designed to be subclassed and used to check for C features. It’s not documented yet, you have to read the source.