Related
I am writing my thesis application. I need linear programming, but my app is written in Elixir, which is really not the language for such operations. That is why I decided to use Erlport as the Elixir dependency, which is capable of connecting Python code with Elixir. I'm also using Pulp as the python library for the optimization.
Elixir version: 1.10.4,
Erlport version: 0.10.1,
Python version: 3.8.5,
PuLP version: 2.3
I've written such a module for Elixir-Python communication, which leverages the GenServer as the main 'communication hub' between Elixir and Python:
defmodule MyApp.PythonHub do
use GenServer
def start_link(_) do
GenServer.start_link(__MODULE__, nil, name: __MODULE__)
end
def init(_opts) do
path = [:code.priv_dir(:feed), "python"]
|> Path.join() |> to_charlist()
{:ok, pid} = :python.start([{ :python_path, path }, { :python, 'python3' }])
{:ok, pid}
end
def handle_call({:call_function, module, function_name, arguments}, _sender, pid) do
result = :python.call(pid, module, function_name, arguments)
{:reply, result, pid}
end
def call_python_function(file_name, function_name, arguments) do
GenServer.call(__MODULE__, {:call_function, file_name, function_name, arguments}, 10_000)
end
end
The GenServer module is calling python file, which contains such a function:
def calculate_meal_4(products_json, diet_json, lower_boundary, upper_boundary, enhance):
from pulp import LpMinimize, LpProblem, LpStatus, lpSum, LpVariable, value
import json
products_dictionary = json.loads(products_json)
print(products_dictionary)
diets_dictionary = json.loads(diet_json)
print(diets_dictionary)
model = LpProblem(name="diet-minimization", sense=LpMinimize)
# ... products setup ...
x = LpVariable("prod_1_100g", lower_boundary, upper_boundary)
y = LpVariable("prod_2_100g", lower_boundary, upper_boundary)
z = LpVariable("prod_3_100g", lower_boundary, upper_boundary)
w = LpVariable("prod_4_100g", lower_boundary, upper_boundary)
optimization_function = # ... optimization function setup ...
model += # ... optimization boundary function setup ...
model += optimization_function
print(model)
solved_model = model.solve()
print(value(model.objective))
return [value(x), value(y), value(z), value(w)]
The call to the GenServer itself looks like that:
PythonHub.call_python_function(:diets, python_function, [products_json, meal_statistics_json, #min_portion, #max_portion, #macro_enhancement])
where python_function is :calculate_meal_4 and products_json and meal_statistic_json are jsons containing required data.
While calling calculate_meal_4 via python3 diets.py, which launches the python script above with some example, but real (taken from the app), data everything works fine - I've got the minimized result in almost no time. The problem occurs while calling the python script via Elixir Erlport. Looking at the printed outputs I can tell that it seems working until
solved_model = model.solve()
is called. Then the script seems to freeze and GenServer finally reaches the timeout on GenServer.call function.
I've tested also the call on a simple python test file:
def pass_var(a):
print(a)
return [a, a, a]
and it worked fine.
That is why I am really consterned right now and I am looking for any advices. Shamefully I found nothing yet.
Hmm, it might be that calling an external solver freezes the process.
Given that you can execute bash scripts using elixir, you can easily change the python script to be command line executable (I recommend click). Then, you can write the output to a .json or .csv file and read it back in with Elixir when you're done.
#click.group()
def cli():
pass
#cli.command()
#click.argument('products_json', help='your array of products')
#click.argument('diet_json', help='your dietary wishes')
#click.option('--lower-bound', default=0, help='your minimum number of desired calories')
#click.option('--upper-bound', default=100, help='your maximum number of desired calories')
#click.option('--enhance', default=False, help="whether you'd like to experience our enhanced experience")
def calculate_meal_4(products_json, diet_json, lower_boundary, upper_boundary, enhance):
pass
if __name__ == '__main__':
cli()
which you can then call using python3 my_file.py <products_json> <diet_json> ... et cetera.
You can even validate the JSON and then return the parsed data directly.
The Problem:
Every time I restart my computer my Windows 8 power settings go back to default and puts my computer to sleep after an hour. I don't want my computer to ever go to sleep unless I say so... I have to go in Control Panel > System and Security > Power Options > Edit Plan Settings and manually edit the put the computer to sleep setting to Never.
What I want:
A Python script to edit the Power Options in Windows 8. I will set it to run every time I reboot.
I've searched for a Python module to edit Windows settings but couldn't find what I was looking for. I've played with win32api to control my courser a while back but couldn't find Power Options in its documentation.
Julius Caesar's hint about using powercfg command-line options was perfect.
Here is the simple script I ended up using:
import subprocess
subprocess.call("powercfg -change -standby-timeout-ac 0")
The -standby-timeout-ac option is set to zero so my computer will Never go to sleep
I think You should look into powercfg Windows' command and set whatever You like with python's subprocess.call, for example:
import subprocess
subprocess.call("powercfg -change -monitor-timeout-ac 666")
I guess it is pretty self-explanatory: change '-ac' to '-dc' for battery setting, value is in minutes and zero stands for infinity, obviously.
I had a solution which heavily involve with win32com.client
It is work on windows 10
import win32com.client
class PowerPlan(Computer):
def __init__(self, mk="//./root/cimv2/power"):
super(PowerPlan, self).__init__(mk)
self.power_info = None
self.power_plan = None
def get_active_power_plan(self) -> str:
power_plans = self.wmi.InstancesOf("Win32_powerplan")
for plan in power_plans:
if plan.IsActive:
match = re.search(r'\{(.+?)\}', plan.InstanceID)
self.power_plan = plan
return match.group(1)
def get_power_plan_index(self, guid_id):
unknown_list = []
current_power_plan_index = {"AC": {}, "DC": {}}
power_index = self.wmi.InstancesOf("Win32_powersettingdataindex")
for power_value in power_index:
# print(type(power_value))
# print(dir(power_value))
match = re.search(guid_id, power_value.InstanceID)
if match is not None:
match = re.search(guid_id + r'\}\\(\w{2})\\\{(.+?)\}', power_value.InstanceID)
power_mode = match.group(1)
power_tag = match.group(2)
try:
power_word = PowerPlanGUID(power_tag).name
except Exception as Err:
# print("Unknown Tag GUID: " + power_tag)
if power_tag not in unknown_list:
unknown_list.append(power_tag)
continue
# power_info = {power_word : power_value.settingindexvalue}
current_power_plan_index[power_mode][power_word] = power_value.settingindexvalue
self.power_info = current_power_plan_index
self._json_dump(self.power_info)
def set_power_plan_value(self, act_plan_guid, power_mode, power_plan_value_guid, value):
power_index = self.wmi.InstancesOf("Win32_powersettingdataindex")
for power_setting in power_index:
match = re.search(act_plan_guid + r'\}\\' + power_mode + r'\\\{' + power_plan_value_guid + r'\}',
power_setting.InstanceID)
# match = re.search(power_plan_value_guid, power_setting.InstanceID)
if match is not None:
print(power_setting.InstanceID)
print(power_setting.settingindexvalue)
# Properties_
power_setting.Properties_("SettingIndexValue").Value = value
# How to make the changed value work
power_setting.Put_()
act_method = self.power_plan.Methods_("Activate")
self.power_plan.ExecMethod_("Activate")
else:
pass
I know it's a bit late, but there has been a new module published named "powerplan".
pip install powerplan
Then import the module:
import powerplan
To get your current power plan scheme use:
print(powerplan.get_current_scheme_name())
print(powerplan.get_current_scheme_guid())
To change your power plan scheme use:
powerplan.change_current_scheme_to_powersaver()
powerplan.change_current_scheme_to_balanced()
powerplan.change_current_scheme_to_high()
I've been looking around for an answer to my original issue.. how do i determine (programmatically) that my win32api.ShellExecute statement executed successfully, and if a successful execution occurs, execute an os.remove() statement.
Researching I found out that the ShellExecute() call returns the HINSTANCE. Further digging I found that ShellExecute() will return an HINSTANCE > 32 if it was successful. My problem/question now is, how do i use it to control the rest of my program's flow? I tried using an if HINSTANCE> 32: statement to control the next part, but I get a NameError: name 'hinstance' is not defined message. Normally this wouldn't confuse me because it means i need to define the variable 'hinstance' before referencing it; however, because i thought ShellExecute is supposed to return HINSTANCE, i thought that makes it available for use?
Here is my full code where i am trying to implement this. Note that in my print_file() def i am assigning hinstance to the full win32api.ShellExecute() command in attempt to capture the hinstance along with explicitly returning it at the end of the function.. this isn't working either.
import win32print
import win32api
from os.path import isfile, join
import glob
import os
import time
source_path = "c:\\temp\\source\\"
def main():
printer_name = win32print.GetDefaultPrinter()
while True:
file_queue = [f for f in glob.glob("%s\\*.txt" % source_path) if isfile(f)]
if len(file_queue) > 0:
for i in file_queue:
print_file(i, printer_name)
if hinstance > 32:
time.sleep(.25)
delete_file(i)
print "Filename: %r has printed" % i
print
time.sleep(.25)
print
else:
print "No files to print. Will retry in 15 seconds"
time.sleep(15)
def print_file(pfile, printer):
hinstance = win32api.ShellExecute(
0,
"print",
'%s' % pfile,
'/d:"%s"' % printer,
".",
0
)
return hinstance
def delete_file(f):
os.remove(f)
print f, "was deleted!"
def alert(email):
pass
main()
With ShellExecute, you will never know when the printing is complete, it depends on the size of the file and whether the printer driver buffers the contents (the printer might be waiting for you to fill the paper tray, for example).
According to this SO answer, it looks like subprocess.call() is a better solution, since it waits for the command to complete, only in this case you would need to read the registry to obtain the exe associated with the file.
ShellExecuteEx is available from pywin32, you can do something like:
import win32com.shell.shell as shell
param = '/d:"%s"' % printer
shell.ShellExecuteEx(fmask = win32com.shell.shellcon.SEE_MASK_NOASYNC, lpVerb='print', lpFile=pfile, lpParameters=param)
EDIT: code for waiting on the handle from ShellExecuteEx()
import win32com.shell.shell as shell
import win32event
#fMask = SEE_MASK_NOASYNC(0x00000100) = 256 + SEE_MASK_NOCLOSEPROCESS(0x00000040) = 64
dict = shell.ShellExecuteEx(fMask = 256 + 64, lpFile='Notepad.exe', lpParameters='Notes.txt')
hh = dict['hProcess']
print hh
ret = win32event.WaitForSingleObject(hh, -1)
print ret
The return value of ShellExecute is what you need to test. You return that from print_file, but you then ignore it. You need to capture it and check that.
hinstance = print_file(i, printer_name)
if hinstance > 32:
....
However, having your print_file function leak implementation detail like an HINSTANCE seems bad. I think you would be better to check the return value of ShellExecute directly at the point of use. So try to move the > 32 check inside print_file.
Note that ShellExecute has very weak error reporting. If you want proper error reporting then you should use ShellExecuteEx instead.
Your delete/sleep loop is very brittle indeed. I'm not quite sure I can recommend anything better since I'm not sure what you are trying to achieve. However, expect to run into trouble with that part of your program.
I've written a Nautilus extension which reads picture's metadata (executing exiftool), but when I open folders with many files, it really slows down the file manager and hangs until it finishes reading the file's data.
Is there a way to make Nautilus keep its work while it runs my extension? Perhaps the Exif data could appear gradually in the columns while I go on with my work.
#!/usr/bin/python
# Richiede:
# nautilus-python
# exiftool
# gconf-python
# Versione 0.15
import gobject
import nautilus
from subprocess import Popen, PIPE
from urllib import unquote
import gconf
def getexiftool(filename):
options = '-fast2 -f -m -q -q -s3 -ExifIFD:DateTimeOriginal -IFD0:Software -ExifIFD:Flash -Composite:ImageSize -IFD0:Model'
exiftool=Popen(['/usr/bin/exiftool'] + options.split() + [filename],stdout=PIPE,stderr=PIPE)
#'-Nikon:ShutterCount' non utilizzabile con l'argomento -fast2
output,errors=exiftool.communicate()
return output.split('\n')
class ColumnExtension(nautilus.ColumnProvider, nautilus.InfoProvider, gobject.GObject):
def __init__(self):
pass
def get_columns(self):
return (
nautilus.Column("NautilusPython::ExifIFD:DateTimeOriginal","ExifIFD:DateTimeOriginal","Data (ExifIFD)","Data di scatto"),
nautilus.Column("NautilusPython::IFD0:Software","IFD0:Software","Software (IFD0)","Software utilizzato"),
nautilus.Column("NautilusPython::ExifIFD:Flash","ExifIFD:Flash","Flash (ExifIFD)","Modalit\u00e0 del flash"),
nautilus.Column("NautilusPython::Composite:ImageSize","Composite:ImageSize","Risoluzione (Exif)","Risoluzione dell'immagine"),
nautilus.Column("NautilusPython::IFD0:Model","IFD0:Model","Fotocamera (IFD0)","Modello fotocamera"),
#nautilus.Column("NautilusPython::Nikon:ShutterCount","Nikon:ShutterCount","Contatore scatti (Nikon)","Numero di scatti effettuati dalla macchina a questo file"),
nautilus.Column("NautilusPython::Mp","Mp","Megapixel (Exif)","Dimensione dell'immagine in megapixel"),
)
def update_file_info_full(self, provider, handle, closure, file):
client = gconf.client_get_default()
if not client.get_bool('/apps/nautilus/nautilus-metadata/enable'):
client.set_bool('/apps/nautilus/nautilus-metadata/enable',0)
return
if file.get_uri_scheme() != 'file':
return
if file.get_mime_type() in ('image/jpeg', 'image/png', 'image/gif', 'image/bmp', 'image/x-nikon-nef', 'image/x-xcf', 'image/vnd.adobe.photoshop'):
gobject.timeout_add_seconds(1, self.update_exif, provider, handle, closure, file)
return Nautilus.OperationResult.IN_PROGRESS
file.add_string_attribute('ExifIFD:DateTimeOriginal','')
file.add_string_attribute('IFD0:Software','')
file.add_string_attribute('ExifIFD:Flash','')
file.add_string_attribute('Composite:ImageSize','')
file.add_string_attribute('IFD0:Model','')
file.add_string_attribute('Nikon:ShutterCount','')
file.add_string_attribute('Mp','')
return Nautilus.OperationResult.COMPLETE
def update_exif(self, provider, handle, closure, file):
filename = unquote(file.get_uri()[7:])
data = getexiftool(filename)
file.add_string_attribute('ExifIFD:DateTimeOriginal',data[0].replace(':','-',2))
file.add_string_attribute('IFD0:Software',data[1])
file.add_string_attribute('ExifIFD:Flash',data[2])
file.add_string_attribute('Composite:ImageSize',data[3])
file.add_string_attribute('IFD0:Model',data[4])
#file.add_string_attribute('Nikon:ShutterCount',data[5])
width, height = data[3].split('x')
mp = float(width) * float(height) / 1000000
mp = "%.2f" % mp
file.add_string_attribute('Mp',str(mp) + ' Mp')
Nautilus.info_provider_update_complete_invoke(closure, provider, handle, Nautilus.OperationResult.COMPLETE)
return false
That happens because you are invoking update_file_info, which is part of the asynchronous IO system of Nautilus. Therefore, it blocks nautilus if the operations are not fast enough.
In your case it is exacerbated because you are calling an external program, and that is an expensive operation. Notice that update_file_info is called once per file. If you have 100 files, then you will call 100 times the external program, and Nautilus will have to wait for each one before processing the next one.
Since nautilus-python 0.7 are available update_file_info_full and cancel_update, which allows you to program async calls. You can check the documentation of Nautilus 0.7 for more details.
It worth to mention this was a limitation of nautilus-python only, which previously did not expose those methods available in C.
EDIT: Added a couple of examples.
The trick is make the process as fast as possible or make it asynchronous.
Example 1: Invoking an external program
Using a simplified version of your code, we make asynchronous using
GObject.timeout_add_seconds in update_file_info_full.
from gi.repository import Nautilus, GObject
from urllib import unquote
from subprocess import Popen, PIPE
def getexiftool(filename):
options = '-fast2 -f -m -q -q -s3 -ExifIFD:DateTimeOriginal'
exiftool = Popen(['/usr/bin/exiftool'] + options.split() + [filename],
stdout=PIPE, stderr=PIPE)
output, errors = exiftool.communicate()
return output.split('\n')
class MyExtension(Nautilus.ColumnProvider, Nautilus.InfoProvider, GObject.GObject):
def __init__(self):
pass
def get_columns(self):
return (
Nautilus.Column(name='MyExif::DateTime',
attribute='Exif:Image:DateTime',
label='Date Original',
description='Data time original'
),
)
def update_file_info_full(self, provider, handle, closure, file_info):
if file_info.get_uri_scheme() != 'file':
return
filename = unquote(file_info.get_uri()[7:])
attr = ''
if file_info.get_mime_type() in ('image/jpeg', 'image/png'):
GObject.timeout_add_seconds(1, self.update_exif,
provider, handle, closure, file_info)
return Nautilus.OperationResult.IN_PROGRESS
file_info.add_string_attribute('Exif:Image:DateTime', attr)
return Nautilus.OperationResult.COMPLETE
def update_exif(self, provider, handle, closure, file_info):
filename = unquote(file_info.get_uri()[7:])
try:
data = getexiftool(filename)
attr = data[0]
except:
attr = ''
file_info.add_string_attribute('Exif:Image:DateTime', attr)
Nautilus.info_provider_update_complete_invoke(closure, provider,
handle, Nautilus.OperationResult.COMPLETE)
return False
The code above will not block Nautilus, and if the column 'Date Original' is available in the column view, the JPEG and PNG images will show the 'unknown' value, and slowly they will being updated (the subprocess is called after 1 second).
Examples 2: Using a library
Rather than invoking an external program, it could be better to use a library. As the example below:
from gi.repository import Nautilus, GObject
from urllib import unquote
import pyexiv2
class MyExtension(Nautilus.ColumnProvider, Nautilus.InfoProvider, GObject.GObject):
def __init__(self):
pass
def get_columns(self):
return (
Nautilus.Column(name='MyExif::DateTime',
attribute='Exif:Image:DateTime',
label='Date Original',
description='Data time original'
),
)
def update_file_info_full(self, provider, handle, closure, file_info):
if file_info.get_uri_scheme() != 'file':
return
filename = unquote(file_info.get_uri()[7:])
attr = ''
if file_info.get_mime_type() in ('image/jpeg', 'image/png'):
metadata = pyexiv2.ImageMetadata(filename)
metadata.read()
try:
tag = metadata['Exif.Image.DateTime'].value
attr = tag.strftime('%Y-%m-%d %H:%M')
except:
attr = ''
file_info.add_string_attribute('Exif:Image:DateTime', attr)
return Nautilus.OperationResult.COMPLETE
Eventually, if the routine is slow you would need to make it asynchronous (maybe using something better than GObject.timeout_add_seconds.
At last but not least, in my examples I used GObject Introspection (typically for Nautilus 3), but it easy to change it to use the module nautilus directly.
The above solution is only partly correct.
Between state changes for file_info metadata, the user should call file_info.invalidate_extension_info() to notify nautilus of the change.
Failing to do this could end up with 'unknown' appearing in your columns.
file_info.add_string_attribute('video_width', video_width)
file_info.add_string_attribute('video_height', video_height)
file_info.add_string_attribute('name_suggestion', name_suggestion)
file_info.invalidate_extension_info()
Nautilus.info_provider_update_complete_invoke(closure, provider, handle, Nautilus.OperationResult.COMPLETE)
Full working example here:
Fully working example
API Documentation
thanks to Dave!
i was looking for a solution to the 'unknown' text in the column for ages
file_info.invalidate_extension_info()
Fixed the issue for me right away :)
Per the api API Documentation
https://projects-old.gnome.org/nautilus-python/documentation/html/class-nautilus-python-file-info.html#method-nautilus-python-file-info--invalidate-extension-info
Nautilus.FileInfo.invalidate_extension_info
def invalidate_extension_info()
Invalidates the information Nautilus has about this file, which causes it to request new information from its Nautilus.InfoProvider providers.
Is there a Pythonic way to have only one instance of a program running?
The only reasonable solution I've come up with is trying to run it as a server on some port, then second program trying to bind to same port - fails. But it's not really a great idea, maybe there's something more lightweight than this?
(Take into consideration that program is expected to fail sometimes, i.e. segfault - so things like "lock file" won't work)
The following code should do the job, it is cross-platform and runs on Python 2.4-3.2. I tested it on Windows, OS X and Linux.
from tendo import singleton
me = singleton.SingleInstance() # will sys.exit(-1) if other instance is running
The latest code version is available singleton.py. Please file bugs here.
You can install tend using one of the following methods:
easy_install tendo
pip install tendo
manually by getting it from http://pypi.python.org/pypi/tendo
Simple, cross-platform solution, found in another question by zgoda:
import fcntl
import os
import sys
def instance_already_running(label="default"):
"""
Detect if an an instance with the label is already running, globally
at the operating system level.
Using `os.open` ensures that the file pointer won't be closed
by Python's garbage collector after the function's scope is exited.
The lock will be released when the program exits, or could be
released if the file pointer were closed.
"""
lock_file_pointer = os.open(f"/tmp/instance_{label}.lock", os.O_WRONLY)
try:
fcntl.lockf(lock_file_pointer, fcntl.LOCK_EX | fcntl.LOCK_NB)
already_running = False
except IOError:
already_running = True
return already_running
A lot like S.Lott's suggestion, but with the code.
This code is Linux specific. It uses 'abstract' UNIX domain sockets, but it is simple and won't leave stale lock files around. I prefer it to the solution above because it doesn't require a specially reserved TCP port.
try:
import socket
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
## Create an abstract socket, by prefixing it with null.
s.bind( '\0postconnect_gateway_notify_lock')
except socket.error as e:
error_code = e.args[0]
error_string = e.args[1]
print "Process already running (%d:%s ). Exiting" % ( error_code, error_string)
sys.exit (0)
The unique string postconnect_gateway_notify_lock can be changed to allow multiple programs that need a single instance enforced.
I don't know if it's pythonic enough, but in the Java world listening on a defined port is a pretty widely used solution, as it works on all major platforms and doesn't have any problems with crashing programs.
Another advantage of listening to a port is that you could send a command to the running instance. For example when the users starts the program a second time, you could send the running instance a command to tell it to open another window (that's what Firefox does, for example. I don't know if they use TCP ports or named pipes or something like that, 'though).
Never written python before, but this is what I've just implemented in mycheckpoint, to prevent it being started twice or more by crond:
import os
import sys
import fcntl
fh=0
def run_once():
global fh
fh=open(os.path.realpath(__file__),'r')
try:
fcntl.flock(fh,fcntl.LOCK_EX|fcntl.LOCK_NB)
except:
os._exit(0)
run_once()
Found Slava-N's suggestion after posting this in another issue (http://stackoverflow.com/questions/2959474). This one is called as a function, locks the executing scripts file (not a pid file) and maintains the lock until the script ends (normal or error).
Use a pid file. You have some known location, "/path/to/pidfile" and at startup you do something like this (partially pseudocode because I'm pre-coffee and don't want to work all that hard):
import os, os.path
pidfilePath = """/path/to/pidfile"""
if os.path.exists(pidfilePath):
pidfile = open(pidfilePath,"r")
pidString = pidfile.read()
if <pidString is equal to os.getpid()>:
# something is real weird
Sys.exit(BADCODE)
else:
<use ps or pidof to see if the process with pid pidString is still running>
if <process with pid == 'pidString' is still running>:
Sys.exit(ALREADAYRUNNING)
else:
# the previous server must have crashed
<log server had crashed>
<reopen pidfilePath for writing>
pidfile.write(os.getpid())
else:
<open pidfilePath for writing>
pidfile.write(os.getpid())
So, in other words, you're checking if a pidfile exists; if not, write your pid to that file. If the pidfile does exist, then check to see if the pid is the pid of a running process; if so, then you've got another live process running, so just shut down. If not, then the previous process crashed, so log it, and then write your own pid to the file in place of the old one. Then continue.
The best solution for this on windows is to use mutexes as suggested by #zgoda.
import win32event
import win32api
from winerror import ERROR_ALREADY_EXISTS
mutex = win32event.CreateMutex(None, False, 'name')
last_error = win32api.GetLastError()
if last_error == ERROR_ALREADY_EXISTS:
print("App instance already running")
Some answers use fctnl (included also in #sorin tendo package) which is not available on windows and should you try to freeze your python app using a package like pyinstaller which does static imports, it throws an error.
Also, using the lock file method, creates a read-only problem with database files( experienced this with sqlite3).
Here is my eventual Windows-only solution. Put the following into a module, perhaps called 'onlyone.py', or whatever. Include that module directly into your __ main __ python script file.
import win32event, win32api, winerror, time, sys, os
main_path = os.path.abspath(sys.modules['__main__'].__file__).replace("\\", "/")
first = True
while True:
mutex = win32event.CreateMutex(None, False, main_path + "_{<paste YOUR GUID HERE>}")
if win32api.GetLastError() == 0:
break
win32api.CloseHandle(mutex)
if first:
print "Another instance of %s running, please wait for completion" % main_path
first = False
time.sleep(1)
Explanation
The code attempts to create a mutex with name derived from the full path to the script. We use forward-slashes to avoid potential confusion with the real file system.
Advantages
No configuration or 'magic' identifiers needed, use it in as many different scripts as needed.
No stale files left around, the mutex dies with you.
Prints a helpful message when waiting
This may work.
Attempt create a PID file to a known location. If you fail, someone has the file locked, you're done.
When you finish normally, close and remove the PID file, so someone else can overwrite it.
You can wrap your program in a shell script that removes the PID file even if your program crashes.
You can, also, use the PID file to kill the program if it hangs.
For anybody using wxPython for their application, you can use the function wx.SingleInstanceChecker documented here.
I personally use a subclass of wx.App which makes use of wx.SingleInstanceChecker and returns False from OnInit() if there is an existing instance of the app already executing like so:
import wx
class SingleApp(wx.App):
"""
class that extends wx.App and only permits a single running instance.
"""
def OnInit(self):
"""
wx.App init function that returns False if the app is already running.
"""
self.name = "SingleApp-%s".format(wx.GetUserId())
self.instance = wx.SingleInstanceChecker(self.name)
if self.instance.IsAnotherRunning():
wx.MessageBox(
"An instance of the application is already running",
"Error",
wx.OK | wx.ICON_WARNING
)
return False
return True
This is a simple drop-in replacement for wx.App that prohibits multiple instances. To use it simply replace wx.App with SingleApp in your code like so:
app = SingleApp(redirect=False)
frame = wx.Frame(None, wx.ID_ANY, "Hello World")
frame.Show(True)
app.MainLoop()
Using a lock-file is a quite common approach on unix. If it crashes, you have to clean up manually. You could stor the PID in the file, and on startup check if there is a process with this PID, overriding the lock-file if not. (However, you also need a lock around the read-file-check-pid-rewrite-file). You will find what you need for getting and checking pid in the os-package. The common way of checking if there exists a process with a given pid, is to send it a non-fatal signal.
Other alternatives could be combining this with flock or posix semaphores.
Opening a network socket, as saua proposed, would probably be the easiest and most portable.
I'm posting this as an answer because I'm a new user and Stack Overflow won't let me vote yet.
Sorin Sbarnea's solution works for me under OS X, Linux and Windows, and I am grateful for it.
However, tempfile.gettempdir() behaves one way under OS X and Windows and another under other some/many/all(?) *nixes (ignoring the fact that OS X is also Unix!). The difference is important to this code.
OS X and Windows have user-specific temp directories, so a tempfile created by one user isn't visible to another user. By contrast, under many versions of *nix (I tested Ubuntu 9, RHEL 5, OpenSolaris 2008 and FreeBSD 8), the temp dir is /tmp for all users.
That means that when the lockfile is created on a multi-user machine, it's created in /tmp and only the user who creates the lockfile the first time will be able to run the application.
A possible solution is to embed the current username in the name of the lock file.
It's worth noting that the OP's solution of grabbing a port will also misbehave on a multi-user machine.
Building upon Roberto Rosario's answer, I come up with the following function:
SOCKET = None
def run_single_instance(uniq_name):
try:
import socket
global SOCKET
SOCKET = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
## Create an abstract socket, by prefixing it with null.
# this relies on a feature only in linux, when current process quits, the
# socket will be deleted.
SOCKET.bind('\0' + uniq_name)
return True
except socket.error as e:
return False
We need to define global SOCKET vaiable since it will only be garbage collected when the whole process quits. If we declare a local variable in the function, it will go out of scope after the function exits, thus the socket be deleted.
All the credit should go to Roberto Rosario, since I only clarify and elaborate upon his code. And this code will work only on Linux, as the following quoted text from https://troydhanson.github.io/network/Unix_domain_sockets.html explains:
Linux has a special feature: if the pathname for a UNIX domain socket
begins with a null byte \0, its name is not mapped into the
filesystem. Thus it won’t collide with other names in the filesystem.
Also, when a server closes its UNIX domain listening socket in the
abstract namespace, its file is deleted; with regular UNIX domain
sockets, the file persists after the server closes it.
Late answer, but for windows you can use:
from win32event import CreateMutex
from win32api import CloseHandle, GetLastError
from winerror import ERROR_ALREADY_EXISTS
import sys
class singleinstance:
""" Limits application to single instance """
def __init__(self):
self.mutexname = "testmutex_{D0E858DF-985E-4907-B7FB-8D732C3FC3B9}"
self.mutex = CreateMutex(None, False, self.mutexname)
self.lasterror = GetLastError()
def alreadyrunning(self):
return (self.lasterror == ERROR_ALREADY_EXISTS)
def __del__(self):
if self.mutex:
CloseHandle(self.mutex)
Usage
# do this at beginnig of your application
myapp = singleinstance()
# check is another instance of same program running
if myapp.alreadyrunning():
print ("Another instance of this program is already running")
sys.exit(1)
Here is a cross platform example that I've tested on Windows Server 2016 and Ubuntu 20.04 using Python 3.7.9:
import os
class SingleInstanceChecker:
def __init__(self, id):
if isWin():
ensure_win32api()
self.mutexname = id
self.lock = win32event.CreateMutex(None, False, self.mutexname)
self.running = (win32api.GetLastError() == winerror.ERROR_ALREADY_EXISTS)
else:
ensure_fcntl()
self.lock = open(f"/tmp/isnstance_{id}.lock", 'wb')
try:
fcntl.lockf(self.lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
self.running = False
except IOError:
self.running = True
def already_running(self):
return self.running
def __del__(self):
if self.lock:
try:
if isWin():
win32api.CloseHandle(self.lock)
else:
os.close(self.lock)
except Exception as ex:
pass
# ---------------------------------------
# Utility Functions
# Dynamically load win32api on demand
# Install with: pip install pywin32
win32api=winerror=win32event=None
def ensure_win32api():
global win32api,winerror,win32event
if win32api is None:
import win32api
import winerror
import win32event
# Dynamically load fcntl on demand
# Install with: pip install fcntl
fcntl=None
def ensure_fcntl():
global fcntl
if fcntl is None:
import fcntl
def isWin():
return (os.name == 'nt')
# ---------------------------------------
Here is it in use:
import time, sys
def main(argv):
_timeout = 10
print("main() called. sleeping for %s seconds" % _timeout)
time.sleep(_timeout)
print("DONE")
if __name__ == '__main__':
SCR_NAME = "my_script"
sic = SingleInstanceChecker(SCR_NAME)
if sic.already_running():
print("An instance of {} is already running.".format(SCR_NAME))
sys.exit(1)
else:
main(sys.argv[1:])
I use single_process on my gentoo;
pip install single_process
example:
from single_process import single_process
#single_process
def main():
print 1
if __name__ == "__main__":
main()
refer: https://pypi.python.org/pypi/single_process/
I keep suspecting there ought to be a good POSIXy solution using process groups, without having to hit the file system, but I can't quite nail it down. Something like:
On startup, your process sends a 'kill -0' to all processes in a particular group. If any such processes exist, it exits. Then it joins the group. No other processes use that group.
However, this has a race condition - multiple processes could all do this at precisely the same time and all end up joining the group and running simultaneously. By the time you've added some sort of mutex to make it watertight, you no longer need the process groups.
This might be acceptable if your process only gets started by cron, once every minute or every hour, but it makes me a bit nervous that it would go wrong precisely on the day when you don't want it to.
I guess this isn't a very good solution after all, unless someone can improve on it?
I ran into this exact problem last week, and although I did find some good solutions, I decided to make a very simple and clean python package and uploaded it to PyPI. It differs from tendo in that it can lock any string resource name. Although you could certainly lock __file__ to achieve the same effect.
Install with: pip install quicklock
Using it is extremely simple:
[nate#Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> # Let's create a lock so that only one instance of a script will run
...
>>> singleton('hello world')
>>>
>>> # Let's try to do that again, this should fail
...
>>> singleton('hello world')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/nate/live/gallery/env/lib/python2.7/site-packages/quicklock/quicklock.py", line 47, in singleton
raise RuntimeError('Resource <{}> is currently locked by <Process {}: "{}">'.format(resource, other_process.pid, other_process.name()))
RuntimeError: Resource <hello world> is currently locked by <Process 24801: "python">
>>>
>>> # But if we quit this process, we release the lock automatically
...
>>> ^D
[nate#Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> singleton('hello world')
>>>
>>> # No exception was thrown, we own 'hello world'!
Take a look: https://pypi.python.org/pypi/quicklock
linux example
This method is based on the creation of a temporary file automatically deleted after you close the application.
the program launch we verify the existence of the file;
if the file exists ( there is a pending execution) , the program is closed ; otherwise it creates the file and continues the execution of the program.
from tempfile import *
import time
import os
import sys
f = NamedTemporaryFile( prefix='lock01_', delete=True) if not [f for f in os.listdir('/tmp') if f.find('lock01_')!=-1] else sys.exit()
YOUR CODE COMES HERE
On a Linux system one could also ask
pgrep -a for the number of instances, the script
is found in the process list (option -a reveals the
full command line string). E.g.
import os
import sys
import subprocess
procOut = subprocess.check_output( "/bin/pgrep -u $UID -a python", shell=True,
executable="/bin/bash", universal_newlines=True)
if procOut.count( os.path.basename(__file__)) > 1 :
sys.exit( ("found another instance of >{}<, quitting."
).format( os.path.basename(__file__)))
Remove -u $UID if the restriction should apply to all users.
Disclaimer: a) it is assumed that the script's (base)name is unique, b) there might be race conditions.
Here's a good example for django with contextmanager and memcached:
https://docs.celeryproject.org/en/latest/tutorials/task-cookbook.html
Can be used to protect simultaneous operation on different hosts.
Can be used to manage multiple tasks.
Can also be changed for simple python scripts.
My modification of the above code is here:
import time
from contextlib import contextmanager
from django.core.cache import cache
#contextmanager
def memcache_lock(lock_key, lock_value, lock_expire):
timeout_at = time.monotonic() + lock_expire - 3
# cache.add fails if the key already exists
status = cache.add(lock_key, lock_value, lock_expire)
try:
yield status
finally:
# memcache delete is very slow, but we have to use it to take
# advantage of using add() for atomic locking
if time.monotonic() < timeout_at and status:
# don't release the lock if we exceeded the timeout
# to lessen the chance of releasing an expired lock owned by someone else
# also don't release the lock if we didn't acquire it
cache.delete(lock_key)
LOCK_EXPIRE = 60 * 10 # Lock expires in 10 minutes
def main():
lock_name, lock_value = "lock_1", "locked"
with memcache_lock(lock_name, lock_value, LOCK_EXPIRE) as acquired:
if acquired:
# single instance code here:
pass
if __name__ == "__main__":
main()
Here is a cross-platform implementation, creating a temporary lock file using a context manager.
Can be used to manage multiple tasks.
import os
from contextlib import contextmanager
from time import sleep
class ExceptionTaskInProgress(Exception):
pass
# Context manager for suppressing exceptions
class SuppressException:
def __init__(self):
pass
def __enter__(self):
return self
def __exit__(self, *exc):
return True
# Context manager for task
class TaskSingleInstance:
def __init__(self, task_name, lock_path):
self.task_name = task_name
self.lock_path = lock_path
self.lock_filename = os.path.join(self.lock_path, self.task_name + ".lock")
if os.path.exists(self.lock_filename):
raise ExceptionTaskInProgress("Resource already in use")
def __enter__(self):
self.fl = open(self.lock_filename, "w")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.fl.close()
os.unlink(self.lock_filename)
# Here the task is silently interrupted
# if it is already running on another instance.
def main1():
task_name = "task1"
tmp_filename_path = "."
with SuppressException():
with TaskSingleInstance(task_name, tmp_filename_path):
print("The task `{}` has started.".format(task_name))
# The single task instance code is here.
sleep(5)
print("The task `{}` has completed.".format(task_name))
# Here the task is interrupted with a message
# if it is already running in another instance.
def main2():
task_name = "task1"
tmp_filename_path = "."
try:
with TaskSingleInstance(task_name, tmp_filename_path):
print("The task `{}` has started.".format(task_name))
# The single task instance code is here.
sleep(5)
print("Task `{}` completed.".format(task_name))
except ExceptionTaskInProgress as ex:
print("The task `{}` is already running.".format(task_name))
if __name__ == "__main__":
main1()
main2()
import sys,os
# start program
try: # (1)
os.unlink('lock') # (2)
fd=os.open("lock", os.O_CREAT|os.O_EXCL) # (3)
except:
try: fd=os.open("lock", os.O_CREAT|os.O_EXCL) # (4)
except:
print "Another Program running !.." # (5)
sys.exit()
# your program ...
# ...
# exit program
try: os.close(fd) # (6)
except: pass
try: os.unlink('lock')
except: pass
sys.exit()