What is the best/easiest why to check if a specific Python script already running in Windows?
I have a script that goes over all files in a folder and copies them to another folder (sort to Movie or TV Shows folder).
I want to make sure when the script starts that there isn't another process (of the same script) that is already running, so I wouldn't have issues with 2 scripts that are trying to move the same files.
I have tried to create a file in the start of the script and deleting it when the script finishes, but I got into problems when the script crashes and/or throws an error.
I know that I can use psutil, but then I will get the process name (python.exe) and I'm looking for a why to distinguish if the Python process is running my script or another program.
You can use psutil.Process().cmdline() to see the complete command line of a process.
Alternatively, you could lock the files you're working on. See the answer to this question how to do this on ms-windows. The thing with locks is that you have to be careful to remove them, especially when an error occurs.
For Windows o.s. you can use timestamp.txt file
timestamp = 'timestamp.txt'
...
elif windows:
try:
new_timestamp = False
if not os.path.exists(timestamp):
new_timestamp = True
try:
with open(timestamp, 'a') as f_timestamp:
f_timestamp.write(str(int_t0))
except IOError as e:
out1 = 'M. Cannot open file for writing. Error: %s - %s.' \
% (e.logfile, e.strerror) + ' -> Exit code 3'
logging.error(out1)
sys.exit(3)
if not new_timestamp and os.path.exists(timestamp):
out1 = 'N. Script ' + __file__ + ' is already running.'
print(out1)
logging.error(out1)
sys.exit(0)
except IOError as e:
out1 = 'J. Cannot open file. Error: %s - %s.' \
% (e.filepath, e.strerror) + ' -> Exit code 4'
logging.error(out1)
...
try:
f_timestamp.close()
os.remove(timestamp)
except OSError as e:
logging.error('B. Cannot delete ' + timestamp + \
' Error: %s - %s.' % (e.filename, e.strerror))
Use lockfile. It is cross-platform, uses native OS functions and much more robust than any home-brewn lock file creation schemas
I have solved it by using an empty dummy file.
At the start of the process I check if the file exists, if not I create a new file, run the process and delete it in the end (even if the process failed), if the file does exist, that means that the process is running now so I terminate the current (new) process.
Related
I'm nearly finished with a PyQt application for Windows, which I would like to have automatically update from a ZIP archive hosted on a remote server.
I have most of the update script complete, with it correctly downloading the new source and extracting it. My final step is to stop the software, replace the old source and restart the app.
My question is about which of the following is more appropriate:
to run the updater script through a system call to python and killing the software using python.
run the updater script using a system call to a batch file that kills the main software before overwriting it's source.
import the updater as a module and perform all of it's actions in the same process as the main software.
I can provide the scripts if necessary.
UPDATE:
So I've been exploring all the ways to do this including using multiprocessing (which produced a child process that was killed along with the parent) and subprocess.
The latter apparently can run the child process separately, which will allow me to shutdown the main app before extracting the new source. Here's what I have working:
#staticmethod
def install(folder):
# stop Pierre, unpack newest version, then restart Pierre.
try:
with open('pierre.pid', mode='r') as pid:
os.kill(int(pid.read()), signal.SIGINT)
with zipfile.ZipFile(file=folder) as zipped:
zipped.extractall(path='src')
try:
pierre = os.path.join(os.path.abspath(os.getcwd()), 'src/pierre.py')
exec(pierre)
except OSError as exc:
logging.error("Restarting Pierre failed. " + str(exc))
try:
os.remove('src.zip')
except OSError as exc:
logging.error("Deletion of zip failed. " + str(exc))
except zipfile.BadZipFile:
logging.error("Pierre update file is corrupt.")
except Exception as exc:
logging.error("Pierre update install failed. " + str(exc))
What's not working:
#staticmethod
def update_process():
# Begin the update process by spawning the updater script.
script = 'python ' + os.getcwd() + '\\updater.py'
subprocess.Popen([script])
The subprocess is producing a FileNotFoundError despite the path working when run manually in command prompt. (The second method is what starts the script, leading to the first method.)
I've got it. Here's what the process spawner looks like now:
#staticmethod
def update_process():
# Begin the update process by spawning the updater script.
script = os.path.join(os.getcwd() + '/updater.py')
script = script.replace('\\', '/')
subprocess.Popen([sys.executable, script], shell=True)
This launches the separate updater script.
I have the following code written in Python 2.7 on Windows. I want to check for updates for the current python script and update it, if there is an update, with a new version through ftp server preserving the filename and then executing the new python script after terminating the current through the os.kill with SIGNTERM.
I went with the exit function approach but I read that in Windows this only works with the atexit library and default python exit methods. So I used a combination of the atexit.register() and the signal handler.
***necessary libraries***
filematch = 'test.py'
version = '0.0'
checkdir = os.path.abspath(".")
dircontent = os.listdir(checkdir)
r = StringIO()
def exithandler():
try:
try:
if filematch in dircontent:
os.remove(checkdir + '\\' + filematch)
except Exception as e:
print e
ftp = FTP(ip address)
ftp.login(username, password)
ftp.cwd('/Test')
for filename in ftp.nlst(filematch):
fhandle = open(filename, 'wb')
ftp.retrbinary('RETR ' + filename, fhandle.write)
fhandle.close()
subprocess.Popen([sys.executable, "test.py"])
print 'Test file successfully updated.'
except Exception as e:
print e
ftp = FTP(ip address)
ftp.login(username, password)
ftp.cwd('/Test')
ftp.retrbinary('RETR version.txt', r.write)
if(r.getvalue() != version):
atexit.register(exithandler)
somepid = os.getpid()
signal.signal(SIGTERM, lambda signum, stack_frame: exit(1))
os.kill(somepid, signal.SIGTERM)
print 'Successfully replaced and started the file'
Using the:
signal.signal(SIGTERM, lambda signum, stack_frame: exit(1))
I get:
Traceback (most recent call last):
File "C:\Users\STiX\Desktop\Python Keylogger\test.py", line 50, in <module>
signal.signal(SIGTERM, lambda signum, stack_frame: exit(1))
NameError: name 'SIGTERM' is not defined
But I get the job done without a problem except if I use the current code in a more complex script where the script give me the same error but terminates right away for some reason.
On the other hand though, if I use it the correct way, signal.SIGTERM, the process goes straight to termination and the exit function never executed. Why is that?
How can I make this work on Windows and get the outcome that I described above successfully?
What you are trying to do seems a bit complicated (and dangerous from an infosec-perspective ;-). I would suggest to handle the reload-file-when-updated part of the functionality be adding a controller class that imports the python script you have now as a module and, starts it and the reloads it when it is updated (based on a function return or other technique) - look this way for inspiration - https://stackoverflow.com/a/1517072/1010991
Edit - what about exe?
Another hacky technique for manipulating the file of the currently running program would be the shell ping trick. It can be used from all programming languages. The trick is to send a shell command that is not executed before after the calling process has terminated. Use ping to cause the delay and chain the other commands with &. For your use case it could be something like this:
import subprocess
subprocess.Popen("ping -n 2 -w 2000 1.1.1.1 > Nul & del hack.py & rename hack_temp.py hack.py & hack.py ", shell=True)
Edit 2 - Alternative solution to original question
Since python does not block write access to the currently running script an alternative concept to solve the original question would be:
import subprocess
print "hello"
a = open(__file__,"r")
running_script_as_string = a.read()
b = open(__file__,"w")
b.write(running_script_as_string)
b.write("\nprint 'updated version of hack'")
b.close()
subprocess.Popen("python hack.py")
I want to call a Python script from Jenkins and have it build my app, FTP it to the target, and run it.
I am trying to build and the subprocess command fails. I have tried this with both subprocess.call() and subprocess.popen(), with the same result.
When I evaluate shellCommand and run it from the command line, the build succeeds.
Note that I have 3 shell commands: 1) remove work directory, 2) create a fresh, empty, work directory, then 3) build. The first two commands return from subprocess, but the third hangs (although it completes when run from the command line).
What am I doing wrongly? Or, what alternatives do I have for executing that command?
# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
def ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand):
try:
process = subprocess.call(shellCommand, shell=True, stdout=subprocess.PIPE)
#process.wait()
return process #.returncode
except KeyboardInterrupt, e: # Ctrl-C
raise e
except SystemExit, e: # sys.exit()
raise e
except Exception, e:
print 'Exception while executing shell command : ' + shellCommand
print str(e)
traceback.print_exc()
os._exit(1)
# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
def BuildApplciation(arguments):
# See http://gnuarmeclipse.github.io/advanced/headless-builds/
jenkinsWorkspaceDirectory = arguments.eclipseworkspace + '/jenkins'
shellCommand = 'rm -r ' + jenkinsWorkspaceDirectory
ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand)
shellCommand = 'mkdir ' + jenkinsWorkspaceDirectory
if not ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand) == 0:
print "Error making Jenkins work directory in Eclipse workspace : " + jenkinsWorkspaceDirectory
return False
application = 'org.eclipse.cdt.managedbuilder.core.headlessbuild'
shellCommand = 'eclipse -nosplash -application ' + application + ' -import ' + arguments.buildRoot + '/../Project/ -build myAppApp/TargetRelease -cleanBuild myAppApp/TargetRelease -data ' + jenkinsWorkspaceDirectory + ' -D DO_APPTEST'
if not ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand) == 0:
print "Error in build"
return False
I Googled further and found this page, which, at 1.2 says
One way of gaining access to the output of the executed command would
be to use PIPE in the arguments stdout or stderr, but the child
process will block if it generates enough output to a pipe to fill up
the OS pipe buffer as the pipes are not being read from.
Sure enough, when I deleted the , stdout=subprocess.PIPE from the code above, it worked as expected.
As I only want the exit code from the subprocess, the above code is enough for me. Read the linked page if you want the output of the command.
I am trying to to run rsync for each folder in a folder.
__author__ = 'Alexander'
import os
import subprocess
root ='/data/shares'
arguments=["--verbose", "--recursive", "--dry-run", "--human-readable", "--remove-source-files"]
remote_host = 'TL-AS203'
for folder in os.listdir(root):
print 'Sync Team ' + folder.__str__()
path = os.path.join(root,folder, 'in')
if os.path.exists(path):
folder_arguments = list(arguments)
print (type(folder_arguments))
folder_arguments.append("--log-file=" + path +"/rsync.log")
folder_arguments.append(path)
folder_arguments.append("transfer#"+remote_host+":/data/shares/"+ folder+"/out")
print "running rsync with " + str(folder_arguments)
returncode = subprocess.call(["rsync",str(folder_arguments)])
if returncode == 0:
print "pull successfull"
else:
print "error during rsync pull"
else:
print "not a valid team folder, in not found"
If I run this I get the following output:
Sync Team IT-Systemberatung
<type 'list'>
running rsync with ['--verbose', '--recursive', '--dry-run', '--human-readable', '--remove-source-files', '--log-file=/data/shares/IT-Systemberatung/in/rsync.log', '/data/shares/IT-Systemberatung/in', 'transfer#TL-AS203:/data/shares/IT-Systemberatung/out']
rsync: change_dir "/data/shares/IT-Systemberatung/['--verbose', '--recursive', '--dry-run', '--human-readable', '--remove-source-files', '--log-file=/data/shares/IT-Systemberatung/in/rsync.log', '/data/shares/IT-Systemberatung/in', 'transfer#TL-AS203:/data/shares/IT-Systemberatung" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1040) [sender=3.0.4]
error during rsync pull
Sync Team IT-Applikationsbetrieb
not a valid team folder, in not found
transfer#INT-AS238:/data/shares/IT-Systemberatung
If i manually start rsync from bash with these arguments, everything works fine. I also tried it with shell=true but with the same result.
You need to do:
returncode = subprocess.call(["rsync"] + folder_arguments)
Calling str() on a list will return the string represention of the python list which is not what you want to pass in as an argument to rsync
You do a os.chdir(os.path.join(root,folder)), but never go back.
In order to properly resume operation on the next folder, you should either remember the last os.getpwd() and return to it, or just do os.chdir('..') at the end of one loop run.
Why isn't os.remove(-string-) working for me? I have the code written as follows:
try:
os.remove(a)
output = current_time() + "\trmv successful"
message = message + '\n' + output
message = "".join(message)
return message
except OSError:
try:
os.removedirs(a)
output = current_time() + "\trmv successful"
message = message + '\n' + output
message = "".join(message)
return message
except OSError:
output = current_time() + "\trmv failed: [?]"
message = message + '\n' + output
message = "".join(message)
return message
And it would return 21:32:53 rmv failed: [?] every time I perform the rmv command in the client. My Python version is 2.6.1 if that helps.
Exceptions are there to be looked at! Check this:
try:
os.remove(a)
except OSError as e: # name the Exception `e`
print "Failed with:", e.strerror # look what it says
print "Error code:", e.code
Modify your code to display the error message and you'll know why it failed. The docs can help you.
Why don't you try printing out the error?
try:
os.remove(a)
output = current_time() + "\trmv successful"
message = message + '\n' + output
message = "".join(message)
return message
except OSError, e:
print ("Failed to remove %s\nError is: %s" % (a,e))
try:
os.removedirs(a)
output = current_time() + "\trmv successful"
message = message + '\n' + output
message = "".join(message)
return message
except OSError, e:
print ("Failed twice to remove %s\nError is: %s" % (a,e))
output = current_time() + "\trmv failed: [?]"
message = message + '\n' + output
message = "".join(message)
return message
The error could be literally anything you see... A permissions issue for example?
try putting some delay time.sleep(0.2) after opening / removing files Or
It seems a windows and/or antivirus issue
Josh Rosenberg on this error tracking on python development points out the same:
Short version: Indexing and anti-virus tools prevent deletion from occurring.
Longer version:
DeleteFile (and all the stuff that ultimately devolves to DeleteFile) operate in a funny way on Windows. Internally, it opens a HANDLE to the file, marks it as pending deletion, and closes the HANDLE. If no one snuck in and grabbed another HANDLE to the file during that time, then the file is deleted when DeleteFile's hidden HANDLE is closed. Well designed anti-virus/indexing tools use oplocks ( http://blogs.msdn.com/b/oldnewthing/archive/2013/04/15/10410965.aspx ) so they can open a file, but seamlessly get out of the way if a normal process needs to take exclusive control of a file or delete it. Sadly "well-designed" is not a term usually associated with anti-virus tools, so errors like this are relatively commonplace.
Workarounds like using GetTempFileName() and MoveFile() to move the file out of the way will work, though I believe they introduce their own race conditions (the temp file itself is created but the HANDLE is closed immediately, which could mean a race to open the empty file by the bad anti-virus that would block MoveFile()).
Basically, if you're running on Windows, and you're using unfriendly anti-virus/indexing tools, there is no clean workaround that maintains the same behavior. You can't keep creating and deleting a file of the same name over and over without risking access denied errors.
That said, you could probably get the same results by opening and closing the file only once. Change from the original pseudocode: