So I've researched and tested multiple methods to get this to work, but both our IT guy and myself are unsure how to proceed. Ultimately I need to run three python scripts on a server which look at a set of Excel files and scans them for errors. If there are errors, it will email the appropriate people to fix them. The scripts themselves work fine when run through the CMD prompt, either individually or in a batch file. These scripts work perfectly when run manually from the command prompt.
However, when I try to schedule one of the scripts or the batch file, they do not run in their entirety. I have a print statement early on in the scripts and that runs fine in the Python.exe, but then the script abruptly quits. This makes me think that Python runs fine, but the contents of the script aren't running correctly through the scheduler.
To simplify troubleshooting I created two test scripts, test.py and test2.py, and placed them in a simple directory of the server, C:\Store. All this first script does is print something to the console, and write something to a file and save it to the root directory, after which it will wait for user input. The second script is simpler and just prints the architecture of Python and it's build path. Both scripts work fine when run from the command prompt manually.
**test.py** Does NOT Work as scheduled task
import io
print('JUST WORRRRRKKK!')
with open("//kaicmapp/Store/foobar.txt", "w") as file:
file.write("Hello!")
file.close()
raw_input("press any button")
**test2.py** Works as scheduled task
import sys
import platform
print("Python EXE : " + sys.executable)
print("Architecture : " + platform.architecture()[0])
raw_input("\n\nPress ENTER to quit")
What I am using for these test scripts:
Python 2.7 32 bit
Windows Server 2012
Windows User Profile with Admin Rights
Python 2.7 exe added as environment variable. Renamed to python2.exe in installation directory, as I also have Python 3 installed on the serverand want to be able to run them exclusively
Things I've tried:
Running Task Scheduler as Admin
Run with/without highest privileges
Run only when current user is logged in
Run regardless of current user logged in, password entered correctly
Run the scripts in a batch file
The solution listed here, results in same outcome in both scripts
Things I've noticed:
Using python2 "C:\Store\test2.py" in the scheduler will run the script successfully, with the prompt waiting for user input. Run result from the scheduler says The operation completed successfully. (0x0)
Using the same task as above, but running test.py instead, will not result any output from the Python shell, it simply pops up and disappears. Run result from scheduler says Incorrect function. (0x80070001)
Any ideas on how to proceed? I am not very familiar with server administrative tasks, so I feel pretty lost here.
This is a Microsoft scheduler issue.
https://social.technet.microsoft.com/Forums/windows/en-US/c8ad90e1-8435-4e13-9af9-098e72a1926d/error-code-incorrect-function-0x80070001-when-running-a-scheduled-task?forum=winserver8gen
Please check if this solution works for you.
Related
I have a batch file to execute Python and run a Python script that works fine when run manually. But when run using Windows Task Scheduler, the Python Script gives me a traceback because it can't find a .TXT file that's in the same file as the Python script.
There are many flavors of the "batch file works manually but not with Task Scheduler" problem. I think my problem may be related to the fact that my Python script is in a virtual environment. It may also be related to Windows user accounts and permissions. Past posts about Task Scheduler have been resolved with changing settings related to user accounts but so far that hasn't worked for me.
More details on Windows Task Scheduler settings:
Here are the settings I've chosen for the task:
My task gets closest to working when I use the "run only when user is logged on option. If I choose "run whether user is logged on or not" the task opens a Command Prompt but doesn't appear to execute anything.
More details about batch file
My batch file looks like this:
"C:\Users\MyPathtovirtualenvironment\virtualenv\Scripts\python.exe" "C:\Users\fried\Desktop\calendaralert\court_calendar_alert1.3_automatic.py"
pause
When run manually, it does what it's supposed to: it executes Python from within the virtual environment, then loads the Python script, which runs correctly except when it's run from Windows Task Scheduler.
More details on Python script
The Python script works fine when run by itself in the command prompt and as a batch file, so I don't think there's anything directly wrong with it. Here's the part of the script that causes the traceback.
with open('replacements.txt') as f:
replacements = dict(x.rstrip().split("!") for x in f)
The replacements.txt file is in the same folder as my Python program. I also tried putting copies of the file within the Virtual Environment folder and the /scripts subfolder without success.
Just came across this problem, while running a python script on a windows ec2 instance. The problem got solved by adding this at the top:
import os
path = "your-directory-path"
os.chdir(path)
I have a python script that starts a program and automates through it, continuously processing new data and saves to a pre-set directory.
What is the recommended way to run the Python script forever, logging errors when they occur, and restarting when it crashes?
so far I've came across the os.execv and have this to start:
import sys
import os
def pyexcept(t, v, tb):
import traceback
## restarts the script
os.execv( sys.executable, '')
but I often get stuck trying to figure out the next step, could someone explain the next steps i could take, ty!
When the python script crashes, the program is not running anymore, therefore the script cannot execute more lines of code.
You have 2 options:
Make sure your python script doesn't crash, which is very much recommended. You can do this by handling the exceptions thrown by your program.
Option 1
I assume you are new to python, so here is an example of a python script that handles an exception calls the same function again.
from time import sleep
def run_forever():
try:
# Create infinite loop to simulate whatever is running
# in your program
while True:
print("Hello!")
sleep(10)
# Simulate an exception which would crash your program
# if you don't handle it!
raise Exception("Error simulated!")
except Exception:
print("Something crashed your program. Let's restart it")
run_forever() # Careful.. recursive behavior
# Recommended to do this instead
handle_exception()
def handle_exception():
# code here
pass
run_forever()
If you want to restart the python script you would need another python script (assuming you want to do this with python) that checks if the process is still alive and if not then run it again with python.
Option 2
This is the script that starts another python script called 'test.py' via the command python test.py.
Make sure you have the right file path, if you put the scripts in the same folder, you usually don't need the full path and only the script name.
Notably, make sure that command 'python' is recognized by your system, it could in some cases by 'python3'
script_starter.py
from subprocess import run
from time import sleep
# Path and name to the script you are trying to start
file_path = "test.py"
restart_timer = 2
def start_script():
try:
# Make sure 'python' command is available
run("python "+file_path, check=True)
except:
# Script crashed, lets restart it!
handle_crash()
def handle_crash():
sleep(restart_timer) # Restarts the script after 2 seconds
start_script()
start_script()
In case you are interested in the code I used for the test file: 'test.py', I post it here.
test.py
from time import sleep
while True:
sleep(1)
print("Hello")
raise Exception("Hello")
It should be self-evident that if the Python script crashes, it cannot restart itself as it is no longer resident in memory. The answer, then, must come from something outside of Python operating at a lower level of abstraction than an application.
Linux often solves this with systemd. You could create a unit file describing the run of your application and start that service. systemd will then monitor the running process and restart it if it dies.
Windows usually solves this with Windows Services, which are executables with a special win32 API wrapper around them to allow the system to make calls into them to check their status, stop them, or restart them. NSSM (Non-Sucking Service Manager) is a helpful utility to create a service from any arbitrary command.
You have to use some external service to watch your python application. In windows you can run your python application as a windows service using Windows Service Wrapper.
Windows service wrapper (WinSW) is originally developed for run Jenkins client and server as a Windows service in Windows machine. But unlike other service wrappers, WinSW allow you to run any other applications like python.
You can provide all the configuration details as a XML file.
Few configurations you may interested
executable --> you can define which executable you wish to run (python in your case)
aruments --> you can provide arguments for the executable (ex - you can provide your application)
onFailure --> You can use onfailure configuration to specify what to do if your applications fails.
logpath --> you can specify the location for your logs.
and many more. Please read this file to know about all the configurations.
You should place your configuration XML file in the same directory where the WinSW executable located. Also there are few must implemented configurations which need to manage windows service (id, name, description, executable).
You can download the latest version from this link.
I am currently working on a python script to automatically parse a large amount of emails created by a registration form. Due to constraints outside my control, the way I chose to do this is to mirror all mails through offlineimap, run a script that processes all mails and files the relevant info into the right places and then run offlineimap again to move all processed mail into a different folders. Being on Windows(7), I run offlineimap through the Cygwin Terminal.
In order to allow not-so-tech-versed colleagues use my script, I wanted to include both runs of offlineimap into the script using:
import subprocess
subprocess.run("offlineimap", shell=True, timeout=20)
I get as a result:
CompletedProcess(args='offlineimap', returncode=1)
Sadly I'm rather new to Python and the subprocess.run command appears to be so new that google didn't yield too many examples. But the problem appears to be that offlineimap is run in the Windows command shell and it fails.
So the solutions to my problem would be either:
a) utilize the cygwin terminal through python
b) run offlineimap through the windows command shell
Sadly the documentation of offlineimap only mentions Windows in passing and the tool seems to be written to be run on linux machines. So I'm not holding my breath for option b) but any advise would be appreciated.
I just set up my first aws server for a personal project I'm doing. I'm running ubuntu linux, and I have a python script that accesses an sqlite database file in order to send email. I have these same files on my own ubuntu machine and the script works fine. I'm having trouble, however, figuring out how to run my script from the terminal in my aws vm. Normally I use idle to run my python script on my linux machine, so I'm trying to figure out how to run it from the terminal and it's giving me some trouble.
I tried
python script.py
which did nothing, so I converted it to an executable, ran it, and got
./script.py: line 1: import: command not found
...and so on
I realized that I had to add
#!/usr/bin/env python3
to my script, so I did that, converted to executable again, and ran it by entering
./script.py
which also did nothing. If the program had run, it would have delivered an email to my inbox. Is there any way I can tell if it's actually trying to run my script? Or am I trying to run it incorrectly?
You can modify the script to add verbose that prints out to the console the status of the script, or if you just want to know whether your script is running in the background, you can check whether the process is active using ps (process name would be the name of the script) :
ps aux | grep "script.py"
Anyways the former is a better practice, since you can exactly know execution flow of your script.
Not sure if anyone has run into this, but I'll take suggestions for troubleshooting and/or alternative methods.
I have a Windows 2008 server on which I am running several scheduled tasks. One of those tasks is a python script that uses pscp to log into a linux box, checks for new files and if there is anything new, copies them down to a local directory on the C: drive. I've put some logging into the script at key points as well and I'm using logging.basicConfig(level=DEBUG).
I built the command using a variable, command = 'pscp -pw xxxx name#ip:/ c:\local_dir' and then I use subprocess.call(command) to execute the command.
Now here's the weird part. If I run the script manually from the command line, it works fine. New files are downloaded and processed. However, if the Task Scheduler runs the script, no new files are downloaded. The script is running under the same user, but yet yields different results.
According to the log files created by the script and on the linux box, the script successfully logs into the linux box. However, no files are downloaded despite there being new files. Again, when I run it via the command line, files are downloaded.
Any ideas? suggestions, alternative methods?
Thanks.
You can use the windows Task Scheduler, but make sure the "optional" field "Start In" is filled in.
In the Task Scheduler app, add an action that specifies your python file to run "doSomeWork" and fill in the Start in (optional) input with the directory that contains the file.. So for example if you have a python file in:
C:\pythonProject\doSomeWork.py
You would enter:
Program/Script: doSomeWork.py
Start in (optional): C:\pythonProject
I had the same issue when trying to open an MS Access database on a Linux VM. Running the script at the Windows 7 command prompt worked but running it in Task Scheduler didn't. With Task Scheduler it would find the database and verify it existed but wouldn't return the tables within it.
The solution was to have Task Scheduler run cmd as the Program/Script with the arguments /c python C:\path\to\script.py (under Add arguments (optional)).
I can't tell you why this works but it solved my problem.
I'm having a similar issue. In testing I found that any type of call with subprocess stops the python script when run in task scheduler but works fine when run on the command line.
import subprocess
print('Start')
test = subprocess.check_output(["dir"], shell=True)
print('First call finished')
When run on command line this outputs:
Start
First call finished
When run from task scheduler the output is:
Start
In order to get the output from task scheduler I run the python script from a batch file as follows:
python test.py >> log.txt
I run the script through the batch file both on command line and through task scheduler.
Brad's answer is right. Subprocess needs the shell context to work and the task manager can launch python without that. Another way to do it is to make a batch file that is launched by the task scheduler that calls python c:\path\to\script.py etc. The only difference to this is that if you run into a script that has a call to os.getcwd() you will always get the root where the script is but you get something else when you make the call to cmd from task scheduler.
Last edit - start
After experiments... If you put there full path to python program it works without highest privileges (as admin). Meaning task settings like this:
program: "C:\Program Files\Python37\python.exe"
arguments: "D:\folder\folder\python script.py"
I have no idea why, but it works even if script uses subprocess and multiple threads.
Last edit - end
What I did is I changed task settings: checked Run with highest privileges. And task started to work perfectly while running python [script path].
But keep in mind, that title contains "Administrator: " at the begining... always...
P.S. Thanks guys for pointing out that subprocess is a problem. It made me think of task settings.
I had similar problem when one script is running from Windows Task Scheduler, and another one doesn't.
Running cmd with python [script path] didn't work for me on Windows 8.1 Embedded x64. Not sure why. Probably because of necessity to have spaces in path and issue with quotes.
Hope my answer helps someone. ;)
Create a batch file add your python script in your batch file and then schedule that batch file .it will work .
Example : suppose your python script is in folder c:\abhishek\script\merun.py
first you have to go to directory by cd command .so your batch file would be like :
cd c:\abhishek\script
python merun.py
it work for me .
Just leaving this for posterity: A similar issue I faced was resolved by using the UNC (\10.x.xx.xx\Folder\xxx)path everywhere in my .bat and .py scripts instead of the letter assigned to the drive (\K:\Folder\xxx).
I had this issue before. I was able to run the task manually in Windows Task Scheduler, but not automatically. I remembered that there was a change in the time made by another user, maybe this change made the task scheduler to error out. I am not sure. Therefore, I created another task with a different name, for the same script, and the script worked automatically. Try to create a test task running the same script. Hopefully that works!
For Anaconda installation of python in windows below solution worked for me
create a batch file with below
"C:\Users\username\Anaconda3\condabin\activate" && python "script.py" &&
deactivate
Setup task to run this batch file