Python script cannot find file only sometimes - python

I am running a Python script multiple times and the only thing that changes each time is the name of my run.
Approximately 50% of the time, the runs fail and I get the error output:
IOError: Cannot find file filename.txt
But the file is certainly in that location when I check. And the same file is also used by all the other runs that succeed.
So I'm just trying to understand what if there is anything that could be producing this error only sometimes.
I don't know if it's relevant but I am submitting the jobs via Slurm on a remote machine.

Related

Windows Task Schedule will not run .py file

I have received a ton of errors trying to run my py script via task scheduler. This is on my personal computer and all files are on local drives. The last error I get is The System cannot find the file specified. Even when I move the file to my desktop
Script: ‪C:\Users\S\AppData\Local\Programs\Python\Python37-32\python.exe
Argument: Bot.py
Start in: ‪C:\Users\S\Desktop\Bot.py
There is nothing else on my desktop. I have no idea what the issue could be. I am running on highest permissions and I have deselected the Ac power checkbox. I am at a loss and any help would be appreciated. It took me a really long time to get the script right and find python on my PC but I think this error has to do with the py file itself.

Task Scheduler returns 0xFFFFFFFF in Windows 7

I created a Python script that collects data from a website and generates an Excel file based on a table in that website. I used pyinstaller with -w -F parameters to generate a .exe file.
I ran this file a few times and it worked perfect so I decided to use Task Scheduler to run it every hour. Two days after the task worked every hour, while I was using the computer the Task Scheduler returned this error when it tried to run the .exe: 0xFFFFFFFF and a pop-up saying: Failed to "something"
Given the fact that I needed data every hour, I ran the file manually and again... it worked!
Is there any way I can fix this? How can I make sure that it won't fail again when I leave my computer online for 1 week but I'm not there to manually start it in case it fails...
Here are the settings for the Task Scheduler:
Actions:
Program/script: C:\path1\path2\path3\Script_G1.exe
/ Start in (optional): C:\path1\path2\path3\
Settings:
Allow task to be run on demand
We had a similar problem where we'd get a 0xFFFFFFFF error when running our custom .exe from Task Scheduler, but it would work fine outside of Task Scheduler.
The workaround was to create a .bat file to run the .exe and make the scheduled task call the .bat file. Not a solution, obviously, but works in a pinch.
We had a similar problem. The program accessed shared disk F:\SomeFolder\File.log and copied a file from it to the local folder. I had to change the shared disk path name in the program to use full server path.
From
F:\SomeFolder\File.log
to
\\serverName\\docs\\SomeFolder\File.log
and then it worked.
In the "Action" check the "Start in (optional)" field. It makes a difference in these situations.

How to find a real cause for Python segmentation fault

Python script generates segmentation fault
I cannot find the source of the problem.
How is it to debug a segfault simply in Python ?
Are there recommended coding practices to avoid Segmentation Faults ?
I wrote a script (1600 lines) to systematically download CSV files from a source and format the collected data. The script works very well for 100-200 files, but it systematically crashes with a segmentation fault message after a while.
-Crash always happens at the same spot in the script, with no understandable cause
-I run it on Mac OSX but the crash also happens on Ubuntu Linux and Debian 9
-If I run the Pandas Routines that crash during my script on single files, they work properly. The crash only happens when I loop the script 100-200 times.
-I checked every variable content, constructors (init) and they seem to be fine.
Code is too long to be pasted, but available on demand
Expected result should be execution to the end. Instead, it crashes after 100-200 iterations

Run same python code in two terminals, will them interfere each other?

I have a python script which takes a while to finish its executing depending on the passed argument. So if I run them from two terminals with different arguments, do they get their own version of the code? I can't see two .pyc files being generated.
Terminal 1 runs: python prog.py 1000 > out_1000.out
Before the script running on terminal 1 terminate, i start running an another; thus terminal 2 runs: python prog.py 100 > out_100.out
Or basically my question is could they interfere with each other?
If you are writing the output to the same file in disk, then yes, it will be overwritten. However, it seems that you're actually printing to the stdout and then redirect it to a file. So that is not the case here.
Now answer to your question is simple: there is no interaction between two different executions of the same code. When you execute a program or a script OS will load the code to the memory and execute it and subsequent changes to code has nothing to do with the code that is already running. Technically a program that is running is called a process. Also when you run a code on two different terminals there will be two different processes on the OS one for each of them and there is no way for two process to interfere unless you explicitly do that (IPC or inter-process communication) which you are doing here.
So in summary you can run your code simultaneously on different terminals they will be completely independent.
Each Python interpreter process is independent. How the script reacts to itself being run multiple times depends on the exact code in use, but in general they should not interfere.
.pyc file reference http://effbot.org/pyfaq/how-do-i-create-a-pyc-file.htm
Python automatically compiles your script to compiled code, so called
byte code, before running it. When a module is imported for the first
time, or when the source is more recent than the current compiled
file, a .pyc file containing the compiled code will usually be created
in the same directory as the .py file.
If you afraid your code get overwritten due to whatever mistake, you should learn to put your code under VERSION CONTROL. Register github and use git to do that.
bigger sign ">" will send the output to the right handler. It you specify file name, it will push the output to that file name. Even in different terminal, if you run the code inside the same folder, use the ">" point to SAME file name, the file on the right of the ">" definitely get overwrite.
Program SOURCE CODE ARE NOT mutable during execution. Unless you acquire high level program hacking skill.
Each program will run inside its "execution workspace". Unless you make a code that tap into same resources(like change same file,shared reources ), otherwise there is no interference. (except if one exhaust all CPU, Memory resources, the second one will be interfere, but that is other story)

Turning Selenium script into executable and running remotely return inconsistent results

I am trying to run a Selenium script that I created using Python bindings as an executable on a remote headless machine. The problem is that the scripts behaves differently when executed remotely.
I have done the following steps:
Created a Selenium script (with Python bindings) that works perfectly on my local virtual machine.
Transformed the above mentioned script into an standalone executable using py2exe. This one works on my local machine as expected as well.
Run the created standalone executable on a remote VM that is absolutely similar to the one I was developing and testing the initial script on (in step 1). The only difference is that it's headless. For remote run I am using PsExec from PSTools utilities set. I run it in the following way:
PsExec.exe \vmcle001iso -u rhdwa\he119712a -p Password123 -h c:\dist\test.exe
This is when my problems start. The first thing I've mention that some of Python commands stopped working. For example, usual Pythin exit(0) return with error:
Traceback (most recent call last):
File "nacs.py", line 103, in <module>
NameError: name 'exit' is not defined
But my main problem that some of the elements become unable to find or interact with. Thus, certain buttons that worked perfectly before, return with Selenium exceptions like:
WebDriverException: Message: u'unknown error: Element is not clickable at point (960, 23). Other element would receive the click:
So just to emphasize that again, it all happens just when I run the executable remotely. Most of the code works as expected though.
Any hints to what might cause such behavior are very welcome.
I feel this information is best contained in a comment, but I don't have those privileges yet, so this is what I found thus far.
I found this link which talked about the exit function- essentially, it's not recommended for scripts not run in the main interpreter because not as many modules are loaded upon startup, causing it not to be defined. (so it seems is not always built-in, as you had thought)
As far as the second error message, it seems that your code is loading some things faster/slower than before so it is getting to certain points before it can load a certain element- if I had to guess what would cause that, it would likely be too little time for a driver.sleep() command. Instead of changing these times manually, however, I would advise inserting WebDriverWaits (That's a link you can click on) to ensure that the element is clickable at that time. If the waits don't work and time out, it could also be something else affected.
Please update your post whenever you can so we can give more relevant information.

Categories

Resources