SOLVED
chkdsk /f and rebooting fixed the problem.
Problem. Python script freezes when doing multiple writings in files located inside 'Desktop' folder. This happens at around 10-15's file. Doesn't matter what kind of file, but amount of files, that script have modified (around 10-15).
If they are moved outside 'Desktop' folder (C:\Users\User\Desktop) or python script is running by CMD with admin rights, then there is no freeze.
EDIT This behaviour has nothing to do with specific file. Because I've experimented with any random files I found. Also I tried to delete file at which script froze with prior file. The result was the same — freeze after exact amount of files.
Also when I deleted first 9 files. Then script succesfully modified files where previosly it froze. But again when around 10-15 files passed script freeze.
Here is an example of code where script freezes:
root = ...
for path, subdirs, files in os.walk(root):
for name in files:
file = (os.path.join(path, name))
print("1")
with open(file, 'r+b') as f: #Here script freezes
print("2") #No matter what the file is
f.seek(1) #but iterations of loop are limited to 10-15
f.write(b'\x0a')
Python 3.10, Windows 10(updated recently), Windows Defender deactivated.
Replacing with() by open()+close() doesnt help at all.
I tried script on .txt, .css, .bin files — everywhere the same freeze occurs, but at different amount of files that script went through (~10-20 cycles in the above code example before freeze)
Q. Why does this happen without Exceptions, Errors or notifications by System? What causes a script to freeze? And why the freeze happens after 10-20 files were successfully modified?
Related
I'm working with HPC and am trying to run a python script that reads and modify every tif files in a folder and its subfolders. Right now I'm running a for loop:
files = glob.glob("./Examples/**/*.tif", recursive = True)
for file in files:
But it doesn't modify every file in the subfolder, only the final one. I run the exact same script locally and it worked just fine. So I'm thinking it's something to do with SLURM sitting on top of the code and maybe controlling how and when outputs get written. So could there be a way to write a python/slurm script that looks at the files in the directory and calls a slurm script, passing it a file name? Or any other solutions are also welcomed.
Thanks.
I am using pyinstaller to create one file executable with command line. Like all "one file" exeutables, it extracts binaries/dependencies in one temporary folder of OS whenever opened. It generally clean those after closing the application properly but when I close the command line (or when the app crashes) then it leaves all those extracted temp files behind in that temp folder. And it creates new temp folder (with name "_MEIxxxxxx") every time I open the app, so the old crashed temp stays there if not cleaned manually.
I used the --runtime-tmpdir parameter to specify one separate temp folder, but the same is happening there also. Any fix to this?
This issue was there from a very long time, but it's finally fixed in the new 5.3 version.
So for those who are facing this issue, just update pyinstaller to the latest version (5.3+).
Now the "_MEIxxxxxx" folder gets deleted if we close the parent console window or terminate the process in task-manager or even if we shutdown the system.
You can view the full changelog here
I have a Python script that tags MP3's and renames them. It works when assigning sys args via script parameters in Pycharm.
However, when I call the Python file from a bat, parsing in the same two parameters, I get problems with the file rename.
When printing os.listdir debugging the script, I see a bunch of old test MP3's that are no longer in the dir.
I've checked I'm referencing the correct dir. But it seems like somehow the list of files is cached and being passed over (since I have a validation check - abstracted function in the second part of the if statement blow).
Could it be possible Windows is caching something, or perhaps the .pyc is?
Here is part of the Python in question...
os.chdir(directory)
for files in os.listdir("."):
if files.endswith(".mp3") and not the_file_name_is_valid(files):
log.info("Starting work on: " + files)...
When run from the below bat file, os.listdir(".") produces a list of files that are not current for the working directory. Yet when the script is run from Pycharm it works perfectly.
Here is the bat
echo off
set SET_PREP="set_prep.py"
set DIR=%cd%
set IMAGE="cover_artwork.jpg"
c:\python27\python.exe %SET_PREP% %DIR% %IMAGE%
Pause
Why does running my script with command parameters behave differently to using Script Parameters in Pycharm?
Python novice here.
I have a Python script that performs some geodatabase management (reconcile/post versions, compress, etc). I have the following line of code in my script:
createLog = open(str(datetime.date.today()) + ".txt", "w")
along each step of the script I add to the text file with the following statements:
createLog.write("Database connections blocked.\n")
When I run the script in my IDE (PyCharm) I get the desired result: A text file with each step written to the .txt file. When I run it in Task Scheduler no .txt file is created and therefore no log. Everything else runs as far as I can tell. I'm able to track edits made to the data.
I have experienced things like this before with task scheduler but have never been able to resolve the problem in the past.
Any ideas?
I think this is a working directory problem. Python's open function opens a file in the current working directory, NOT in the same folder as the script. This is a common misconception! (Which confused me for ages when learning Python...)
So what is a working directory? Well to quote my good friend Wikipedia:
In computing, the working directory of a process is a directory of a hierarchical file system, if any,[1] dynamically associated with each process. When the process refers to a file using a simple file name or relative path (as opposed to a file designated by a full path from a root directory), the reference is interpreted relative to the current working directory of the process. So for example a process with working directory /rabbit-shoes that asks to create the file foo.txt will end up creating the file /rabbit-shoes/foo.txt.
(Source: https://en.wikipedia.org/wiki/Working_directory)
So how is this working directory selected?
Well it is selected by the parent process of that processes! When you run a program from a shell like bash, the shell (the parent process) helpfully sets the working directory of the program you are running (the child process) to the directory you are currently in. (That is, the directory you cd'd to.)
Since your IDE is smart and helpful, it is starting your Python script process and setting the working directory to the same place the script itself is located. The task scheduler is less helpful... I have absolutely no idea what it is setting the working directory to. However if you search your system, I am sure you will find the log file lying about somewhere!
I'm making a chat client for OSX, and I wanted it to be able to run as both a .app (for my less technologically inclined users) and as a .py file. I made a workflow app that contained two .py files (an auto-updater and the client itself), run by a python script in the .wflow file. This worked well. However, I couldn't update the updater or workflow script, and the icon was the Python rocket instead of the icon I had chosen. Then, I combined the client .py file with the updater .py file. This still worked, and now I could update the updater. I still couldn't update the python script in the workflow, though, and the icon was still wrong. So, I modified the updater to open the .wflow file, split it into a list (based on python comments in the workflow's python script, such as "#Start") of the stuff before the script, the script's modification time, and the stuff after the script. If the modification time isn't the same as the modification time of the remote file (the one that the updater updates from), then the script downloads the remote .py file, replaces characters (<, >, &) that .wflow files replace ('<' -> "<"), and opens document.wflow with the "w" (write/replace) flag. Then, the stuff that was before the old script, the downloaded script, and the stuff that was after the old script (using file.write(''.join(list))) are all put into document.wflow. This should work, but OSX no longer sees document as an automator file.
As you can see, OSX thinks that the old file is a Workflow, while the new file is a "Microsoft Excel 97-2004 workbook". The IMClient.app (the application that contains document.wflow) gives this message when I try to run it: "The document "IMClient" could not be opened because it is damaged or incomplete." Does anyone know how to fix this?
I'm using python 2.7 and OSX 10.7. The updater is downloading files via FTP.
If clarification is necessary, just ask. I'll post the working and nonworking files if anyone wants them.
EDIT: the file command gives "document.wflow: XML document text" for both the old and new file.