Develop Reference

Python FileNotFoundError: [Errno 2] despite giving full filepath

So I have written a piece of code which first runs a powershell command to generate a UTF-8 version of a DAT file (have been having special character issues with the original file, hence the step). Following which I try to open the newly created file. But the issue is, I keep getting 'FileNotFoundError: [Errno 2]' Initially I was only trying with the file name since the newly created file was in the same folder, but then i tried to generate the absolute path as well.
import os
import subprocess
subprocess.Popen('powershell.exe -Command "Get-Content .\Own.DAT | Set-Content -Encoding utf8 Own1.dat"')
filepath = __file__
filepath = filepath[:-7]
with open(filepath+"Own1.dat", "r") as f:
I can confirm that filepath+"Own1.dat" is fetching the correct filepath. Yet can't figure out what the issue could be.
Edit: Someone asked for confirmation, here is the message i am getting:
C:\Users\Debojit\MiniConda3\python.exe "E:/My Documents/Work/essbase/ownership/test.py"
Traceback (most recent call last):
File "E:/My Documents/Work/essbase/ownership/test.py", line 18, in <module>
with open(filepath+"Own1.dat", "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'E:/My Documents/Work/essbase/ownership/Own1.dat'
Process finished with exit code 1
Note: Curiously enough if i put the powershell command into a separate batch file, write a code in the python script to run it, the works without any issues. Here is the code i am talking about:
import os
import subprocess
from subprocess import Popen
p = Popen("conversion.bat", cwd=r"E:\My Documents\Work\essbase\ownership")
stdout, stderr = p.communicate()
filepath = __file__
filepath = filepath[:-7]
with open(filepath+"Own1.dat", "r") as f:
The conversion.bat file contains the following
powershell.exe -Command "Get-Content .\Own.DAT | Set-Content -Encoding utf8 Own1.DAT"
But I don't want to include a separate batch file to go with the python script.
Any idea what might be causing the issue?

Your error is unrelated to powershell. Popen runs asynchronously. In one command, you are using communicate(), but in the other, you are not.

You're using Popen() incorrectly.
If you want run a command and also pass arguments to it, you have to pass them as a list, like so:
subprocess.Popen(['powershell.exe', '-Command', ...])
In your code, popen tries to run a command literally named powershell.exe -Command "Get-Content ... which of course doesn't exist.
To use a simpler example, this code won't work:
subprocess.Popen('ls -l')
because it's trying to run a command literally named ls -l.
But this does work:
subprocess.Popen(['ls', '-l'])

I still couldn't figure out why the error was happening. But I found a workaround
with open("conversion.bat","w") as f:
f.writelines("powershell.exe -Command \"Get-Content '" +fileName+ "' | Set-Content -Encoding utf8 Own1.dat\"")
from subprocess import Popen
p = Popen("conversion.bat", cwd=os.path.dirname(os.path.realpath(__file__)))
stdout, stderr = p.communicate()
os.remove("conversion.bat")
Basically I would create the batch file, run it and then delete it once the file has been created. Don't why I have to use this route, but it works.

Subprocess.Popen error

Proc = subprocess.Popen ([ 'FileName'])
The FileName is a variable which stores "/home/USER/exec.sh &", the program searches for the exec.sh file in the home folder and stores the path in FileName.I am unable to start this exec.sh process.It gives me the following error
OSError: [Errno 2] No such file or directory
I initially used::
os.system(FileName)
It worked perfectly but didn't return the pid. Thus, I switched to Popen.

just:
fileName = "/home/USER/exec.sh"
proc = subprocess.Popen(fileName)
pid = proc.pid

Why is my script attempting to unzip files that haven't been downloaded yet?

I have a Python 3 script that I am writing to do three things:
1) Determine which Retrosheets data files are supposed to be downloaded
2) Create wget commands to retrieve the files and download them
3) Unzip the files after they have been downloaded.
When testing each function in the Python Console, I have no problems. But, when I try to do everything automatically, I get the following output:
Start Decade: 1930
End Decade: 1950
Creating wget commands...
Commands created...
Downloaded 3 files.
Unzipping files...
Traceback (most recent call last):
File "import_pbp.py", line 54, in <module>
unzip_data(decade_files)
File "import_pbp.py", line 39, in unzip_data
with zipfile.ZipFile('zip' + file, 'r') as zip_ref:
File "/usr/local/Cellar/python3/3.5.2_1/Frameworks/Python.framework/Versions/3.5 /lib/python3.5/zipfile.py", line 1009, in __init__
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: 'zip1930seve.zip'
The files are downloaded after this output to the console. This would seem to indicate that the unzip function is running before the files are downloaded. How do I make sure that my files are downloaded before the unzip function is called? Code below:
Download function:
# define function to execute the download commands
def download_data(commands):
for command in commands:
os.popen(command)
print('Downloaded ' + str(len(commands)) + ' files.')
Unzip Function:
# Unzip the data files into the 'unparsed' folder.
def unzip_data(file_list):
print('Unzipping files...')
for file in file_list:
with zipfile.ZipFile('zip' + file, 'r') as zip_ref:
zip_ref.extractall('unparsed/')
print(file + ' unzipped')
print('All files unzipped...')
EDIT: I looked at the response in this thread but it didn't quite explain what I needed like tdelaney did below. They are similar, but for my purposes, different. Especially since that question is 6 years old and I'm guessing there may have been significant changes to the language since then.
EDIT 2: Removed non-essential code to shorten the post.

os.popen doesn't wait for the process to complete so you launch all of the commands at once then try the unzips before they are done. Since you don't read the stdout pipe returned from os.popen, you also risk the program hanging if the output pipe fills.
The subprocess module has several functions for calling programs. Assuming you really do want all of the commands to run in parallel and that you just want to discard any output data from the commands, you could reimplement that function as
import subprocess as subp
import os
# define function to execute the download commands and unzip
def download_data(commands):
procs = []
for command in commands:
procs.append(subp.Popen(command, shell=True,
stdout=open(os.devnull, 'wb')))
for proc in procs:
proc.wait()
print('Downloaded ' + str(len(commands)) + ' files.')

file line writing closes off script

I'm looking to make a file that when you enter a directory, it creates a folder inside that directory. I'm using a batch file to create the folder, so I'm making it when you enter the directory, it will write that directory to the batch file, then run it. However I'm having some errors. Here's my code
directory = str(input())
text = 'cd ' + directory
lines = open('new.bat', 'r').readlines()
lines[2] = text
out = open('new.bat', 'w')
out.writelines(lines)
out.close()
call('new.bat')
out.close() exits my python script before it can have a chance to call 'new.bat', however if I move the out.close() to after it calls new.bat, it gives me an error that the file is currently being used. How do I fix this?

permission denied while creating a file object in python

i have two python scripts
script1.py1:-
def somefunction()
.................
out_file = open("out.txt","w")
#creates a file out.txt and write something into it
out_file.close()
.................
.................
somefunction() #calling somefunction
out_file = open("out.txt","r")
output = open("Final_out.txt","w")
for line in out_file:
match_for_pattern = re.search(r"^\(.*affected\)$",line)
if match_for_pattern:
output.write("")
else:
output.write("%s" % line)
out_file.close()
output.close()
I wanted to change something in out.txt file, i.e removing some unwanted lines, So i thought to create a new file Final_out.txt with all the changes done in it.
but i'm getting permission denied error when i run the code
ERROR:-
output = open("Final_out.txt","w")
IOError: [Errno 13] Permission denied: 'Final_out.txt'
I have full permission chmod 777 in my present working directory,
If i take out the code where i am changing something in my file(out.txt) and place it into a new python script (script2.py) then it is working.
scritp2.py:-
out_file = open("out.txt","r")
output = open("Final_out.txt","w")
for line in out_file:
match_for_pattern = re.search(r"^\(.*affected\)$",line)
if match_for_pattern:
output.write("")
else:
output.write("%s" % line)
out_file.close()
output.close()
script2.py creates a file Final_out.txt in my present directory with all the changes that i want.
how's this possible that my script1.py is not able to do this but script2.py does it without any errors?