I have a python script that doesn't seem to be opening the files.
The folder in the script is defined like this:
logdir = "C:\\Programs\\CommuniGate Files\\SystemLogs\\"
submitdir = "C:\\Programs\\CommuniGate Files\\Submitted\\"
This is how the paths are being used:
filenames = os.listdir(logdir)
fnamewithpath = logdir + fname
I'm running this script in Windows 7 sp1
Does this look correct?
Is there something I can put into the code to debug it to make sure the files are opening?
Thank you,
Docfxit
Edited to provide more clarification:
The actual code to open and close the file is here:
# read all the log parts
for fname in logfilenames :
fnamewithpath = logdir + fname
try :
inputFile = open(fnamewithpath, "r")
except IOError as reason :
print("Error: " + str(reason))
return
if testing :
print("Reading file '%s'" % (fname))
reporter.munchFile(inputFile)
inputFile.close()
# open output file
if testing :
outfilename = fullLognameWithPath + ".summary"
fullOutfilename = outfilename
else :
outfilename = submitdir + "ls" + str(time.time()) + "-" + str(os.getpid())
fullOutfilename = outfilename + ".tmp"
try :
outfile = open(fullOutfilename, "w")
except IOError :
print("Can't open output file " + fullOutfilename)
return
if not testing :
# add the mail headers first
outfile.write("To: " + reportToAddress + "\n")
outfile.write("From: " + reportFromAddress + "\n")
outfile.write("Subject: CGP Log Summary new for " + logname + "\n")
if useBase64 :
outfile.write("Content-Transfer-Encoding: base64\n")
outfile.write("\n")
# save all this as a string so that we can base64 encode it
outstring = ""
outstring += "Summary of file: " + fullLogname + partAddendum + "\n"
outstring += "Generated " + time.asctime() + "\n"
outstring += reporter.generateReport()
if useBase64 :
outstring = base64.encodestring(outstring)
outfile.write(outstring)
outfile.close()
if not testing :
# rename output file to submit it
try :
os.rename(outfilename + ".tmp", outfilename + ".sub")
except OSError :
print("Can't rename mail file to " + outfilename + ".sub")
I was originally wondering if the double back slashes included in the path were correct.
I can't figure out why it isn't producing the output correctly.
Just in case someone would like to see the entire script I posted it:
The first half is at:
http://pastebin.ws/7ipf3
The second half is at:
http://pastebin.ws/2fuu3n
It was too large to post all in one.
This is being run in Python 3.2.2
Thank you very much for looking at it,
Docfxit
The code as written above does not actually open either file.
os.listdir returns a list of the files (technically entries, as non-files like . and .. are also included) in the specified path, but does not actually open them. For that you would need to call the open function on one of the paths.
If you wanted to open all the files in filenames for write, you could do something like this:
fileList = []
for f in filenames:
if os.path.isfile(fullPath):
fullPath = os.path.join(logdir, f)
fileList.append(open(fullPath, 'w')
After this, the list fileList would contain the open file handles for all of the files, which could then be iterated over (and, for example, all written to if you wanted to multiplex the output).
Note that when done, you should loop through the list and explicitly close them all (the with syntax that automatically closes them has additional complexities/limitations when it comes to dynamically sized lists, and is best avoided here, IMO).
For more info on files, see:
Reading and Writing Files
Also, it's best to use os.path.join to combine components of a path. That way it can be portable across supported platforms, and will automatically use the correct path separators, and such.
Edit in response to comment from OP:
I would recommend you step through the code using a debugger to see exactly what's going wrong. I use pudb, which is an enhanced command-line debugger, and find it invaluable. You can install it via pip into your system/virtualenv Python environment.
Related
I'm not sure if there's an answer to this but I thought I'd ask. This could very well be something Python just won't do.
I've written a small function to create a date/time stamped text file in a specific folder as a sort of log. I'm writing a lot of file transfer scripts that will be automated so I wanted a way to keep track of when they've actually run. I've also set it up to split a script name, passed into the function using os.path.basename(__file__). Here's my code for creating the log files (filepath replaced as it's a business file structure):
def loggerauto(fileNameRaw):
name, ext = fileNameRaw.split('.')
jobName = name
now = datetime.now()
dateStamp = now.strftime("%y.%m.%d-%H.%M")
fileNam = jobName + " " + dateStamp
logPath = "PATH"
fileList = glob.glob(logPath + jobName + "*", recursive=True)
for filePath in fileList:
try:
os.remove(filePath)
except OSError:
print("Error deleting old file!")
print("Writing " + jobName + " log file...")
logfile = open(logPath + fileNam + ".txt", "w")
logfile.close
print("Done!")
In a nutshell, it takes in the file name, splits it to remove the extension. Uses that as the name of the log file and adds the time and date. It checks for a file with the same job name in the folder and deletes it if it exists. Then creates a new one with the current time and date.
I've set this up in a separate module so I can just call it in all my scripts (much easier than pasting this code into 20 scripts) but I currently have to call it like this:
tt.loggerauto(os.path.basename(__file__))
I want to make it more user friendly and have it automatically get the name of the running script and pass it into the function, so you can call it like this:
tt.loggerauto()
If I put the os.path.etc line in the function code, it gets the name of the module the function is in, which makes sense. I can't think of a way to get the running script file name automatically. Is there any way to do this?
Importing inspect helps in identifying the calling function's filename.
inspect.stack() returns following data
[FrameInfo(frame=<frame object at 0x2817808>, filename='/home/logeshwaran/ins2.py', lineno=6, function='loggerauto', code_context=[' print(inspect.stack())\n'], index=0), FrameInfo(frame=<frame object at 0x7f0286681828>, filename='ins1.py', lineno=2, function='<module>', code_context=['loggerauto()\n'], index=0)]
Calling the jobname with inspect does the task
from datetime import datetime
import glob
import os
import inspect
def loggerauto():
jobName = inspect.stack()[1][1]
now = datetime.now()
dateStamp = now.strftime("%y.%m.%d-%H.%M")
fileNam = jobName + " " + dateStamp
logPath = "PATH"
fileList = glob.glob(logPath + jobName + "*", recursive=True)
for filePath in fileList:
try:
os.remove(filePath)
except OSError:
print("Error deleting old file!")
print("Writing " + jobName + " log file...")
logfile = open(logPath + fileNam + ".txt", "w")
logfile.close
print("Done!")
This can now be called as below
loggerauto()
Reference: https://docs.python.org/3/library/inspect.html
I have a txt-file called "odbList.txt" which contains the names of several odb-files.
plate_2mm.odb
plate_4mm.odb
plate_6mm.odb
Now I wrote a Python Script, where I want to open each of these files in a loop.
# list of ODB-Files
odbList = [ ]
f = file( 'W:/someDirectory/odbList.txt' , 'r')
count = 0
for line in f.readlines() :
odbList.append (line)
count = count + 1
def getSIF(case, i):
odb = openOdb(path = 'W:/someDirectory/' + case)
# start analyses for each case
for i in xrange(0,count):
getSIF(odbList[i], i)
I get the following error message:
OdbError: Cannot open file W:/someDirectory/plate_2mm.odb
. *** ERROR: No such file: W:/someDirectory/plate_2mm.odb
The weird thing however is that it works perfectly fine when I hardcode the complete path.
Another weird thing. If I use this line instead:
odb = openOdb(path = case)
I get following error message:
OdbError: Cannot open file C:/Temp/plate_2mm.odb
. *** ERROR: No such file: C:/Temp/plate_2mm.odb
And if I transfer all my files into C:/Temp everything works fine. But why doesn't it work if I use the first version / a different folder? Especially since it is working when hardcoded.
Most of the times when I open a file I use the following:
f=open(file_name,'r')
s=f.read().splitlines()
f.close()
while '' in s:
s.remove('')
You will now have a list in s without enters.
Alternatively you can use something like
import os
odbList=[]
for fileN in os.listdir("."):
if '.odb' in fileN:
odbList.append(fileN)
This will find all files containing .odb in the name in the script's directory/working directory
Have you tried entering your string as a raw string like thisodb = openOdb(path = r'W:/someDirectory/' + case) or usining the os.sep character like this: odb = openOdb(path = 'W:someDirectory' + os.sep + case)
I am trying to include a binary file within a zip file and below is the code snippet:
I first unzip the zip contents into a temporary location and add few more files and zip it back to a new archive.
import zipfile
def test(fileName, tempDir):
# unzip the file contents,may contain binary files
myZipFile=zipfile.ZipFile(fileName,'r')
for name in myZipFile.namelist():
toFile = tempDir + '/' + name
fd = open(toFile, "w")
fd.write(myZipFile.read(name))
fd.close()
myZipFile.close()
# code which post processes few of the files goes here
#zip it back
newZip = zipfile.ZipFile(fileName, mode='w')
try:
fileList = os.listdir(tempDir)
for name in fileList:
name = tempDir + '/' + name
newZip.write(name,os.path.basename(name))
newZip.close()
except Exception:
print 'Exception occured while writing to PAR file: ' + fileName
Some of the files may be binary files. The zipping code works fine but when i try to unzip it using linux ' unzip or python's zip module , i get the below error:
zipfile corrupt. (please check that you have transferred or
created the zipfile in the appropriate BINARY mode and that you have
compiled UnZip properly)
And am using python 2.3
What's going wrong here ?
You might want to upgrade, as Python 2.3 is really outdated. 2.7.3 is the latest one from the 2.x-versions and 3.2.3 the latest python version.
See docs.python.org:
| extractall(self, path=None, members=None, pwd=None)
| Extract all members from the archive to the current working
| directory. `path' specifies a different directory to extract to.
| `members' is optional and must be a subset of the list returned
| by namelist().
(New in version 2.6)
Take a look at Zip a folder and its content.
You might also be interested in distutlis.archive_util.
Hmm not sure if its a bug in python 2.3. Current work environment do not allow me to upgrade to a higher version of python :-( :-( :-(
The below workaround worked:
import zipfile
def test(fileName, tempDir):
# unzip the file contents,may contain binary files
myZipFile=zipfile.ZipFile(fileName,'r')
for name in myZipFile.namelist():
toFile = tempDir + '/' + name
# check if the file is a binary file
#if binary file, open it in "wb" mode
fd = open(toFile, "wb")
#else open in just "w" mode
fd = open(toFile, "w")
fd.write(myZipFile.read(name))
fd.close()
myZipFile.close()
# code which post processes few of the files goes here
#zip it back
newZip = zipfile.ZipFile(fileName, mode='w')
try:
fileList = os.listdir(tempDir)
for name in fileList:
name = tempDir + '/' + name
newZip.write(name,os.path.basename(name))
newZip.close()
except Exception:
print 'Exception occured while writing to PAR file: ' + fileName
According to all the sources I've read, the open method creates a file or overwrites one with an existing name. However I am trying to use it and i get an error:
File not found - newlist.txt (Access is denied)
I/O operation failed.
I tried to read a file, and couldn't. Are you sure that file exists? If it does exist, did you specify the correct directory/folder?
def getIngredients(path, basename):
ingredient = []
filename = path + '\\' + basename
file = open(filename, "r")
for item in file:
if item.find("name") > -1:
startindex = item.find("name") + 5
endindex = item.find("<//name>") - 7
ingredients = item[startindex:endindex]
ingredient.append(ingredients)
del ingredient[0]
del ingredient[4]
for item in ingredient:
printNow(item)
file2 = open('newlist.txt', 'w+')
for item in ingredient:
file2.write("%s \n" % item)
As you can see i'm trying to write the list i've made into a file, but its not creating it like it should. I've tried all the different modes for the open function and they all give me the same error.
It looks like you do not have write access to the current working directory. You can get the Python working directory with import os; print os.getcwd().
You should then check whether you have write access in this directory. This can be done in Python with
import os
cwd = os.getcwd()
print "Write access granted to current directory", cwd, '>', os.access(cwd, os.W_OK)
If you get False (no write access), then you must put your newfile.txt file somewhere else (maybe at path + '/newfile.txt'?).
Are you certain the directory that you're trying to create the folder in exists?
If it does NOT... Then the OS won't be able to create the file.
This looks like a permissions problem.
either the directory does not exist or your user doesn't have the permissions to write into this directory .
I guess the possible problems may be:
1) You are passing the path and basename as parameters. If you are passing the parameters as strings, then you may get this problem:
For example:
def getIngredients(path, basename):
ingredient = []
filename = path + '\\' + basename
getIngredients("D","newlist.txt")
If you passing the parameters the above way, this means you are doing this
filename = "D" + "\\" + "newlist.txt"
2) You did not include a colon(:) after the path + in the filename.
3) Maybe, the file does not exist.
I want to create symlinks for each file in a nested directory structure, where all symlinks will be put in one large flat folder, and have the following code by now:
# loop over directory structure:
# for all items in current directory,
# if item is directory, recurse into it;
# else it's a file, then create a symlink for it
def makelinks(folder, targetfolder, cmdprocess = None):
if not cmdprocess:
cmdprocess = subprocess.Popen("cmd",
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
print(folder)
for name in os.listdir(folder):
fullname = os.path.join(folder, name)
if os.path.isdir(fullname):
makelinks(fullname, targetfolder, cmdprocess)
else:
makelink(fullname, targetfolder, cmdprocess)
#for a given file, create one symlink in the target folder
def makelink(fullname, targetfolder, cmdprocess):
linkname = os.path.join(targetfolder, re.sub(r"[\/\\\:\*\?\"\<\>\|]", "-", fullname))
if not os.path.exists(linkname):
try:
os.remove(linkname)
print("Invalid symlink removed:", linkname)
except: pass
if not os.path.exists(linkname):
cmdprocess.stdin.write("mklink " + linkname + " " + fullname + "\r\n")
So this is a top-down recursion where first the folder name is printed, then the subdirectories are processed. If I run this now over some folder, the whole thing just stops after 10 or so symbolic links.
The program still seems to run but no new output is generated. It created 9 symlinks for some files in the # tag & reencode and the first three files in the ChillOutMix folder. The cmd.exe Window is still open and empty, and shows in its title bar that it is currently processing the mklink command for the third file in ChillOutMix.
I tried to insert a time.sleep(2) after each cmdprocess.stdin.write in case Python is just too fast for the cmd process, but it doesn't help.
Does anyone know what the problem might be?
Why not just execute mklink directly?
Try this at the end:
if not os.path.exists(linkname):
fullcmd = "mklink " + linkname + " " + fullname + "\r\n"
print fullcmd
cmdprocess.stdin.write(fullcmd)
See what commands it prints. You may see a problem.
It may need double quotes around mklink's arg, since it contains spaces sometimes.