I am trying to read in a series of DICOM files in a folder tree and I am using the below code to run through the tree, reading in each file as I go. The problem is I am getting IOErrors for files that definitely exist, I have checked file permissions and other SO threads such as Python: IOError: [Errno 2] No such file or directory but I haven't managed to get it working without these IOErrors yet. Does anyone have any ideas?
for root, dirs, files in os.walk(path):
for fname in files:
name = os.path.basename(os.path.abspath(fname))
if name.startswith('.') == True:
pass
else:
try:
plan=dicom.read_file(fname)
ds=dicom.read_file(fname, stop_before_pixels = True)
kVp = TagChecker([0x18,0x60]) #KVP
Target = TagChecker([0x18,0x1191]) #ANODE
Filter = TagChecker([0x18,0x7050]) #
write_results.writerow([Survey_Number, Patient_ID, View_Protocol, int(kVp), Target, Filter, Thickness, mAs_Exposure, LPad_Yes_No, autoorman, AECMode, AECDset, Patient_Age, Comment, Compression_Force])
#print(fname)
except IOError:
print "IOError: ", "//" + os.path.join(root, fname) + "//"
except InvalidDicomError:
# This exception line prints an error message to the command line, checks to see if an error log
# has been generated for this session, writes a new one if not and then writes the error to the log file
print "Invalid Dicom File: ", fname
Usually a method that takes a filename, like dicom.read_file(fname), will take an absolute filename (or assume that the filename is relative to the dir that your main python program is running in, the cwd()). Can I suggest that you put this line in front of the first read_file() call:
print "reading: %s" % os.path.abspath(fname)
Then you'll see the filename that you're actually trying to read. I'm guessing it's not the file (or droids) you think you're looking for.
In order to fix your problem.. join the dir and the fname before you read.. e.g.
full_fname = os.path.join(dir, fname)
dicom.read_file(full_fname)
In other words, I think you're reading files with relative paths and you want to be using absolute paths.
Related
My English is very poor, and the use of the Google translation, I am sorry for that. :)
Unable to save filename, error indicating no directory exists, but directory exists.
1.You can manually create the file in the resource manager --> the file name is legal.
2.You can manually create a directory in the resource manager --> the directory name is legal
3.You can save other file names such as aaa.png to this directory, that is, this directory can be written to other files --> The path path is legal, there is no permission problem, and there is no problem with the writing method.
4.The file can be written to the upper-level directory download_pictures --> It's not a file name problem.
thank you!!!
import os
path = 'download_pictures\\landscape[or]no people[or]nature[OrderBydata]\\'
download_name = '[6]772803-2500x1459-genshin+impact-lumine+(genshin+impact)-arama+(genshin+impact)-aranara+(genshin+impact)-arabalika+(genshin+impact)-arakavi+(genshin+impact).png'
filename = path + download_name
print('filename = ', filename)
# Create the folder make sure the path exists
if not os.path.exists(path):
os.makedirs(path)
try:
with open(filename, 'w') as f:
f.write('test')
except Exception as e:
print('\n【error!】First save file, failed, caught exception:', e)
print(filename)
filename = path + 'aaa.png'
with open(filename, 'w') as f:
print('\nThe second save file, changed the file name aaa.png, the path remains unchanged')
f.write('test')
print(filename)
path = 'download_pictures\\'
filename = path + download_name
with open(filename, 'w') as f:
print('\nThe third save file, the file name is unchanged, but the directory has changed')
f.write('test')
console
filename = download_pictures\landscape[or]no people[or]nature[OrderBydata]\[6]772803-2500x1459-genshin+impact-lumine+(genshin+impact)-arama+(genshin+impact)-aranara+(genshin+impact)-arabalika+(genshin+impact)-arakavi+(genshin+impact).png
【error!】First save file, failed, caught exception: [Errno 2] No such file or directory: 'download_pictures\\landscape[or]no people[or]nature[OrderBydata]\\[6]772803-2500x1459-genshin+impact-lumine+(genshin+impact)-arama+(genshin+impact)-aranara+(genshin+impact)-arabalika+(genshin+impact)-arakavi+(genshin+impact).png'
download_pictures\landscape[or]no people[or]nature[OrderBydata]\[6]772803-2500x1459-genshin+impact-lumine+(genshin+impact)-arama+(genshin+impact)-aranara+(genshin+impact)-arabalika+(genshin+impact)-arakavi+(genshin+impact).png
The second save file, changed the file name aaa.png, the path remains unchanged
download_pictures\landscape[or]no people[or]nature[OrderBydata]\aaa.png
The third save file, the file name is unchanged, but the directory has changed
Process finished with exit code 0
I couldn't replicate your error (I'm using linux and I think you have a Windows system), but anyway, you should not try to join paths manually. Instead try to use os.path.join to join multiple paths to one valid path. This will also ensure that based on your operating system the correct path separators are used (forward slash on unix and backslash on Windows).
I have adapted the code until the first saving attempt accordingly and it writes a file correctly. Also, the code gets cleaner this way and it's easier to see the separate folder names.
import os
if __name__ == '__main__':
path = os.path.join('download_pictures', 'landscape[or]no people[or]nature[OrderBydata]')
download_name = '[6]772803-2500x1459-genshin+impact-lumine+(genshin+impact)-arama+(genshin+impact)-aranara+(genshin+impact)-arabalika+(genshin+impact)-arakavi+(genshin+impact).png'
filename = os.path.join(path, download_name)
print('filename = ', filename)
# Create the folder make sure the path exists
if not os.path.exists(path):
os.makedirs(path)
try:
with open(filename, 'w') as f:
f.write('test')
except Exception as e:
print('\n【error!】First save file, failed, caught exception:', e)
print(filename)
I hope this helps. I think the issue with your approach is related to the path separators \\ under Windows.
I've managed to find out the method to convert a file from one file extension to another (.evtx to .xml) using an external script. Below is what I am using:
os.system("file_converter.py file1.evtx > file1.xml")
This successfully converts a file from .txt to .xml using the external script I called (file_converter.py).
I am now trying to find out a method on how I can use 'os.system' or perhaps another method to convert more than one file at once, I would like for my program to dive into a folder and convert all of the 10 files I have at once to .xml format.
The questions I have are how is this possible as os.system only takes 1 argument and I'm not sure on how I could make it locate through a directory as unlike the first file I converted was on my standard home directory, but the folder I want to access with the 10 files is inside of another folder, I am trying to find out a way to address this argument and for the conversion to be done at once, I also want the file name to stay the same for each individual file with the only difference being the '.xml' being changed from '.evtx' at the end.
The file "file_converter.py" is downloadable from here
import threading
import os
def file_converter(file):
os.system("file_converter.py {0} > {1}".format(file, file.replace(".evtx", ".xml")))
base_dir = "C:\\Users\\carlo.zanocco\\Desktop\\test_dir\\"
for file in os.listdir(base_dir):
threading.Thread(target=file_converter, args=(file,)).start()
Here my sample code.
You can generate multiple thread to run the operation "concurrently". The program will check for all files in the directory and convert it.
EDIT python2.7 version
Now that we have more information about what you want I can help you.
This program can handle multiple file concurrently from one folder, it check also into the subfolders.
import subprocess
import os
base_dir = "C:\\Users\\carlo.zanocco\\Desktop\\test_dir\\"
commands_to_run = list()
#Search all files
def file_list(directory):
allFiles = list()
for entry in os.listdir(directory):
fullPath = os.path.join(directory, entry)
#if is directory search for more files
if os.path.isdir(fullPath):
allFiles = allFiles + file_list(fullPath)
else:
#check that the file have the right extension and append the command to execute later
if(entry.endswith(".evtx")):
commands_to_run.append("C:\\Python27\\python.exe file_converter.py {0} > {1}".format(fullPath, fullPath.replace(".evtx", ".xml")))
return allFiles
print "Searching for files"
file_list(base_dir)
print "Running conversion"
processes = [subprocess.Popen(command, shell=True) for command in commands_to_run]
print "Waiting for converted files"
for process in processes:
process.wait()
print "Conversion done"
The subprocess module can be used in two ways:
subprocess.Popen: it run the process and continue the execution
subprocess.call: it run the process and wait for it, this function return the exit status. This value if zero indicate that the process terminate succesfully
EDIT python3.7 version
if you want to solve all your problem just implement the code that you share from github in your program. You can easily implement it as function.
import threading
import os
import Evtx.Evtx as evtx
import Evtx.Views as e_views
base_dir = "C:\\Users\\carlo.zanocco\\Desktop\\test_dir\\"
def convert(file_in, file_out):
tmp_list = list()
with evtx.Evtx(file_in) as log:
tmp_list.append(e_views.XML_HEADER)
tmp_list.append("<Events>")
for record in log.records():
try:
tmp_list.append(record.xml())
except Exception as e:
print(e)
tmp_list.append("</Events>")
with open(file_out, 'w') as final:
final.writelines(tmp_list)
#Search all files
def file_list(directory):
allFiles = list()
for entry in os.listdir(directory):
fullPath = os.path.join(directory, entry)
#if is directory search for more files
if os.path.isdir(fullPath):
allFiles = allFiles + file_list(fullPath)
else:
#check that the file have the right extension and append the command to execute later
if(entry.endswith(".evtx")):
threading.Thread(target=convert, args=(fullPath, fullPath.replace(".evtx", ".xml"))).start()
return allFiles
print("Searching and converting files")
file_list(base_dir)
If you want to show your files generate, just edit as above:
def convert(file_in, file_out):
tmp_list = list()
with evtx.Evtx(file_in) as log:
with open(file_out, 'a') as final:
final.write(e_views.XML_HEADER)
final.write("<Events>")
for record in log.records():
try:
final.write(record.xml())
except Exception as e:
print(e)
final.write("</Events>")
UPDATE
If you want to delete the '.evtx' files after the conversion you can simply add the following rows at the end of the convert function:
try:
os.remove(file_in)
except(Exception, ex):
raise ex
Here you just need to use try .. except because you run the thread only if the input value is a file.
If the file doesn't exist, this function throws an exception, so it's necessary to check os.path.isfile() first.
import os, sys
DIR = "D:/Test"
# ...or as a command line argument
DIR = sys.argv[1]
for f in os.listdir(DIR):
path = os.path.join(DIR, f)
name, ext = os.path.splitext(f)
if ext == ".txt":
new_path = os.path.join(DIR, f"{name}.xml")
os.rename(path, new_path)
Iterates over a directory, and changes all text files to XML.
My relevant code block is as follows:
path = "\\Users\\Harmless\\Documents\\untitled"
cellorder = []
cellcont = []
for roots, dirs, files, in os.walk(path):
for file in natural_sort(files):
if "Row" in file:
cellorder.append(file)
with open(file,'r') as txt:
print txt.readlines
#print "file = %s" % file
This would successfully list all the files I want to open (as commented out), but when I try to pass in the filename the same way it was printed in order to read it:
IOError: [Errno 2] No such file or directory: 'Row0Col0Heat.txt'
How can I fix this? Do I need to reference the entire path name and string substitute in each filename? If so, why? Is there a better way to reference/utilize paths?
Try using the absolute path of the file, you can get the absolute path by
abs_file_path = os.path.abspath(file)
since you already have the base path, you can also use:
abs_file_path = os.path.join(path, file)
Hope it helps
I have this code that will let the user choose which file he wants to update by passing an argument in the command line, and then it do some more things but I have not included that here:
import sys
import os
from sys import argv
path = "/home/Desktop/python/test"
files = os.walk( path )
filename = argv[1]
if filename in files:
inputFile = open(filename, 'r')
else:
print "no match found"
sys.exit()
inputFile.close()
When I run the script it keeps giving me "no match found" but im pretty sure the file is there. I cant see what Im doing wrong
os.walk() returns a generator, one that produces tuples with (root, directories, files) values for each iteration.
You can't use that generator to test for a single file, not with a simple in membership test.
You'll also need to re-instate the whole path; you can't just open an unclassified filename without the directory it lives in. Just use a for loop here, and break once you found it. The else suite on a for loop only executes when you did not use break (e.g. the file was not found):
path = "/home/Desktop/python/test"
filename = argv[1]
for root, directories, files in os.walk(path):
if filename in files:
full_path = os.path.join(root, filename)
break
else:
print "no match found"
sys.exit()
with open(full_path) as input_file:
# do something with the file
I added a with statement to handle the lifetime of the file object; once the with block is exited the file is automatically closed for you.
Alternatively, you may use following code snippet.
import os.path
filename = argv[1]
path = "/home/Desktop/python/test/"
if os.path.isfile(path + filename):
inputFile = open(path + filename, "r")
else:
print "File Not Found"
I've written the following program in Python:
import re
import os
import string
folder = 'C:\Users\Jodie\Documents\Uni\Final Year Project\All Data'
folderlisting = os.listdir(folder)
for eachfolder in folderlisting:
print eachfolder
if os.path.isdir(folder + '\\' + eachfolder):
filelisting = os.listdir('C:\Users\Jodie\Documents\Uni\Final Year Project\All Data\\' + eachfolder)
print filelisting
for eachfile in filelisting:
if re.search('.genbank.txt$', eachfile):
genbankfile = open(eachfile, 'r')
print genbankfile
if re.search('.alleles.txt$', eachfile):
allelesfile = open(eachfile, 'r')
print allelesfile
It looks through a lot of folders, and prints the following:
The name of each folder, without the path
A list of all files in each folder
Two specific files in each folder (Any files containing ".genbank.txt" and ".alleles.txt").
The code works until it reaches a certain directory, and then fails with the following error:
Traceback (most recent call last):
File "C:\Users\Jodie\Documents\Uni\Final Year Project\Altering Frequency Practice\Change_freq_data.py", line 16, in <module>
genbankfile = open(eachfile, 'r')
IOError: [Errno 2] No such file or directory: 'ABP1.genbank.txt'
The problem is:
That file most definitely exists, since the program lists it before it tries to open the file.
Even if I take that directory out of the original group of directories, the program throws up the same error for the next folder it iterates to. And the next, if that one is removed. And so on.
This makes me think that it's not the folder or any files in it, but some limitation of Python? I have no idea. It has stumped me.
Any help would be appreciated.
You should use os.walk() http://docs.python.org/library/os.html#os.walk
Also, you need to read the contents of the file, you don't want to print the file object. And you need to close the file when you're done or use a context manager to close it for you.
would look something like:
for root, dirs, files in os.walk(folder):
for file_name in files:
if re.search('.genbank.txt$', file_name) or \
re.search('.alleles.txt$', file_name):
with open(os.path.join(root, f), 'r') as f:
print f.read()
Keep in mind this is not 'exactly' what you're doing, this will walk the entire tree, you may just want to walk a single level like you are already doing.