Am doing a bit of unit testing on a function which attempts to open a new file, but should fail if the file already exists. when the function runs sucessfully, the new file is created, so i want to delete it after every test run, but it doesn't seem to be working:
class MyObject_Initialisation(unittest.TestCase):
def setUp(self):
if os.path.exists(TEMPORARY_FILE_NAME):
try:
os.remove(TEMPORARY_FILE_NAME)
except WindowsError:
#TODO: can't figure out how to fix this...
#time.sleep(3)
#self.setUp() #this just loops forever
pass
def tearDown(self):
self.setUp()
any thoughts? The Windows Error thrown seems to suggest the file is in use... could it be that the tests are run in parallel threads?
I've read elsewhere that it's 'bad practice' to use the filesystem in unit testing, but really? Surely there's a way around this that doesn't invole dummying the filesystem?
If you're just looking for a temporary file, have a look at tempfile - this should handle the clean-up all on its own.
Do you remember to explicitly close file handler that operates on TEMPORARY_FILE_NAME?
From Python Documentation:
On Windows, attempting to remove a
file that is in use causes an
exception to be raised;
Related
I try to use the behave before_tag callback.
But is seems to be never called.
So I try to put any other before_XXX function in my environment.py file:
def before_all(context):
raise ValueError("Here we are")
def before_scenario(context, scenario):
raise ValueError("Here we are")
def before_step(context, step):
raise ValueError("Here we are")
But steps are still executed without exception.
As inserting invalid syntax in the same python file result in "Exception SyntaxError" it is interpreted by behave.
Do you have any idea of what can I do to identify my usage problem?
(I'm currently using behave 1.2.6)
Thanks to behave code deployed in python; I could investigate with python debuger:
environment.py shall be stored in the current directory.
I was storing it in a sub directory named steps near my other python files.
I'm now able to attach fixtures to a tag!
Imagine one writes a unit test for handle for a case when path does not exist:
def handle(path):
try:
with open(path) as f:
pass
except FileNotFoundError:
raise FileNotFoundError(path)
I would write something like below for such test:
import pytest
def test_handle_on_non_existent_path():
x = "abc" # some unbelievable string
with pytest.raises(FileNotFoundError):
handle(x)
My question is what is a better way to generate non-existent path for a unit-test.
My ideas are:
force delete a temporary file
genereate a random string like uuid?
"abc" is fairly concise, but in principle does guarantee path does not exist.
Update: in this question x is "no_exist.txt"
With respect to unit-testing, it seems your intent is to test the behaviour of your code for the case that open(path) will throw a FileNotFoundError. Your approach is to have the code actually perform the open call, but with a non-existent path name. This has some disadvantages: As you already noticed, the dependency on the real file system brings the question how to create a value for path that reliably does not exist as a file on the file system. But, there is another point, namely that you are not even sure that there could not exist another problem with the file system, for example some permission related problem, which could cause some other exception to be raised (OSError).
Put together, performing the call to open itself means you are not in full control of what happens. Therefore, a better approach can be, for this unit test case, to mock open and make your mock raise the FileNotFoundError.
I am very new to python, but I have written a simple python script tool for automating the process of updating mosaic datasets at my job. The tool runs great, but sometimes I get the dreaded 9999999 error, or "the geodatase already exists" when I try to overwrite the data.
The file structure is c:\users\my.name\projects\ImageryMosaic\Alachua_2014\Alachua_2014_mosaic.gdb. After some research, I determined that the lock was being placed on the FGDB whenever I opened the newly created mosaic dataset inside of the FGDB to check for errors after running the tool. I would like to be able to overwrite the data instead of having to delete it, so I am using the arcpy.env.overwriteOutput statement in my script. This works fine unless I open the dataset after running the tool. Since other people will be using this tool, I don't want them scratching thier heads for hours like me, so it would be nice if the script tool could look for the presence of a .Lock file in the geodatabase. That way I could at least provide a statement in the script as to why the tool failed in lieu of the unhelpful 9999999 error. I know about arcpy.TestSchemaLock, but I don't think that will work in this case since I am not trying to place a lock and I want to overwrite the FGDB, not edit it.
Late but this function below will check for lock files in given (gdb) path.
def lockFileExist(path = None):
if path == None:
import traceback
raise Exception("Invalid Path!")
else:
import glob
full_file_paths = glob.glob(path+"\\*.lock")
for f in full_file_paths:
if f.endswith(".lock"):
return True
return False
if lockFileExist(r"D:\sample.gdb"):
print "Lock file found in gdb. Aborting..."
else:
print "No lock files found!. Ready for processing..."
edit:OK, I could swear that the way I'd tested it showed that the getcwd was also causing the exception, but now it appears it's just the file creation. When I move the try-except blocks to their it actually does catch it like you'd think it would. So chalk that up to user error.
Original Question:
I have a script I'm working on that I want to be able to drop a file on it to have it run with that file as an argument. I checked in this question, and I already have the mentioned registry keys (apparently the Python 2.6 installer takes care of it.) However, it's throwing an exception that I can't catch. Running it from the console works correctly, but when I drop a file on it, it throws an exception then closes the console window. I tried to have it redirect standard error to a file, but it threw the exception before the redirection occurred in the script. With a little testing, and some quick eyesight I saw that it was throwing an IOError when I tried to create the file to write the error to.
import sys
import os
#os.chdir("C:/Python26/Projects/arguments")
try:
print sys.argv
raw_input()
os.getcwd()
except Exception,e:
print sys.argv + '\n'
print e
f = open("./myfile.txt", "w")
If I run this from the console with any or no arguments, it behaves as one would expect. If I run it by dropping a file on it, for instance test.txt, it runs, prints the arguments correctly, then when os.getcwd() is called, it throws the exception, and does not perform any of the stuff from the except: block, making it difficult for me to find any way to actually get the exception text to stay on screen. If I uncomment the os.chdir(), the script doesn't fail. If I move that line to within the except block, it's never executed.
I'm guessing running by dropping the file on it, which according to the other linked question, uses the WSH, is somehow messing up its permissions or the cwd, but I don't know how to work around it.
Seeing as this is probably not Python related, but a Windows problem (I for one could not reproduce the error given your code), I'd suggest attaching a debugger to the Python interpreter when it is started. Since you start the interpreter implicitly by a drag&drop action, you need to configure Windows to auto-attach a debugger each time Python starts. If I remember correctly, this article has the needed information to do that (you can substitute another debugger if you are not using Visual Studio).
Apart from that, I would take a snapshot with ProcMon while dragging a file onto your script, to get an idea of what is going on.
As pointed out in my edit above, the errors were caused by the working directory changing to C:\windows\system32, where the script isn't allowed to create files. I don't know how to get it to not change the working directory when started that way, but was able to work around it like this.
if len(sys.argv) == 1:
files = [filename for filename in os.listdir(os.getcwd())
if filename.endswith(".txt")]
else:
files = [filename for filename in sys.argv[1:]]
Fixing the working directory can be managed this way I guess.
exepath = sys.argv[0]
os.chdir(exepath[:exepath.rfind('\\')])
I am on windows with Python 2.5. I have an open file for writing. I write some data. Call file close. When I try to delete the file from the folder using Windows Explorer, it errors, saying that a process still holds a handle to the file.
If I shutdown python, and try again, it succeeds.
It does close them.
Are you sure f.close() is getting called?
I just tested the same scenario and windows deletes the file for me.
Are you handling any exceptions around the file object? If so, make sure the error handling looks something like this:
f = open("hello.txt")
try:
for line in f:
print line
finally:
f.close()
In considering why you should do this, consider the following lines of code:
f = open('hello.txt')
try:
perform_an_operation_that_causes_f_to_raise_an_exception()
f.close()
except IOError:
pass
As you can see, f.close will never be called in the above code. The problem is that the above code will also cause f to not get garbage collected. The reason is that f will still be referenced in sys.traceback, in which case the only solution is to manually call close on f in a finally block or set sys.traceback to None (and I strongly recommend the former).
Explained in the tutorial:
with open('/tmp/workfile', 'r') as f:
read_data = f.read()
It works when you writing or pickling/unpickling, too
It's not really necessary that try finally block: Java way of doing things, not Python
I was looking for this, because the same thing happened to me. The question didn't help me, but I think I figured out what happened.
In the original version of the script I wrote, I neglected to add in a 'finally' clause to the file in case of an exception.
I was testing the script from the interactive prompt and got an exception while the file was open. What I didn't realize was that the file object wasn't immediately garbage-collected. After that, when I ran the script (still from the same interactive session), even though the new file objects were being closed, the first one still hadn't been, and so the file handle was still in use, from the perspective of the operating system.
Once I closed the interactive prompt, the problem went away, at which I remembered that exception occurring while the file was open and realized what had been going on. (Moral: Don't try to program on insufficient sleep. : ) )
Naturally, I have no idea if this is what happened in the case of the original poster, and even if the original poster is still around, they may not remember the specific circumstances, but the symptoms are similar, so I thought I'd add this as something to check for, for anyone caught in the same situation and looking for an answer.
I did it using intermediate file:
import os
f = open("report.tmp","w")
f.write("{}".format("Hello"))
f.close()
os.system("move report.tmp report.html") #this line is for Windows users