I am trying to understand the docs.
My Python script generates Python code that runs much later, so I want to check now if what I generated is valid.
The docs say py_compile.compile(file[, cfile[, dfile[, doraise]]]) and
If doraise is true, a PyCompileError is raised when an error is encountered while compiling file
So, I tried
source = open(generatedScriptPath, 'rt').read() + '\n'
try:
import py_compile
x = py_compile.compile(source, '', '', True)
except py_compile.PyCompileError, e:
print str(e)
but the inner Exception is never hit, instead an outer Exception is being caught:
Traceback (most recent call last): File
"H:/code/testgen/testGen.py", line 293, in <module>
x = py_compile.compile(source, '', '', True) File "C:\Python27\lib\py_compile.py", line 106, in compile
with open(file, 'U') as f: IOError: [Errno 2] No such file or directory: '# This script was auto-generated ...
How can I fix this? Note that I am open to alternatives, I just want to ask the simplest way. "Is the code I just generated syntactically valid Python?"
Read the error message: "No such file or directory". The first argument is supposed to be a file name to open. Or read the docs: "The source code is loaded from the file name /file/." You might prefer the "compile" builtin, which can compile a string of Python.
Related
I want to check if a file exists before returning it. So inside my script, I did:
print("***********")
print(args.src_path)
print(os.path.isdir(args.src_path))
But the terminal returns that:
(val_env) jovyan#jupyter-me:~/use-cases/UC_Scene_understanding/Code_Woodscape/scripts$ python semantic_map_generator.py --src_path ../data/instance_annotations/ --dst_path ../data/semantic_annotations --semantic_class_mapping configs/semantic_mapping_9_classes.json --instance_class_mapping scripts/mappers/class_names.json
***********
../data/instance_annotations/
False
Traceback (most recent call last):
File "/home/jovyan/use-cases/UC_Scene_understanding/Code_Woodscape/scripts/semantic_map_generator.py", line 69, in <module>
src_path, dst_path, semantic_classes_mapping, instance_classes_mapping = parser_arguments()
File "/home/jovyan/use-cases/_Scene_understanding/Code_Woodscape/scripts/semantic_map_generator.py", line 63, in parser_arguments
raise Exception("Error: Check if the files or dirs in arguments exit!")
Exception: Error: Check if the files or dirs in arguments exit!
However, to me this ../data/instance_annotations/ is an example of dir that os.path.isdir(args.src_path) should return True.
Furthermore this directory exists:
$find
...
./scripts/semantic_map_generator.py
./scripts/polygon_generator.py
./scripts/mappers
./scripts/mappers/class_names.json
...
./scripts/box_2d_generator.py
./data
./data/semantic_annotations
./data/download.txt
Considering that this error is not related to this part and os.path.isdir() method returns if False the file or directory doesn't exist, this error is related to the later lines of this code (probably somewhere in this code you open and read a file) but
for checking existence of a file or directory I recommend you to try .exists() method instead of .isdir() method.
import os
os path.exists("blob/blob/blobfile.txt")
Reference: os.path.exists(path) [Python-doc]
What specific Python 3 syntax must be changed below in order to successfully print a trackback form a successfully running function into a file that is located at aValidFileNameAndPath?
We need this to work in normal running functions where there is NO exception.
The Python 3 code we are using is:
import traceback
traceback.print_stack(file=aValidFileNameAndPath)
The error thrown by the above code is:
File "C:\path\to\script\in\our\app\ourScriptName.py", line 69, in ourFunctionName
traceback.print_stack(file=aValidFileNameAndPath)
File "C:\Program Files\Python39\lib\traceback.py", line 190, in print_stack
print_list(extract_stack(f, limit=limit), file=file)
File "C:\Program Files\Python39\lib\traceback.py", line 25, in print_list
print(item, file=file, end="")
AttributeError: 'str' object has no attribute 'write'
The other postings we have found on Stack Overflow have to do with printing exceptions. We do NOT want to print an exception. Instead, we just want to print out the chain of functions that called a specific function during normal functioning of the app when there is no exception to be thrown.
You are getting this error because where you are using the file path the code wants a file object.
To make a file object, use open, e.g. myfile = open(aValidFileNameAndPath)
You will also want to set the file to writing mode, e.g. open(path, 'w')
Then you can pass myfile as a paremeter, e.g. traceback.print_stack(file=myfile)
Then make sure to close the file with close
Here is a full example:
import traceback
myfile = open(aValidFileNameAndPath, 'w') # note that this will delete anything that alredy exists in the file
traceback.print_stack(file=myfile)
myfile.close()
I am working on html parser, it uses Python multiprocessing Pool, because it runs through huge number of pages. The output from every page is saved to a separate CSV file. The problem is sometimes I get unexpected error and whole program crashes and I have errors handling almost everywhere - reading pages, parsing pages, even writing files. Moreover it looks like the script crashes after it finishes writing a batch of files, so it shouldn't be anything to crush on. Thus after whole day of debugging I am left clueless.
Error:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 44, in mapstar
return list(map(*args))
File "D:\ppp\Python\parser\run.py", line 244, in media_process
save_media_product(DIRECTORY, category, media_data)
File "D:\ppp\Python\parser\manage_output.py", line 180, in save_media_product
_file_manager(target_file, temp, temp2)
File "D:\ppp\Python\store_parser\manage_output.py", line 214, in _file_manager
file_to_write.close()
UnboundLocalError: local variable 'file_to_write' referenced before assignment
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\ppp\Python\store_parser\run.py", line 356, in <module>
main()
File "D:\Rzeczy Mariusza\Python\store_parser\run.py", line 318, in main
process.map(media_process, batch)
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 644, in get
raise self._value
UnboundLocalError: local variable 'file_to_write' referenced before assignment
It look like, there is an error with variable assignment, but it is not:
try:
file_to_write = open(target_file, 'w')
except OSError:
message = 'OSError while writing file name - {}'.format(target_file)
log_error(message)
except UnboundLocalError:
message = 'UnboundLocalError while writing file name - {}'.format(target_file)
log_error(message)
except Exception as e:
message = 'Total failure "{}" while writing file name - {}'.format(e, target_file)
log_error(message)
else:
file_to_write.write(temp)
file_to_write.write(temp2)
finally:
file_to_write.close()
Line - except Exception as e:, does not help with anything, the whole thing still crashes. So far i have excluded only Out Of Memory scenario, because this script is designed to be handled on low spec VPS, but in testing stage I run it in environment with 8 GB of ram. So if You have any theories please share.
The exception really says what is happening.
This part is telling you obvious issue:
UnboundLocalError: local variable 'file_to_write' referenced before assignment
Even you have try/except blocks that catches various exceptions, else/finally doesn't.
More specifically in finally block you reference variable that might not exist since exception with doing: file_to_write = open(target_file, 'w') is being handled by at least last except Exception as e block, but then finally is run too.
Since exception happened as a result of not being able to open target file, you do not have anything assigned to file_to_write and that variable doesn't exist after exception is handled. That is why finally block crashes.
Given the following code:
# Edit build number in test report
print(path) # TODO remove
html_report = fileinput.input(path, inplace=True)
for line in html_report:
print(line.replace('$BUILD_NUMBER',
args.number).rstrip())
html_report.close()
I get the following output:
/home/jenkins/workspace/reports/report201610261053.html
Traceback (most recent call last):
File "report_generator.py", line 58, in <module>
for line in html_report:
File "/usr/lib/python2.7/fileinput.py", line 252, in next
line = self.readline()
File "/usr/lib/python2.7/fileinput.py", line 321, in readline
os.rename(self._filename, self._backupfilename)
OSError: [Errno 2] No such file or directory
If I just use the command:
gedit /home/jenkins/workspace/reports/report201610261053.html
I can check that the file exists. In fact, if it didn't I would expect this error to be raised in the fileinput.input() line, not in the line loop.
Any idea of what's wrong?
What is your "path" value?
I think you should try to use absolute path
You can also check the user permissions to file.
I don't see anything wrong in the code that you have shown.
I can check that the file exists. In fact, if it didn't I would expect
this error to be raised in the fileinput.input() line, not in the line
loop.
The error is reported only when an attempt is made to open file and it happens in the for loop.
Is it possible that your code is running under a different user and doesn't see the file on that path as compared to you manually verifying file existence?
def functION():
Source_obj = path.relpath("WebSource\EXAMPLE SOURCE.htm")
data = Source_obj.read()
I am having trouble opening this file while located in a sub-directory directly underneath my Python file... is there a better way to open files from ANY directory on my computer?
FileNotFoundError: [Errno 2] No such file or directory: 'WebSource\\EXAMPLE SOURCE.htm'
I can't read from the file because I get the following error:
C:\python34\python.exe G:\Robot\test.py
Process started >>>
Traceback (most recent call last):
File "G:\Robot\test.py", line 118, in <module>
functION()
File "G:\Robot\test.py", line 64, in functION
data = Source_obj.read()
AttributeError: 'str' object has no attribute 'read'
<<< Process finished. (Exit code 1)
================ READY ================
BTW: The file to be read is just a source file from an HTML Chrome webpage.
EDIT
I'm looking for more help with the path and wondering why I get the first mentioned Traceback regarding the path
os.path.relpath() returns a string, not an open file object. You'll need to open a file first; use the open() function:
def functION():
Source_obj = path.relpath(r"WebSource\EXAMPLE SOURCE.htm")
with open(Source_obj) as fileobj:
data = fileobj.read()
with here treats the file object as a context manager; when the indented codeblock under the statement is exited (either because the code completed or an exception occurred), the file object will automatically be closed.
Your Source_obj is just a string, not a file.
def functION():
Source_obj = path.relpath("WebSource\EXAMPLE SOURCE.htm")
with open(Source_obj) as f:
data = f.read()
By open()-ing it you can read from the file. Using the with context manager, the file will be properly closed for you when you leave that block of code.