File not found with Python's fileinput - python

Given the following code:
# Edit build number in test report
print(path) # TODO remove
html_report = fileinput.input(path, inplace=True)
for line in html_report:
print(line.replace('$BUILD_NUMBER',
args.number).rstrip())
html_report.close()
I get the following output:
/home/jenkins/workspace/reports/report201610261053.html
Traceback (most recent call last):
File "report_generator.py", line 58, in <module>
for line in html_report:
File "/usr/lib/python2.7/fileinput.py", line 252, in next
line = self.readline()
File "/usr/lib/python2.7/fileinput.py", line 321, in readline
os.rename(self._filename, self._backupfilename)
OSError: [Errno 2] No such file or directory
If I just use the command:
gedit /home/jenkins/workspace/reports/report201610261053.html
I can check that the file exists. In fact, if it didn't I would expect this error to be raised in the fileinput.input() line, not in the line loop.
Any idea of what's wrong?

What is your "path" value?
I think you should try to use absolute path
You can also check the user permissions to file.

I don't see anything wrong in the code that you have shown.
I can check that the file exists. In fact, if it didn't I would expect
this error to be raised in the fileinput.input() line, not in the line
loop.
The error is reported only when an attempt is made to open file and it happens in the for loop.
Is it possible that your code is running under a different user and doesn't see the file on that path as compared to you manually verifying file existence?

Related

"directory/" is not recognized as such by os.path.isdir()

I want to check if a file exists before returning it. So inside my script, I did:
print("***********")
print(args.src_path)
print(os.path.isdir(args.src_path))
But the terminal returns that:
(val_env) jovyan#jupyter-me:~/use-cases/UC_Scene_understanding/Code_Woodscape/scripts$ python semantic_map_generator.py --src_path ../data/instance_annotations/ --dst_path ../data/semantic_annotations --semantic_class_mapping configs/semantic_mapping_9_classes.json --instance_class_mapping scripts/mappers/class_names.json
***********
../data/instance_annotations/
False
Traceback (most recent call last):
File "/home/jovyan/use-cases/UC_Scene_understanding/Code_Woodscape/scripts/semantic_map_generator.py", line 69, in <module>
src_path, dst_path, semantic_classes_mapping, instance_classes_mapping = parser_arguments()
File "/home/jovyan/use-cases/_Scene_understanding/Code_Woodscape/scripts/semantic_map_generator.py", line 63, in parser_arguments
raise Exception("Error: Check if the files or dirs in arguments exit!")
Exception: Error: Check if the files or dirs in arguments exit!
However, to me this ../data/instance_annotations/ is an example of dir that os.path.isdir(args.src_path) should return True.
Furthermore this directory exists:
$find
...
./scripts/semantic_map_generator.py
./scripts/polygon_generator.py
./scripts/mappers
./scripts/mappers/class_names.json
...
./scripts/box_2d_generator.py
./data
./data/semantic_annotations
./data/download.txt
Considering that this error is not related to this part and os.path.isdir() method returns if False the file or directory doesn't exist, this error is related to the later lines of this code (probably somewhere in this code you open and read a file) but
for checking existence of a file or directory I recommend you to try .exists() method instead of .isdir() method.
import os
os path.exists("blob/blob/blobfile.txt")
Reference: os.path.exists(path) [Python-doc]

python readline module giving PermissionError: [Errno 1] only when run at startup

I'm making a console based game that saves input history, to help with debugging I created a function that will automatically input commands on start
def __readfile (self) -> None:
lines = None
with open("insts.txt", "r") as f:
lines = f.read().split("\n")
if (lines == None):
print("attempted to read insts.txt for instructions, could not find file")
return
self.__initfile = True
for line in lines:
self.parse_input(line)
self.__initfile = False
in the instance of the error "self.parse_input" ultimately leads to "readline.read_history_file" and none of the code in between the two has any effect on the error
but it gives this error message:
Traceback (most recent call last):
File "main.py", line 9, in <module>
game.start()
File "/Users/tristans/Documents/GitHub/console-rpg/classes.py", line 1353, in start
self.__readfile()
File "/Users/tristans/Documents/GitHub/console-rpg/classes.py", line 1345, in __readfile
self.parse_input(line)
File "/Users/tristans/Documents/GitHub/console-rpg/classes.py", line 1223, in parse_input
self._parse_dialog(text)
File "/Users/tristans/Documents/GitHub/console-rpg/classes.py", line 1128, in _parse_dialog
self._parse_dialog("leave")
File "/Users/tristans/Documents/GitHub/console-rpg/classes.py", line 1101, in _parse_dialog
self._load_hist_scope()
File "/Users/tristans/Documents/GitHub/console-rpg/classes.py", line 1331, in _load_hist_scope
readline.read_history_file("history.txt")
PermissionError: [Errno 1] Operation not permitted
I have looked everywhere for an answer to where this error is coming from and can't find one
I've tried changing the file read operation from the "with open("insts.txt")" to a hardcoded list but that didn't work, os.access("history.txt", os.R_OK) also returns true when called just before "readline.read_history_file"
It appears to be a Mac-specific issue with readline. According to this answer, you need to use gnureadline on the Mac, rather than readline.
import gnureadline as readline

Using VS Code to debug python files. Exception thrown on breakpoint and breakpoint is ignored

Tried with multiple different python files. Every time I try to use the debugger in vs code and set breakpoints the breakpoint gets ignored and exception gets raised and the script continues on. I've been googling and tinkering for over 2 hours and can't seem to figure out what's going on here. Tried rebooting PC, running vs code as admin, uninstall/reinstall the python extension for vs code. Tried to dig into the files mentioned in the traceback and pinpointed the function that seems to be raising the exception but I can't figure out where it's being called from or why it's raising the exception. I'm still new-ish to Python. Debugging works properly on my laptop but for whatever reason my desktop is having this issue.
Traceback (most recent call last):
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 529, in _original_file_to_client
return cache[filename]
KeyError: 'c:\\users\\joel\\local settings\\application data\\programs\\python\\python37-32\\lib\\runpy.py'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py", line 330, in _on_run
self.process_net_command_json(self.py_db, json_contents)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_process_net_command_json.py", line 190, in process_net_command_json
cmd = on_request(py_db, request)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_process_net_command_json.py", line 771, in on_stacktrace_request
self.api.request_stack(py_db, request.seq, thread_id, fmt=fmt, start_frame=start_frame, levels=levels)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_api.py", line 214, in request_stack
if internal_get_thread_stack.can_be_executed_by(get_current_thread_id(threading.current_thread())):
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py", line 661, in can_be_executed_by
py_db, self.seq, self.thread_id, frame, self._fmt, must_be_suspended=not timed_out, start_frame=self._start_frame, levels=self._levels)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_net_command_factory_json.py", line 213, in make_get_thread_stack_message
py_db, frames_list
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_net_command_factory_xml.py", line 175, in _iter_visible_frames_info
new_filename_in_utf8, applied_mapping = pydevd_file_utils.norm_file_to_client(filename_in_utf8)
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 531, in _original_file_to_client
translated = _path_to_expected_str(get_path_with_real_case(_AbsFile(filename)))
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 221, in _get_path_with_real_case
return _resolve_listing(drive, iter(parts))
File "c:\Users\Joel\.vscode\extensions\ms-python.python-2020.7.96456\pythonFiles\lib\python\debugpy\_vendored\pydevd\pydevd_file_utils.py", line 184, in _resolve_listing
dir_contents = cache[resolved_lower] = os.listdir(resolved)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\Joel\\Local Settings'
So I get this traceback every time a breakpoint is hit. Taking a peek at the "_original_file_to_client" function in "pydevd_file_utils.py" we get this:
def _original_file_to_client(filename, cache={}):
try:
return cache[filename]
except KeyError:
translated = _path_to_expected_str(get_path_with_real_case(_AbsFile(filename)))
cache[filename] = (translated, False)
return cache[filename]
I wasn't able to figure out where this function was being called from or what the expected output was supposed to be. Any help with this would be greatly appreciated!
Edit: Forgot to mention I'm using Windows 10 if it wasn't obvious from the trace
This is a similar question. The spaces in the filename cause this problem:
"Local Settings", "application data"

Python exclude CWD file in proc

I have a script called client that does a few things around the OS and it's failing when it comes to /proc/1/cwd
I'm trying to ignore these types of files but the script is still crashing when it comes to this file. I get Memory Map output and also a Stack Trace before the script crashes.
I did a file /proc/1/cwd command on the file and I can see it's a symlink, where I then included an if os.path.islink(file) is False: IF statement to skip it and proceed.
However, I seem to still be hitting it.
The below is the Traceback after the crash.
Traceback (most recent call last):
File "client.py", line 235, in <module>
File "client.py", line 120, in runner
OSError: [Errno 13] Permission denied: '/proc/1/cwd'
Failed to execute script client
The error suggests the script client is working on /proc/1/cwd and then right after seems to say that it can't execute itself?
Any help would be appreciated.
-----Additional -------
Code lines from error:
like 235 is literally the line runner(path)
runner() represents a function that has a for loop that cycles through files in a folder to find the correct file. During this process, runner will do the following:
do line 120 mode = os.stat(fullfilename).st_mode
check if os.path.isdir(fullfilename) is False:
then if stat.S_ISREG(mode) is True:
then if os.path.islink(fullfilename) is False:
then if fullfilename not in blacklist:
Line 120 is mode = os.stat(fullfilename).st_mode

PyXB - AssertionError: No element bindings in http://www.w3.org/1999/xhtml

I am attempting to generate bindings for a WSDL with PyXB, and it is giving the AssertionError exception in the title.
My understanding, based on the PyXB documentation, is that the bundle archive for http://www.w3.org/1999/xhtml is included with PyXB. However, something appears to be wrong. It either does not get used, or it has incorrect contents.
I use the following command line to attempt to generate the bindings:
python c:\Python27\Scripts\pyxbgen.py --wsdl-location=http://xx.xxx.xxx.xxx/YYY.asmx?WSDL --module=client --write-for-customization
The traceback:
Traceback (most recent call last):
File "c:\Python27\Scripts\pyxbgen.py", line 51, in <module> generator.resolveExternalSchema()
File "c:\Python27\lib\site-packages\pyxb\binding\generate.py", line 2647, in resolveExternalSchema
schema = converter(self, sl)
File "c:\Python27\Scripts\pyxbgen.py", line 28, in WSDLToSchema
spec = wsdl.definitions.createFromDOM(pyxb.utils.domutils.StringToDOM(xmld,
location_base=wsdl_uri), process_schema=True, generation_uid=generator.generationUID())
File "c:\Python27\lib\site-packages\pyxb\binding\basis.py", line 1767, in createFromDOM
return self._createFromDOM(node, expanded_name, **kw)
File "c:\Python27\lib\site-packages\pyxb\binding\basis.py", line 1791, in _createFromDOM
return element.CreateDOMBinding(node, self.elementForName(expanded_name), **kw)
File "c:\Python27\lib\site-packages\pyxb\binding\basis.py", line 1735, in elementForName
assert 'elementBinding' in elt_en.namespace()._categoryMap(), 'No element bindings in %s' % (elt_en.namespace(),)
AssertionError: No element bindings in http://www.w3.org/1999/xhtml
In addition, I set the PYXB_ARCHIVE_PATH environment variable to:
C:\Python27\Lib\site-packages\pyxb\bundles\common\raw
I am not sure if this is the correct way to do this. I also tried specifying the --archive-path command line option as well, but I got the same error back.
Probably you need to use:
--archive-path=${PYXB_ROOT}/pyxb/bundles/common//:+
as the argument. This recursively searches for available namespaces in the common bundles first, then includes any other search paths. There's an example in the manual that's close to this.

Categories

Resources