I'm having an issue with web2py. I have a text file called defVals.txt that's in the modules folder. I try to read from it, using open("defVals.txt") (in a Module in the same director as defVals.txt), but I get the error:
Traceback (most recent call last):
File "/home/jordan/web2py/gluon/restricted.py", line 212, in restricted
exec ccode in environment
File "/home/jordan/web2py/applications/randommotif/controllers/default.py", line 67, in <module>
File "/home/jordan/web2py/gluon/globals.py", line 188, in <lambda>
self._caller = lambda f: f()
File "/home/jordan/web2py/applications/randommotif/controllers/default.py", line 13, in index
defaultData = parse('defVals.txt')
File "applications/randommotif/modules/defaultValParser.py", line 6, in parse
lines = open(fileName)
IOError: [Errno 2] No such file or directory: 'defVals.txt'
What am I doing wrong? Where should I place defVals.txt
I'm using Ubuntu 12.10
Thanks,
Jordan
Update:
This is the source code to defaultValParser.py:
import itertools
import string
import os
from gluon import *
from gluon.custom_import import track_changes; track_changes(True)
#this returns a dictionary with the variables in it.
def parse(fileName):
moduleDir = os.path.dirname(os.path.abspath('defaultValParser.py'))
filePath = os.path.join(moduleDir, fileName)
lines = open(filePath, 'r')
#remove lines that are comments. Be sure to remove whitespace in the beginning and end of line
real = filter(lambda x: (x.strip())[0:2] != '//', lines)
parts = (''.join(list(itertools.chain(*real)))).split("<>")
names = map(lambda x: (x.split('=')[0]).strip(), parts)
values = map(lambda x: eval(x.split('=')[1]), parts)
return dict(zip(names, values))
It works fine if I import it and call it from a terminal (provided I comment out the gluon imports), but if I call it from a web2py controller, it fails completely:
Traceback (most recent call last):
File "/home/jordan/web2py/gluon/restricted.py", line 212, in restricted
exec ccode in environment
File "/home/jordan/web2py/applications/randommotif/controllers/default.py", line 71, in <module>
File "/home/jordan/web2py/gluon/globals.py", line 188, in <lambda>
self._caller = lambda f: f()
File "/home/jordan/web2py/applications/randommotif/controllers/default.py", line 17, in index
defaultData = parse('defVals.txt')
File "applications/randommotif/modules/defaultValParser.py", line 6, in parse
IOError: [Errno 2] No such file or directory: 'defVals.txt'
Use an absolute path based on the __file__ path of the module:
moduledir = os.path.dirname(os.path.abspath('__file__'))
# ..
defaultData = parse(os.path.join(moduledir, 'defVals.txt'))
__file__ is the filename of the current module, using the .dirname() of that gives you the directory the module is in. I used .abspath() to make sure you have an absolute path for your module file at all times, heading off some edgecases you could hit otherwise.
moduledir is a global in your module.
Related
I have a list (tuple), where is local path and img url. Works fine until now.
On Python 3.11 and Windows 11 still works well (no error at all), but on Ubuntu 20.04 with python 3.11 throwing FileNotFoundError.
I'm downloading more then 30 images, but i will cut off the list here.
I can't figure out what I'm missing?
/media/hdd/img/ directory exists with 755 permission.
from multiprocessing.pool import ThreadPool
import os
import requests
list2 = [('/media/hdd/img/tt24949716.jpg', 'https://www.cdnzone.org/uploads/2023/01/29/Devil-Beneath-2023-Cover.jpg'),
('/media/hdd/img/tt11231356.jpg', 'https://www.cdnzone.org/uploads/2023/01/29/Beneath-the-Green-2022-Cover.jpg'),
('/media/hdd/img/tt13103732.jpg', 'https://www.cdnzone.org/uploads/2023/01/29/The-Offering-2022-Cover.jpg'),
('/media/hdd/img/tt14826022.jpg', 'https://www.cdnzone.org/uploads/2023/01/29/You-People-2023-Cover.jpg'),
('/media/hdd/img/tt18252340.jpg', 'https://www.cdnzone.org/uploads/2023/01/29/Carnifex-2022-Cover.jpg')]
def download_images(list):
results = ThreadPool(8).imap_unordered(fetch_url, list)
for path in results:
print(path)
def fetch_url(entry):
path, uri = entry
if not os.path.exists(path):
r = requests.get(uri, stream=True)
if r.status_code == 200:
with open(path, 'wb') as f:
for chunk in r:
f.write(chunk)
return path
>>> download_images(list2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in download_images
File "/usr/lib/python3.11/multiprocessing/pool.py", line 930, in __init__
File "/usr/lib/python3.11/multiprocessing/pool.py", line 196, in __init__
File "/usr/lib/python3.11/multiprocessing/context.py", line 113, in SimpleQueue
File "/usr/lib/python3.11/multiprocessing/queues.py", line 341, in __init__
File "/usr/lib/python3.11/multiprocessing/context.py", line 68, in Lock
File "/usr/lib/python3.11/multiprocessing/synchronize.py", line 162, in __init__
File "/usr/lib/python3.11/multiprocessing/synchronize.py", line 57, in __init__
FileNotFoundError: [Errno 2] No such file or directory
I have written this Python script that checks if a file exists in a web server. And if it does exist, it copies is in the "found" directory on the local machine. However, since the web server is running Linux, and the full file path has slashes in it, it shows some errors. I have tried to solve it with "os.path" module but have been unsuccessful in my attempts.
import requests
import sys
import re
regex = re.compile(r"header(.*) footer", re.DOTALL)
data = {
"name":"user01",
"password":"secret"
}
r = requests.post('http://10.10.10.10/posts', data=data)
file = re.search(regex, r.text)
filename = sys.argv[1]
if file.group(1) != 'None':
with open('found/' + filename, 'w') as f:
f.write(file.group(1))
print("Found: ", filename)
I get the following error:
ERROR
Traceback (most recent call last):
File "/home/shit.py", line 18, in <module>
with open('found/' + filename, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'found//boot/grub/grub.cfg'
Traceback (most recent call last):
File "/home/shit.py", line 18, in <module>
with open('found/' + filename, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'found//etc/adduser.conf'
Traceback (most recent call last):
File "/home/shit.py", line 18, in <module>
with open('found/' + filename, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'found//etc/apache2/apache2.conf'
I have come up with a workaround. All I have to do is use:
filename = sys.argv[1].replace("/", "_")
It saves the files with an "_" instead of "/". For example, it saves "/etc/passwd" as "_etc_passwd".
I have a code where I call a Node script. The javaScript named preprocess_latex.js is in the same directory as filter_tokenize.py. Every directory has __init__.py so I can call the modules.
My code looks like this:
def tokenize(latex:str,kind:str='normalize')->list:
'''
Tokenize the incoming LaTex. This uses subscripts based on Node and Perl which uses bash call to KaTex. It also creates 2 .lst files in the same directory where the function is being called
args:
latex: Latex string
out:
List of cleaned tokens per LaTex
'''
path = pathlib.Path(__file__).parent.absolute() # When you call this module, it'll give the path of THIS file instead of the Current Working Directory
output_file = os.path.join(path,'out.lst')
input_file = os.path.join(path,'input_file.lst')
js_file_path = os.path.join(path,"preprocess_latex.js").replace(' ','\\ ')
with open(input_file,'w') as f:
f.write(latex+'\n')
cmd = "perl -pe 's|hskip(.*?)(cm\\|in\\|pt\\|mm\\|em)|hspace{\\1\\2}|g' %s > %s"%(input_file, output_file) # Call Perl subscript
ret = subprocess.call(cmd, shell=True)
cmd = f"cat {temp_file} | node {js_file_path} {kind} > {output_file} "
ret = subprocess.call(cmd, shell=True)
Problem is that I have spaces in my directories and when I do:
from latex_utils.filter_tokenize import *
tokenize('some_string')
it gives me error as:
Can't open Improve/Latex/latex_simialarity/latex_utils/input_file.lst: No such file or directory.
Can't open Improve/Latex/latex_simialarity/latex_utils/out.lst: No such file or directory.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/admin1/Desktop/Work/Search Improve/Latex/latex_simialarity/latex_utils/filter_tokenize.py", line 31, in tokenize
with open(output_file) as fin:
FileNotFoundError: [Errno 2] No such file or directory: '/home/admin1/Desktop/Work/Search Improve/Latex/latex_simialarity/latex_utils/out.lst'
I tried with output_file.replace(' ','\\ ') and there was a different error as:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/admin1/Desktop/Work/Search Improve/Latex/latex_simialarity/latex_utils/filter_tokenize.py", line 23, in tokenize
with open(input_file,'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/admin1/Desktop/Work/Search\\ Improve/Latex/latex_simialarity/latex_utils/input_file.lst'
I am creating a program that loads and runs python scripts from a compressed file. Along with those python scripts, I have a config file that I previously used configparser to load info from in an uncompressed version of the program.
Is it possible to directly read config files in zip files directly with configparser? or do I have to unzip it into a temp folder and load it from there?
I have tried directly giving the path:
>>> sysconf = configparser.ConfigParser()
>>> sysconf.read_file("compressed.zip/config_data.conf")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/local/lib/python3.4/configparser.py", line 1058, in _read
raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '<???>', line: 1
Didn't work. no surprises there.
Then I tried using zipfile
>>> zf = zipfile.ZipFile("compressed.zip")
>>> data = zf.read("config_data.conf")
>>> sysconf = configparser.ConfigParser()
>>> sysconf.read_file(data)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/local/lib/python3.4/configparser.py", line 1009, in _read
if line.strip().startswith(prefix):
AttributeError: 'int' object has no attribute 'strip'
and found that it didn't work either.
so I've resorted to creating a temp folder, uncompressing to it, and reading the conf file there. I would really like to avoid this if possible as the conf files are the only limiting factor. I can (and am) loading the python modules from the zip file just fine at this point.
I can get the raw text of the file if there's a way to pass that directly to configparser, but searching the docs I came up empty handed.
Update:
I tried using stringIO as a file object, and it seems to work somewhat.
configparser doesn't reject it, but it doesn't like it either.
>>> zf = zipfile.ZipFile("compressed.zip")
>>> data = zf.read(config_data.conf)
>>> confdata = io.StringIO(str(data))
>>> sysconf = configparser.ConfigParser()
>>> sysconf.readfp(confdata)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 736, in readfp
self.read_file(fp, source=filename)
File "/usr/local/lib/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/local/lib/python3.4/configparser.py", line 1058, in _read
raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '<???>', line: 1
(continues to spit out the entire contents of the file)
If I use read_file instead, it doesn't error out, but doesn't load anything either.
>>> zf = zipfile.ZipFile("compressed.zip")
>>> data = zf.read(config_data.conf)
>>> confdata = io.StringIO(str(data))
>>> sysconf = configparser.ConfigParser()
>>> sysconf.read_file(confdata)
>>> sysconf.items("General") #(this is the main section in the file)
Traceback (most recent call last):
File "/usr/local/lib/python3.4/configparser.py", line 824, in items
d.update(self._sections[section])
KeyError: 'General'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 827, in items
raise NoSectionError(section)
configparser.NoSectionError: No section: 'General'
can get the raw text of the file if there's a way to pass that directly to configparser
Try configparser.ConfigParser.read_string
When coupled with an appropriate ZIP file, this code works for me:
import zipfile
import configparser
zf = zipfile.ZipFile("compressed.zip")
zf_config = zf.open("config_data.conf", "rU")
zf_config_data = zf_config.read().decode('ascii')
config = configparser.ConfigParser()
config.read_string(zf_config_data)
assert config['today']['lunch']=='cheeseburger'
Upon reflection, the following might be more appropriate:
import zipfile
import configparser
import io
zf = zipfile.ZipFile("compressed.zip")
zf_config = zf.open("config_data.conf", "rU")
zf_config = io.TextIOWrapper(zf_config)
config = configparser.ConfigParser()
config.read_file(zf_config)
assert config['today']['lunch']=='cheeseburger'
As written in comments, #matthewatabet answer won't work with Python 3.4 (and newer vesions). It's because ZipFile.open now returns a "bytes-like" object and not a "file-like" object anymore. You can use:
codecs.getreader("utf-8")(config_file)
To convert the config_file bytes-like object into a file-like object using the UTF-8 encoding. The code is now:
import zipfile, configparser, codecs
# Python >= 3.4
with zipfile.ZipFile("compressed.zip") as zf:
config_file = zf.open("config_data.conf") # binary mode
sysconfig = configparser.ConfigParser()
sysconfig.read_file(codecs.getreader("utf-8")(config_file))
That seems more satisfactory than creating a string, but I don't know if it's more efficient...
EDIT Since Python 3.9, the zipfile module provides a zipfile.Path.open method that can handle text and binary modes. Default is text mode. The following code works fine:
# Python >= 3.9
with zipfile.ZipFile("compressed.zip") as zf:
zip_path = zipfile.Path(zf)
config_path = zip_path / "config_data.conf"
config_file = config_path.open() # text mode
sysconfig = configparser.ConfigParser()
sysconfig.read_file(config_file)
ZipFile not only supports read but also open, which returns a file-like object. So, you could do something like this:
zf = zipfile.ZipFile("compressed.zip")
config_file = zf.open("config_data.conf")
sysconfig = configparser.ConfigParser()
sysconfig.readfp(config_file)
I know you can open files, browsers, and URLs in the Python GUI. However, I don't know how to apply this to programs. For example, none of the below work. (The below are snippets from my growing chat bot program):
def browser():
print('OPENING FIREFOX...')
handle = webbroswer.get() # webbrowser is imported at the top of the file
handle.open('http://youtube.com')
handle.open_new_tab('http://google.com')
and
def file():
file = str(input('ENTER THE FILE\'S NAME AND EXTENSION:'))
action = open(file, 'r')
actionTwo = action.read()
print (actionTwo)
These errors occur, in respect to the above order, but in individual runs:
OPENING FIREFOX...
Traceback (most recent call last):
File "C:/Users/RCOMP/Desktop/Programming/Python Files/AI/COMPUTRON_01.py", line 202, in <module>
askForQuestions()
File "C:/Users/RCOMP/Desktop/Programming/Python Files/AI/COMPUTRON_01.py", line 64, in askForQuestions
browser()
File "C:/Users/RCOMP/Desktop/Programming/Python Files/AI/COMPUTRON_01.py", line 38, in browser
handle = webbroswer.get()
NameError: global name 'webbroswer' is not defined
>>>
ENTER THE FILE'S NAME AND EXTENSION:file.txt
Traceback (most recent call last):
File "C:/Users/RCOMP/Desktop/Programming/Python Files/AI/COMPUTRON_01.py", line 202, in <module>
askForQuestions()
File "C:/Users/RCOMP/Desktop/Programming/Python Files/AI/COMPUTRON_01.py", line 66, in askForQuestions
file()
File "C:/Users/RCOMP/Desktop/Programming/Python Files/AI/COMPUTRON_01.py", line 51, in file
action = open(file, 'r')
IOError: [Errno 2] No such file or directory: 'file.txt'
>>>
Am I handling this wrong, or can I just not use open() and webbrowser in a program?
You should read the errors and try to understand them - they are very helpful in this case - as they often are:
The first one says NameError: global name 'webbroswer' is not defined.
You can see here that webbrowser is spelled wrong in the code. It also tells you the line it finds the error (line 38)
The second one IOError: [Errno 2] No such file or directory: 'file.txt' tells you that you're trying to open a file that doesn't exist. This does not work because you specified
action = open(file, 'r')
which means that you're trying to read a file. Python does not allow reading from a file that does not exist.