I am trying to read a file path using config parser and later read from that file
>>> cfg_file = './crawler.config'
>>> config = SafeConfigParser()
>>> config.read(cfg_file)
['./crawler.config']
>>> f = config.get('default', 'sites_file')
>>> with open(f) as fp:
... print fp.read()
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: u'"/home/avi/src/typo.csv"'
>>>
I think there is a problem with the Unicode. I can't figure out a solution. If I pass the filename directly as a string it works fine. Any help in resolving this would be appreciated.
Try changing the value of 'sites_file' in the config to /home/avi/src/typo.csv from "/home/avi/src/typo.csv"
or:
Replace the quotes before opening the file.
Ex:
with open(f.replace('"', '')) as fp:
print fp.read()
f = config.get('default', 'sites_file')
may be f's value will be "/home/avi/src/typo.csv"
which does not exist
with open(f) as fp:
print fp.read()
You are trying to read a non existing file
Related
I am creating a program that loads and runs python scripts from a compressed file. Along with those python scripts, I have a config file that I previously used configparser to load info from in an uncompressed version of the program.
Is it possible to directly read config files in zip files directly with configparser? or do I have to unzip it into a temp folder and load it from there?
I have tried directly giving the path:
>>> sysconf = configparser.ConfigParser()
>>> sysconf.read_file("compressed.zip/config_data.conf")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/local/lib/python3.4/configparser.py", line 1058, in _read
raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '<???>', line: 1
Didn't work. no surprises there.
Then I tried using zipfile
>>> zf = zipfile.ZipFile("compressed.zip")
>>> data = zf.read("config_data.conf")
>>> sysconf = configparser.ConfigParser()
>>> sysconf.read_file(data)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/local/lib/python3.4/configparser.py", line 1009, in _read
if line.strip().startswith(prefix):
AttributeError: 'int' object has no attribute 'strip'
and found that it didn't work either.
so I've resorted to creating a temp folder, uncompressing to it, and reading the conf file there. I would really like to avoid this if possible as the conf files are the only limiting factor. I can (and am) loading the python modules from the zip file just fine at this point.
I can get the raw text of the file if there's a way to pass that directly to configparser, but searching the docs I came up empty handed.
Update:
I tried using stringIO as a file object, and it seems to work somewhat.
configparser doesn't reject it, but it doesn't like it either.
>>> zf = zipfile.ZipFile("compressed.zip")
>>> data = zf.read(config_data.conf)
>>> confdata = io.StringIO(str(data))
>>> sysconf = configparser.ConfigParser()
>>> sysconf.readfp(confdata)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 736, in readfp
self.read_file(fp, source=filename)
File "/usr/local/lib/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/local/lib/python3.4/configparser.py", line 1058, in _read
raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '<???>', line: 1
(continues to spit out the entire contents of the file)
If I use read_file instead, it doesn't error out, but doesn't load anything either.
>>> zf = zipfile.ZipFile("compressed.zip")
>>> data = zf.read(config_data.conf)
>>> confdata = io.StringIO(str(data))
>>> sysconf = configparser.ConfigParser()
>>> sysconf.read_file(confdata)
>>> sysconf.items("General") #(this is the main section in the file)
Traceback (most recent call last):
File "/usr/local/lib/python3.4/configparser.py", line 824, in items
d.update(self._sections[section])
KeyError: 'General'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.4/configparser.py", line 827, in items
raise NoSectionError(section)
configparser.NoSectionError: No section: 'General'
can get the raw text of the file if there's a way to pass that directly to configparser
Try configparser.ConfigParser.read_string
When coupled with an appropriate ZIP file, this code works for me:
import zipfile
import configparser
zf = zipfile.ZipFile("compressed.zip")
zf_config = zf.open("config_data.conf", "rU")
zf_config_data = zf_config.read().decode('ascii')
config = configparser.ConfigParser()
config.read_string(zf_config_data)
assert config['today']['lunch']=='cheeseburger'
Upon reflection, the following might be more appropriate:
import zipfile
import configparser
import io
zf = zipfile.ZipFile("compressed.zip")
zf_config = zf.open("config_data.conf", "rU")
zf_config = io.TextIOWrapper(zf_config)
config = configparser.ConfigParser()
config.read_file(zf_config)
assert config['today']['lunch']=='cheeseburger'
As written in comments, #matthewatabet answer won't work with Python 3.4 (and newer vesions). It's because ZipFile.open now returns a "bytes-like" object and not a "file-like" object anymore. You can use:
codecs.getreader("utf-8")(config_file)
To convert the config_file bytes-like object into a file-like object using the UTF-8 encoding. The code is now:
import zipfile, configparser, codecs
# Python >= 3.4
with zipfile.ZipFile("compressed.zip") as zf:
config_file = zf.open("config_data.conf") # binary mode
sysconfig = configparser.ConfigParser()
sysconfig.read_file(codecs.getreader("utf-8")(config_file))
That seems more satisfactory than creating a string, but I don't know if it's more efficient...
EDIT Since Python 3.9, the zipfile module provides a zipfile.Path.open method that can handle text and binary modes. Default is text mode. The following code works fine:
# Python >= 3.9
with zipfile.ZipFile("compressed.zip") as zf:
zip_path = zipfile.Path(zf)
config_path = zip_path / "config_data.conf"
config_file = config_path.open() # text mode
sysconfig = configparser.ConfigParser()
sysconfig.read_file(config_file)
ZipFile not only supports read but also open, which returns a file-like object. So, you could do something like this:
zf = zipfile.ZipFile("compressed.zip")
config_file = zf.open("config_data.conf")
sysconfig = configparser.ConfigParser()
sysconfig.readfp(config_file)
I have passed .csv file to post request,
input_file = data.get('file', None)
with input_file as datasheet:
header = datasheet.readline()
Always I am getting error on second line. Also my file type is Unicode thats why it again giving error on third line for readline()
>>> with "test1.html" as fp:
... header = fp.readline()
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: __exit__
>>>
How to Read file with with stament:
code:
>>> with open("test1.html") as fp:
... header = fp.readline()
...
Check file is exits or not before doing any process.
Use os module
Demo:
>>> os.path.isfile("test1.html")
True
>>> os.path.isfile("nofile.html")
False
>>>
File Upload to server via post request in API testing using tastypie
fp = open("C:\sample_datasheet.csv", 'rb')
content = fp.read()
fp.close()
fd ={'file': "C:\sample_datasheet.csv", "content": content}
self.assertHttpOK(self.api_client.post('api of upload', format='json',\
org_id=2, content_type="multipart/form-data",\
data=fd))
and Save content from the data to server location in the view.
Considering that {u'file': u'C:\\sample_datasheet.csv'} is returned by the data.get() function, you have to obtain the file name and open it:
data = data.get('file', None)
fname = data["file"]
with open(fname, "r") as datasheet:
header = datasheet.readline()
I have files on the linux server and their file names are broken because of the middle \r character. I couldn't download those files by using WinScp or Filezilla on windows.
Moreover I couldn't rename or process them properly in python.
On command
files = os.listdir("2014/")
I got this list set.
['16963_6_iris2570_20150110_052515\r_172518.gpx', '29174_3_Sunnam0223_20150114_0 10833\r_130835.gpx', '35767_3_samsi2_20150117_035045\r_155047.gpx', '36581_4_kix ing_20150117_045424\r_165425.gpx', '33383_4_rnrghk10kr_20150117_101618\r_101619. gpx']
On command:
file1 = files[0]
output: _172518.gpxs2570_20150110_052515
Then I try to replace \r
file2 = files[0].replace('\r', '')
output: 16963_6_iris2570_20150110_052515_172518.gpx
That's good but when I try to rename:
os.rename("2014/"+file1, "2014/"+file2)
f = open(file2, "r")
data = f.readlines()
f.close()
output:
Traceback (most recent call last):
File "test.py", line 25, in <module>
f = open(file2, "r")
IOError: [Errno 2] No such file or directory: '29174_3_Sunnam0223_20150114_010833_130835.gpx'
Did you try:
f = open("2014/"+file2, "r")
In your example code above, you included the 2014 folder name in your rename, but not in your open call.
I want to create and write a .txt file in a hidden folder using python. I am using this code:
file_name="hi.txt"
temp_path = '~/.myfolder/docs/' + file_name
file = open(temp_path, 'w')
file.write('editing the file')
file.close()
print 'Execution completed.'
where ~/.myfolder/docs/ is a hidden folder. I ma getting the error:
Traceback (most recent call last):
File "test.py", line 3, in <module>
file = open(temp_path, 'w')
IOError: [Errno 2] No such file or directory: '~/.myfolder/docs/hi.txt'
The same code works when I save the file in some non-hidden folder.
Any ideas why open() is not working for hidden folders.
The problem isn't that it's hidden, it's that Python cannot resolve your use of ~ representing your home directory. Use os.path.expanduser,
>>> with open('~/.FDSA', 'w') as f:
... f.write('hi')
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: '~/.FDSA'
>>>
>>> import os
>>> with open(os.path.expanduser('~/.FDSA'), 'w') as f:
... f.write('hi')
...
>>>
I'm working on a python script to thoroughly mutilate images, and to do so, I'm replacing every "g" in the file's text with an "h" (for now, it will likely change). Here's the beginning, and it's not working:
pathToFile = raw_input('File to corrupt (drag the file here): ')
x = open(pathToFile, 'r')
print x
After giving the path (dragging file into the terminal), this is the result:
File to corrupt (drag the file here): /Users/me/Desktop/file.jpeg
Traceback (most recent call last):
File "/Users/me/Desktop/corrupt.py", line 7, in <module>
x = open(pathToFile, 'r')
IOError: [Errno 2] No such file or directory: '/Users/me/Desktop/file.jpeg '
How can the file not exist if it's right there, and I'm using the exact filename?
Look closely: '/Users/me/Desktop/file.jpeg '. There's a space in your filename. open doesn't do any stripping.
>>> f = open('foo.txt', 'w')
>>> f.write('a')
>>> f.close()
>>> f = open('foo.txt ', 'r')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'foo.txt '