Python config file for script - python

I have a problem with reading config file from file.
It looks pretty basic but as I am new in python for I'm missing something.
Config file looks like this
CCWD_HOME=/batch/ccwd
#Temporary location for daemon job listing
CCWD_TEMP=/tmp
#Directory for job definitions
CCWD_PROD=/batch/PRD
The problem is that syntax of this file has to stay this way.
Assigning string to variable needs quota marks ("").
Is there any easy possible way to read variables from config file as above?
e.g. I have script
#!/bin/python
import conf
print CCWD_TEMP
And got this error
Traceback (most recent call last):
File "./testconf", line 2, in <module>
import conf
File "/app/test/conf.py", line 6
CCWD_HOME=/batch/ccwd
^
SyntaxError: invalid syntax

It looks like you are trying to import the config file. But you can't do that: import is for importing Python modules, so the file you import is expected to be valid Python, which CCWD_HOME=/batch/ccwd is not. That is what the syntax error means.
You can use the module configparser to read the file, but it requires the settings to be grouped in sections headed by a section name in square brackets, like this:
[MyStuff]
CCWD_HOME=/batch/ccwd
#Temporary location for daemon job listing
CCWD_TEMP=/tmp
#Directory for job definitions
CCWD_PROD=/batch/PRD
If you can't change the config file you will have to parse it yourself.
with open("./testconf") as configs:
for config in configs:
if config.startswith("#"):
continue
keyword, value = config.split("=")

The directories need to be strings. You can achieve that by not importing the config file as module but open the file as text file (for example by numpys loadtxt()). Afterwards you can read out the directories by carefully scanning the lines.

Change it to the following:
CCWD_HOME="/batch/ccwd"
#Temporary location for daemon job listing
CCWD_TEMP="/tmp"
#Directory for job definitions
CCWD_PROD="/batch/PRD"
Since you have made conf.py a python file, it has to adhere to python standards. Hence, it has to be in above format
If you want it to be like this only, then you'll have to use it in a diff manner. Instead of importing it, read it as a file, extract the contents and then do the required operations. Following is the code for that:
>>> with open("a.txt") as f:
... content = f.readlines()
...
>>>
# you may also want to remove whitespace characters like `\n` at the end of each line
>>> content = dict(x.split("=") for x in content if "=" in x)
>>> content
{'CCWD_PROD': '/batch/PRDw\n', 'CWD_HOME': '/batch/ccwd\n', 'CCWD_TEMP': '/tmp\n'}
>>>

You are trying to execute your conf file as a python script. Instead, it would be better to write a simple parser of your config file and inport all config values into a dict like so:
conf = {}
with open(r"PathToFile", "r") as f:
for confline in f.readlines():
if "=" in confline:
conf[confline.partition("=")[0]] = confline.partition("=")[2]
output is
print(conf)
{'CCWD_PROD': '/batch/PRD', 'CCWD_HOME': '/batch/ccwd\n', 'CCWD_TEMP': '/tmp\n'}

Related

How to include another python file from the same folder

I was wondering if it's possible for me to include another python file in the actual one?
I'm running an app.py from the terminal with Flask and I want, when I click on submit that python run 5 lines of code, like nothing. But when I do the variable from the first python file can't be read on the second and even if I put the same variable into the second python file then it still doesn't work.
I want to run a "for", this is the code
for line in fin:
line = re.sub('APP:NAME', name, line)
line = re.sub('APP:USERNAME', username, line)
line = re.sub('APP:TEXT', tweet, line)
fout.write(line)
I checked all the forums and I didn't find the solution.
Thank you
Method 1
I think your need is to include a python file to another python file in a same directory.
You can import a python file as shown below
import filename
here, the filename is the filename of the file in the same location without file extension.
and you can call functions inside that file by calling like this.
filename.function()
Method 2
if you which to rename that filename and use it as something else:
import filename as fn
here, the filename is the filename of the file in the same location without file extension.
and you can call functions inside that file by calling like this.
fn.functionName()
Method 3
or you can simply use
from filename import *
and call the functions in that file as normal functions in the current python file like
functionname()
but Better method is method 1.

Read config file in python without any section

I want to read a config file in python. My file contents look like this
name:age:location
abc:1:US
pqr:2:UK
I tried to use the config parser to read the data. Since the file does not contain any section, I added it on the fly
from configparser import ConfigParser
parser = ConfigParser()
with open("foo.conf") as stream:
parser.read_string("[top]\n" + stream.read())
I am not sure how can I iterate the whole file data and read.

Python Error:CSV File Error, Read/Print CSV file

I am getting this error when trying to print the contents of a CSV file in Python.
Traceback (most recent call last):
File "/Users/cassandracampbell/Library/Preferences/PyCharmCE2018.2/scratches/Player.py", line 5, in
with open('player.csv') as csvfile:
FileNotFoundError: [Errno 2] No such file or directory: 'player.csv'
Get the exact file path to the csv, if you are on a windows get the entire folder path and then the name, and then do:
with open(r'C:\users\path\players.csv') as csvfile:
If you're using a windows and the exact path, easiest to put the r before the path like I did because it is a literal which will allow the string to be interpreted and the path to be found.
You must put player.csv to the same location with your script Player.py
Example like your code, both files should be here: /Users/cassandracampbell/Library/Preferences/PyCharmCE2018.2/scratches/
Or you can put the specific directory of player.csv,
Ex:
with open("/Users/cassandracampbell/Library/Preferences/PyCharmCE2018.2/scratches/player.csv") as csvfile:
...
Check that you have the file in question in same directory as your .py file you're working on. If that doesn't work you might want to use the full path to the file:
with open ('/Users/cassandracampbell/Library/Preferences/PyCharmCE2018.2/scratches/player.CSV') as csvfile:
And you should try to check the name of the file too should be case sensitive otherwise, let's say you have Player.csv then don't try to open player.csv all lower case won't work!
Plus I don't know what you're trying to do with your CSV file, just print the raw content? You might like using pandas.

How to store a dictionary as a separate file, and read the file in a python script to use the variables

I would like some help using python to open a file and use the contents of the file as a variables.
I have a script that looks like this.
#!/usr/bin/env python
with open("seqnames-test1-iso-legal-temp.txt") as f:
gene_data = {'ham_pb_length':2973, 'ham_pb_bitscore':5664,'cg2225_ph_length':3303, 'cg2225_ph_bitscore':6435,'lrp1_pf_length':14259, 'lrp1_pf_bitscore':28010,}
for line in f:
if not line.isspace():
bitscore = gene_data[line.rstrip()+'_bitscore']
length = gene_data[line.rstrip()+'_length']
if (2*0.95*length <= bitscore/2 <= 2*1.05*length):
print line
Where the file "seqnames-test1-iso-legal-temp.txt" is a list of gene names ham_pb, cg2225, lrp1_pf, etc. I only included the first 6 values of the dictionary, but it has a total of 600 keys. Each in the form 'name'_length, 'name'_bitscore for the 300 gene names in the file "seqnames-test1-iso-legal-temp.txt".
For this reason, I would like to save the dictionary gene_data as a separate text file, and read the file while executing the script. Is there a way to do this. I tried to make a text file "gene_data1.txt" that just included the dictionary. So, the contents of the text file are:
gene_data = { 'ham_pb_length':2973, 'ham_pb_bitscore':5664,'cg2225_ph_length':3303, 'cg2225_ph_bitscore':6435,'lrp1_pf_length':14259, 'lrp1_pf_bitscore':28010,}
And I tried to use the open function to open the file, so my script looked like this:
#!/usr/bin/env python
gene_data = open("gene_data1.txt", "r")
with open("seqnames-test1-iso-legal-temp.txt") as f:
for line in f:
if not line.isspace():
bitscore = gene_data[line.rstrip()+'_bitscore']
length = gene_data[line.rstrip()+'_length']
if (2*0.95*length <= bitscore/2 <= 2*1.05*length):
print line
But this just gave me the error message:
Traceback (most recent call last):
File "fixduplicatebittest1.py", line 6, in <module>
bitscore = gene_data[line.rstrip()+'_bitscore']
NameError: name 'gene_data' is not defined
Is there a simple way to make this script?
Any help would be greatly appreciated.
You can replace this line:
gene_data = open("gene_data1.txt", "r")
with this:
import ast
with open('dict.txt') as f:
gene_data = f.read()
gene_data = ast.literal_eval(gene_data)
but make sure the text file just contains the dictionary, not the assignment of the dictionary:
{ 'ham_pb_length':2973, 'ham_pb_bitscore':5664,'cg2225_ph_length':3303, 'cg2225_ph_bitscore':6435,'lrp1_pf_length':14259, 'lrp1_pf_bitscore':28010,}
As pointed out by others, allowing your script to execute any command in a file can be dangerous. With this method, at least it won't execute anything in the external file, if the contents don't evaluate nicely the script will just throw an error.
The simplest way would be to put the dictionary as you wrote it in its own .py file and import it like any other module.
from <filename without .py> import gene_data
Then you can use it as if you had typed it in the importing module.
This is very unsafe to do if you do not control the data file.
Either execfile or import will let you run it as text inside your file. Be mindful of security implications though. import gives you more control over the execution process, but at the expense of more involved syntax.

Issues reading and writing txt files line by line

The script is written using PyQt4.10.1 and Python2.7
I have been working on a simple tool to do allow a user to search for paths and then save them out to a config file for another program to read later. If there is already a config file then the script reads it and displays the existing paths for the user to edit or add to. I wrote a gui to make it as user friendly as possible. There are a couple issues I am having with it.
First, when I read in the config file I am using the following code:
try:
self.paths = open(configFile, "r")
self.data = self.paths.readlines()
self.paths.close()
except:
self.data = None
if self.data is not None:
for line in self.data:
print line
#self.listDelegate is the model for my QListView
self.listDelegate.insertRows(0, 1, line)
When I do that I get the following in my gui:
This (above) is how it looks when you first input the data (before the data is saved and then reopened)
This (above) is how the data looks after the config file is saved and then read back in (note the extra space below the path).
The config file is only read in when the script is first opened.
the following is how the config file looks when it is written out.
C:\Program Files
C:\MappedDrives
C:\NVIDIA
Now all of that wouldnt be a big deal but when I open the config file to edit it with this tool then the extra space in the gui is read as another line break. so the config file is then printed as:
C:\Program Files
C:\MappedDrives
C:\NVIDIA
Then the problem just gets bigger and bigger every time I edit the file.
This issue leads me to the second issue (which I think may be the culprit). When I write the lines from the gui to the config file I use the following code:
rowCount = self.listDelegate.rowCount()
if rowCount > 0:
myfile = open(configFile, 'w')
for i in range(rowCount):
myfile.write(str(self.listDelegate.index(i).data(role = QtCore.Qt.DisplayRole).toPyObject()))
myfile.write("\n")
myfile.close()
I am assuming that the issue with the extra line breaks is because I am adding the line breaks in manually. The problem is that I need each path to be on its own line for the config file to be usable later. I don't have a lot of experience writing out text files and everyone says that the easiest way to write them out line by line is to add in the line breaks by hand. If anyone has any better ideas I would love to hear them.
Sorry for the long winded explanation. If I am not being clear enough please tell me and I will try to explain myself better.
Thanks for the help!
The problem is that every time you read the file the line break remains at the end of the line. From the description of readline:
f.readline() reads a single line from the file; a newline character (\n) is left at the end of the string, and is only omitted on the last line of the file if the file doesn’t end in a newline.
If you try
self.paths = open(configFile, "r")
self.data = self.paths.readlines()
for line in self.data:
print repr(line)
which prints the representation of every line as python code you will get something like
'C:\\Program Files\n'
'C:\\MappedDrives\n'
'C:\\NVIDIA\n'
As you later insert further newlines the easiest fix is probably to remove the trailing newline:
for line in self.data:
strippedLine = line.rstrip('\n')

Categories

Resources