extracting source code line before and after logging statement - python

I am trying to extract the source code line before and after the logging statement using astor and python AST, thanks to the answers available on my previous StackOverflow question:
import ast
import astor
import json
class LogPrinter(astor.TreeWalk):
def pre_Call(self):
if (isinstance(self.cur_node.func, ast.Attribute) and
isinstance(self.cur_node.func.value, ast.Name) and
(self.cur_node.func.value.id.lower() == "logging" or self.cur_node.func.value.id.lower() == "log" or self.cur_node.func.value.id.lower() == "logger")
):
print(astor.to_source(self.cur_node))
def post_Call(self):
if (isinstance(self.cur_node.func, ast.Attribute)):
print(astor.to_source(self.cur_node))
I can get logging statements from the above code but I am not able to get complete preceding and following statements:
example:
def db_contecton(string url):
if URL:
#multiline logger
logger.debug(
"connecting to
URL : " +url)
#multiline source code
db = db
.conn.
init()
db.connect(url)
#multiline logger
logger.
info("connection
to DB
successful")
The output that I am trying to get is:
1)
if url:
logger.debug("connecting to URL : " +url)
db = db.conn.init()
and
2)
db = db.conn.init()
db.connect(url)
logger.info("connection to DB successful")
NOTE: multiline loggers and source code is converted into single line
At this point I can extract loggers and get the following statement which does not have the complete source code:
my current output is
#the statement above logger is missing
logger.debug("connecting to URL : " +url)
db.conn.init() #the db variable is missing should be db = db.conn.init()
The above are just example snippets of python source code that may not be logically correct.

I'm not sure i understood your question: Do you want to parse python files as text and extract the 1 above and 1 below lines wherever it has some logging related code?
You could read the file as text, break it into lines, match with your desired words (e.g.: logger, logging, log), keep the indexes of those lines and then do whatever you want with it:
log_words = 'logger logging log'.split()
python_files = ['file1.py', 'file2.py']
keep = dict()
for f in python_files:
keep[f] = []
with open(f, 'r') as f:
lines = f.readlines()
for i in range(len(lines)):
for w in log_words:
if w in lines[i]:
keep[f].append(i - 1) # previous line
keep[f].append(i) # actual line
keep[f].append(i + 1) # next line
It checks each line for each desired word for each given file.
It's not performatic though.
But i believe it solves your problem.

Related

How do I run python '__main__' program file from bash prompt in Windows10?

I am trying to run a python3 program file and am getting some unexpected behaviors.
I'll start off first with my PATH and env setup configuration. When I run:
which Python
I get:
/c/Program Files/Python36/python
From there, I cd into the directory where my python program is located to prepare to run the program.
Roughly speaking this is how my python program is set up:
import modulesNeeded
print('1st debug statement to show program execution')
# variables declared as needed
def aFunctionNeeded():
print('2nd debug statement to show fxn exe, never prints')
... function logic...
if __name__ == '__main__':
aFunctionNeeded() # Never gets called
Here is a link to the repository with the code I am working with in case you would like more details as to the implementation. Keep in mind that API keys are not published, but API keys are in local file correctly:
https://github.com/lopezdp/API.Mashups
My question revolves around why my 1st debug statements inside the files are printing to the terminal, but not the 2nd debug statements inside the functions?
This is happening in both of the findRestaurant.py file and the geocode.py file.
I know I have written my if __name__ == '__main__': program entry point correctly as this is the same exact way I have done it for other programs, but in this case I may be missing something that I am not noticing.
If this is my output when I run my program in my bash terminal:
$ python findRestaurant.py
inside geo
inside find
then, why does it appear that my aFunctionNeeded() method shown in my pseudo code is not being called from the main?
Why do both programs seem to fail immediately after the first debug statements are printed to the terminal?
findRestaurant.py File that can also be found in link above
from geocode import getGeocodeLocation
import json
import httplib2
import sys
import codecs
print('inside find')
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
sys.stderr = codecs.getwriter('utf8')(sys.stderr)
foursquare_client_id = "..."
foursquare_client_secret = "..."
def findARestaurant(mealType,location):
print('inside findFxn')
#1. Use getGeocodeLocation to get the latitude and longitude coordinates of the location string.
latitude, longitude = getGeocodeLocation(location)
#2. Use foursquare API to find a nearby restaurant with the latitude, longitude, and mealType strings.
#HINT: format for url will be something like https://api.foursquare.com/v2/venues/search?client_id=CLIENT_ID&client_secret=CLIENT_SECRET&v=20130815&ll=40.7,-74&query=sushi
url = ('https://api.foursquare.com/v2/venues/search?client_id=%s&client_secret=%s&v=20130815&ll=%s,%s&query=%s' % (foursquare_client_id, foursquare_client_secret,latitude,longitude,mealType))
h = httplib2.Http()
result = json.loads(h.request(url,'GET')[1])
if result['response']['venues']:
#3. Grab the first restaurant
restaurant = result['response']['venues'][0]
venue_id = restaurant['id']
restaurant_name = restaurant['name']
restaurant_address = restaurant['location']['formattedAddress']
address = ""
for i in restaurant_address:
address += i + " "
restaurant_address = address
#4. Get a 300x300 picture of the restaurant using the venue_id (you can change this by altering the 300x300 value in the URL or replacing it with 'orginal' to get the original picture
url = ('https://api.foursquare.com/v2/venues/%s/photos?client_id=%s&v=20150603&client_secret=%s' % ((venue_id,foursquare_client_id,foursquare_client_secret)))
result = json.loads(h.request(url, 'GET')[1])
#5. Grab the first image
if result['response']['photos']['items']:
firstpic = result['response']['photos']['items'][0]
prefix = firstpic['prefix']
suffix = firstpic['suffix']
imageURL = prefix + "300x300" + suffix
else:
#6. if no image available, insert default image url
imageURL = "http://pixabay.com/get/8926af5eb597ca51ca4c/1433440765/cheeseburger-34314_1280.png?direct"
#7. return a dictionary containing the restaurant name, address, and image url
restaurantInfo = {'name':restaurant_name, 'address':restaurant_address, 'image':imageURL}
print ("Restaurant Name: %s" % restaurantInfo['name'])
print ("Restaurant Address: %s" % restaurantInfo['address'])
print ("Image: %s \n" % restaurantInfo['image'])
return restaurantInfo
else:
print ("No Restaurants Found for %s" % location)
return "No Restaurants Found"
if __name__ == '__main__':
findARestaurant("Pizza", "Tokyo, Japan")
geocode.py File that can also be found in link above
import httplib2
import json
print('inside geo')
def getGeocodeLocation(inputString):
print('inside of geoFxn')
# Use Google Maps to convert a location into Latitute/Longitute coordinates
# FORMAT: https://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&key=API_KEY
google_api_key = "..."
locationString = inputString.replace(" ", "+")
url = ('https://maps.googleapis.com/maps/api/geocode/json?address=%s&key=%s' % (locationString, google_api_key))
h = httplib2.Http()
result = json.loads(h.request(url,'GET')[1])
latitude = result['results'][0]['geometry']['location']['lat']
longitude = result['results'][0]['geometry']['location']['lng']
return (latitude,longitude)
The reason you're not seeing the output of the later parts of your code is that you've rebound the standard output and error streams with these lines:
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
sys.stderr = codecs.getwriter('utf8')(sys.stderr)
I'm not exactly sure why those lines are breaking things for you, perhaps your console does not expect utf8 encoded output... But because they don't work as intended, you're not seeing anything from the rest of your code, including error messages, since you rebound the stderr stream along with the stdout stream.

Getting file input into Python script for praw script

So I have a simple reddit bot set up which I wrote using the praw framework. The code is as follows:
import praw
import time
import numpy
import pickle
r = praw.Reddit(user_agent = "Gets the Daily General Thread from subreddit.")
print("Logging in...")
r.login()
words_to_match = ['sdfghm']
cache = []
def run_bot():
print("Grabbing subreddit...")
subreddit = r.get_subreddit("test")
print("Grabbing thread titles...")
threads = subreddit.get_hot(limit=10)
for submission in threads:
thread_title = submission.title.lower()
isMatch = any(string in thread_title for string in words_to_match)
if submission.id not in cache and isMatch:
print("Match found! Thread ID is " + submission.id)
r.send_message('FlameDraBot', 'DGT has been posted!', 'You are awesome!')
print("Message sent!")
cache.append(submission.id)
print("Comment loop finished. Restarting...")
# Run the script
while True:
run_bot()
time.sleep(20)
I want to create a file (text file or xml, or something else) using which the user can change the fields for the various information being queried. For example I want a file with lines such as :
Words to Search for = sdfghm
Subreddit to Search in = text
Send message to = FlameDraBot
I want the info to be input from fields, so that it takes the value after Words to Search for = instead of the whole line. After the information has been input into the file and it has been saved. I want my script to pull the information from the file, store it in a variable, and use that variable in the appropriate functions, such as:
words_to_match = ['sdfghm']
subreddit = r.get_subreddit("test")
r.send_message('FlameDraBot'....
So basically like a config file for the script. How do I go about making it so that my script can take input from a .txt or another appropriate file and implement it into my code?
Yes, that's just a plain old Python config, which you can implement in an ASCII file, or else YAML or JSON.
Create a subdirectory ./config, put your settings in ./config/__init__.py
Then import config.
Using PEP-18 compliant names, the file ./config/__init__.py would look like:
search_string = ['sdfghm']
subreddit_to_search = 'text'
notify = ['FlameDraBot']
If you want more complicated config, just read the many other posts on that.

try: and exception: error

So i am working on this code below. It complied alright when my Reff.txt has more than one line. But it doesnt work when my Reff.txt file has one line. Why is that? I also wondering why my code doesn't run "try" portion of my code but it always run only "exception" part.
so i have a reference file which has a list of ids (one id per line)
I use the reference file(Reff.txt) as a reference to search through the database from the website and the database from the server within my network.
The result i should get is there should be an output file and file with information of that id; for each reference id
However, this code doesn't do anything on my "try:" portion at all
import sys
import urllib2
from lxml import etree
import os
getReference = open('Reff.txt','r') #open the file that contains list of reference ids
global tID
for tID in getReference:
tID = tID.strip()
try:
with open(''+tID.strip()+'.txt') as f: pass
fileInput = open(''+tID+'.txt','r')
readAA = fileInput.read()
store_value = (readAA.partition('\n'))
aaSequence = store_value[2].replace('\n', '') #concatenate lines
makeList = list(aaSequence)#print makeList
inRange = ''
fileAddress = '/database/int/data/'+tID+'.txt'
filename = open(fileAddress,'r')#name of the working file
print fileAddress
with open(fileAddress,'rb') as f:
root = etree.parse(f)
for lcn in root.xpath("/protein/match[#dbname='PFAM']/lcn"):#find dbname =PFAM
start = int(lcn.get("start"))#if it is PFAM then look for start value
end = int(lcn.get("end"))#if it is PFAM then also look for end value
while start <= end:
inRange = makeList[start]
start += 1
print outputFile.write(inRange)
outputFile.close()
break
break
break
except IOError as e:
newURL ='http://www.uniprot.org/uniprot/'+tID+'.fasta'
print newURL
response = urllib2.urlopen(''+newURL) #go to the website and grab the information
creatNew = open(''+uniprotID+'.txt','w')
html = response.read() #read file
creatNew.write(html)
creatNew.close()
So, when you do Try/Except - if try fails, Except runs. Except is always running, because Try is always failing.
Most likely reason for this is that you have this - "print outputFile.write(inRange)", but you have not previously declared outputFile.
ETA: Also, it looks like you are only interested in testing to the first pass of the for loop? You break at that point. Your other breaks are extraneous in that case, because they will never be reached while that one is there.

How Do I Suppress or Disable Warnings in reSTructuredText?

I'm working on a CMS in Python that uses reStructuredText (via docutils) to format content. Alot of my content is imported from other sources and usually comes in the form of unformatted text documents. reST works great for this because it makes everything look pretty sane by default.
One problem I am having, however, is that I get warnings dumped to stderr on my webserver and injected into my page content. For example, I get warnings like the following on my web page:
System Message: WARNING/2 (, line 296); backlink
My question is: How do I suppress, disable, or otherwise re-direct these warnings?
Ideally, I'd love to write these out to a log file, but if someone can just tell me how to turn off the warnings from being injected into my content then that would be perfect.
The code that's responsible for parsing the reST into HTML:
from docutils import core
import reSTpygments
def reST2HTML( str ):
parts = core.publish_parts(
source = str,
writer_name = 'html')
return parts['body_pre_docinfo'] + parts['fragment']
def reST2HTML( str ):
parts = core.publish_parts(
source = str,
writer_name = 'html',
settings_overrides={'report_level':'quiet'},
)
return parts['body_pre_docinfo'] + parts['fragment']
It seems the report_level accept string is an old version. Now, the below is work for me.
import docutils.core
import docutils.utils
from pathlib import Path
shut_up_level = docutils.utils.Reporter.SEVERE_LEVEL + 1
docutils.core.publish_file(
source_path=Path(...), destination_path=Path(...),
settings_overrides={'report_level': shut_up_level},
writer_name='html')
about level
# docutils.utils.__init__.py
class Reporter(object):
# system message level constants:
(DEBUG_LEVEL,
INFO_LEVEL,
WARNING_LEVEL,
ERROR_LEVEL,
SEVERE_LEVEL) = range(5)
...
def system_message(self, level, message, *children, **kwargs):
...
if self.stream and (level >= self.report_level # self.report_level was set by you. (for example, shut_up_level)
or self.debug_flag and level == self.DEBUG_LEVEL
or level >= self.halt_level):
self.stream.write(msg.astext() + '\n')
...
return msg
According to the above code, you know that you can assign the self.report_level (i.e. settings_overrides={'report_level': ...}) let the warning not show.
and I set it to SERVER_LEVEL+1, so it will not show any error. (you can set it according to your demand.)

Properties file in python (similar to Java Properties)

Given the following format (.properties or .ini):
propertyName1=propertyValue1
propertyName2=propertyValue2
...
propertyNameN=propertyValueN
For Java there is the Properties class that offers functionality to parse / interact with the above format.
Is there something similar in python's standard library (2.x) ?
If not, what other alternatives do I have ?
I was able to get this to work with ConfigParser, no one showed any examples on how to do this, so here is a simple python reader of a property file and example of the property file. Note that the extension is still .properties, but I had to add a section header similar to what you see in .ini files... a bit of a bastardization, but it works.
The python file: PythonPropertyReader.py
#!/usr/bin/python
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read('ConfigFile.properties')
print config.get('DatabaseSection', 'database.dbname');
The property file: ConfigFile.properties
[DatabaseSection]
database.dbname=unitTest
database.user=root
database.password=
For more functionality, read: https://docs.python.org/2/library/configparser.html
For .ini files there is the configparser module that provides a format compatible with .ini files.
Anyway there's nothing available for parsing complete .properties files, when I have to do that I simply use jython (I'm talking about scripting).
I know that this is a very old question, but I need it just now and I decided to implement my own solution, a pure python solution, that covers most uses cases (not all):
def load_properties(filepath, sep='=', comment_char='#'):
"""
Read the file passed as parameter as a properties file.
"""
props = {}
with open(filepath, "rt") as f:
for line in f:
l = line.strip()
if l and not l.startswith(comment_char):
key_value = l.split(sep)
key = key_value[0].strip()
value = sep.join(key_value[1:]).strip().strip('"')
props[key] = value
return props
You can change the sep to ':' to parse files with format:
key : value
The code parses correctly lines like:
url = "http://my-host.com"
name = Paul = Pablo
# This comment line will be ignored
You'll get a dict with:
{"url": "http://my-host.com", "name": "Paul = Pablo" }
A java properties file is often valid python code as well. You could rename your myconfig.properties file to myconfig.py. Then just import your file, like this
import myconfig
and access the properties directly
print myconfig.propertyName1
if you don't have multi line properties and a very simple need, a few lines of code can solve it for you:
File t.properties:
a=b
c=d
e=f
Python code:
with open("t.properties") as f:
l = [line.split("=") for line in f.readlines()]
d = {key.strip(): value.strip() for key, value in l}
If you have an option of file formats I suggest using .ini and Python's ConfigParser as mentioned. If you need compatibility with Java .properties files I have written a library for it called jprops. We were using pyjavaproperties, but after encountering various limitations I ended up implementing my own. It has full support for the .properties format, including unicode support and better support for escape sequences. Jprops can also parse any file-like object while pyjavaproperties only works with real files on disk.
This is not exactly properties but Python does have a nice library for parsing configuration files. Also see this recipe: A python replacement for java.util.Properties.
i have used this, this library is very useful
from pyjavaproperties import Properties
p = Properties()
p.load(open('test.properties'))
p.list()
print(p)
print(p.items())
print(p['name3'])
p['name3'] = 'changed = value'
Here is link to my project: https://sourceforge.net/projects/pyproperties/. It is a library with methods for working with *.properties files for Python 3.x.
But it is not based on java.util.Properties
This is a one-to-one replacement of java.util.Propeties
From the doc:
def __parse(self, lines):
""" Parse a list of lines and create
an internal property dictionary """
# Every line in the file must consist of either a comment
# or a key-value pair. A key-value pair is a line consisting
# of a key which is a combination of non-white space characters
# The separator character between key-value pairs is a '=',
# ':' or a whitespace character not including the newline.
# If the '=' or ':' characters are found, in the line, even
# keys containing whitespace chars are allowed.
# A line with only a key according to the rules above is also
# fine. In such case, the value is considered as the empty string.
# In order to include characters '=' or ':' in a key or value,
# they have to be properly escaped using the backslash character.
# Some examples of valid key-value pairs:
#
# key value
# key=value
# key:value
# key value1,value2,value3
# key value1,value2,value3 \
# value4, value5
# key
# This key= this value
# key = value1 value2 value3
# Any line that starts with a '#' is considerered a comment
# and skipped. Also any trailing or preceding whitespaces
# are removed from the key/value.
# This is a line parser. It parses the
# contents like by line.
You can use a file-like object in ConfigParser.RawConfigParser.readfp defined here -> https://docs.python.org/2/library/configparser.html#ConfigParser.RawConfigParser.readfp
Define a class that overrides readline that adds a section name before the actual contents of your properties file.
I've packaged it into the class that returns a dict of all the properties defined.
import ConfigParser
class PropertiesReader(object):
def __init__(self, properties_file_name):
self.name = properties_file_name
self.main_section = 'main'
# Add dummy section on top
self.lines = [ '[%s]\n' % self.main_section ]
with open(properties_file_name) as f:
self.lines.extend(f.readlines())
# This makes sure that iterator in readfp stops
self.lines.append('')
def readline(self):
return self.lines.pop(0)
def read_properties(self):
config = ConfigParser.RawConfigParser()
# Without next line the property names will be lowercased
config.optionxform = str
config.readfp(self)
return dict(config.items(self.main_section))
if __name__ == '__main__':
print PropertiesReader('/path/to/file.properties').read_properties()
If you need to read all values from a section in properties file in a simple manner:
Your config.properties file layout :
[SECTION_NAME]
key1 = value1
key2 = value2
You code:
import configparser
config = configparser.RawConfigParser()
config.read('path_to_config.properties file')
details_dict = dict(config.items('SECTION_NAME'))
This will give you a dictionary where keys are same as in config file and their corresponding values.
details_dict is :
{'key1':'value1', 'key2':'value2'}
Now to get key1's value :
details_dict['key1']
Putting it all in a method which reads that section from config file only once(the first time the method is called during a program run).
def get_config_dict():
if not hasattr(get_config_dict, 'config_dict'):
get_config_dict.config_dict = dict(config.items('SECTION_NAME'))
return get_config_dict.config_dict
Now call the above function and get the required key's value :
config_details = get_config_dict()
key_1_value = config_details['key1']
-------------------------------------------------------------
Extending the approach mentioned above, reading section by section automatically and then accessing by section name followed by key name.
def get_config_section():
if not hasattr(get_config_section, 'section_dict'):
get_config_section.section_dict = dict()
for section in config.sections():
get_config_section.section_dict[section] =
dict(config.items(section))
return get_config_section.section_dict
To access:
config_dict = get_config_section()
port = config_dict['DB']['port']
(here 'DB' is a section name in config file
and 'port' is a key under section 'DB'.)
create a dictionary in your python module and store everything into it and access it, for example:
dict = {
'portalPath' : 'www.xyx.com',
'elementID': 'submit'}
Now to access it you can simply do:
submitButton = driver.find_element_by_id(dict['elementID'])
My Java ini files didn't have section headers and I wanted a dict as a result. So i simply injected an "[ini]" section and let the default config library do its job.
Take a version.ini fie of the eclipse IDE .metadata directory as an example:
#Mon Dec 20 07:35:29 CET 2021
org.eclipse.core.runtime=2
org.eclipse.platform=4.19.0.v20210303-1800
# 'injected' ini section
[ini]
#Mon Dec 20 07:35:29 CET 2021
org.eclipse.core.runtime=2
org.eclipse.platform=4.19.0.v20210303-1800
The result is converted to a dict:
from configparser import ConfigParser
#staticmethod
def readPropertyFile(path):
# https://stackoverflow.com/questions/3595363/properties-file-in-python-similar-to-java-properties
config = ConfigParser()
s_config= open(path, 'r').read()
s_config="[ini]\n%s" % s_config
# https://stackoverflow.com/a/36841741/1497139
config.read_string(s_config)
items=config.items('ini')
itemDict={}
for key,value in items:
itemDict[key]=value
return itemDict
This is what I'm doing in my project: I just create another .py file called properties.py which includes all common variables/properties I used in the project, and in any file need to refer to these variables, put
from properties import *(or anything you need)
Used this method to keep svn peace when I was changing dev locations frequently and some common variables were quite relative to local environment. Works fine for me but not sure this method would be suggested for formal dev environment etc.
import json
f=open('test.json')
x=json.load(f)
f.close()
print(x)
Contents of test.json:
{"host": "127.0.0.1", "user": "jms"}
I have created a python module that is almost similar to the Properties class of Java ( Actually it is like the PropertyPlaceholderConfigurer in spring which lets you use ${variable-reference} to refer to already defined property )
EDIT : You may install this package by running the command(currently tested for python 3).
pip install property
The project is hosted on GitHub
Example : ( Detailed documentation can be found here )
Let's say you have the following properties defined in my_file.properties file
foo = I am awesome
bar = ${chocolate}-bar
chocolate = fudge
Code to load the above properties
from properties.p import Property
prop = Property()
# Simply load it into a dictionary
dic_prop = prop.load_property_files('my_file.properties')
Below 2 lines of code shows how to use Python List Comprehension to load 'java style' property file.
split_properties=[line.split("=") for line in open('/<path_to_property_file>)]
properties={key: value for key,value in split_properties }
Please have a look at below post for details
https://ilearnonlinesite.wordpress.com/2017/07/24/reading-property-file-in-python-using-comprehension-and-generators/
you can use parameter "fromfile_prefix_chars" with argparse to read from config file as below---
temp.py
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('--a')
parser.add_argument('--b')
args = parser.parse_args()
print(args.a)
print(args.b)
config file
--a
hello
--b
hello dear
Run command
python temp.py "#config"
You could use - https://pypi.org/project/property/
eg - my_file.properties
foo = I am awesome
bar = ${chocolate}-bar
chocolate = fudge
long = a very long property that is described in the property file which takes up \
multiple lines can be defined by the escape character as it is done here
url=example.com/api?auth_token=xyz
user_dir=${HOME}/test
unresolved = ${HOME}/files/${id}/${bar}/
fname_template = /opt/myapp/{arch}/ext/{objid}.dat
Code
from properties.p import Property
## set use_env to evaluate properties from shell / os environment
prop = Property(use_env = True)
dic_prop = prop.load_property_files('my_file.properties')
## Read multiple files
## dic_prop = prop.load_property_files('file1', 'file2')
print(dic_prop)
# Output
# {'foo': 'I am awesome', 'bar': 'fudge-bar', 'chocolate': 'fudge',
# 'long': 'a very long property that is described in the property file which takes up multiple lines
# can be defined by the escape character as it is done here', 'url': 'example.com/api?auth_token=xyz',
# 'user_dir': '/home/user/test',
# 'unresolved': '/home/user/files/${id}/fudge-bar/',
# 'fname_template': '/opt/myapp/{arch}/ext/{objid}.dat'}
I did this using ConfigParser as follows. The code assumes that there is a file called config.prop in the same directory where BaseTest is placed:
config.prop
[CredentialSection]
app.name=MyAppName
BaseTest.py:
import unittest
import ConfigParser
class BaseTest(unittest.TestCase):
def setUp(self):
__SECTION = 'CredentialSection'
config = ConfigParser.ConfigParser()
config.readfp(open('config.prop'))
self.__app_name = config.get(__SECTION, 'app.name')
def test1(self):
print self.__app_name % This should print: MyAppName
This is what i had written to parse file and set it as env variables which skips comments and non key value lines added switches to specify
hg:d
-h or --help print usage summary
-c Specify char that identifies comment
-s Separator between key and value in prop file
and specify properties file that needs to be parsed eg : python
EnvParamSet.py -c # -s = env.properties
import pipes
import sys , getopt
import os.path
class Parsing :
def __init__(self , seprator , commentChar , propFile):
self.seprator = seprator
self.commentChar = commentChar
self.propFile = propFile
def parseProp(self):
prop = open(self.propFile,'rU')
for line in prop :
if line.startswith(self.commentChar)==False and line.find(self.seprator) != -1 :
keyValue = line.split(self.seprator)
key = keyValue[0].strip()
value = keyValue[1].strip()
print("export %s=%s" % (str (key),pipes.quote(str(value))))
class EnvParamSet:
def main (argv):
seprator = '='
comment = '#'
if len(argv) is 0:
print "Please Specify properties file to be parsed "
sys.exit()
propFile=argv[-1]
try :
opts, args = getopt.getopt(argv, "hs:c:f:", ["help", "seprator=","comment=", "file="])
except getopt.GetoptError,e:
print str(e)
print " possible arguments -s <key value sperator > -c < comment char > <file> \n Try -h or --help "
sys.exit(2)
if os.path.isfile(args[0])==False:
print "File doesnt exist "
sys.exit()
for opt , arg in opts :
if opt in ("-h" , "--help"):
print " hg:d \n -h or --help print usage summary \n -c Specify char that idetifes comment \n -s Sperator between key and value in prop file \n specify file "
sys.exit()
elif opt in ("-s" , "--seprator"):
seprator = arg
elif opt in ("-c" , "--comment"):
comment = arg
p = Parsing( seprator, comment , propFile)
p.parseProp()
if __name__ == "__main__":
main(sys.argv[1:])
Lightbend has released the Typesafe Config library, which parses properties files and also some JSON-based extensions. Lightbend's library is only for the JVM, but it seems to be widely adopted and there are now ports in many languages, including Python: https://github.com/chimpler/pyhocon
You can use the following function, which is the modified code of #mvallebr. It respects the properties file comments, ignores empty new lines, and allows retrieving a single key value.
def getProperties(propertiesFile ="/home/memin/.config/customMemin/conf.properties", key=''):
"""
Reads a .properties file and returns the key value pairs as dictionary.
if key value is specified, then it will return its value alone.
"""
with open(propertiesFile) as f:
l = [line.strip().split("=") for line in f.readlines() if not line.startswith('#') and line.strip()]
d = {key.strip(): value.strip() for key, value in l}
if key:
return d[key]
else:
return d
this works for me.
from pyjavaproperties import Properties
p = Properties()
p.load(open('test.properties'))
p.list()
print p
print p.items()
print p['name3']
I followed configparser approach and it worked quite well for me. Created one PropertyReader file and used config parser there to ready property to corresponding to each section.
**Used Python 2.7
Content of PropertyReader.py file:
#!/usr/bin/python
import ConfigParser
class PropertyReader:
def readProperty(self, strSection, strKey):
config = ConfigParser.RawConfigParser()
config.read('ConfigFile.properties')
strValue = config.get(strSection,strKey);
print "Value captured for "+strKey+" :"+strValue
return strValue
Content of read schema file:
from PropertyReader import *
class ReadSchema:
print PropertyReader().readProperty('source1_section','source_name1')
print PropertyReader().readProperty('source2_section','sn2_sc1_tb')
Content of .properties file:
[source1_section]
source_name1:module1
sn1_schema:schema1,schema2,schema3
sn1_sc1_tb:employee,department,location
sn1_sc2_tb:student,college,country
[source2_section]
source_name1:module2
sn2_schema:schema4,schema5,schema6
sn2_sc1_tb:employee,department,location
sn2_sc2_tb:student,college,country
You can try the python-dotenv library. This library reads key-value pairs from a .env (so not exactly a .properties file though) file and can set them as environment variables.
Here's a sample usage from the official documentation:
from dotenv import load_dotenv
load_dotenv() # take environment variables from .env.
# Code of your application, which uses environment variables (e.g. from `os.environ` or
# `os.getenv`) as if they came from the actual environment.

Categories

Resources