Develop Reference

Python - Parse INI text (not INI file)

I have a case where I get a string version of an INI file and not the file itself. I am getting it this way from an external source and hence cannot be changed.
Example of the string version:
'[DEFAULT]\nScore = 0.1\n\n[dev]\nHost = abc.com\nPort = 0000\n\n[qa]\nHost = xyz.com\nEsPort = 1000\n\n[main]\nHost = pqr.com\nPort = 2000\n'
I tried parsing this string using the configParser library in python:
>>> config = configparser.ConfigParser()
>>> config.read(my_ini_str)
[]
I get back nothing with the read function. Is there any other way to achieve this?
Thanks in advance!
UPDATE - Forgot to add that I am using Python 2.

This is what the read_string() method is for.
Parse configuration data from a string.
Hence:
>>> config = configparser.ConfigParser()
>>> config.read_string(my_ini_str)

you have to use read_string in order to parse a string.
then you have to use getters on "config" object, such as :
>>> config.get('dev', 'Host')
'abc.com'
or
>>> config.getfloat(config.default_section, 'Score')
0.1

read from text file and send in to aws sqs Fifo queue

I have a little issue here. I want to read from a text file using python and create queue and then send these lines from the text file into Amazon web services SQS(Simple Queue service). First of all, I`ve actually managed to do this using boto, but the problem is that the lines dont come in order, just randomly, like line 4,line 1, line 5 etc etc..
Here is my code:
import boto.sqs
conn = boto.sqs.connect_to_region("us-east-2",
aws_access_key_id='AKIAIJIQZG5TR3NMW3LQ',
aws_secret_access_key='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')
q = conn.create_queue('test')
with open('read.txt', 'r') as read_file:
from boto.sqs.message import RawMessage
for line in read_file:
m = RawMessage()
m.set_body(line)
q.write(m)
So, what to do? Well, we need to create an FIFO queue(which I also managed to do using boto3 in python), but now the problem is that I have problems reading the text file.. Here is the code i used to create a FIFO queue in SQS:
import boto3
AWS_ACCESS_KEY = 'AKIAIJIQZG5TR3NMW3LQ'
AWS_SECRET_ACCESS_KEY = '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'
sqs_client = boto3.resource(
'sqs',
aws_access_key_id=AWS_ACCESS_KEY,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
region_name='us-east-2'
)
queue_name = ('demo_queue.fifo')
response = sqs_client.create_queue(
QueueName=queue_name,
Attributes={
'FifoQueue': 'true',
'ContentBasedDeduplication': 'true'
}
)
with open('read.txt', 'r') as read_file:
from boto.sqs.message import RawMessage
for line in read_file:
m = RawMessage()
m.set_body(line)
queue_name.write(m)
Somebody know how to solve this? thanks.

Try replacing
queue_name.write(m)
with
response.write(m)
in your first piece of code. You should use the actual queue returned by get_queue_by_name
Also, when only specifying MessageBody and MessageGroupID in boto3, make sure that Content-Based deduplication is enabled for the queue, or specify a MessageDeduplicationId string, otherwise it will fail

Python Json Config 'Extended Interpolation'

I am currently using the Python library configparser:
from configparser import ConfigParser, ExtendedInterpolation
I find the ExtendedInterpolation very useful because it avoids the risk of having to reenter constants in multiple places.
I now have a requirement to use a Json document as the basis of the configuration as it provides more structure.
import json
from collections import OrderedDict
def get_json_config(file):
"""Load Json into OrderedDict from file"""
with open(file) as json_data:
d = json.load(json_data, object_pairs_hook=OrderedDict)
return d
Does anyone have any suggestions as to the best way to implement configparser style ExtendedInterpolation?
For example if a node in the Json contains the value ${home_dir}/lumberjack this would copy root node home_dir and take value 'lumberjack'?

Try to use string.Template. But I'm not sure whether it's your need. There is one package can do this may be. Bellow is what i should do.
config.json
{
"home_dir": "/home/joey",
"class_path": "/user/local/bin",
"dir_one": "${home_dir}/dir_one",
"dir_two": "${home_dir}/dir_two",
"sep_path_list": [
"${class_path}/python",
"${class_path}/ruby",
"${class_path}/php"
]
}
python code:
import json
from string import Template
with open("config.json", "r") as config_file:
config_content = config_file.read()
config_template = Template(config_content)
mid_json = json.loads(config_content)
config = config_template.safe_substitute(mid_json)
print config
This can substitute the defined key in json file.

Came in very useful; however, I found "config" is a unicode string; I resolved with:
# proper code
return json.loads(config)

Python- how can I set the value for params from another file

I am new to Python. I have to edit some test I have.
I have a file which inclused all the details for a test I have to execute. The file name for example is "test 1"
In this file there are few global params:
LINK_IP_ADDR = "10.8.22.89"
STABILITY_SLEEP = 5
MAIN_STATE_SLEEP = 30
For making the tests automaticly and generic I want to set the the value from the params from another python file (for example config_file). How should I do it?

You should probably use configuration files for this. The ConfigParser package would do the job.
If you want a quick and dirty version, the following could be for you.
In your 'test files', write the following:
class Params:
LINK_IP_ADDR = "10.8.22.89"
STABILITY_SLEEP = 5
MAIN_STATE_SLEEP = 30
Then in your main program you import the parameters you want and just use them:
from test1 import Params
# from test2 import Params
print Params.LINK_IP_ADDR
You can then switch your import statements to the file you want to use. I would use a normal .ini file with different sections though.

Here is a quick n' dirty way from a plain file:
params = dict()
with open('config_file') as f:
for line in f:
param, value = line.split("=")
params[param.strip()] = value.strip()
print(params)
config_file:
LINK_IP_ADDR = "10.8.22.89"
STABILITY_SLEEP = 5
MAIN_STATE_SLEEP = 30

for config file i used .yaml file. This format stores your config like json or python-dict. U can store complex structure (array, hashmap, ...) very good
config.yaml
LINK_IP_ADDR: 10.8.22.89
STABILITY_SLEEP: 5
MAIN_STATE_SLEEP: 30
another.py
import yaml
configs = yaml.safe_load(file("path_to_config.yaml"))
# to use you configs:
configs['LINK_IP_ADDR']
#.....

Django Imagekit - how to get the url not in the template?

I have been using django imagekit without any problems in the templates but now I need to get the url in the view not the template, following this example in the imagekit docs:
source_file = open('/path/to/myimage.jpg')
image_generator = Thumbnail(source=source_file)
result = image_generator.generate()
So I did this:
course_img = open(settings.MEDIA_ROOT+str(course.image), 'rb+')
image_generator = myapp_images.Thumbnail100x100(source=course_img)
result = image_generator.generate()
And then, I try to get the url from the "result" variable, but I don't know how:
details_html += "<img class='img-rounded' src='"+ str(result) +"'/>
I have been trying with str(result), result.url, result.name, etc... with no luck, any idea how to get it?
Thx

I got the solution from the imagekit author in the google imagekit group:
In my case I wanted to interact with the Cache File, in order to do that we have to wrap my generator with an ImageCacheFile, like this:
from imagekit.cachefiles import ImageCacheFile
image_generator = myapp_images.Thumbnail00x100(source=course.image)
result = ImageCacheFile(image_generator)
so the result object it is what I was expecting, now I can read the url just using result.url.

Generator itself doesn't allow you to do that on the fly, it generates file-like object, which means it doesn't save the file on disk, which means no filename and so, no URL name..
Either you save the object and generate url manually or you can use ImageCacheFile like this:
from imagekit.cachefiles import ImageCacheFile
file = ImageCacheFile(image_generator, name='my_image.thumb.jpg')
file.generate()
print file.url