I have a bunch of JSON objects that I need to update in order to use the CLI for AWS.
Here is an example of the JSON format. I will need to update lbtest, lbtest-cookie-pol and 80 with different values.
{
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod":80
}
In some cases, there will be multiple values here for each Load Balancer Name.
The output will need to look like this:
{
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod":80
}
{
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol2",
"CookieExpirationPeriod":8080.
}
Suppose I had a CSV file with all these entries, what kind of python script can I write to loop through these and print out my JSON output? The part where I am having issues is the printing of the nested JSON object. print doesn't seem to like multiple lines or the curly braces that I have. Newbie here so I would appreciate any kind of solution.
you can use json.dumps method and it's options mentioned in documentations:
for example with using indent option you get this on python 2.7 :
>>> dictionary = {
"LoadBalancerName": "lbtest",
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod":80 } #a dictionary object made from csv
>>> print dictionary
{'PolicyName': 'lbtest-cookie-pol', 'CookieExpirationPeriod': 80, 'LoadBalancerName': 'lbtest'}
>>> import json
>>> jobj = json.dumps(dictionary,indent=4, separators=(',', ': '))
>>> print jobj
{
"PolicyName": "lbtest-cookie-pol",
"CookieExpirationPeriod": 80,
"LoadBalancerName": "lbtest"
}
>>> f = open(r'jtest.txt','w') #save our json object to file
>>> json.dump(dictionary,fp,indent =4 , seperators = (',',': '))
>>> f.close()
>>> f = open(r'jtest.txt!','r') #read our object from file
>>> test = json.load(f)
>>> test
{u'PolicyName': u'lbtest-cookie-pol', u'CookieExpirationPeriod': 80, u'LoadBalancerName': u'lbtest'}
>>> dict(test)
{u'PolicyName': u'lbtest-cookie-pol', u'CookieExpirationPeriod': 80, u'LoadBalancerName': u'lbtest'}
here is how our file looks like:
jtest.txt file
Related
I've been trying to get a more readable output from a JSON list. I have not yet been successful. I hard-coded some data to see if I can get it as I want. This is what I did:
import json
jsonData = {
"person": {"FirstName": "Kwin", "LastName": "Harley", "Age": 25},
"DoB": {"DateOfBirth": "19/12/1996", "Birthplace": "Belgium"},
"insurer":{"id":"12345","contractNumber":"98765432",
"valid_from":"2020-10-01T00:00:00.000Z","valid_until":"2021-01-30T00:00:00.000Z",
"representativeID":"135792468",
"representativeEmail":"sample#test.com"}
}
jsonString = json.dumps(jsonData, sort_keys=False, indent=4)
print(jsonString)
Output 1
As you can see, the data is structured nicely.
Now, when I use my main code, the output looks like this:
Output 2
It just returns the data in 1 row :(
Is there a way to fix that? This is the code I have for that:
qrType = qr.type
qrData = json.dumps(qr.data.decode('utf-8'),sort_keys=True)
# print the QR type and data to the terminal
print("[INFORMATION] Found {} barcode:\n{}".format(qrType, qrData))
I don't think you're passing a dict to json.dumps() at all. qr.data is clearly a string, as you .decode() it. Presumably it's a json string, so you want to do something like this:
formatted_data = json.dumps(json.load(qr.data.decode()), indent=2)
print(formatted_data)
Problem statement:
This is how I am invoking my prepare_payload.py ,
python3 prepare_payload.py ['Test_B1','Test_B2','Test_B3'] [https://10.5.5.1,https://10.5.5.2,https://10.5.5.3] ['abc','efg','sss']
my json payload which I am trying to prepare:
{
"billing_account_number": "",
"vcenter_url": "",
"cred_header": "",
}
Expected output:
{
"billing_account_number": "Test_B1",
"vcenter_url": "https://10.5.5.1",
"cred_header": "abc",
}
{
"billing_account_number": "Test_B2",
"vcenter_url": "https://10.5.5.2",
"cred_header": "efg",
}
{
"billing_account_number": "Test_B3",
"vcenter_url": "https://10.5.5.3",
"cred_header": "sss",
}
my code:
import json
import os
import sys
master_list = []
billing_account_number = sys.argv[1]
ip_addr = sys.argv[2]
cred_header = sys.argv[3]
res = list(map(str, billing_account_number.strip('[]').split(',')))
ip = list(map(str, ip_addr.strip('[]').split(',')))
cred_headers = list(map(str, cred_header.strip('[]').split(',')))
master_list.append(res)
master_list.append(ip)
master_list.append(cred_headers)
def prepare_payload():
with open("rabbitmq_payload.json") as fh:
data = json.load(fh)
print('================================================')
return data
data = prepare_payload()
for i in master_list:
for j in i:
data['billing_account_number'] = j
data['vcenter_url'] = j
data['cred_header'] = j
print(data)
I am not able to figure if I have to merge these individual list such as res, IP, cred_headers into a single list and then iterate like main list [res[0],IP[0],cred_headers[0]] and then start replacing key value pair in my data dictionary?
Please help me if there is any built in function I can rely on or any efficient approach to solve this problem. Thank you in advance for all the awesome python coders!
It's kind of inconvenient to pass in lists as command line arguments. Better to use standard input, but nonetheless, take a look at this code:
import sys
import ast
billing_account_number = ast.literal_eval(sys.argv[1])
ip_addr = ast.literal_eval(sys.argv[2])
cred_header = ast.literal_eval(sys.argv[3])
output = []
for i in range(len(billing_account_number)):
output.append({"billing_account_number": billing_account_number[i], "ip_addr": ip_addr[i], "cred_header": cred_header[i]})
print(output)
The output:
[{'billing_account_number': 'Test_B1', 'ip_addr': 'https://10.5.5.1', 'cred_header': 'abc'}, {'billing_account_number': 'Test_B2', 'ip_addr': 'https://10.5.5.2', 'cred_header': 'efg'}, {'billing_account_number': 'Test_B3', 'ip_addr': 'https://10.5.5.3', 'cred_header': 'sss'}]
You will need to wrap your command line arguments in quotes so that it parses correctly. Also, this code assumes that all 3 lists are the same length. If that's not guaranteed, then you'll need to iterate over the longest list and set the missing values to empty string.
(It seems like you're asking different questions. How to take argv better is different from how to merge lists. This is for the question in your text. It's also hard to tell what you're trying to do with your file.)
You have content in 3 lists that you want to both merge, and convert to dictionaries.
The builtin way to merge lists is zip()
zip(res, ips, cred_headers)
There are multiple ways to convert this to dictionaries. One is a list comprehension:
result = [{
"billing_account_number": res_item,
"ip_addr": ips_item,
"cred_header": cred_item,
} for res_item, ips_item, cred_item in zip(res, ips, cred_headers)]
Basically I want to read a string from a text file and store it as orderedDict.
My file contains the following content.
content.txt:
variable_one=OrderedDict([('xxx', [['xxx_a', 'xxx_b'],['xx_c', 'xx_d']]),('yyy', [['yyy_a', 'yyy_b'],['yy_c', 'yy_d']]))])
variable_two=OrderedDict([('xxx', [['xxx_a', 'xxx_b'],['xx_c', 'xx_d']]),('yyy', [['yyy_a', 'yyy_b'],['yy_c', 'yy_d']]))])
how will I retrieve values in python as:
xxx
xxx_a -> xxx_b
xxx_c -> xxx_d
import re
from ast import literal_eval
from collections import OrderedDict
# This string is slightly different from your sample which had an extra bracket
line = "variable_one=OrderedDict([('xxx', [['xxx_a', 'xxx_b'],['xx_c', 'xx_d']]),('yyy', [['yyy_a', 'yyy_b'],['yy_c', 'yy_d']])])"
match = re.match(r'(\w+)\s*=\s*OrderedDict\((.+)\)\s*$', line)
variable, data = match.groups()
# This allows safe evaluation: data can only be a basic data structure
data = literal_eval(data)
data = [(key, OrderedDict(val)) for key, val in data]
data = OrderedDict(data)
Verification that it works:
print variable
import json
print json.dumps(data, indent=4)
Output:
variable_one
{
"xxx": {
"xxx_a": "xxx_b",
"xx_c": "xx_d"
},
"yyy": {
"yyy_a": "yyy_b",
"yy_c": "yy_d"
}
}
Having said all that, your request is very odd. If you can control the source of the data, use a real serialisation format that supports order (so not JSON). Don't output Python code.
I am writing a test in Python where i am specifying the JSON sting in a parameter as follows :
json = '...[{"MemberOperand":{
"AttributeName":"TEST",
"Comparison":"=",
"Value":"Test"}
}]...'
In this example i have the value as "Test" however i want to run the test with several values. Could you guys tell me how can i parameterize The values of "Value"?
You can construct proper JSON:
import json
the_value = 'Test'
data = [{"MemberOperand": {
"AttributeName":"TEST",
"Comparison":"=",
"Value": the_value}
}]
json_text = json.dumps(data)
This is regular dictionary (nested) formatted as string -
def changer(x):
import json
d=json.loads(json.loads(json.dumps('[{"MemberOperand":{"AttributeName":"TEST","Comparison":"=","Value":"Test"}}]')))
d[0]['MemberOperand']['AttributeName']=x
return d
print changer('New_TEST')
Output-
[{'MemberOperand': {'Comparison': '=', 'AttributeName': 'New_TEST', 'Value': 'Test'}}]
Add function which return you different json string all the time by provided value as parameter:
def get_mock_json(value='Test'):
return '...[{"MemberOperand":{"AttributeName":"TEST","Comparison":"=","Value":%s}}]...'%value
print get_mock_json('test')
print get_mock_json('ttttttest')
I am trying to write a json file. The json file should look something like this -
{
"roster":[
{"name":"Andy","age":11},
{"name":"Nathan","age":10},
{"name":"Amy","age":12}
],
"links":[
{"source":1,"target":0,"value":1},
{"source":2,"target":0,"value":8},
{"source":3,"target":0,"value":10}
]
}
I am trying to create the roster part of the json by running through a for loop. In each iteration I am trying to append a line to the json file as follows -
wf = open("abc.json", "w")
wf.write('{\n"Roster":[\n')
for example in data:
name = ----some code here ----
group = ----some code here ----
wf.write('{"name":"'+name+'","group":'+group+'},\n')
I am getting a typeError - str and int objects cannot be concatenated. I understand why I am getting that error. I was just wondering if there is a better way to do it.
Python is Strongly Typed which means:
s = "foo"
s += 123
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: cannot concatenate 'str' and 'int' objects
If you are adding strings (which btw are immutable) you need to "cast" the int to a str:
s = "foo"
s += str(123)
You are getting the error:
typeError - str and int objects cannot be concatenated
Because strings and integers cannot be concatenated.
Use str() to cast the integer to a string for concatenation instead.
wf.write('{"name":"'+name+'","group":'+str(group)+'},\n')
Or even better read up on string formatting in Python.
Or even more betterer read up on the Python JSON module.
Checking your string. You have to ensure string concatenation.
Try:
'{"name":"'+str(name)+'","group":'+str(group)+'},\n'
Or use json module to support you to output json file easily.
import json
output = {}
output['roster'] = [
{'name':'Andy', 'age':11},
{'name': 'Nathan', 'age': 10}
]
print(json.dumps(output))
Output:
{"roster": [{"name": "Andy", "age": 11}, {"name": "Nathan", "age": 10}]}
If you really want to write the json by hand, then the way to get around the concatenation error is to simply typecast the number to a string:
wf.write('{"name" : "'+str(name)+'","group":'+str(group)'},\n')
It would definitely be advisable to use the json module in python, it's a pretty great library.
For example, to get the json example you gave us:
{
"roster":[
{"name":"Andy","age":11},
{"name":"Nathan","age":10},
{"name":"Amy","age":12}
],
"links":[
{"source":1,"target":0,"value":1},
{"source":2,"target":0,"value":8},
{"source":3,"target":0,"value":10}
]
}
You could use the following:
import json
result = {}
result['roster'] = []
result['roster'].append({'name' : 'Andy', 'age' : 11})
result['roster'].append({'name' : 'Nathan', 'age' : 10})
result['roster'].append({'name' : 'Amy', 'age' : 12})
result['links'] = []
result['links'].append({'source' : 1, 'target' : 0, 'value' : 1})
...
(Obviously you would probably do this with a for loop instead of by hand in your program).
And then to convert this python dictionary to a json object you just use:
json.dumps(result)