Develop Reference

How can i create multiple json payload efficiently by reading input from sys.argv in python

Problem statement:
This is how I am invoking my prepare_payload.py ,
python3 prepare_payload.py ['Test_B1','Test_B2','Test_B3'] [https://10.5.5.1,https://10.5.5.2,https://10.5.5.3] ['abc','efg','sss']
my json payload which I am trying to prepare:
{
"billing_account_number": "",
"vcenter_url": "",
"cred_header": "",
}
Expected output:
{
"billing_account_number": "Test_B1",
"vcenter_url": "https://10.5.5.1",
"cred_header": "abc",
}
{
"billing_account_number": "Test_B2",
"vcenter_url": "https://10.5.5.2",
"cred_header": "efg",
}
{
"billing_account_number": "Test_B3",
"vcenter_url": "https://10.5.5.3",
"cred_header": "sss",
}
my code:
import json
import os
import sys
master_list = []
billing_account_number = sys.argv[1]
ip_addr = sys.argv[2]
cred_header = sys.argv[3]
res = list(map(str, billing_account_number.strip('[]').split(',')))
ip = list(map(str, ip_addr.strip('[]').split(',')))
cred_headers = list(map(str, cred_header.strip('[]').split(',')))
master_list.append(res)
master_list.append(ip)
master_list.append(cred_headers)
def prepare_payload():
with open("rabbitmq_payload.json") as fh:
data = json.load(fh)
print('================================================')
return data
data = prepare_payload()
for i in master_list:
for j in i:
data['billing_account_number'] = j
data['vcenter_url'] = j
data['cred_header'] = j
print(data)
I am not able to figure if I have to merge these individual list such as res, IP, cred_headers into a single list and then iterate like main list [res[0],IP[0],cred_headers[0]] and then start replacing key value pair in my data dictionary?
Please help me if there is any built in function I can rely on or any efficient approach to solve this problem. Thank you in advance for all the awesome python coders!

It's kind of inconvenient to pass in lists as command line arguments. Better to use standard input, but nonetheless, take a look at this code:
import sys
import ast
billing_account_number = ast.literal_eval(sys.argv[1])
ip_addr = ast.literal_eval(sys.argv[2])
cred_header = ast.literal_eval(sys.argv[3])
output = []
for i in range(len(billing_account_number)):
output.append({"billing_account_number": billing_account_number[i], "ip_addr": ip_addr[i], "cred_header": cred_header[i]})
print(output)
The output:
[{'billing_account_number': 'Test_B1', 'ip_addr': 'https://10.5.5.1', 'cred_header': 'abc'}, {'billing_account_number': 'Test_B2', 'ip_addr': 'https://10.5.5.2', 'cred_header': 'efg'}, {'billing_account_number': 'Test_B3', 'ip_addr': 'https://10.5.5.3', 'cred_header': 'sss'}]
You will need to wrap your command line arguments in quotes so that it parses correctly. Also, this code assumes that all 3 lists are the same length. If that's not guaranteed, then you'll need to iterate over the longest list and set the missing values to empty string.

(It seems like you're asking different questions. How to take argv better is different from how to merge lists. This is for the question in your text. It's also hard to tell what you're trying to do with your file.)
You have content in 3 lists that you want to both merge, and convert to dictionaries.
The builtin way to merge lists is zip()
zip(res, ips, cred_headers)
There are multiple ways to convert this to dictionaries. One is a list comprehension:
result = [{
"billing_account_number": res_item,
"ip_addr": ips_item,
"cred_header": cred_item,
} for res_item, ips_item, cred_item in zip(res, ips, cred_headers)]

Python list formatting with JSON

I'm a newbie in Python trying to turn information from an Excel file into JSON output.
I'm trying to parse this Python list:
value = ['Position: Backstab, Gouge,', 'SumPosition: DoubleParse, Pineapple']
into this JSON format:
"value": [
{
"Position": [
"Backstab, Gouge,"
]
},
{
"SumPosition": [
"DoubleParse, Pineapple"
]
}
]
Please note:
This list was previously a string:
value = 'Position: Backstab, Gouge, SumPosition: DoubleParse, Pineapple'
Which I turned into a list by using re.split().
I've already turned the string into a list by using re.split, but I still can't turn the inside of the string into a dict, and the value from the dict into a list.
Is that even possible? Is it the case to format the list/string with JSON or previously prepare the string itself so it can receive the json.dump method?
Thanks in advance!

You can iterate over the list to achieve desired result.
d = {'value': []}
for val in value:
k, v = val.split(':')
tmp = {k.strip() : [v.strip()]}
d['value'].append(tmp)
print(d)
{'value': [{'Position': ['Backstab, Gouge,']},
{'SumPosition': ['DoubleParse, Pineapple']}]}

Here is a quick way.
value = ['Position: Backstab, Gouge,',
'SumPosition: DoubleParse, Pineapple']
dictionary_result = {}
for line in value:
key, vals = line.split(':')
vals = vals.split(',')
dictionary_result[key] = vals
Remaining tasks for you: trim off empty strings from result lists like [' Backstab', ' Gouge', ''], and actually convert the data from a Python dict to a JSON file

Getting size of JSON array and read it's elements Python

I have an json array that looks like this:
{
"inventory": [
{
"Name": "katt"
},
{
"Name": "dog"
}
]
}
And now I want to access this array in a program that I'm creating and remove a element, for example "Name": "dog".
I'm not super familiar with how to work with json in python, but so far I have tried something like this:
import json
jsonfile = open("viktor.json", "r")
jsonObj = json.load(jsonfile)
jsonfile.close()
counter = 0
for item in range(len(jsonObj["inventory"])):
print(jsonObj["inventory"][counter])
print(type(jsonObj["inventory"][counter]))
if jsonObj["inventory"][counter] == argOne:
print("hej")
counter += 1
So first I read from the json and stores the data in a variable.
Then I want to loop through the whole variable and see if I can find any match, and if so, I want to remove it. I think I can use a pop() method here or something?
But I can't seem to get my if-statement to work properly since the jsonObj["inventory"][counter] is a dict and argOne is a string.
What can I do instead of this? Or what I'm missing?

Making the change suggested by #arvindpdmn (to be more pythonic).
for index, item in enumerate(jsonObj["inventory"]):
print(item)
print(type(item)) # Here we have item is a dict object
if item['Name'] == argOne: # So we can access their elements using item['key'] syntax
print(index, "Match found")
The for loop is responsible to go through the array, which contains dict objects, and for each dict it will create a item variable that we use to try to get a match.
edit
In order to remove the element if it is in the list, I suggest you to use this:
new_list = []
for item in jsonObj["inventory"]:
if item['Name'] is not argOne: # add the item if it does not match
new_list.append(item)
This way you will end with the list you want (new_list).
# Or shorter.. and more pythonic with comprehensions lists.
new_list = [item for item in jsonObj['inventory'] if item['Name'] is not argOne]

You can use filter:
In [11]: import json
In [12]: with open("viktor.json", "r") as f:
...: jsonObj = json.load(f)
...:
In [13]: argOne = 'katt' #Let's say
In [14]: jsonObj['inventory'] = list(filter(lambda x: x['Name'] != argOne, jsonObj['inventory']))
In [15]: jsonObj
Out[15]: {'inventory': [{'Name': 'dog'}]}

Write a list objects in JSON using Python

I am trying to output the following JSON from my python (2.7) script:
[
{
"id": "1002-00001",
"name": "Name 1"
},
{
"id": "1002-00002",
"display": "Name 2"
},
]
What data structure in Python will output this when using json.dumps?
The outermost item is a python list, but what should be the type of items inside the list? It looks like a dictionary with no keys?

Hopefully this clarifies the notes in comments that are not clear for you. It's achieved by appending (in this case small) dictionaries into a list.
import json
#Added an extra entry with an integer type. Doesn't have to be string.
full_list = [['1002-00001', 'Name 1'],
['1002-00002', 'Name 2'],
['1002-00003', 2]]
output_list = []
for item in full_list:
sub_dict = {}
sub_dict['id'] = item[0] # key-value pair defined
sub_dict['name'] = item[1]
output_list.append(sub_dict) # Just put the mini dictionary into a list
# See Python data structure
print output_list
# Specifically using json.dumps as requested in question.
# Automatically adds double quotes to strings for json formatting in printed
# output but keeps ints (unquoted)
json_object = json.dumps(output_list)
print json_object
# Writing to a file
with open('SO_jsonout.json', 'w') as outfile:
json.dump(output_list, outfile)
# What I think you are confused about with the "keys" is achieved with an
# outer dictionary (but isn't necessary to make a valid data structure, just
# one that you might be more used to seeing)
outer_dict = {}
outer_dict['so_called_missing_key'] = output_list
print outer_dict

how to delete json object using python?

I am using python to delete and update a JSON file generated from the data provided by user, so that only few items should be stored in the database. I want to delete a particular object from the JSON file.
My JSON file is:
[
{
"ename": "mark",
"url": "Lennon.com"
},
{
"ename": "egg",
"url": "Lennon.com"
}
]
I want to delete the JSON object with ename mark.
As I am new to python I tried to delete it by converting objects into dict but it is not working. Is there any other way to do it?
i tried this one:
index=0
while index < len(data):
next=index+1
if(data[index]['ename']==data[next]['ename']):
print "match found at"
print "line %d and %d" %(next,next+1)
del data[next]
index +=1

Here's a complete example that loads the JSON file, removes the target object, and then outputs the updated JSON object to file.
#!/usr/bin/python
# Load the JSON module and use it to load your JSON file.
# I'm assuming that the JSON file contains a list of objects.
import json
obj = json.load(open("file.json"))
# Iterate through the objects in the JSON and pop (remove)
# the obj once we find it.
for i in xrange(len(obj)):
if obj[i]["ename"] == "mark":
obj.pop(i)
break
# Output the updated file with pretty JSON
open("updated-file.json", "w").write(
json.dumps(obj, sort_keys=True, indent=4, separators=(',', ': '))
)
The main point is that we find the object by iterating through the objects in the loaded list, and then pop the object off the list once we find it. If you need to remove more than one object in the list, then you should store the indices of the objects you want to remove, and then remove them all at once after you've reached the end of the for loop (you don't want to modify the list while you iterate through it).

The proper way to json is to deserialize it, modify the created objects, and then, if needed, serialize them back to json.
To do so, use the json module. In short, use <deserialized object> = json.loads(<some json string>) for reading json and <json output> = json.dumps(<your object>) to create json strings.
In your example this would be:
import json
o = json.loads("""[
{
"ename": "mark",
"url": "Lennon.com"
},
{
"ename": "egg",
"url": "Lennon.com"
}
]""")
# kick out the unwanted item from the list
o = filter(lambda x: x['ename']!="mark", o)
output_string = json.dumps(o)

Your json file contains in a list of objects, which are dictionaries in Python. Just replace the list with a new one that doesn't have the object in it:
import json
with open('testdata.json', 'rb') as fp:
jsondata = json.load(fp)
jsondata = [obj for obj in jsondata if obj['ename'] != 'mark']
print(json.dumps(jsondata, indent=4))

You need to use the json module. I'm assuming python2. Try this:
import json
json_data = json.loads('<json_string>')
for i in xrange(len(json_data)):
if(json_data[i]["id"] == "mark"):
del json_data[i]
break

You have a list there with two items, which happen to be dictionaries. To remove the first, you can use list.remove(item) or list.pop(0) or del list[0].
http://docs.python.org/2/tutorial/datastructures.html#more-on-lists