Develop Reference

Python - get a LIST value from field in JSON

I have a JSON file containing three fields: 2 are strings and third one is field containing a list of values.
{ "STREAM": "stream",
"BASIS_STREAM": "basis",
"PATHS": "[/opt/path1,/opt/path2]"
}
Now I load that JSON
with open('/pathToJsonFile.json', 'r') as f:
data = json.load(f)
Now I want to get those values.
stream=str(data["STREAM"])
basis=str(data["BASIS_STREAM"])
paths=data["BASE_PATHS"]
The issue is that paths is also threated as String, although I have to use it as a list. I am converting with str function other fields because of the Unicode. Code must be in python2.
Thanks a lot!

Say you have a file called data.json with the following contents:
{
"STREAM": "stream",
"BASIS_STREAM": "basis",
"PATHS": "[/opt/path1,/opt/path2]"
}
Maybe you could use str.split after calling json.load:
with open('data.json', 'r') as f:
data = json.load(f)
print 'data = %s' % data
stream = str(data['STREAM'])
basis = str(data['BASIS_STREAM'])
paths = [str(u_s) for u_s in data['PATHS'][1:-1].split(',')]
print 'stream = %s' % stream
print 'basis = %s' % basis
print 'paths = %s' % paths
Output:
data = {u'PATHS': u'[/opt/path1,/opt/path2]', u'BASIS_STREAM': u'basis', u'STREAM': u'stream'}
stream = stream
basis = basis
paths = ['/opt/path1', '/opt/path2']

Your /opt/path1 and /opt/path2 should be in a quotation marks to be converted in a list. If your PATHS always have a similar template such as "[/XXX,/YYY,/ZZZ,/TTT,/KKK]" the following code should also help. I have converted your data as "['/XXX','/YYY','/ZZZ','/TTT','/KKK']" so that it can be easily converted to a list using ast library. Please see the code as following:
import json
import ast
with open("text_text.json") as f:
data = json.load(f)
print(data["PATHS"]) # Your data
for i in data["PATHS"]:
if i == "[":
data["PATHS"] = data["PATHS"].replace("[", "['")
elif i == ",":
data["PATHS"] = data["PATHS"].replace(",/", "','/")
elif i == "]":
data["PATHS"] = data["PATHS"].replace("]", "']")
#print(data["PATHS"])
print(type(data["PATHS"]))
print(data["PATHS"]) #converted to a data which can be converted to a list.
data_paths = ast.literal_eval(data["PATHS"]) # ast is used to convert str to list.
print(data_paths) # 'list' data
print(type(data_paths))
See the output of the code:
It should also work if your PATH has more data as following:

How to Change dictionary values in python file from another file

I would like to change values in a Dict in another file. File1.py contains the code to edit the Dict, File2.py contains the Dict itself.
File1.py is generating a code to replace BTOK values only.
File1.py:
with open('file2.py', 'r') as file :
filedata = file.read()
print (filedata.str(BTK['btk1']))
for line in filedata:
line['btk1'] = BTok
with open('file2.py', 'w') as file:
file.write(line)
File2.py:
c = {
'id' : 'C80e3ce43c3ea3e8d1511ec',
'secret' : 'c10c371b4641010a750073925b0857'
}
rk = {
't1' : 'ZTkwMGE1MGEt',
}
BTK = {
'BTok' : '11eyJhbGc'
}

If you want to do this reliably, that is, so it works whether your strings are quoted with ', " or """, for whatever values they have and whatever newlines you want to put around values, then you may want to use ast to parse the source code and modify it. The only inconvenient with this is that module cannot, by itself, generate code, so you would need to install some additional dependency such as astor, for what is essentially a rather menial task. In any case, here is how you could do it that way:
import ast
import astor
# To read from file:
# with open('file2.py', 'r') as f: code = f.read()
code = """
c = {
'id' : 'C80e3ce43c3ea3e8d1511ec',
'secret' : 'c10c371b4641010a750073925b0857'
}
rk = {
't1' : 'ZTkwMGE1MGEt',
}
BTK = {
'BTok' : '11eyJhbGc'
}
"""
# Value to replace
KEY = 'BTok'
NEW_VALUE = 'new_btok'
# Parse code
m = ast.parse(code)
# Go through module statements
for stmt in m.body:
# Only look at assignments
if not isinstance(stmt, ast.Assign): continue
# Take right-hand side of the assignment
value = stmt.value
# Only look at dict values
if not isinstance(value, ast.Dict): continue
# Look for keys that match what we are looking for
replace_idx = [i for i, k in enumerate(value.keys)
if isinstance(k, ast.Str) and k.s == KEY]
# Replace corresponding values
for i in replace_idx:
value.values[i] = ast.Str(NEW_VALUE)
new_code = astor.to_source(m)
# To write to file:
# with open(`file2.py', 'w') as f: f.write(new_code)
print(new_code)
# c = {'id': 'C80e3ce43c3ea3e8d1511ec', 'secret':
# 'c10c371b4641010a750073925b0857'}
# rk = {'t1': 'ZTkwMGE1MGEt'}
# BTK = {'BTok': 'new_btok'}

Converting data to JSON in same format as CSV

I have the following code which prints the object as CSV:
title = ['Username', 'Name', 'Job']
for x in title:
print(x, end =",")
for d in data:
line = d.get_username() + "," + d.get_name() + "," + d.get_role()
print(line)
I get:
Username,Name,Job
rob,robert,developer
danny21,danny,developer
I want to print the same data as JSON in order to get:
[
{
"Username":"rob",
"Name":"robert",
"Job":"developer"
},
{
"Username":"danny21",
"Name":"danny",
"Job":"developer"
}
]
From previous topics I learn that we can use json.dumps but I'm not sure if it helps in this case.
What is the proper way to achieve it?

You could simply do:
l = []
for d in data:
user_dictionary = {}
user_dictionary[title[0]] = d.get_username()
user_dictionary[title[1]] = d.get_name()
user_dictionary[title[2]] = d.get_role()
l.append(user_dictionary)
to get a json like file.
You can also avoid appending and do:
def get_user_data(user):
user_dictionary = {}
user_dictionary[title[0]] = d.get_username()
user_dictionary[title[1]] = d.get_name()
user_dictionary[title[2]] = d.get_role()
return user_dictionary
l = list(map(get_user_data, data))
You can use json.dump to dump l in a file
import json
with open('data.json', 'w') as outfile:
json.dump(l, outfile)

Retrieve yum repos and save as list in json format

I am new to the site and have just started with Python. I am trying to think how to begin working on this problem...basically I need Python to retrieve a list of all yum repos in /etc/yum.repos.d and then save the list in a json format such as below:
{
"[repo_name]" : {
"name" : "repo_name",
"baseurl" : "http://example.com",
"enabled" : "1",
"gpgcheck" : "0"
}
"[next_repo]...
}
I managed to get something working, but it doesn't really do what it was intended to do. Here is the code I have:
#!/usr/bin/python
import json
mylist = []
lines = open('/etc/yum.repos.d/repo_name.repo').read().split('\n')
for line in lines:
if line.strip() != '':
if '[' in line:
mylist.append("{")
repo_name = line.translate(None,'[]')
mylist.append(repo_name + ':')
mylist.append("{")
elif 'gpgcheck' in line:
left, right = line.split('=')
mylist.append(left + ':' + right)
mylist.append("}")
else:
left, right = line.split('=')
mylist.append(left + ':' + right)
out_file = open('test.json','w')
out_file.write(json.dumps(mylist))
out_file.close()
And here is what it returns:
["{", "repo_name:", "{", "name:repo_name", "baseurl:http://www.example.com", "enabled:1", "gpgcheck:0", "}"]
I haven't coded in for multiple repos yet, since I just wanted to get one working first. Am I approaching this correctly or is there a better way? OS is RHEL and python version is 2.6.6. Any help is greatly appreciated!

This a example file structure
[examplerepo]
name=Example Repository
baseurl=http://mirror.cisp.com/CentOS/6/os/i386/
enabled=1
gpgcheck=1
gpgkey=http://mirror.cisp.com/CentOS/6/os/i386/RPM-GPG-KEY-CentOS-6
And this is a code I used
#!/usr/bin/python
import json
test_dict = dict()
lines = open('test', 'r').read().split('\n')
current_repo = None
for line in lines:
if line.strip() != '':
if '[' in line:
current_repo = line
test_dict[current_repo] = dict()
else:
k, v = line.split("=")
test_dict[current_repo][k] = v
out_file = open('test.json', 'w')
out_file.write(json.dumps(test_dict))
out_file.close()
I think that using dictionaries is more natural way to do this.

python: print dictionary keys to a file

I'd like to figure out how to take all dictionary keys from an API call, and insert them into a flat file.
#!/usr/bin/env python
import requests
import json
import time
import urllib3
from base64 import b64encode
requests.packages.urllib3.disable_warnings()
#
# GET /dashboards/{dashboardId}/widgets/{widgetId}/value
test_dashboard = "557750bee4b0033aa111a762"
test_widget = "8bad2fc0-5c9b-44f2-a54b-05c8c6f9552b"
apiserver = "http://serveraddress"
userpass = b64encode(b"myuser:mypass").decode("ascii")
headers = { 'Authorization' : 'Basic %s' % userpass }
def get_apicall(dashboardId, widgetId):
response = requests.get(
apiserver + "/dashboards/" +
dashboardId + "/widgets/" +
widgetId + "/value",
headers=headers,
verify=False)
json_data = json.loads(response.text)
print(json.dumps(json_data["result"]["terms"], indent=2))
get_apicall(test_dashboard, test_widget)
which outputs something like:
[user#host ]$ ./shunhosts.py
{
"71.6.216.39": 2,
"71.6.158.166": 2,
"71.6.216.55": 2,
"71.6.216.56": 2
}
I would like the code to write/append each dictionary key to new line in a flat text file: i.e.
71.6.216.39
71.6.158.166
71.6.216.55
71.6.216.56

If you have a dictionary as
d = {
"71.6.216.39": 2,
"71.6.158.166": 2,
"71.6.216.55": 2,
"71.6.216.56": 2
}
You can get your keys with keys():
d.keys()
dict_keys(['71.6.216.56', '71.6.216.39', '71.6.158.166', '71.6.216.55'])
Make it to a string that is new-line separated:
s = '\n'.join(d.keys())
print(s)
71.6.216.39
71.6.158.166
71.6.216.55
71.6.216.56
Then write it to a file:
with open('some_file.txt', 'w') as fw:
fw.write(s)
You can now further simplify this to:
with open('some_file.txt', 'w') as fw:
fw.write('\n'.join(d.keys()))

json_data["result"]["terms"].keys() should give you all the keys.
You should read the documentation for how to open and write to file. It is very straight forward in python. Assuming you are using python 2.7, here is the link: https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files
It would be something like this:
f = open(filename, 'w')
f.write(json_data["result"]["terms"].keys())
f.close()

A dictionary has, by definition, an arbitrary number of keys. There is no "the key". You have the keys() method, which gives you a python list of all the keys, and you have theiteritems() method, which returns key-value pairs, so
for key, value in mydic.iteritems() :
thefile.write("%s\n" % key)
Or simply you can do this
for key in mydic.keys():
thefile.write("%s\n" % key)