I've got a nested dict format that I wanted to setup in an environment.
It looks like this:
DEPARTMENTS_INFORMATION={
"Pants": {
"name": "Pants Department",
"email": "pants#department.com",
"ext": "x2121"
},
"Shirts": {
"name": "Shirt Department",
"email": "shirts#department.com",
"ext": "x5151"
},
"Socks": {
"name": "Sock Department",
"email": "socks#department.com",
"ext": " "
}
}
I am using django-environ for this and tried using it like this:
DEPARTMENTS = env.dict("DEPARTMENTS_INFORMATION", default={})
But it's giving me this error:
ValueError: dictionary update sequence element #0 has length 1; 2 is required
I'm not sure how to make the nested dictionary an environment variable - any help appreciated!
You can create 2 files:
file.env (you have to write the dict in one line)
DEPARTMENTS_INFORMATION={"Pants": {"name": "Pants Department","email": "pants#department.com","ext": "x2121"},"Shirts": {"name": "Shirt Department","email": "shirts#department.com","ext": "x5151"},"Socks": {"name": "Sock Department","email": "socks#department.com","ext": " "}}
main.py
import environ
#start the environ
env = environ.Env()
#load the env file
environ.Env.read_env("file.env")
#read the data
data = env.json("DEPARTMENTS_INFORMATION")
print(data)
Hope this helps.
Related
I have dicts with
I get a dictionary from the database with about this structure. And also the path to the desired field of the form - ['extra_fields']['period'] - as a string. Can you please tell me how to turn this string into the path to the data in the dictionary, i.e. into this piece of code in python clt['extra_fields']['period'] or clt.get('extra_fields').get('period'). I'm sitting here thinking and can't figure out the best way to do it? The line with the path to the data can be anything, not just as in the example (so hardcoded is not an option)
clt = {
"first_name": "test",
"last_name": "test",
"phone": "123",
"client_id": 123,
"email": "test#mail.ru",
"father_name": "",
"source": "test",
"extra_fields": {
"amount": 50000,
"period": 90,
"utm_source": "TEST_SOURCE",
"utm_campaign": "TEST_CAMPAIGN",
"keyword": "TEST_CREATIVE_ID"
}
}
lets say path is correct so you can use it with eval or you can parse the keys and then using those parsed keys you can access the data.
example with eval is as, (considering dictionary access path is correct
>>> clt = {
... "first_name": "test",
... "last_name": "test",
... "phone": "123",
... "client_id": 123,
... "email": "test#mail.ru",
... "father_name": "",
... "source": "test",
... "extra_fields": {
... "amount": 50000,
... "period": 90,
... "utm_source": "TEST_SOURCE",
... "utm_campaign": "TEST_CAMPAIGN",
... "keyword": "TEST_CREATIVE_ID"
... }
... }
>>>
>>> path = "['extra_fields']['period']"
>>>
>>> data = eval(f"clt{path}")
>>> data
90
>>>
You can convert the path to a list of paths using regex, and get the value using recursion
def get_value(_clt, path):
value = _clt.get(path[0])
if type(value) is dict:
return get_value(value, path[1:])
return value
path = "['extra_fields']['period']"
path = re.findall(r'\[(.*?)]', path.replace("'", ''))
print(get_value(clt, path)) # 90
path = "['last_name']"
path = re.findall(...)
print(get_value(clt, path)) # test
I am trying to use Python to extract pricePerUnit from JSON. There are many entries, and this is just 2 of them -
{
"terms": {
"OnDemand": {
"7Y9ZZ3FXWPC86CZY": {
"7Y9ZZ3FXWPC86CZY.JRTCKXETXF": {
"offerTermCode": "JRTCKXETXF",
"sku": "7Y9ZZ3FXWPC86CZY",
"effectiveDate": "2020-11-01T00:00:00Z",
"priceDimensions": {
"7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7": {
"rateCode": "7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7",
"description": "Processed translation request in AWS GovCloud (US)",
"beginRange": "0",
"endRange": "Inf",
"unit": "Character",
"pricePerUnit": {
"USD": "0.0000150000"
},
"appliesTo": []
}
},
"termAttributes": {}
}
},
"CQNY8UFVUNQQYYV4": {
"CQNY8UFVUNQQYYV4.JRTCKXETXF": {
"offerTermCode": "JRTCKXETXF",
"sku": "CQNY8UFVUNQQYYV4",
"effectiveDate": "2020-11-01T00:00:00Z",
"priceDimensions": {
"CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7": {
"rateCode": "CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7",
"description": "$0.000015 per Character for TextTranslationJob:TextTranslationJob in EU (London)",
"beginRange": "0",
"endRange": "Inf",
"unit": "Character",
"pricePerUnit": {
"USD": "0.0000150000"
},
"appliesTo": []
}
},
"termAttributes": {}
}
}
}
}
}
The issue I run into is that the keys, which in this sample, are 7Y9ZZ3FXWPC86CZY, CQNY8UFVUNQQYYV4.JRTCKXETXF, and CQNY8UFVUNQQYYV4.JRTCKXETXF.6YS6EN2CT7 are a changing string that I cannot just type out as I am parsing the dictionary.
I have python code that works for the first level of these random keys -
with open('index.json') as json_file:
data = json.load(json_file)
json_keys=list(data['terms']['OnDemand'].keys())
#Get the region
for i in json_keys:
print((data['terms']['OnDemand'][i]))
However, this is tedious, as I would need to run the same code three times to get the other keys like 7Y9ZZ3FXWPC86CZY.JRTCKXETXF and 7Y9ZZ3FXWPC86CZY.JRTCKXETXF.6YS6EN2CT7, since the string changes with each JSON entry.
Is there a way that I can just tell python to automatically enter the next level of the JSON object, without having to parse all keys, save them, and then iterate through them? Using JQ in bash I can do this quite easily with jq -r '.terms[][][]'.
If you are really sure, that there is exactly one key-value pair on each level, you can try the following:
def descend(x, depth):
for i in range(depth):
x = next(iter(x.values()))
return x
You can use dict.values() to iterate over the values of a dict. You can also use next(iter(dict.values())) to get a first (only) element of a dict.
for demand in data['terms']['OnDemand'].values():
next_level = next(iter(demand.values()))
print(next_level)
If you expect other number of children than 1 in the second level, you can just nest the fors:
for demand in data['terms']['OnDemand'].values():
for sub_demand in demand.values()
print(sub_demand)
If you are insterested in the keys too, you can use dict.items() method to iterate over dict keys and values at the same time:
for demand_key, demand in data['terms']['OnDemand'].items():
for sub_demand_key, sub_demand in demand.items()
print(demand_key, sub_demand_key, sub_demand)
I have an inventory which I collected from ansible and I'm reading it with python but I have no sucess to print the values here, so I have the following:
the content below is in a file called "localhost".
{
"ansible_facts": {
"facter_architecture": "x86_64",
"facter_bios_release_date": "12/01/2006",
"facter_bios_vendor": "innotek GmbH",
"facter_bios_version": "VirtualBox",
"facter_blockdevice_sda_model": "VBOX HARDDISK",
"facter_blockdevice_sda_size": 10737418240,
"facter_blockdevice_sda_vendor": "ATA",
"facter_blockdevice_sr0_model": "CD-ROM",
"facter_blockdevice_sr0_size": 1073741312,
"facter_blockdevice_sr0_vendor": "VBOX",
"facter_blockdevices": "sda,sr0",
"facter_boardmanufacturer": "Oracle Corporation",
"facter_boardproductname": "VirtualBox",
"facter_boardserialnumber": "0",
"facter_dhcp_servers": {
"enp0s3": "10.0.2.2",
"enp0s8": "192.168.1.1",
"system": "10.0.2.2"
},
"facter_domain": "home",
"facter_facterversion": "2.4.1",
"facter_filesystems": "xfs",
"facter_fqdn": "mylab.home",
"facter_gid": "root",
"facter_hardwareisa": "x86_64",
"facter_hardwaremodel": "x86_64",
"facter_hostname": "mylab",
"facter_id": "root",
"facter_interfaces": "enp0s3,enp0s8,lo",
"facter_ipaddress": "10.0.2.15",
"facter_ipaddress_enp0s3": "10.0.2.15",
"facter_ipaddress_enp0s8": "192.168.1.101",
"facter_ipaddress_lo": "127.0.0.1",
"facter_is_virtual": true,
"facter_kernel": "Linux",
"facter_kernelmajversion": "3.10",
"facter_kernelrelease": "3.10.0-1127.13.1.el7.x86_64",
"facter_kernelversion": "3.10.0",
"facter_macaddress": "08:00:27:dd:47:a8",
"facter_macaddress_enp0s3": "08:00:27:dd:47:a8",
"facter_macaddress_enp0s8": "08:00:27:12:ce:46",
"facter_manufacturer": "innotek GmbH",
"facter_memoryfree": "730.39 MB",
"facter_memoryfree_mb": "730.39",
"facter_memorysize": "990.98 MB",
"facter_memorysize_mb": "990.98",
"facter_mtu_enp0s3": 1500,
"facter_mtu_enp0s8": 1500,
"facter_mtu_lo": 65536,
"facter_netmask": "255.255.255.0",
"facter_netmask_enp0s3": "255.255.255.0",
"facter_netmask_enp0s8": "255.255.255.0",
"facter_netmask_lo": "255.0.0.0",
"facter_network_enp0s3": "10.0.2.0",
"facter_network_enp0s8": "192.168.1.0",
"facter_network_lo": "127.0.0.0",
"facter_operatingsystem": "CentOS",
"facter_operatingsystemmajrelease": "7",
"facter_operatingsystemrelease": "7.8.2003",
"facter_os": {
"family": "RedHat",
"name": "CentOS",
"release": {
"full": "7.8.2003",
"major": "7",
"minor": "8"
}
},
"facter_osfamily": "RedHat",
"facter_partitions": {
"sda1": {
"filesystem": "xfs",
"mount": "/boot",
"size": "2097152",
"uuid": "987fb5e2-f636-423b-997d-c2654993708c"
},
"sda2": {
"filesystem": "LVM2_member",
"size": "18872320"
}
},
"facter_path": "/root/.rbenv/shims:/root/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin/python3:/root/bin",
"facter_physicalprocessorcount": 1,
"facter_processor0": "Intel(R) Core(TM) i5-5350U CPU # 1.80GHz",
"facter_processorcount": 1,
"facter_processors": {
"count": 1,
"models": [
"Intel(R) Core(TM) i5-5350U CPU # 1.80GHz"
],
"physicalcount": 1
},
"facter_productname": "VirtualBox",
"facter_ps": "ps -ef",
"facter_rubyplatform": "x86_64-linux",
"facter_rubysitedir": "/usr/local/share/ruby/site_ruby/",
"facter_rubyversion": "2.0.0",
"facter_selinux": false,
"facter_serialnumber": "0",
"facter_sshecdsakey": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8u+HCceoSA9mrw1oboY4sAXhkgp4CBTe8QjuW2OAeYIQS7LblPztfQmJFkXpHTWhLvSBYglzcuZiJOzUbZQ/0=",
"facter_sshed25519key": "AAAAC3NzaC1lZDI1NTE5AAAAIDutjvzHFEKqc0bprmbkm9ZUoADflkan6dnCVcYsGOTT",
"facter_sshfp_ecdsa": "SSHFP 3 1 2190a1073f110b50ed6fb912cd04144603a85098\nSSHFP 3 2 3a708ee555593b0ad5a2f1992ae949d56d2f3556c37b201ef683d4a3ea850660",
"facter_sshfp_ed25519": "SSHFP 4 1 695978669c105b4a3a06c4b9a685020363b72c67\nSSHFP 4 2 794f24ad5bd7b91a1d3a0f484ceb166088ac8d57e3e1682e8b8fe006fde1c169",
"facter_sshfp_rsa": "SSHFP 1 1 07afd9583d0785ac923230bef3b0d0ffeefad097\nSSHFP 1 2 29158514b311cc7687fa1c3aab1fa1abee0f2f581eb3d607a6b4ffb8ff258d59",
"facter_sshrsakey": "AAAAB3NzaC1yc2EAAAADAQABAAABAQCtgyTEG+VnEnXiiaP4tFpIiWwWfqxdW8BCATa5W9QE0AsfY1OiFoLRXYGqhL72q0N+VTHQGB7eB1sd9Nas48erDzZXpgLoIDqM1pa/vT/j/SygQB2rwgo2wga0tw+zW1cw+sELjXHAYsi8DADKbGlX2cCeT3MKeWdkg+BQogf74Sy4NEPbYhILXPfvt3cJxCM02sIn/eQL+n06iSzesUIEy5n+AlRgACR3zHnk5rtHipj/RzmPv+J0V3du7+g6/3TiKDcBTNHtb8QSa4DSGkmbW7Wdhvnw9GyhO5ySGB2G3rmLiVIm9vdjB9L/X/L2g8TB5+/dO52UxUSX17nwzuJB",
"facter_swapfree": "1024.00 MB",
"facter_swapfree_mb": "1024.00",
"facter_swapsize": "1024.00 MB",
"facter_swapsize_mb": "1024.00",
"facter_system_uptime": {
"days": 0,
"hours": 3,
"seconds": 12984,
"uptime": "3:36 hours"
},
"facter_timezone": "CEST",
"facter_type": "Other",
"facter_uniqueid": "000a0f02",
"facter_uptime": "3:36 hours",
"facter_uptime_days": 0,
"facter_uptime_hours": 3,
"facter_uptime_seconds": 12984,
"facter_uuid": "b64ed9b0-7168-4e49-a34a-90e6ea6f751a",
"facter_virtual": "kvm"
},
"changed": false
}
I have the following code.
I tried a lot of deferents stuffs, but somehow it complains about "list indices must be integers or slices, not str"
import json
dict = []
with open('localhost', 'r') as jsonfile:
myfile = json.load(jsonfile)
result = json.dumps(myfile, indent=2, sort_keys=True)
dict.append(result)
print(dict['ansible_facts'])
From this list I need to extract
facter_system_uptime and facter_hostname only but I had no success.
I tried to put in a loop, but still same issue.
for data in dict['ansible_facts']:
print(data)
What's wrong here?
you don't need to use json.dumps because you've already used json.load to deserialize the JSON document to a Python object.
import json
with open("localhost") as jsonfile:
myfile = json.load(jsonfile)
print(myfile["ansible_facts"]["facter_system_uptime"])
print(myfile["ansible_facts"]["facter_hostname"])
Don't use dict as the name for a variable - it shadows the builtin and will cause you problems.
You created a list (mydict = []) and then you appended your parsed json to it. That means, to access it, you must use mydict[0]['ansible_facts'].
I do not know what ansible is, but to my knowledge, your error is because you initilized dict as a list:
dict = []
which you can access its elements by numerical indices, but instead you gave it a string:
print(dict['ansible_facts'])
this is how you initialize a dictionary:
dict = {}
There are two issues here.
You have created a list instead of dict
You have used reserved key(dict) as variable name
So the final code should be something like
import json
with open('localhost', 'r') as jsonfile:
myfile = json.load(jsonfile)
print(myfile['ansible_facts']['facter_system_uptime'])
print(myfile['ansible_facts']['facter_hostname'])
If you want to print all the data using loop
You can iterate on myfile['ansible_facts'] like this.
import json
with open('localhost', 'r') as jsonfile:
myfile = json.load(jsonfile)
for key, value in myfile['ansible_facts'].items():
print(key, value)
Is there any way to pull the key from JSON if the only thing I know is the value? (In groovy or python)
An example:
I know the "_number" value and I need a key.
So let's say, known _number is 2 and as an output, I should get dsf34f43f34f34f
{
"id": "8e37ecadf4908f79d58080e6ddbc",
"project": "some_project",
"branch": "master",
"current_revision": "3rtgfgdfg2fdsf",
"revisions": {
"43g5g534534rf34f43f": {
"_number": 3,
"created": "2019-04-16 09:03:07.459000000",
"uploader": {
"_account_id": 4
},
"description": "Rebase"
},
"dsf34f43f34f34f": {
"_number": 2,
"created": "2019-04-02 10:54:14.682000000",
"uploader": {
"_account_id": 2
},
"description": "Rebase"
}
}
}
With Groovy:
def json = new groovy.json.JsonSlurper().parse("x.json" as File)
println(json.revisions.findResult{ it.value._number==2 ? it.key : null })
// => dsf34f43f34f34f
Python 3: (assuming that data is saved in data.json):
import json
with open('data.json') as f:
json_data = json.load(f)
for rev, revdata in json_data['revisions'].items():
if revdata['_number'] == 2:
print(rev)
Prints all revs where _number equals 2.
using dict-comprehension:
print({k for k,v in d['revisions'].items() if v.get('_number') == 2})
OUTPUT:
{'dsf34f43f34f34f'}
I have a complex json file it included nested dics in it.
it looks like this
{
"objectivelist": [{
"measureid": "1122",
"gradeID": "4222332",
"graduationdate": "May",
"system": {
"platform": "MAC",
"TeacherName": "Mike",
"manager": "Jim",
"studentinfomation": {
"ZIP": "94122",
"city": "SF"
}
},
"measureid": "1122",
"gradeID": "4222332",
"graduationdate": "May",
"system": {
"platform": "MAC",
"TeacherName": "joshe",
"manager": "steven"
},
"studentinfomation": {
"ZIP": "94122",
"city": "SF"
}
}]
}
Here the grade ID and Measured ID are the same, so the result should only need to show one times, and my result should be like this:
{"measureid":"1122","gradeID"4222332","graduationdate":"May"}
I do not need the managername, teachername etc.
not sure how to do this. I try to use comprehensation but do not know who to use it in nest dictionary.
Thank you guys.
Depending on how huge the json file is you may need better solution. We will hash fields which are of interest to us and build the unique json iteratively.
check_set = set()
output = []
interesting_fields = ['measureid', 'gradeID', 'graduationdate']
for dat in X['objectivelist']:
m = hashlib.md5()
m.update(dat['measureid'].encode('utf-8'))
m.update(dat['gradeID'].encode('utf-8'))
m.update(dat['graduationdate'].encode('utf-8'))
digest = m.hexdigest()
if digest not in check_set:
output.append({key: dat[key] for key in ['measureid', 'gradeID', 'graduationdate']})
check_set.add(digest)
And you can find your output in output.