Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 17 days ago.
Improve this question
Suppose that I have a dictionary dict = {"fadfafa": {"price": 5}} in python. If I do json.dump(dict), then I will end up getting {{"fadfafa":"{\"price\" : 5}"} in the json file. Is there any way to store it in the json file just as how it originally looks like dict = {"fadfafa": {"price": 5}}? So a minimal working example would be:
dict = {"fadfafa": {"price": 5}}
with open('temp.json') as output_file:
json.dump(dict)
The problem with this output is that when I do
with open('temp.json') as input_file:
json.load(temp.json = object_pairs_hook = dict_raise_on_duplicates")
It reads back the object not as a dictionary of dictionary but one single dictionary with its value being a string.
Firstly, don't name your dict 'dict' because that is a reserved word and will cause you problems. Secondly that last bit of code doesn't run so I'm not sure what you want it to do.
But let us see if you assumption is correct:
import json
from io import StringIO
io = StringIO()
sample_dict = {"fadfafa":{"price" : 5}}
json.dump(sample_dict, io)
print(io.getvalue())
This gives the output:
{"fadfafa": {"price": 5}}
So the file contents do not have the extra brackets as you state. Let us read it back and compare the result with the original dictionary:
io.seek(0)
new_dict = json.load(io)
print(new_dict)
is_same = all((new_dict.get(k) == v for k, v in sample_dict.items()))
print(is_same)
This gives the output:
{'fadfafa': {'price': 5}}
True
So it seems to behave fine.
Related
I currently have a list variable that looks like this:
list_of_dicts = [{"Away_Team":"KC", "Home_Team":"NYY"},
{"Away_Team":"TB", "Home_Team":"MIA"},
{"Away_Team":"TOR", "Home_Team":"BOS"},
]
As you can see, there are multiple keys with the same names, pertaining to the game matchups.
When I try to use:
print(json.dumps(list_of_dicts[0], indent=4, sort_keys=True))
...it only prints out the first matchup due to the same keys:
{
"Away_Team": "KC",
"Home_Team": "NYY"
}
How can I convert this list_of_dicts variable into something like the following output so I can use it like a valid dictionary or json object?
{
"Away_Team_1":"KC", "Home_Team_1":"NYY",
"Away_Team_2":"TB", "Home_Team_2":"MIA",
"Away_Team_3":"TOR", "Home_Team_3":"BOS",
}
This output doesn't need to be exactly that if a better solution is available, this is just to give you an idea of how I'd like to be able to parse the data.
The list_of_dicts variable can be of varying sizes, I've shown 3 here, but it could contain 1 or 10 matchups, it varies, so the solution needs to be dynamic to that.
You can add suffixes to the keys with enumerate:
list_of_dicts2 = [{f"{k}_{i}":v for k,v in d.items()} for i,d in enumerate(list_of_dicts, start=1)]
One option is to use pandas:
pd.DataFrame(list_of_dicts).to_csv('filename.csv', index=False)
gives
Away_Team,Home_Team
KC,NYY
TB,MIA
TOR,BOS
Now the index is implied by the row, and if you load it back in you'll have those indices. Pandas also supports to_json if you are hard set on using json though. You can even recover your original list from a dataframe using .to_dict(orient='records')
Data structure is important. You really don't need a dictionary for this. Simply reducing to a list of tuples the first slot always the away team, and the second the home team.
list_of_dicts = [{"Away_Team":"KC", "Home_Team":"NYY"},
{"Away_Team":"TB", "Home_Team":"MIA"},
{"Away_Team":"TOR", "Home_Team":"BOS"},
]
l = [tuple(l.values()) for l in list_of_dicts]
output:
[('KC', 'NYY'), ('TB', 'MIA'), ('TOR', 'BOS')]
The problem with your proposed solution is iterating through dicts where you don't know the key name is cumbersome, this solution makes the data structure easy to decipher, transform, or manipulate.
I have a JSON that looks like the following (this example is just a toy example to illustrate the point, default iteration might work by luck).
{ "A": 1, "B": 2, "C": 3 }
I can load this JSON into a Python (v2.7) dictionary.
with open('/path/to/json/file') as f:
json_data = json.load(f)
I then iterate over the key-value pairs as follows.
for k, v in json_data.iteritems():
print('{}, {}'.format(k, v))
What is printed is the following.
A, 1
C, 3
B, 2
What I want is to be able to iterate over the items in the order that was specified in the JSON. So, my expected output should be.
A, 1
B, 2
C, 3
Is there any way to read in and store the key-value pairs from JSON to a Python dictionary such that when I iterate over them, the order specified by JSON is preserved and read back in such order?
I do not have control with the producer of the JSON to store the data in an array (discussed here for Java). I am wondering if there is a LinkedHashMap like in Java with Python that perhaps I may use with json.load to preserve the insertion order for reading.
Any help is appreciated.
You can get the json loader to build an OrderedDict:
import json
import collections
with open('example.json') as f:
json_object = json.load(f,object_pairs_hook=collections.OrderedDict)
A json-file which has unique markers (or [more appropriate] field-names) preceeding the values is (rather) easy to dissect, because you can perform a string search on the unique markers/field-names to find within the string the first and last position of the characters of the value, and with that info you can pinpoint the position of the value, and extract the value.
Have performed that function with various lua-scripts and Python-scripts (also on xml-files).
Now need to extract values from a json-file which does not have unique markers/ field-names, but just a multiple occurrence of "value_type" and "value", preceeding the 'name', respectively the 'value': see below.
{
"software_version": "NRZ-2017-099",
"age":"78",
"sensordatavalues":[
{"value_type":"SDS_P1","value":"4.43"},
{"value_type":"SDS_P2","value":"3.80"},
{"value_type":"temperature","value":"20.10"},
{"value_type":"humidity","value":"44.50"},
{"value_type":"samples","value":"614292"},
{"value_type":"min_micro","value":"233"},
{"value_type":"max_micro","value":"25951"},
{"value_type":"signal","value":"-66"}
]
}
Experience as described above does not provide working solution.
Question: In this json-filelayout, how to directly extract the specific, individual values (preferably by lua-script)?
[Or might XML-parsing provide an easier solution?]
Here is Python to read the JSON file and make it more convenient:
import json
import pprint
with open("/tmp/foo.json") as j:
data = json.load(j)
for sdv in data.pop('sensordatavalues'):
data[sdv['value_type']] = sdv['value']
pprint.pprint(data)
The results:
{'SDS_P1': '4.43',
'SDS_P2': '3.80',
'age': '78',
'humidity': '44.50',
'max_micro': '25951',
'min_micro': '233',
'samples': '614292',
'signal': '-66',
'software_version': 'NRZ-2017-099',
'temperature': '20.10'}
You might want to have a look into filter functions.
E.g. in your example json to get only the dict that contains the value for samples you could go by:
sample_sensordata = list(filter(lambda d: d["value_type"] == "samples", your_json_dict["sensordatavalues"]))
sample_value = sample_sensordata["value"]
To make a dictionary like Ned Batchelder said you could also go with a dict comprehension like this:
sensor_data_dict = {d['value_type']: d['value'] for d in a}
and then get the value you want just by sensor_data_dict['<ValueTypeYouAreLookingFor>']
A little bit late and I'm trying Anvil in which the previous answers didn't work. just for the curious people.
resp = anvil.http.request("http://<ipaddress>/data.json", json=True)
#print(resp) # prints json file
tempdict = resp['sensordatavalues'][2].values()
humiddict = resp['sensordatavalues'][3].values()
temperature = float(list(tempdict)[1])
humidity = float(list(humiddict)[1])
print(temperature)
print(humidity)
I am facing some trouble with trying to add more key:value pairs to a dictionary object that is itself nested within another dictionary object. Also, the usual way of doing dict[key] = value to assign additional key:value pairs to the dictionary is not suitable for my case here (I'll explain why later below), and thus this makes my objective a lot more challenging to achieve.
I'll illustrate what I'm trying to achieve with some statements from my source code.
First, I have a dictionary object that contains nesting:
environment = { 'v' :
{
'SDC_PERIOD':'{period}s'.format(period = self.period),
'FAMILY':'{legup_family}s'.format(legup_family = self.legup_family),
'DEVICE_FAMILY':'"{fpga_family}s"'.format(fpga_family = self.fpga_family)
}
}
and then following this line, I will do an if test that, if passed, will require me to add this other dictionary:
environment_add = { 'v' : {'LM_LICENSE_FILE' : '1800#adsc-linux'} ,
'l' : 'quartus_full' }
to ultimately form this complete dictionary:
environment = { 'v' :
{
'SDC_PERIOD':'{period}s'.format(period = self.period),
'FAMILY':'{legup_family}s'.format(legup_family = self.legup_family),
'DEVICE_FAMILY':'"{fpga_family}s"'.format(fpga_family = self.fpga_family),
'LM_LICENSE_FILE' : '1800#adsc-linux'
} ,
'l' : 'quartus_full'
}
As you can see, if I were to try and assign a new key:value pair using the dict[key] = value syntax, it would not work for me because it would end up either creating an new key:value pair for me, or overwrite the existing dictionary object and the key:value pairs that are nested under the 'v' key.
So far, in order to accomplish the creation of the dictionary, I've been using the following:
environment = """{ v: {'SDC_PERIOD':'%(period)s','FAMILY':'%(legup_family)s','DEVICE_FAMILY':'"%(fpga_family)s"'}}""" % self
if self.require_license: # this is the if statement test that I was referring to earlier
environment = environment.replace('}', '')
environment += """ ,'LM_LICENSE_FILE':'1800#adsc-linux'}, 'l': 'quartus_full'}"""
and then obtaining the dictionary object later with:
import ast
env_dict = ast.literal_eval(environment)
which gives effectively converts the environment string into a dictionary object stored under a new variable name of env_dict.
My teammates think that this is much too overkill, especially since the environment or env_dict object will be parsed in 2 separate modules later on. In the first module, the key-value pairs will be broken up and reconstructed to form strings that look like '-v SDC_PERIOD=2500s, LM_LICENSE_FILE=1800#adsc-linux' , while in the second module, the dictionary nested under the 'v' key (of the environment/env_dict dictionary object) will be extracted out and directly fed as an argument to a function that accepts a mapping object.
So as you can see, there is quite a lot of precise parsing required to do the job, and although my method fulfills the objective, it is not accepted by my team and they think that there must be a better way to do this directly from environment being a dictionary object and not a string object.
Thank you very much for studying my detailed post, and I will greatly appreciate any help or suggestions to move forward on this!
for k,v in environment_add.iteritems(): # .items() in Python 3
if k in environment:
environment[k].update(v)
else:
environment[k] = v
That is, for each item to add, check if it exists, and update it if so, or simply create it. This assumes the items being added, if they exist, will be dicts (you can't update a string like quartus_full).
Why not just use update
In [4]: dict_ = {"a": {"b": 2, "c": 3}}
In [5]: dict_["a"].update(d=4)
In [6]: dict_
Out[6]: {'a': {'b': 2, 'c': 3, 'd': 4}}
Say for example i want to count how many times bob visits sears and walmart how would i do this by creating dictionaries?
information given:
bob:oct1:sears
bob:oct1:walmart
mary:oct2:walmart
don:oct2:sears
bob:oct4:walmart
mary:oct4:sears
Okay, as this might be homework, I’ll try to give you some hints on how to do this. If this is not homework, please say so, and I’ll restore my original answer and example code.
So first of all, you have your data set in a way that each entry is in single line. As we want to work with each data entry on its own, we have to split the original data into each lines. We can use str.split for that.
Each entry is constructed in a simple format name:date:location. So to get each of those segments again, we can use str.split again. Then we end up with separated content for each entry.
To store this, we want to sort the data by the name first. So we choose a dictionary taking the name as the key, and put in the visits as the data. As we don’t care about the date, we can forget about it. Instead we want to count how often a single location occurs for a given name. So what we do, is keep another dictionary using the locations as the key and the visit count as the data. So we end up with a doubled dictionary, looking like this:
{
'bob': {
'sears': 1,
'walmart': 1,
},
'mary': {
...
}
}
So to get the final answers we just look into that dictionary and can immediately read out the values.
#poke provided a nice explanation, here's a corresponding code:
Read input from files provided on command-line or stdin and dump occurrences in json format:
#!/usr/bin/env python
import csv
import fileinput
import json
import sys
from collections import defaultdict
visits = defaultdict(lambda: defaultdict(int))
for name, _, shop in csv.reader(fileinput.input(), delimiter=':'):
visits[name][shop] += 1
# pretty print
json.dump(visits, sys.stdout, indent=2)
Output
{
"bob": {
"sears": 1,
"walmart": 2
},
"don": {
"sears": 1
},
"mary": {
"sears": 1,
"walmart": 1
}
}
This representation allows easily to find out how many visits and where a person had.
If you always know both name and location then you could use a simpler representation:
visits = defaultdict(int)
for name, _, shop in csv.reader(fileinput.input(), delimiter=':'):
visits[name,shop] += 1
print(visits['bob','walmart'])
# -> 2