I am trying to append value inside the list,but whenever I re run the server it overwrites the value. How can I effectively write in the json file.
{
"heart_rate": [
72.18
],
"Experiment_time": [
01/22/2023 11:59:59
]
}
I want to add values in json file, When ever I get new value it should get append inside the heartrate and Experiment_time list inside the corresponding keys, similar to below mentioned format.
{
"heart_rate": [
72.18,73.44
],
"Experiment_time": [
01/22/2023 11:59:59,01/22/2023 11:60:00
]
}
What I am tring is to first read the json file if it exists, If it doesn't create one.
try:
with open('Accounts.json','r') as fp:
data = json.load(fp)
f = data
print(f)
heartrate,dt_string = perform_all()
lst1 = []
if f != '':
print("something")
else:
lst1.append(heartrate)
except :
print("File Not found. Creating One")
data = {}
data['heart_rate'] = [None]
data['Experiment_time'] = [None]
with open('Accounts.json','w') as fp:
json.dump(data,fp,indent = 4)
heartrate,dt_string = perform_all()
is a function that returns 1 value of heartrate and 1 value of datetime when called
Close, just append to the right dictionary key. Note that JSON should be encoded in UTF-8 and not all OSes default to that encoding. It will matter if you write non-ASCII characters. Also bare except is a bad practice. Put the exception you expect.
import json
import datetime as dt
def perform_all():
return 85, dt.datetime.now().strftime('%m/%d/%y %H:%M:%S')
try:
with open('Accounts.json', encoding='utf8') as fp:
data = json.load(fp)
heartrate, dt_string = perform_all()
data['heart_rate'].append(heartrate)
data['Experiment_time'].append(dt_string)
except FileNotFoundError:
print('File not found. Creating initial data...')
data = {'heart_rate': [], 'Experiment_time': []}
with open('Accounts.json', 'w', encoding='utf8') as fp:
json.dump(data, fp, indent=2)
Output:
C:\>test
File not found. Creating one...
C:\>test
C:\>test
C:\>type Accounts.json
{
"heart_rate": [
85,
85
],
"Experiment_time": [
"01/26/23 21:34:22",
"01/26/23 21:34:23"
]
}
Related
I'm trying to process a json file like below and extract its data in the below output format for further processing.
json file
{
"application_robotics-2.1.610.80350109": [
"/home/machine_process/application_robotics/services/linear_service/4.106.50109987/robotics.yaml",
"/home/machine_process/application_robotics/services/linear_service/4.106.50109987/application_robotics-4.106.50109987.zip"
],
"web_robotics-3.116.50100987": [
"/home/machine_process/application_robotics/services/web_robotics/3.116.50100987/robotics.yaml",
"/home/machine_process/application_robotics/services/web_robotics/3.116.50100987/web_robotics-3.116.50100987.zip"
]
}
Expected output format
name = "application_robotics-2.1.610.80350109" # where name is a variable to be used in the other portion of the code.
yaml = "/home/machine_process/application_robotics/services/linear_service/4.106.50109987/robotics.yaml" # where yaml is a variable.
zip = "/home/machine_process/application_robotics/services/linear_service/4.106.50109987/application_robotics-4.106.50109987.zip" # where zip is a variable.
same format applied for other entries.
Below is the snippet code I've come up with and I'm not exactly getting the logic. Any help will be really helpful here. Thanks.
with concurrent.futures.ProcessPoolExecutor() as executor:
with open(file_path, "r") as input_json:
json_data = json.load(input_json)
for key, value in json_data.items():
name = json_data[key]
yaml = json_data[value]
zip = json_data[value]
file_location = os.path.dirname(tar)
futures = executor.submit(
other_function_name, yaml, zip, file_location, name
)
results.append(futures)
Current Output:
['home/machine_process/application_robotics/services/linear_service/4.106.50109987/robotics.yaml', '/home/machine_process/application_robotics/services/linear_service/4.106.50109987/application_robotics-4.106.50109987.zip']
Since name corresponds to the keys; yaml to the first element of lists; and zip_ to the second elements (note that zip is a python builtin, so avoid using it as a variable name), we can directly unpack it as we loop over the dictionary and pass these to executor.
with concurrent.futures.ProcessPoolExecutor() as executor:
with open(file_path, "r") as input_json:
json_data = json.load(input_json)
for name, (yaml, zip_) in json_data.items():
file_location = os.path.dirname(tar)
futures = executor.submit(other_function_name, yaml, zip_, file_location, name)
results.append(futures)
I have a JSON file containing three fields: 2 are strings and third one is field containing a list of values.
{ "STREAM": "stream",
"BASIS_STREAM": "basis",
"PATHS": "[/opt/path1,/opt/path2]"
}
Now I load that JSON
with open('/pathToJsonFile.json', 'r') as f:
data = json.load(f)
Now I want to get those values.
stream=str(data["STREAM"])
basis=str(data["BASIS_STREAM"])
paths=data["BASE_PATHS"]
The issue is that paths is also threated as String, although I have to use it as a list. I am converting with str function other fields because of the Unicode. Code must be in python2.
Thanks a lot!
Say you have a file called data.json with the following contents:
{
"STREAM": "stream",
"BASIS_STREAM": "basis",
"PATHS": "[/opt/path1,/opt/path2]"
}
Maybe you could use str.split after calling json.load:
with open('data.json', 'r') as f:
data = json.load(f)
print 'data = %s' % data
stream = str(data['STREAM'])
basis = str(data['BASIS_STREAM'])
paths = [str(u_s) for u_s in data['PATHS'][1:-1].split(',')]
print 'stream = %s' % stream
print 'basis = %s' % basis
print 'paths = %s' % paths
Output:
data = {u'PATHS': u'[/opt/path1,/opt/path2]', u'BASIS_STREAM': u'basis', u'STREAM': u'stream'}
stream = stream
basis = basis
paths = ['/opt/path1', '/opt/path2']
Your /opt/path1 and /opt/path2 should be in a quotation marks to be converted in a list. If your PATHS always have a similar template such as "[/XXX,/YYY,/ZZZ,/TTT,/KKK]" the following code should also help. I have converted your data as "['/XXX','/YYY','/ZZZ','/TTT','/KKK']" so that it can be easily converted to a list using ast library. Please see the code as following:
import json
import ast
with open("text_text.json") as f:
data = json.load(f)
print(data["PATHS"]) # Your data
for i in data["PATHS"]:
if i == "[":
data["PATHS"] = data["PATHS"].replace("[", "['")
elif i == ",":
data["PATHS"] = data["PATHS"].replace(",/", "','/")
elif i == "]":
data["PATHS"] = data["PATHS"].replace("]", "']")
#print(data["PATHS"])
print(type(data["PATHS"]))
print(data["PATHS"]) #converted to a data which can be converted to a list.
data_paths = ast.literal_eval(data["PATHS"]) # ast is used to convert str to list.
print(data_paths) # 'list' data
print(type(data_paths))
See the output of the code:
It should also work if your PATH has more data as following:
Currently, I store just the device_id of a certain object with an assigned sACN universe. The function looks like this:
for device_index in range(device_count):
device_name = sdk.get_device_info(device_index)
device_type = device_name.type
if device_name.id not in conf:
universe = get_free_universe()
conf.update({device_name.id: universe})
print(f"conf= {conf}")
else:
universe = conf[device_name.id] #main.py line 368
save_config(DEVICE_PATH)
Which produces this JSON:
{
"3192efa109cfb5d86f09a82a7cc00c5d": 4,
"42aa42a0bb5fcee780fb1be13dfcb873": 5,
"4b80e1817076307b36c58c31118f6696": 1,
"62c13e2db726382e9c66d9f69020ab5e": 6,
"a51da6fe155f299a3fc474c22310cde9": 2,
"b5ff59af43d6c3572a41d7693b5bec1c": 3
}
Now I wanna store together with the device_id not only the universe, but its device_name.model attribute, kinda in this format:
{
"3192efa109cfb5d86f09a82a7cc00c5d":
"universe": 4,
"model": "Vengance RGB PRO"
"42aa42a0bb5fcee780fb1be13dfcb873":
"universe": 5,
"model": "M65 PRO"
}
I have absolutely no clue how to do it, afaik my conf is a python dict where I cant do something like this and I need to use python lists. I define it like this: conf = load_config(DEVICE_PATH), the function load_config()looks like this:
def load_config(config_path):
if not os.path.isfile(config_path): #Create the file if not present
open(config_path, "w+")
if config_path == MQTT_PATH:
with open(config_path) as f: #load the config file
try:
return json.load(f)
except json.JSONDecodeError:
data = {}
data['enable_MQTT'] = True
data['ip'] =""
data['port']= 1883
data['username'] =""
data['password'] =""
data['base_topic'] =""
with open(config_path, "w", encoding="utf-8") as f: # Save config
json.dump(
data,
f,
ensure_ascii=False,
sort_keys=False,
indent=4
)
print(f"MQTT Config Created, please edit {MQTT_PATH} and restart this program!")
print("For Home Assistant Auto Discovery, set base_topic to homeassistant!")
sys.exit()
with open(config_path) as f: #load the config file
try:
return json.load(f)
except json.JSONDecodeError:
return {}
When I try to convert the list via conf = list(conf), i just get the runtime error
File "c:\Users\tenn0\Documents\Projects\iCue2sACN-mqtt\src\main.py", line 368, in <module>
universe = conf[device_name.id]
TypeError: list indices must be integers or slices, not str
Ive marked in the first snipped the correct line
Edit: As pointed out in the comments, i can simply do a dict inside a dict. conf.update({device_name.id: {"model": device_name.model, "universe": universe}}) did the trick for me.
I have a file (my_file.json) has contents as below;
[
{
"use":"abcd",
"contact":"xyz",
"name":"my_script.py",
"time":"11:22:33"
},
{
"use":"abcd"
"contact":"xyz",
"name":"some_other_script.py",
"time":"11:22:33"
},
{
"use":"apqwkndf",
"contact":"xyz",
"name":"my_script.py",
"time":"11:22:33"
},
{
"use":"kjdshfjkasd",
"contact":"xyz",
"name":"my_script.py",
"time":"11:22:33"
}
]
I used following python code to delete the objects that has "name":"my_script.py",
#!/bin/usr/python
impoty json
obj = json.load(open("my_file.json"))
index_list = []
for i in xrange(len(obj)):
if obj[i]["name"] == ["my_script.py"]
index_list.append(i)
for x in range(len(index_list)):
obj.pop(index_list[x])
open("output_my_file.json","w".write(json.dumps(obj, indent=4, separators=(',',': ')))
but it seems I am stuck, because after popping an index the index position in actual obj gets changed, which leads to wrong index deletion or sometimes pop index gets out of range. Any other solution?
Try popping in reverse order:
for x in reversed(range(len(index_list))):
This will create a new list and assign only those without "name": "my_script.py" to the new list.
obj = [i for i in obj if i["name"] != "my_script.py"]
import json
with open('my_file.json') as f:
data = json.load(f)
data = [item for item in data if item.get('name') != 'my_script.py']
with open('output_my_file.json', 'w') as f:
json.dump(data, f, indent=4)
Try:
import json
json_file = json.load(open("file.json"))
for json_dict in json_file:
json_dict.pop("name",None)
print(json.dumps(json_file, indent=4))
You don't need the last line where it says 'json.dumps' I just have it there so it looks more readable when printed.
As a general rule of thumb, you usually don't want to ever change an iterable while iterating over it.
I suggest you save the elements you do want in the first loop:
import json
with open('path/to/file', 'r') as f:
data = json.load(f)
items_to_keep = []
for item in data:
if item['name'] != 'my_script.py':
items_to_keep.append(item)
with open('path/to/file', 'w') as f:
json.dump(items_to_keep, f, ...)
The filtering can be reduced into a single line (called list-comprehension)
import json
with open('path/to/file', 'r') as f:
data = json.load(f)
items_to_keep = [item for item in data if item['name'] != 'my_script.py']
with open('path/to/file', 'w') as f:
json.dump(items_to_keep, f, ...)
Below, is the json structure I am pulling from my online weather station. I am also including a json_to_csv python script that is supposed to convert json data to csv output, but only returns a "Key" error. I want to pull data from "current_observation": only.
{
"response": {
"features": {
"conditions": 1
}
}
, "current_observation": {
"display_location": {
"latitude":"40.466442",
"longitude":"-85.362709",
"elevation":"280.4"
},
"observation_time_rfc822":"Fri, 26 Jan 2018 09:40:16 -0500",
"local_time_rfc822":"Sun, 28 Jan 2018 11:22:47 -0500",
"local_epoch":"1517156567",
"local_tz_short":"EST",
"weather":"Clear",
"temperature_string":"44.6 F (7.0 C)",
}
}
import csv, json, sys
inputFile = open("pywu.cache.json", 'r') #open json file
outputFile = open("CurrentObs.csv", 'w') #load csv file
data = json.load(inputFile) #load json content
inputFile.close() #close the input file
output = csv.writer(outputFile) #create a csv.write
output.writerow(data[0].keys())
for row in data:
output = csv.writer(outputFile) #create a csv.write
output.writerow(data[0].keys())
for row in data:
output.writerow(row.values()) #values row
What's the best method to retrieve the temperature string and convert to .csv format? Thank you!
import pandas as pd
df = pd.read_json("pywu.cache.json")
df = df.loc[["local_time_rfc822", "weather", "temperature_string"],"current_observation"].T
df.to_csv("pywu.cache.csv")
maybe pandas can be of help for you. the .read_json() function creates a nice dataframe, from which you can easily choose the desired rows and columns. and it can save as csv as well.
to add latitude and longitude to the csv-line, you can do this:
df = pd.read_json("pywu.cache.csv")
df = df.loc[["local_time_rfc822", "weather", "temperature_string", "display_location"],"current_observation"].T
df = df.append(pd.Series([df["display_location"]["latitude"], df["display_location"]["longitude"]], index=["latitude", "longitude"]))
df = df.drop("display_location")
df.to_csv("pywu.cache.csv")
to print the location in numeric values, you can do this:
df = pd.to_numeric(df, errors="ignore")
print(df['latitude'], df['longitude'])
This will find all keys (e.g. "temperature_string") specified inside of the json blob and then write them to a csv file. You can modify this code to get multiple keys.
import csv, json, sys
def find_deep_value(d, key):
# Find a the value of keys hidden within a dict[dict[...]]
# Modified from https://stackoverflow.com/questions/9807634/find-all-occurrences-of-a-key-in-nested-python-dictionaries-and-lists
# #param d dictionary to search through
# #param key to find
if key in d:
yield d[key]
for k in d.keys():
if isinstance(d[k], dict):
for j in find_deep_value(d[k], key):
yield j
inputFile = open("pywu.cache.json", 'r') # open json file
outputFile = open("mypws.csv", 'w') # load csv file
data = json.load(inputFile) # load json content
inputFile.close() # close the input file
output = csv.writer(outputFile) # create a csv.write
# Gives you a list of temperature_strings from within the json
temps = list(find_deep_value(data, "temperature_string"))
output.writerow(temps)
outputFile.close()