I wrote the following function that I want to apply to a json file:
import json
def myfunction(dictionary):
#doing things
return new_dictionary
data = """{
#a json file as a dictionary
}"""
info = json.loads(data)
refined = key_replacer(info)
new_data = json.dumps(refined)
print(new_data)
It works fine, but how do I do it when I want to import a file from my computer? json.loads take a string as input and returns a dictionary as output and json.dumps take a dictionary as input and returns a string as output. I tried with:
with open('C:\\Users\\SONY\\Desktop\\test.json', 'r', encoding="utf8") as data:
info = json.loads(data)
But TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper.
You are passing a file object instead of string. To fix that, you need to read the file first json.loads(data.read())
Howerver, you can directly load json from files using json.load(open('myFile','r')) or in your case, json.load(data)
loads and dumps work on strings. If you want to work on files you should use load and dump instead.
Here is an example:
from json import dump, load
with open('myfile.json','r') as my_file:
content = load(my_file)
#do stuff on content
with open('myooutput.json','w') as my_output:
dump(content, my_output)
Related
I'm trying to iterate over specific keys and values from a JSON file and write them to a new JSON file. I'm not getting any errors and the second JSON file is being created successfully, though the data passed onto the second JSON file is a whole object of the first JSON file.
def get_keys_values():
json_file = open("get_failed_archives.json")
json_data = json.load(json_file)
for archive_data in json_data["data"]:
archive_data["latestEvent"]["objectName"]
archive_data["latestEvent"]["time"]
archive_data["latestEvent"]["eventType"]
archive_data["latestEvent"]["eventStatus"]
with open("new_json_file.json", "w") as file:
json.dump(archive_data, file)
Is there a way to write specific keys and values from one JSON file to another JSON file?
as stated by josh, the statement archive_data["latestEvent"]["objectName"] functionally does nothing.
if you want to create a new JSON object, in your case this would functionally be a list of dict items (I think), the approach below would match what (I think) you want it to do.
I've added a pprint statement to print out what the result is.
import json
from pprint import pprint
def get_keys_values():
with open("get_failed_archives.json") as json_file:
json_data = json.load(json_file)
new_data = []
for archive_data in json_data["data"]:
new_data.append({
"objectName": archive_data["latestEvent"]["objectName"],
"time": archive_data["latestEvent"]["time"],
"eventType": archive_data["latestEvent"]["eventType"],
"eventStatus": archive_data["latestEvent"]["eventStatus"]
})
pprint(new_data, indent=4)
with open("new_json_file.json", "w") as file:
json.dump(new_data, file)
I am fetching html source code of many pages from one website, I need to convert it into json object and combine with other elements in json doc. . I have seen many questions on same topic but non of them were helpful.
My code:
url = "https://totalhash.cymru.com/analysis/?1ce201cf28c6dd738fd4e65da55242822111bd9f"
htmlContent = requests.get(url, verify=False)
data = htmlContent.text
print("data",data)
jsonD = json.dumps(htmlContent.text)
jsonL = json.loads(jsonD)
ContentUrl='{ \"url\" : \"'+str(urls)+'\" ,'+"\n"+' \"uid\" : \"'+str(uniqueID)+'\" ,\n\"page_content\" : \"'+jsonL+'\" , \n\"date\" : \"'+finalDate+'\"}'
above code gives me unicode type, however, when I put that output in jsonLint it gives me invalid json error. Can somebody help me understand how can I convert the complete html into a json objet?
jsonD = json.dumps(htmlContent.text) converts the raw HTML content into a JSON string representation.
jsonL = json.loads(jsonD) parses the JSON string back into a regular string/unicode object. This results in a no-op, as any escaping done by dumps() is reverted by loads(). jsonL contains the same data as htmlContent.text.
Try to use json.dumps to generate your final JSON instead of building the JSON by hand:
ContentUrl = json.dumps({
'url': str(urls),
'uid': str(uniqueID),
'page_content': htmlContent.text,
'date': finalDate
})
The correct way to convert HTML source code to a JSON file on the local system is as follows:
import json
import codecs
# Load the JSON file by specifying the location and filename
with codecs.open(filename="json_file.json", mode="r", encoding="utf-8") as jsonf:
json_file = json.loads(jsonf.read())
# Load the HTML file by specifying the location and filename
with codecs.open(filename="html_file.html", mode='r', encoding="utf-8") as htmlf:
html_file = htmlf.read()
# Chose the key name where the HTML source code will live as a string
json_file['Key1']['Key2'] = html_file
# Dump the dictionary to JSON object and save it in a specific location
json_object = json.dumps(json_file, indent=4)
with codecs.open(filename="final_json_file.json", mode="w", encoding="utf-8") as ojsonf:
ojsonf.write(json_object)
Next, open the JSON file in your editor.
Press CTRL + H, and replace \n or \t characters by '' (nothing!).
Now you can parse your HTML file with codecs.open() function and do the operations.
I am trying to write a service to read twitter feed stream data and then write it to a file. I am writing each JSON structure to a line in the file. With a different service I need to read each line of the file and load the json structure for further operations.
My problem is that I can read the first line then the JSON loader says the rest are not JSON structures. They look fine. Not sure what is going on.
Writting file:
self.output = open(os.path.join(self.outputdir,self.filename,'w')
self.output.write(status + "\n")
Reading File:
with open(file) as f:
line = line.replace("\n","")
tweet = json.loads(line)
print tweet['text']
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
Example json file:
JSON File
JSON File
Your json is composed with multiple json objects and empty lines.
You need to load each line as a new json object and ignore empty lines:
>>> with open('streamer.151205-071156.json') as f:
>>> data = [json.loads(l) for l in f if len(l) > 1]
>>> len(data)
7
>>> print(data[0]['text'])
u'Mnjd \U0001f642\U0001f602 https://t.co/BL5Ezxtt0i'
I have a valid JSON (checked using Lint) in a text file.
I am loading the json as follows
test_data_file = open('jsonfile', "r")
json_str = test_data_file.read()
the_json = json.loads(json_str)
I have verified the json data in file on Lint and it shows it as valid. However the json.loads throws
ValueError: No JSON object could be decoded
I am a newbie to Python so not sure how to do it the right way. Please help
(I assume it has something to do it encoding the string to unicode format from utf-8 as the data in file is retrieved as a string)
I tried with open('jsonfile', 'r') and it works now.
Also I did the following on the file
json_newfile = open('json_newfile', 'w')
json_oldfile = open('json_oldfile', 'r')
old_data = json_oldfile.read()
json.dump(old_data, json_newfile)
and now I am reading the new file.
I'm trying to get the values from the json file and the error that I'm getting is TypeError: expected string or buffer. I'm parsing the file correctly and moreover I guess my json file format is also correct. Where I'm going wrong?
Both the files are in the same directory.
Main_file.py
import json
json_data = open('meters_parameters.json')
data = json.loads(json_data) // TypeError: expected string or buffer
print data
json_data.close()
meters_parameters.json
{
"cilantro" : [{
"cem_093":[{
"kwh":{"function":"3","address":"286","length":"2"},
"serial_number":{"function":"3","address":"298","length":"2"},
"slave_id":{"function":"3","address":"15","length":"2"}
}]
}],
"elmeasure" : [{
"lg1119_d":[{
"kwh":{"function":"3","address":"286","length":"2"},
"serial_number":{"function":"3","address":"298","length":"2"},
"slave_id":{"function":"3","address":"15","length":"2"}
}]
}]
}
loads expects a string not a file handle. You need json.load:
import json
with open('meters_parameters.json') as f:
data = json.load(f)
print data
You're trying to load the file object, when you want to load everything in the file. Do:
data = json.loads(json_data.read())
.read() gets everything from the file and returns it as a string.
A with statement is much more pythonic here as well:
with open('meters_parameters.json') as myfile:
data = json.loads(myfile.read())