I am trying to convert JSON data into a CSV in Python and found this code listed on Stack Exchange from a while back (link:How can I convert JSON to CSV?)
It no longer works in Python 3, giving me different errors. Anyone know how to fix for Python 3? Thanks.
import csv
import json
x = """[
{
"pk": 22,
"model": "auth.permission",
"fields": {
"codename": "add_logentry",
"name": "Can add log entry",
"content_type": 8
}
},
{
"pk": 23,
"model": "auth.permission",
"fields": {
"codename": "change_logentry",
"name": "Can change log entry",
"content_type": 8
}
},
{
"pk": 24,
"model": "auth.permission",
"fields": {
"codename": "delete_logentry",
"name": "Can delete log entry",
"content_type": 8
}
}
]"""
x = json.loads(x)
f = csv.writer(open("test.csv", "wb+"))
# Write CSV Header, If you dont need that, remove this line
f.writerow(["pk", "model", "codename", "name", "content_type"])
for x in x:
f.writerow([x["pk"],
x["model"],
x["fields"]["codename"],
x["fields"]["name"],
x["fields"]["content_type"]])
You were opening the file as binary with wb+ whereas you are trying to write str.
f = csv.writer(open("test.csv", "w+"))
Maybe this code will help you in some way to understand that.
import json,csv
data = []
with open('your_json_file_here.json') as file:
for line in file:
data.append(json.loads(line)) length = len(data)
with open('create_new_file.csv','w') as f:
writer = csv.writer(f)
writers = csv.DictWriter(f, fieldnames=['header1','header2'])
writers.writeheader()
for iter in range(length):
writer.writerow((data[iter]['specific_col_name1'],data[iter]['specific_col_name2'])) f.close()
Related
I am biggener with JSON
I have the following in my JSON file (file-1) :
{
"Aps": [
{
"type": "opo",
"price_min": 7,
"price_max":10,
"app_time": 0,
"arguments": {
"prices": 15,
"apps_num": "112"
},
"attributes": {
"name":"user1",
"priority":"100"
}
}
}
How write python code that generates another JSON file that contain the same content of file-1 but duplicated 100 time in each time the name of user is different user2, user3 ... user100 and also it's priority.
I have tried the following but it is not working :
for lp in range(100):
with open("sample.json", "w") as outfile:
outfile.write(json_object)
but it is not working ..
the required output is as follow :
{
"Aps": [
{
"type": "opo",
"price_min": 7,
"price_max":10,
"app_time": 0,
"arguments": {
"prices": 15,
"apps_num": "112"
},
"attributes": {
"name":"user1",
"priority":"100"
}
},
{
"type": "opo",
"price_min": 7,
"price_max":10,
"app_time": 0,
"arguments": {
"prices": 15,
"apps_num": "112"
},
"attributes": {
"name":"user2",
"priority":"90"
}
},
{
"type": "opo",
"price_min": 7,
"price_max":10,
"app_time": 0,
"arguments": {
"prices": 15,
"apps_num": "112"
},
"attributes": {
"name":"user2",
"priority":"80"
}
},
..............
}
I made a little code here using json and copy module
json for reading and writing json files
copy because I had some trouble with reference variables see documentation for copy; if I changed 'temp' dict then it would affect all occurrences in 'file' dict
import json
import copy
repeats = 100
file = json.loads(open('file.json', 'r').read())
temp1 = json.loads(open('file.json', 'r').read())['Aps'][0]
for repeat in range(repeats):
temp = copy.deepcopy(temp1)
temp['attributes']['name'] = f"user{repeat + 2}"
temp['attributes']['priority'] = f"{repeat*10+100 - repeat*20}"
file['Aps'].append(temp)
temp1 = copy.deepcopy(temp)
json.dump(file, open('file1.json', 'w'), indent=4)
You should first convert your json file to a python object (dict):
import json
file = open('sample.json')
data = json.load(file)
file.close()
Now you can do stuff with your Aps list, like appending the first object 100 times to your list.
for dups in range(100):
data['Aps'].append(data['Aps'][0])
Then you save your dict to a json file again:
with open("sample.json", "w") as outputfile:
json.dump(data, outputfile)
I'm trying to update exsisting JSON file when running my code by adding additional data (package_id). this is the exsisting json contents:
{
"1": {
"age": 10,
"name": [
"ramsi",
"jack",
"adem",
"sara",
],
"skills": []
}
}
and I want to insert a new package and should looks like this:
{"1": {
"age": 10,
"name": [
"ramsi",
"jack",
"adem",
"sara",
],
"skills": []
} "2": {
"age": 14,
"name": [
"maya",
"raji",
],
"skills": ["writing"]
}
}
Issue is when I add the new data it adds --> ({) so (one top-level value) is added twice which is not allowed by JSON standards
{"1": {
"age": 10,
"name": [
"ramsi",
"jack",
"adem",
"sara",
],
"skills": []
}} {"2": {
"age": 14,
"name": [
"maya",
"raji",
],
"skills": ["writing"]
}
}
and this is my code to add the new (package_id):
list1[package_id] = {"age": x, "name": y, "skills": z}
ss = json.dumps(list1, indent=2)
data = []
with open('file.json', 'r+') as f:
data = json.loads(f.read())
data1 = json.dumps(data, indent=2)
f.seek(0)
f.write(data1)
f.write(ss)
f.truncate()
I write to the file twice because if I didn't store existing contents and write it again then it will remove old data and keeps only package_id number 2
It doesn't work that way. You can't add to a JSON record by appending another JSON record. A JSON file always has exactly one object. You need to modify that object.
with open('file.json','r') as f:
data = json.loads(f.read())
data[package_id] = {'age':x, 'name':y, 'skills':z}
with open('file.json','w') as f:
f.write(json.dumps(data,indent=2))
I have a JSON file like this.
{
"posts": [{
"id": 0,
"title": "title1",
"author": "CIQ",
"views": 100,
"reviews": 31
}],
"authors": [{
"id": 0,
"first_name": "Commerce",
"last_name": "IQ",
"posts": 45
}]
}
I want to write the values in nested children of JSON eg. Add the following Post and save the same JSON file.
{
"id": 1,
"title": "title2",
"author": "CIQ2",
"views": 100,
"reviews": 31
}
The final file will look like.
{
"posts": [{
"id": 0,
"title": "title1",
"author": "CIQ",
"views": 100,
"reviews": 31
}, {
"id": 1,
"title": "title2",
"author": "CIQ2",
"views": 100,
"reviews": 3
}],
"authors": [{
"id": 0,
"first_name": "Commerce",
"last_name": "IQ",
"posts": 45
}]
}
I am using the below script for writing the file.
with open(file_path, 'r+') as json_file:
data = json.load(json_file)
data['posts'].append(new_post)
json_file.write(json.dumps(data, ensure_ascii=True))
problem with this script is it appends the data after the end of the file. as shown in the image.
Please suggest some apporach.
problem with this script is it appends the data after the end of the file.
Yes, that's what mode r+ means. Please read the documentation.
Please suggest some approach
Read the file with mode r, make the changes, and then overwrite the file using mode w (in a separate with block).
we can make it by reading and writing it separately.
with open(file_path, 'r') as data_file:
data = json.load(data_file)
data['posts'].append(new_post)
with open(file_path, 'w') as data_file:
data = json.dump(data, data_file)
I have a json file saved in local server, such as：
{
"user": "user1",
"id": "21779"
}
and I want to write another dict into this json file, I need new content like this:
{
{
"user": "user1",
"id": "21779"
},
{
"user": "user2",
"id": "21780"
}
}
Or:
[
{
"user": "user1",
"id": "21779"
},
{
"user": "user2",
"id": "21780"
}
]
I try to use json.dump() to add the new element, but is displayed as:
{
"user": "user1",
"id": "21779"
}
{
"user": "user2",
"id": "21780"
}
It is not a valid json file.
How can I do use json.dump, json.load, or other methods?
Thanks for help me!
You have to read your JSON file and then convert it to list instead of dict. Then you just need to append to that list and overwrite your JSON file.
import json
data = json.load(open('data.json'))
# convert data to list if not
if type(data) is dict:
data = [data]
# append new item to data lit
data.append({
"user": "user2",
"id": "21780"
})
# write list to file
with open('data.json', 'w') as outfile:
json.dump(data, outfile)
You can do with the list not with the dict , try the below one solution if its help
import json
def appendList():
with open("test.json", mode='r', encoding='utf-8') as f:
feeds = json.load(f)
print(feeds)
with open("test.json", mode='w', encoding='utf-8') as feedsjson:
entry = { "user": "user3","id": "21574"}
feeds.append(entry)
print(json.dump(feeds, feedsjson))
I've been looking around the web and I cannot find a way on adding new JSON data to array.
Example: I would like to add player_two, player_three through python.
{
"players": {
"player_one": {
"name": "Bob",
"age": "0",
"email": "bob#example.com"
}
}
}
How can I achieve doing this through python?
What I've tried:
with open("/var/www/html/api/toons/details.json", 'w') as outfile:
json.dump(avatarDetails, outfile)
Here is a simple example, read the file as a dict, update the dict, then use json.dumps() to get the json data:
import json
# open your jsonfile in read mode
with open('jsonfile') as f:
# read the data as a dict use json.load()
jsondata = json.load(f)
# add a new item into the dict
jsondata['players']['player_two'] = {'email': 'kevin#example.com', 'name': 'Kevin', 'age': '0'}
# open that file in write mode
with open('jsonfile', 'w') as f:
# write the data into that file
json.dump(jsondata, f, indent=4, sort_keys=True)
Now the file looks like:
{
"players": {
"player_one": {
"age": "0",
"email": "bob#example.com",
"name": "Bob"
},
"player_two": {
"age": "0",
"email": "kevin#example.com",
"name": "Kevin"
}
}
}
Assuming that your file contains this JSON:
{
"players": {
"player_one": {
"name": "Bob",
"age": "0",
"email": "bob#example.com"
}
}
}
You can parse the data into a Python dictionary using json.load():
with open('/var/www/html/api/toons/details.json') as f:
data = json.load(f)
Add your new players:
data['players']['player_two'] = dict(name='Bobbie', age=100, email='b#blah.com')
data['players']['player_three'] = dict(name='Robert', age=22, email='robert#blah.com')
Then save it back to a file:
with open('/var/www/html/api/toons/details.json', 'w') as f:
json.dump(data, f)