I'm transferring my movie ratings from IMDB to Trakt. I use a Python script to do so and can't get it to turn my list into serializable JSON.
My script consists of a JSON uploader and an CSV reader, both work fine separately.
I've looked into list vs. tuple, json.dumps options and syntax and into json.encoder. There is a lot on the topic available online but no complete CSV to JSON example.
The following script includes all steps and a few lines of example data. If you want to test this script, you need the username, pass-SHA1 and API key of your Trakt account.
Current Error:
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: `enter code here`set(['["tt1535108", "Elysium", "8", "2013"]']) is not JSON
serializable
#===============================================================================
# Used CSV file (imdb_ratings.csv)
#===============================================================================
# position,const,created,modified,description,Title,Title type,Directors,You rated,IMDb Rating,Runtime (mins),Year,Genres,Num. Votes,Release Date (month/day/year),URL
# 1,tt1683526,Sat Feb 1 00:00:00 2014,,,Detachment,Feature Film,Tony Kaye,8,7.7,97,2011,drama,36556,2011-04-25,http://www.imdb.com/title/tt1683526/
# 2,tt1205537,Wed Jan 29 00:00:00 2014,,,Jack Ryan: Shadow Recruit,Feature Film,Kenneth Branagh,6,6.6,105,2014,"action, mystery, thriller",11500,2014-01-15,http://www.imdb.com/title/tt1205537/
# 3,tt1535108,Tue Jan 28 00:00:00 2014,,,Elysium,Feature Film,Neill Blomkamp,8,6.7,109,2013,"action, drama, sci_fi, thriller",176354,2013-08-07,http://www.imdb.com/title/tt1535108/
#===============================================================================
# Imports etc.
#===============================================================================
import csv
import json
import urllib2
ifile = open('imdb_ratings.csv', "rb")
reader = csv.reader(ifile)
included_cols = [1, 5, 8, 11]
#===============================================================================
# CSV to JSON
#===============================================================================
rownum = 0
for row in reader:
# Save header row.
if rownum == 0:
header = row
else:
content = list(row[i] for i in included_cols)
print(content)
rownum += 1
ifile.close()
#===============================================================================
# POST of JSON
#===============================================================================
data = {
"username": "<username>",
"password": "<SHA1>",
"movies": [
{
# Expected format:
# "imdb_id": "tt0114746",
# "title": "Twelve Monkeys",
# "year": 1995,
# "rating": 9
json.dumps(content)
}
]
}
req = urllib2.Request('http://api.trakt.tv/rate/movies/<api>')
req.add_header('Content-Type', 'application/json')
response = urllib2.urlopen(req, json.dumps(data))
Construct the dict:
{
"imdb_id": "tt0114746",
"title": "Twelve Monkeys",
"year": 1995,
"rating": 9
}
instead of calling json.dumps(content), which creates a string.
You could create the list of dicts using a list comprehension and a dict comprehension:
movies = [{field:row[i] for field, i in zip(fields, included_cols)} for row in reader]
import csv
import json
import urllib2
with open('imdb_ratings.csv', "rb") as ifile:
reader = csv.reader(ifile)
next(reader) # skip header row
included_cols = [1, 5, 8, 11]
fields = ['imdb_id', 'title', 'rating', 'year']
movies = [{field: row[i] for field, i in zip(fields, included_cols)}
for row in reader]
data = {"username": "<username>",
"password": "<SHA1>",
"movies": movies}
req = urllib2.Request('http://api.trakt.tv/rate/movies/<api>')
req.add_header('Content-Type', 'application/json')
response = urllib2.urlopen(req, json.dumps(data))
Related
I have a .json file , the first few lines are :
{
"global_id": "HICO_train2015_00000001",
"hois": [
{
"connections": [
[
0,
0
]
],
"human_bboxes": [
[
207,
32,
426,
299
]
],
"id": "153",
"invis": 0,
"object_bboxes": [
[
58,
97,
571,
404
]
]
},
I want to print out human_bboxes. id and object_bboxes.
I tried this code:
import json
# Opening JSON file
f = open('anno_list.json',)
# returns JSON object as
# a dictionary
data = json.load(f)
# Iterating through the json
# list
s=data[0]
for i in s:
print(i[1])
# Closing file
f.close()
But, it gave me this output:
l
o
m
m
Do this:
import json
# Opening JSON file
f = open('anno_list.json',)
# returns JSON object as
# a dictionary
data = json.load(f)
# Iterating through the json
# list
s=data[0]
# Do This:
hois_data = s["hois"][0]
print("human_bboxes",hois_data["human_bboxes"])
print("id",hois_data["id"])
print("object_bboxes",hois_data["object_bboxes"])
# Closing file
f.close()
The answer by Behdad Abdollahi Moghadam would print the answer correctly, but only for one set of bboxes and id. The below answer additionally has a for loop which parses the entire file and prints all the human and object bboxes and id into a file.
import json
# Opening JSON file
f = open('anno_list.json',)
# returns JSON object as
# a dictionary
data = json.load(f)
f1 = open("file1.txt", "w")
for annotation in data:
f1.write("==============\n")
f1.write(annotation["global_id"])
for hois in annotation["hois"]:
f1.write("\n")
f1.write("---")
f1.write("\n")
f1.write(hois["id"])
f1.write("\n")
f1.write(str(hois["human_bboxes"]))
f1.write("\n")
f1.write(str(hois["object_bboxes"]))
f1.write("\n")
# Closing file
f.close()
f1.close()
I am trying to convert a CSV file to JSON but there is a header in my csv that is empty. Is there a way to name it when outputting it to JSON?
Example data
"" Calories Fat Sodium
Bread 100 10 23
I got this code from geeksforgeeks
import csv
import json
# Function to convert a CSV to JSON
# Takes the file paths as arguments
def make_json(csvFilePath, jsonFilePath):
# create a dictionary
data = {}
# Open a csv reader called DictReader
with open(csvFilePath, encoding='utf-8') as csvf:
csvReader = csv.DictReader(csvf)
# Convert each row into a dictionary
# and add it to data
for rows in csvReader:
# Assuming a column named 'No' to
# be the primary key
key = rows['']
data[key] = rows
# Open a json writer, and use the json.dumps()
# function to dump data
with open(jsonFilePath, 'w', encoding='utf-8') as jsonf:
jsonf.write(json.dumps(data, indent=4))
# Driver Code
# Decide the two file paths according to your
# computer system
csvFilePath = r'Names.csv'
jsonFilePath = r'Names.json'
# Call the make_json function
make_json(csvFilePath, jsonFilePath)
I did this and it gets the first row, but i'm not sure how to rename it when i output it to JSON.
It appears as "":"Bread" in the JSON file.
key = rows['']
Thanks in advance if anyone can help!
Edit: Expected output
{
"Food": "Bread",
"Calories": "45",
"Fat (g)": "0",
"Carb. (g)": "11",
"Fiber (g)": "0",
"Protein": "0",
"Sodium": "10"
}
I want to print only the number from "PresentValue". But only from "ObjectIdentifier" : 1
I need to be able to specify what "ObjectIdentifier" that is going to be printed.
Here is my json file:
import json
# Data to be written
data = {
"AnalogValues": [
{
"ObjectIdentifier": 1,
"PresentValue": 10.2
},
{
"ObjectIdentifier": 2,
"PresentValue": 20.3
}
]
}
# Serializing json
json_object = json.dumps(data, indent = 4)
# Writing to sample.json
with open("AnalogValues.json", "w") as outfile:
outfile.write(json_object)
This is what I have tried so far (returns the whole json file):
import json
# Opening JSON file
with open('AnalogValues.json', 'r') as openfile:
# Reading from json file
json_object = json.load(openfile)
print(json_object)
print(type(json_object))
You can use function like this:
def get_present_value(no):
for a in data['AnalogValues']:
if a['ObjectIdentifier'] == int(no):
return a['PresentValue']
return None
print(get_present_value(2))
Output:
20.3
I have an excel file in which data is saved in csv format in such a way.This data is present in the excel file as shown below,under column A (The CSV File is generated by LabView Software code which i have written to generate data).I have also attached an image of the csv file for reference at the end of my question.
RPM,Load Current,Battery Output,Power Capacity
1200,30,12,37
1600,88,18,55
I want to create a Json file in such format
{
"power_capacity_data" :
{
"rpm" : ["1200","1600"],
"load_curr" : ["30","88"],
"batt_output" : ["12","18"],
"power_cap" : ["37","55"]
}
}
This is my code
import csv
import json
def main():
#created a dictionary so that i can append data to it afterwards
power_data = {"rpm":[],"load_curr":[],"batt_output":[],"power_cap":[]}
with open('power1.lvm') as f:
reader = csv.reader(f)
#trying to append the data of column "RPM" to dictionary
rowcount = 0
for row in reader:
if rowcount == 0:
#trying to skip the first row
rowcount = rowcount + 1
else:
power_data['rpm'].append(row[0])
print(row)
json_report = {}
json_report['pwr_capacity_data'] = power_data
with open('LVMJSON', "w") as f1:
f1.write(json.dumps(json_report, sort_keys=False, indent=4, separators=(',', ': '),encoding="utf-8",ensure_ascii=False))
f1.close()
if __name__ == "__main__":
main()
The output json file that i am getting is this:(please ignore the print(row) statement in my code)
{
"pwr_capacity_data":
{
"load_curr": [],
"rpm": [
"1200,30,12.62,37.88",
"1600,88,18.62,55.88"
],
"batt_output": [],
"power_cap": []
}
}
The whole row is getting saved in the list,but I just want the values under the column RPM to be saved .Can someone help me out with what I may be doing wrong.Thanks in advance.I have attached an image of csv file to just in case it helps
You could use Python's defaultdict to make it a bit easier. Also a dictionary to map all your header values.
from collections import defaultdict
import csv
import json
power_data = defaultdict(list)
header_mappings = {
'RPM' : 'rpm',
'Load Current' : 'load_curr',
'Battery Output' : 'batt_output',
'Power Capacity' : 'power_cap'}
with open('power1.lvm', newline='') as f_input:
csv_input = csv.DictReader(f_input)
for row in csv_input:
for key, value in row.items():
power_data[header_mappings[key]].append(value)
with open('LVMJSON.json', 'w') as f_output:
json.dump({'power_capacity_data' : power_data}, f_output, indent=2)
Giving you an output JSON file looking like:
{
"power_capacity_data": {
"batt_output": [
"12",
"18"
],
"power_cap": [
"37",
"55"
],
"load_curr": [
"30",
"88"
],
"rpm": [
"1200",
"1600"
]
}
}
I have below data in JSON format, I have started with code below which throws a KEY ERROR.
Not sure how to get all data listed in headers section.
I know I am not doing it right in json_obj['offers'][0]['pkg']['Info']: but not sure how to do it correctly.
how can I get to different nodes like info,PricingInfo,Flt_Info etc?
{
"offerInfo":{
"siteID":"1",
"language":"en_US",
"currency":"USD"
},
"offers":{
"pkg":[
{
"offerDateRange":{
"StartDate":[
2015,
11,
8
],
"EndDate":[
2015,
11,
14
]
},
"Info":{
"Id":"111"
},
"PricingInfo":{
"BaseRate":1932.6
},
"flt_Info":{
"Carrier":"AA"
}
}
]
}
}
import os
import json
import csv
f = open('api.csv','w')
writer = csv.writer(f,delimiter = '~')
headers = ['Id' , 'StartDate', 'EndDate', 'Id', 'BaseRate', 'Carrier']
default = ''
writer.writerow(headers)
string = open('data.json').read().decode('utf-8')
json_obj = json.loads(string)
for pkg in json_obj['offers'][0]['pkg']['Info']:
row = []
row.append(json_obj['id']) # just to test,but I need column values listed in header section
writer.writerow(row)
It looks like you're accessing the json incorrectly. After you have accessed json_obj['offers'], you accessed [0], but there is no array there. json_obj['offers'] gives you another dictionary.
For example, to get PricingInfo like you asked, access like this:
json_obj['offers']['pkg'][0]['PricingInfo']
or 11 from the StartDate like this:
json_obj['offers']['pkg'][0]['offerDateRange']['StartDate'][1]
And I believe you get the KEY ERROR because you access [0] in the dictionary, which since that isn't a key, you get the error.
try to substitute this piece of code:
for pkg in json_obj['offers'][0]['pkg']['Info']:
row = []
row.append(json_obj['id']) # just to test,but I need column values listed in header section
writer.writerow(row)
With this:
for pkg in json_obj['offers']['pkg']:
row.append(pkg['Info']['Id'])
year = pkg['offerDateRange']['StartDate'][0]
month = pkg['offerDateRange']['StartDate'][1]
day = pkg['offerDateRange']['StartDate'][2]
StartDate = "%d-%d-%d" % (year,month,day)
print StartDate
writer.writerow(row)
Try this
import os
import json
import csv
string = open('data.json').read().decode('utf-8')
json_obj = json.loads(string)
print json_obj["offers"]["pkg"][0]["Info"]["Id"]
print str(json_obj["offers"]["pkg"][0]["offerDateRange"]["StartDate"][0]) +'-'+ str(json_obj["offers"]["pkg"][0]["offerDateRange"]["StartDate"][1])+'-'+str(json_obj["offers"]["pkg"][0]
["offerDateRange"]["StartDate"][2])
print str(json_obj["offers"]["pkg"][0]["offerDateRange"]["EndDate"][0]) +'-'+ str(json_obj["offers"]["pkg"][0]["offerDateRange"]["EndDate"][1])+'-'+str(json_obj["offers"]["pkg"][0]
["offerDateRange"]["EndDate"][2])
print json_obj["offers"]["pkg"][0]["Info"]["Id"]
print json_obj["offers"]["pkg"][0]["PricingInfo"]["BaseRate"]
print json_obj["offers"]["pkg"][0]["flt_Info"]["Carrier"]