Develop Reference

Correct way to write list of json to file in Python

Using Python, I'm pulling json data from an API - every time I get a response, I add it to a list named api_responses like so:
api_responses = [{'id':'1', 'first_name':'John', 'last_name':'Smith'},{'id':'2', 'first_name':'Chris', 'last_name':'Clark'}]
I need to write the records to a file so that it's only the records and not a list:
api_responses.json:
{'id':'1', 'first_name':'John', 'last_name':'Smith'}
{'id':'2', 'first_name':'Chris', 'last_name':'Clark'}
I could for-loop the list, write each row and add '\n'.
with open('api_responses.json', 'w') as outfile:
for response in api_responses:
json.dump(response, outfile)
outfile.write('\n')
I don't want to indent or pretty the json - just make it flat as possible. Is there a better way to do this? Is jsonl/jsonlines what I am searching for?

Your code already correctly outputs the JSON objects as individual lines into a file, also known as JSONL as you pointed out.
To make the code slightly more concise and readable, you can take advantage of the print function, which outputs a newline character for you by default:
with open('api_responses.json', 'w') as outfile:
for response in api_responses:
print(json.dumps(response), file=outfile)
With your sample input, this outputs:
{"id": "1", "first_name": "John", "last_name": "Smith"}
{"id": "2", "first_name": "Chris", "last_name": "Clark"}

You are looking for the * decorator for lists on print:
>>> api_responses = [{'id':'1', 'first_name':'John', 'last_name':'Smith'},
{'id':'2', 'first_name':'Chris', 'last_name':'Clark'}]
>>> print(*api_responses,sep="\n", file=open('filename.txt','w'))
{'id': '1', 'first_name': 'John', 'last_name': 'Smith'}
{'id': '2', 'first_name': 'Chris', 'last_name': 'Clark'}
Note that this answer solves perfectly what OP is asking. Open the file filename.txt and see that matches with op requests.
Note: request is to use simple quote ', and this answer match the requierement.

converting a variable length lists into dict with some filters

#I have these two lists and i want to convert these lists such that it shows the following output :
l=['SDL-A-19.0.0-1-WX-ExP.tar','SDL-B-19.0.0-1-WX-ExP.tar','SDL-C-19.0.0-1-WX-ExP.tar','ReadMe']
s=['version','name']
#required output :
{"version": "19.0.0-1", "name": "A"}, {"version": "19.0.0-1", "name": "B"},
{"version": "19.0.0-1", "name": "C"}
Kindly suggest some solutions

You probably need a regex to match the different names those files can take. Based on the example you've provided, you could probably do something along the lines of:
import re
out = []
for s in l:
try:
pat = r'^[A-Z]{3}-(?P<name>[A-Z])-(?P<version>(?:\d+\.){2}\d+-\d+).*?'
match= re.search(pat, s)
out.append({'version':match.group('version'), 'name':match.group('name')})
except AttributeError:
pass
print(out)
[{'vesion': '19.0.0-1', 'name': 'A'}, {'vesion': '19.0.0-1', 'name': 'B'},
{'vesion': '19.0.0-1', 'name': 'C'}]
Though you might need to tweak it a little depending on the actual file names you might have. See the regex demo for the above cases.

You can try the below but this has the following assumptions.
The last element of l is always "Readme" and should be ignored.
The positions of the version and name in l are consistent.
b=[]
for i in range(len(l)-1):
a ={}
a[s[0]] = l[i][6:14]
a[s[1]] = l[i][4]
b.append(a)
# b is your final list of dictionaries
print(b)

Dump each element of list on new line (Python & JSON)

I have a list of dictionaries of the form:
mylist = [{'name': 'Johnny', 'surname': 'Cashew'}, {'name': 'Abraham', 'surname': 'Linfield'}]
and I am trying to dump that to a json this way:
with open('myfile.json', 'w') as f:
json.dump(mylist, f)
but then the entire list is on one line in the json file, making it hardly readable (my list is in reality very long). Is there a way to dump each element of my list on a new line? I have seen this post that suggests using indent in this way:
with open('myfile.json', 'w') as f:
json.dump(mylist, f, indent=2)
but then I get each element within the dictionaries on a new line, like that:
[
{
'name': 'Johnny',
'surname: 'Cashew'
},
{
'name': 'Abraham',
'surname: 'Linfield'
}
]
whereas what I am hoping to obtain is something like that:
[
{'name': 'Johnny', 'surname': 'Cashew'},
{'name': 'Abraham', 'surname': 'Linfield'}
]
Would someone have a hint? Many thanks!

This is a dirty way of doing it but it works for me
import json
my_list = [{'name': 'John', 'surname': 'Doe'}, {'name': 'Jane', 'surname': 'Doe'}]
with open('names.json','w') as f:
f.write('[\n')
for d in my_list:
#dumps() instead of dump() so that we can write it like a normal str
f.write(json.dumps(d))
if d == my_list[-1]:
f.write("\n")
break
f.write(",\n")
f.write(']')

Printing Values in Dictionaries

I cannot get values inside my python dictionairy, created from a json file, to print at all. Here is my JSON:
{
"Questions": [
{
"Q1":"What is capital of egypt?",
"T":"London",
"2":"France",
"3":"Egypt"
},
{
"Q2":"What is capital of USA?",
"T":"London",
"2":"France",
"3":"Egypt"
}
]
}
And here is my python:
import json
with open("questions.json") as f:
data = json.load(f)
print(data["Questions"]["Q1"])
This returns the error: list indices must be integers or slices, not str
What am I doing wrong? I have checked the syntax for printing and it all seems correct.

A good strategy is to start by removing code until it works. For example, instead of
print(data["Questions"]["Q1"])`
You should try
print(data["Questions"])`
The output from this is
[{'Q1': 'What is capital of egypt?', 'T': 'London', '2': 'France', '3': 'Egypt'},
{'Q2': 'What is capital of USA?', 'T': 'London', '2': 'France', '3': 'Egypt'}]
From there, you can try to index like you were doing in the question:
[{'Q1': 'What is capital of egypt?', 'T': 'London', '2': 'France', '3': 'Egypt'},
{'Q2': 'What is capital of USA?', 'T': 'London', '2': 'France', '3': 'Egypt'}]['Q1']
Except that doesn't make sense, because you're trying to index a list, and indexes in a list are ints, not strings.
So to get the result you're expecting, you should use data["Questions"][0] to index the first question.
A better solution, in my opinion, is to change the structure of the JSON to something that makes a little more sense. This is how I would do it:
{
"Questions": {
"Q1": {
"Q":"What is capital of egypt?",
"T":"London",
"2":"France",
"3":"Egypt"
},
"Q2" : {
"Q":"What is capital of USA?",
"T":"London",
"2":"France",
"3":"Egypt"
}
}
}
Now you can perform a lookup with Q1 or Q2 just like you expect.

The value of Questions is a list. Like the error says, the index must be an integer.
The first question is going to be data["Questions"][0], the second is data["Questions"][1] and so on.
If you want the value of "Q1" for the first question, you need to use data["Questions"][0]["Q1"]

Try this piece of code:
import json
with open("questions.json") as f:
data = json.load(f)
print(data["Questions"][0]["Q1"], data["Questions"][1]["Q1"])

How to parse json with ijson and python

I have JSON data as an array of dictionaries which comes as the request payload.
[
{ "Field1": 1, "Feld2": "5" },
{ "Field1": 3, "Feld2": "6" }
]
I tried ijson.items(f, '') which yields the entire JSON object as one single item. Is there a way I can iterate the items inside the array one by one using ijson?
Here is the sample code I tried which is yielding the JSON as one single object.
f = open("metadatam1.json")
objs = ijson.items(f, '')
for o in objs:
print str(o) + "\n"
[{'Feld2': u'5', 'Field1': 1}, {'Feld2': u'6', 'Field1': 3}]

I'm not very familiar with ijson, but reading some of its code it looks like calling items with a prefix of "item" should work to get the items of the array, rather than the top-level object:
for item in ijson.items(f, "item"):
# do stuff with the item dict