Develop Reference

How to insert data in CSV file?

I have a problem with my Python script.
Basically what my script does is first it does a GET request to our API and extracts all ID's from the endpoint and then saves it in a CSV file.
Atm im having problems inserting data in the csv file. What i want is for my csv file to look like this after inserting data in:
id
1
2
3
...
Basically I want every id in their own row.
But what ends up being inserted is this:
id
1,2,3,...
I have tried for looping and few other things and nothing seemed to work out. I would love if anyone can help me with this problem. It's probably something really simple I just missed out.
My script code:
import requests
import json
import csv
from jsonpath_ng import jsonpath, parse
url = 'url'
headers = {
"Authorization": "Bearer token"
}
response = requests.get(url_v1, headers=headers)
JsonResponse = response.json()
converted = json.dumps(JsonResponse)
Data = json.loads(converted)
ParseData = parse('$..id')
Id = ParseData.find(Data)
open_file = open('C:/File/test.csv','w', newline='')
writer = csv.writer(open_file)
list_id = []
fields = ['id']
for i in range(0, len(Id)):
result = Id[i].value
list_id.append(result)
writer.writerow(fields)
writer.writerow(list_id)
open_file.close()

TypeError: expected str, bytes or os.PathLike object, not dict

This is my code:
from os import rename, write
import requests
import json
url = "https://api.github.com/search/users?q=%7Bquery%7D%7B&page,per_page,sort,order%7D"
data = requests.get(url).json()
print(data)
outfile = open("C:/Users/vladi/Desktop/json files Vlad/file structure first attemp.json", "r")
json_object = json.load(outfile)
with open(data,'w') as endfile:
endfile.write(json_object)
print(endfile)
I want to call API request.
I want to take data from this URL: https://api.github.com/search/users?q=%7Bquery%7D%7B&page,per_page,sort,order%7D,
and rewrite it with my own data which is my file called file structure first attemp.json
and update this URL with my own data.

import requests
url = "https://api.github.com/search/usersq=%7Bquery%7D%7B&page,per_page,sort,order%7D"
data = requests.get(url)
with open(data,'w') as endfile:
endfile.write(data.text)
json.loads() returns a Python dictionary, which cannot be written to a file. Simply write the returned string from the URL.
response.json() is a built in feature that requests uses to load the JSON returned from the URL. So you are loading the JSON twice.

How to get response data of a pdf file in python

I am trying to fetch the data of a pdf file available online
I have tried
import requests
response = requests.get("http://imdagrimet.gov.in/sites/default/files/daas_bulletin/District%20Advisory%20patna_17.pdf")
print(response.content)
but it gives a byte object as a response, and I am not able to decode that

You should write the data inside a file in order to be able to get it.
Like this:
with open('/District_Advisory_patna_17.pdf', 'wb') as f:
f.write(response.content)

Try to write your data to file:
import requests
import shutil
url = 'your url'
r = requests.get(url, stream=True)
if r.status_code == 200:
with open(file_path, 'wb') as f:
r.raw.decode_content = True
shutil.copyfileobj(r.raw, f)

Incrementally Append to JSON File in a For Loop

Is there a way to append single JSON objects to a json file while in a for loop in python. I would prefer not store all my data in one giant json object and dump it all at once, as I am planning on performing millions of API requests. I would like to make a single API request, dump the result into a JSON file and then move to the next API request and dump that into the same JSON file.
The below code overwrites the JSON file, I am looking for something that appends.
for url in urls:
r = sesh.get(url)
data = r.json()
with open('data.json', 'w') as outfile:
json.dump(data, outfile)
Such that:
with open('data.json') as outfile:
data = json.load(data, outfile)
type(data)
>> dict
r.json looks something like this:
{'attribute1':1, 'attribute2':10}

Update
Well since I don't have access to your API I just placed some sample responses, in the format you supplied, inside an array.
import json
urls = ['{"attribute1":1, "attribute2":10}', '{"attribute1":67, "attribute2":32}', '{"attribute1":37, "attribute2":12}'];
json_arr = []
for url in urls:
data = json.loads(url)
json_arr.append(data)
with open('data.json', 'w') as outfile:
json.dump(json_arr, outfile)
Basically we keep an array and append each API response to that array. Then, we can write the accumulative JSON to a file. Also if you want to update the same JSON file on different executions of the code, you can just read the existing output file into an array, in the beginning of the code, and then carry on with my example.
Change write mode to append
Try changing this:
with open('data.json', 'w') as outfile:
To this:
with open('data.json', 'a') as outfile:

The previous answer is surprisingly close to what you need to do.
So I will build upon it.
import json
json_arr = ['{"attribute1":1, "attribute2":10}', '{"attribute1":67, "attribute2":32}', '{"attribute1":37, "attribute2":12}'];
with open('data.json', 'w') as outfile:
outfile.write('[')
for element in json_arr:
with open('data.json', 'w') as outfile:
json.dump(element, outfile)
outfile.write(',')
with open('data.json', 'a') as outfile:
outfile.write(']')

printing json value to file

#!/usr/bin/python
import os
import json
import urllib
import urllib2
url = "https://www.example.com"
parameters = {'resource': 'aaaa',
'apikey': '1111'}
data = urllib.urlencode(parameters)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
json_data = response.read()
with open("test.json") as json_file:
json_file.write(json_data)
print json_data
I dont see I use json again it was before json_data was used now.

As Martijn Pieters pointed out, the data is already encoded so you shouldn't need the json module at all in this case
You can just write the output to a file
json_data = response.read()
with open("test.json" , "w") as json_file:
json_file.write(json_data)

Since you already have json,
with open("test.json", "w") as f:
f.write(data)
Another thing to note here, you should not have a variable named json, because
import json
json = 'some string'
json.dumps("{'a':1}")
>> AttributeError: 'str' object has no attribute 'dumps'
if you want a variable named json you can use import json as j