Develop Reference

TypeError: invalid arguments to setopt in pycurl python flask API Development

I am trying to build a Webservice API using python flask. When I execute this code below:
http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&STUDENTNUMBER='1234'&SESSIONKEY=94194202323
...it works fine. But I could not pass STUDENTNUMBER to this function.
I have tried two ways:
Concat build a string and pass it to c.setopt(c.URL,) this function
a. Way 1
b. Way 2
c. Way 3
Via those ways I got the same error:
TypeError: invalid arguments to setopt
Pass the variable using c.setopt(c.POSTFIELDS, post_data)
a. Way 4
This way I got the same error:
Method not allowed. Please see the for constructing valid requests to the service
So that I am going to this link:
b. Way 5
This way I got the same error
TypeError: invalid arguments to setopt
Way 1:
student_url = " http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&STUDENTNUMBER=%s&SESSIONKEY=94194202323"%student_number;
c.setopt(c.URL,student_url)
Way 2:
c.setopt(c.URL,"http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&STUDENTNUMBER=%s&SESSIONKEY=94194202323"%(student_number))
Way 3:
c.setopt(c.URL,"http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&STUDENTNUMBER=%s&SESSIONKEY=94194202323"%student_number)
Way 4:
c.setopt(c.URL,"http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&STUDENTNUMBER=%s&SESSIONKEY=94194202323")
c.setopt(c.POSTFIELDS, 'STUDENTNUMBER = 1234')
Way 5:
post_data ={'STUDENTNUMBER' : '1234'}
c.setopt(c.URL,"http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&SESSIONKEY=94194202323")
c.setopt(c.POSTFIELDS, post_data)
c.setopt(pycurl.POST, 1)
How can I make this work?

This doesn't appear to be a question about Flask. It seems you're trying to write some code to query the API (which may be Flask powered).
I suggest using the Python requests library for this, as you can then define your parameters as a dictionary which is much easier. Not to be confused with Flask.request which is a different thing entirely!
import requests
url = 'http://localhost/Service/API/Services.svc/XMLService/Students'
params = {'SEARCHBY': 'StudentNo',
'STUDENTNUMBER': '1234',
'SESSIONKEY': 94194202323}
r = requests.get(url, params)
The last line above sends the request, and you can see the full URL with:
>>> print (r.url)
http://localhost/Service/API/Services.svc/XMLService/Students?SEARCHBY=StudentNo&STUDENTNUMBER=1234&SESSIONKEY=94194202323
Or print the response code, which is a 404 in my case:
>>> print(r)
<Response [404]>
If the response contains data you can access this through r.text:
>>> print (r.text)
I am the 404's response body.

Python crawler issue

I have a problem that I can't solve myself as it seems, I hope someone here might have another idea that can help me.
My plan is to crawl the data from comtrade for several countries and timeframes, but even my first call isn't working. The URL I want to send a get request to is http://comtrade.un.org/api/get?&r=32&freq=A&ps=2013&px=H4&cc=AG6&type=C&rg=2&p=0&head=M and if I enter this data in postman I get a proper response with plenty datasets, but if I try to from python I get the response
"{'Message': 'Empty parameters or null values are not permitted. For more information please visit http://comtrade.un.org/data/doc/api/'}"
instead. The API doesn't take any authentication and I didn't set any headers or did any other kind of change to postman, but there it works.
Please take a look at my code and tell me what I'm doing wrong. Did I miss something?
You can try it yourself using the above mentioned URL up to 100 times per hour, maybe you find a way to do so :)
My code:
import json
import requests
url = "http://comtrade.un.org/api/get?&r=32&freq=A&ps=2013&px=H4&cc=AG6&type=C&rg=2&p=0&head=M"
f = requests.get(url, timeout=300)
x = json.loads(f.text)
print(x)

The url is malformed, you should replace the ?& with ?, so the correct url becomes:
https://comtrade.un.org/api/get?r=32&freq=A&ps=2013&px=H4&cc=AG6&type=C&rg=2&p=0&head=M

import json
import requests
url = "http://comtrade.un.org/api/get?r=32&freq=A&ps=2013&px=H4&cc=AG6&type=C&rg=2&p=0&head=M"
f = requests.get(url, timeout=300)
x = json.loads(f.text)
print(x)
Hope it helps.

Python request resulting in blank response

I'm relatively new to Python so would like some help, I've created a script which simply use the request library and basic auth to connect to an API and returns the xml or Json result.
# Imports
import requests
from requests.auth import HTTPBasicAuth
# Set variables
url = "api"
apiuser = 'test'
apipass = 'testpass'
# CALL API
r = requests.get(url, auth=HTTPBasicAuth(apiuser, apipass))
# Print Statuscode
print(r.status_code)
# Print XML
xmlString = str(r.text)
print(xmlString)
if but it returns a blank string.
If I was to use a browser to call the api and enter the cretentials I get the following response.
<Response>
<status>SUCCESS</status>
<callId>99999903219032190321</callId>
<result xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Dummy">
<authorFullName>jack jones</authorFullName>
<authorOrderNumber>1</authorOrderNumber>
</result>
</Response>
Can anyone tell me where I'm going wrong.

What API are you connecting to?
Try adding a user-agent to the header:
r = requests.get(url, auth=HTTPBasicAuth(apiuser, apipass), headers={'User-Agent':'test'})

Although this is not an exact answer for the OP, it may solve the issue for someone having a blank response from python-requests.
I was getting a blank response because of the wrong content type. I was expecting an HTML rather than a JSON or a login success. The correct content-type for me was application/x-www-form-urlencoded.
Essentially I had to do the following to make my script work.
data = 'arcDate=2021/01/05'
headers = {
'Content-Type': 'application/x-www-form-urlencoded',
}
r = requests.post('https://www.deccanherald.com/getarchive', data=data, headers=headers)
print(r.status_code)
print(r.text)
Learn more about this in application/x-www-form-urlencoded or multipart/form-data?

Run this and see what responses you get.
import requests
url = "https://google.com"
r = requests.get(url)
print(r.status_code)
print(r.json)
print(r.text)
When you start having to pass things in your GET, PUT, DELETE, OR POST requests, you will add it in the request.
url = "https://google.com"
headers = {'api key': 'blah92382377432432')
r = requests.get(url, headers=headers)
Then you should see the same type of responses. Long story short,
Print(r.text) to see the response, then you once you see the format of the response you get, you can move it around however you want.

I have an empty response only when the authentication failed or is denied.
The HTTP status is still ≤ 400.
However, in the header you can find :
'X-Seraph-LoginReason': 'AUTHENTICATED_FAILED'
or
'X-Seraph-LoginReason': 'AUTHENTICATED_DENIED'

If the request is empty, not even a status code I could suggest waiting some time between printing. Maybe the server is taking time to return the response to you.
import time
time.sleep(5)
Not the nicest thing, but it's worth trying
How can I make a time delay in Python?
I guess there are no errors during execution
EDIT: nvm, you mentioned that you got a status code, I thought you were literally geting nothing.
On the side, if you are using python3 you have to use Print(), it replaced Print

HTTP Delete with python requests module

I would like to do a HTTP DELETE with python requests module that follows the API below;
https://thingspeak.com/docs/channels#create
DELETE https://api.thingspeak.com/channels/4/feeds
api_key=XXXXXXXXXXXXXXXX
I am using python v2.7 and requests module. My python code looks like this;
def clear(channel_id):
data = {}
data['api_key'] = 'DUCYS8xufsV613VX'
URL_delete = "http://api.thingspeak.com/channels/" + str(channel_id) + "/feeds"
r = requests.delete(URL_delete, data)
The code does not work because requests.delete() can only accept one parameter. How should the correct code look like?

You want
import json
mydata = {}
mydata['api_key'] = "Jsa9i23jka"
r = requests.delete(URL_delete, data=json.dumps(mydata))
You have to use the named input, 'data', and I'm guessing that you actually want JSON dumped, so you have to convert your dictionary, 'mydata' to a json string. You can use json.dumps() for that.
I don't know the API you are using, but by the sound of it you actually want to pass URL parameter, not data, for that you need:
r = requests.delete(URL_delete, params=mydata)
No need to convert mydata dict to a json string.

You can send the data params as #Eugene suggested, but conventionally delete requests only contains url and nothing else. The reason is that a RESTful url should uniquely identify the resource, thereby eliminating the need to provide additional parameters for deletion. On the other hand, if your 'APIKEY' has something to do with authentication, then it should be part of headers instead of request data, something like this.
headers = {'APIKEY': 'xxx'}
response = requests.delete(url, data=json.dumps(payload), headers=headers)

Put json data into url with python

I am trying to encode a json parameter within a url for use with the mongolab restAPI.
My url looks something like this
url = 'https://api.mongolab.com/api/1/databases/db/collections/coll?q={"q": "10024"}&apiKey=mykey
I am trying to open it using
urllib2.urlopen(url)
but I run into errors saying that my apikey is incorrect. I know this isn't true because if I copy and paste the url into my browser I get a correct response. I also know that I can access the rest api as long as I don't have the query there (so it must be a json/formatting problem).
So does anyone know how I could encode the json query
{"q": "10024"}
into the url? Thanks!

You'll have to properly URL-encode the string. Use the urllib.quote_plus() function:
url = 'https://api.mongolab.com/api/1/databases/db/collections/coll?q={q}&apiKey={key}'
query = urllib.quote_plus('{"q": "10024"}')
urllib2.urlopen(url.format(q=query, key=your_api_key))

You could also use the requests library.
Example:
import requests
payload = {'q': '10024', 'apiKey': 'mykey'}
r = requests.get("https://api.mongolab.com/api/1/databases/db/collections/coll", params=payload)
print(r.url)
Output:
https://api.mongolab.com/api/1/databases/db/collections/coll?q=10024&apiKey=mykey