Develop Reference

How to properly use POST in an API request for Earth Explorer

So first of all thanks, I'm really new to python and I am trying to understand APIs, I'm currently trying to log in into /inventory/json/v/1.4.1/<request_code>?jsonRequest=<json_request_content> which is the Earth Explorer API, and the first step is to Login, and according to the documentation I am supposed to use POST instead of GET, so here is what I got so far, and it works but this is what I
import requests
import requests
user = 'xxxxx'
psword = 'xxxxx'
input_data= {'username':user,'password':psword,'catalogId':'EE'}
test=requests.post('https://earthexplorer.usgs.gov/inventory/json/v/1.4.0/login?jsonRequest=input_data)')
print(test.text)
print(test.status_code)
{
"errorCode": "AUTH_ERROR",
"error": "Passing credentials via URL is not permitted - use a POST request",
"data": null,
"api_version": "",
"access_level": "guest",
"executionTime": 0
}
200
I have no idea what to do, thank you so much. This is the documentation for the earth explorer API, thank you so much https://earthexplorer.usgs.gov/inventory/documentation/json-api?version=1.4.1#login

I have encountered the same problem while working with Earth Explorer API and managed to solve it by reading usgs package code. Basically, the problem is that you must send the request body as a string. That is, your request body must look like this when printed
{
"jsonRequest": "{\"username\": \"???\", \"password\": \"???\", \"catalogId\": \"???\"}"
}
You can achieve this using
import json
import requests
req_params = {
'username': '???',
'password': '???',
'catalogId': '???',
}
req_txt = json.dumps(req_params)
req_body = {
'jsonRequest': req_txt,
}
resp = requests.post('<LOGIN URL HERE>', req_body)
This code is actually taken from usgs package I mentioned, so you should refer to it if you have any additional questions.

<Twilio/Python> jsonify call logs

I am following the Twilio tutorial to setup an iOS project. Since it requires a backend, I have opted for Python (I am an iOS developer with 'zero' knowledge about Python) So the question might probably be a dumb one, but I do not get the proper syntax to look for.
Objective:
To fetch all the call logs and conference logs & return them as JSON response.
My Code:
#app.route('/getRecordings', methods=['GET'])
def getRecordings():
client = Client(ACCOUNT_SID, ACCOUNT_AUTH_TOKEN)
recordings = []
for record in client.calls.list():
recordings.append(record.sid)
conferences = []
for conf in client.conferences.list():
conferences.append(conf.sid)
return jsonify(calls=recordings, conferences=conferences)
Response:
I am getting the proper response, since I have appended only the SID property of each call.
{
"calls": [
"CAxxx",
"CAxxx",
],
"conferences": [
"CFxxx",
"CFxxx",
]
}
But I would like to get the complete details of each record as shown in the output tab of this sample from Twilio (Reference: https://www.twilio.com/docs/api/voice/conference)
When I try to JSONify the record, it says it cannot JSONify the object of this type.
I understand that I should convert the object to a model and append it, but how can I do that? Any links or leads to help getting this sorted are much appreciated.

You need to create a list of dictionaries with the required values. Something like this:
for record in client.calls.list():
call = {"account_sid": record.account_sid, "api_version": record.api_version, "date_created": record.date_created, "etc": record.etc}
recordings.append(call)
That should give you a response as follows:
{
"calls": [
{
"accound_sid": "1234",
"api_version": "2010-04-01",
"date_created": "Wed, 18 Aug 2010 20:20:06 +0000",
"etc": "etc",
},
{
"accound_sid": "4321",
"api_version": "2010-04-01",
"date_created": "Wed, 18 Aug 2010 20:20:06 +0000",
"etc": "etc",
}
]
}

Twilio developer evangelist here.
If you want the full JSON response proxied to your application, then you may find it easier to avoid the Twilio library and just make requests to the JSON API endpoints and send the response straight on.
For example, to get the list of calls with Python's Requests library you would do something like this:
#app.route('/getCalls', methods=['GET'])
def getCalls():
url = 'https://api.twilio.com/2010-04-01/Accounts/YOUR_ACCOUNT_SID/Calls/.json'
request = requests.get(url, auth=(YOUR_ACCOUNT_SID, YOUR_AUTH_TOKEN)
resp = Response(response=request.text,
status=200,
mimetype="application/json")
return resp
Let me know if that helps at all.

Converting Bottle FORMSDICT to a Python dictionary (in a thread safe way)

I just worked through the [bottle tutorial[1 and found the below helpful table (I hope I get the format right) of where what types of request attributes can be accessed
Attribute GET Form fields POST Form fields File Uploads
BaseRequest.query yes no no
BaseRequest.forms no yes no
BaseRequest.files no no yes
BaseRequest.params yes yes no
BaseRequest.GET yes no no
BaseRequest.POST no yes yes
Of course I want to try it out myself, but because Bottle data structures are special thread-safe versions, and I wanted to use json to print it in a sensible format, I wrote the following (working) test program
from bottle import run, route, request, response, template, Bottle
import uuid
import json
import os
ondersoek = Bottle()
#ondersoek.get('/x')
#ondersoek.post('/x')
def show_everything():
PythonDict={}
PythonDict['forms']={}
for item in request.forms:
PythonDict['forms'][item]=request.forms.get(item)
PythonDict['query']={}
for item in request.forms:
PythonDict['query'][item]=request.query.get(item)
#The below does not work - multipart/form-data doesn't serialize in json
#PythonDict['files']={}
#for item in request.files:
#PythonDict['files'][item]=request.files.get(item)
PythonDict['GET']={}
for item in request.GET:
PythonDict['GET'][item]=request.GET.get(item)
PythonDict['POST']={}
for item in request.POST:
PythonDict['POST'][item]=request.POST.get(item)
PythonDict['params']={}
for item in request.params:
PythonDict['params'][item]=request.params.get(item)
return json.dumps(PythonDict, indent=3)+"\n"
ondersoek.run(host='localhost', port=8080, reloader=True)
This works, I get:
tahaan#Komputer:~/Projects$ curl -G -d dd=dddd http://localhost:8080/x?q=qqq
{
"files": {},
"GET": {
"q": "qqq",
"dd": "dddd"
},
"forms": {},
"params": {
"q": "qqq",
"dd": "dddd"
},
"query": {},
"POST": {}
}
And
tahaan#Komputer:~/Projects$ curl -X POST -d dd=dddd http://localhost:8080/x?q=qqq
{
"files": {},
"GET": {
"q": "qqq"
},
"forms": {
"dd": "dddd"
},
"params": {
"q": "qqq",
"dd": "dddd"
},
"query": {
"dd": null
},
"POST": {
"dd": "dddd"
}
}
I'm quite sure that this is not thread safe because I'm copying the data one item at a time from the Bottle data structure into a Python native data structure. Right now I'm still using the default non-threaded server, but for performance reasons I would want to use a threaded server like CherryPy at some point in the future. The question therefore is How do I get data out of Bottle, or any other similar thread-safe dict into something that can be converted to JSON (easily)? Does Bottle by any chance expose a FormsDict-To-Json function somewhere?

Your code is thread safe. I.e., if you ran it in a multithreaded server, it'd work just fine.
This is because a multithreaded server still only assigns one request per thread. You have no global data; all the data in your code is contained within a single request, which means it's within a single thread.
For example, the Bottle docs for the Request object say (emphasis mine):
A thread-local subclass of
BaseRequest with a different set of attributes for each thread. There
is usually only one global instance of this class (request). If
accessed during a request/response cycle, this instance always refers
to the current request (even on a multithreaded server).
In other words, every time you access request in your code, Bottle does a bit of "magic" to give you a thread-local Request object. This object is not global; it is distinct from all other Request objects that may exist concurrently, e.g. in other threads. As such, it is thread safe.
Edit in response to your question about PythonDict in particular: This line makes your code thread-safe:
PythonDict={}
It's safe because you're creating a new dict every time a thread hits that line of code; and each dict you're creating is local to the thread that created it. (In somewhat more technical terms: it's on the stack.)
This is in contrast to the case where your threads were sharing a global dict; in that case, your suspicion would be right: it would not be thread-safe. But in your code the dict is local, so no thread-safety issues apply.
Hope that helps!

As far as I can see there's no reason to believe that there's a problem with threads, because your request is being served by Bottle in a single thread. Also there are no asynchronous calls in your own code that could spawn new threads that access shared variables.

Need Example of passing Jasper Reports Parameters for REST v2 API using JSON

When I look at the documentation for passing parameters to the Jasper Report REST 2 API here: http://community.jaspersoft.com/documentation/jasperreports-server-web-services-guide/v550/running-report-asynchronously I see that I need to have a "parameters" dict. The example in the link shows the XML which is not all that useful since it's unclear exactly what the equivalent JSON should look like. The closest I could find is in this link: http://community.jaspersoft.com/documentation/jasperreports-server-web-services-guide/v56/modifying-report-parameters. Now, I am sending the equivalent of that to the server (and every other permutation I can think of), and I continue to get a "400 Client Error: Bad Request" back. I could really use an exact example of the python code to generate the required "parameters" parameter for say "my_parameter_1="test_value_1".
Here is my current POST data (with a few params missing for brevity). I know this is correct since the report works fine if I omit the "parameters" parameter:
{
'outputFormat': 'pdf',
'parameters': [{'name': 'ReportID', 'value': ['my_value_1']}],
'async': 'true',
'pages': '',
'interactive': 'false'
}

Nice Job there Staggart. I got it now. Because I wasn't reading with max. scrutinity, I wasted some additional time. So the interested coder is not only advised to be aware of the nested, syntactictally interesting reportParameter-property, but especially that the value-property inside that is an array. I suppose one could pass some form of Lists/Arrays/Collections here?
What irritated me was, if I should construct more than one "reportParameter" property, but that would be nonsense according to
Does JSON syntax allow duplicate keys in an object.
So just for the record, how to post multiple parameters:
{
"reportUnitUri": "/reports/Top10/Top10Customers",
"async": true,
"freshData": true,
"saveDataSnapshot": false,
"outputFormat": "pdf",
"interactive": false,
"ignorePagination": true,
"parameters": {
"reportParameter": [
{
"name": "DATE_START_STRING",
"value": ["14.07.2014"]
},
{
"name": "DATE_END_STRING",
"value": ["14.10.2014"]
}
]
}
}
If someone accidently is struggling with communicating with jasper via REST and PHP. Do yourself a favour and use the Requests for PHP instead of pure CURL. It even has a fallback for internally using Sockets instead of CURL, when latter isn't available.
Upvote for you Staggart.

OK, thanks to rafkacz1 # http://community.jaspersoft.com/questions/825719/json-equivalent-xml-post-reportexecutions-rest-service who posted an answer, I figured it out. As he report there, the required format is:
"parameters":{
"reportParameter":[
{"name":"my_parameter_1","value":["my_value_1"]}
]
}
Pay particular attention to the plurality of "reportParameter".

Here is an example that worked for me. Im using Python 2.7, and the community edition of Jaspersoft. Like the C# example above, this example also uses the rest v2 which made it very simple for me to download a pdf report quickly
import requests
sess = requests.Session()
auth = ('username', 'password')
res = sess.get(url='http://your.jasper.domain:8080/jasperserver/', auth=auth)
res.raise_for_status()
url = 'http://your.jasper.domain:8080/jasperserver/rest_v2/reports/report_folder/sub_folder/report_name.pdf'
params = {'Month':'2', 'Year':'2017','Project': 'ProjectName'}
res = sess.get(url=url, params=params, stream=True)
res.raise_for_status()
path = '/path/to/Downloads/report_name.pdf'
with open(path, "wb") as f:
f.write(res.content)

Here's a full example about generate a report using Rest V2, in my case it's running on C#:
try {
var server = "http://localhost:8080/jasperserver";
var login = server + "/rest/login";
var report = "/rest_v2/reports/organization/Reports/report_name.pdf";
var client = new WebClient();
//Set the content type of the request
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
//Set the username and password
NameValueCollection parametros = new NameValueCollection();
parametros.Add("j_username", "jasperadmin");
parametros.Add("j_password", "123456");
//Request to login
client.UploadValues(login, "POST", parametros);
//Get session cookie
string session = client.ResponseHeaders.Get("Set-Cookie");
//Set session cookie to the next request
client.Headers.Add("Cookie", session);
//Generate report with parameters: "start" and "end"
var reporte = client.DownloadData(server + report + "?start=2015-10-01&end=2015-10-10");
//Returns the report as response
return File(reporte, "application/pdf", "test.pdf");
}catch(WebException e){
//return Content("There was a problem, status code: " + ((HttpWebResponse)e.Response).StatusCode);
return null;
}

QooxDoo FrontEnd + Python BackEnd ( SimpleXMLRPCServer) problems

I have tried Qooxdoo and I made a simple Python server with SimpleXMLRPCServer, with a Python test I get the data without problems, but can I get this data from Qooxdoo? I get lost, and I've searched for 3 days but didn't get solutions.
I try this:
var JSON_lista_empresas = 1000
button1.addListener("execute", function(e)
{
var rpc = new qx.io.remote.Rpc();
rpc.setServiceName("get_data");
//rpc.setCrossDomain(true);
rpc.setUrl("http://192.168.1.54:46000");
rpc.addListener("completed", function(event)
{
console.log(event.getData());
});
rpc.callAsync( JSON_lista_empresas, '');
});
And I tried other options but got nothing :(
The link to files:
http://mieresdelcamin.es/owncloud/public.php?service=files&dir=%2Fjesus%2Ffiles%2FQooxdoo
I tried and read all of qooxdoo-contrib.
Well,
RpcPython --> Ok
and in class/qooxdoo -> test.py
run server [start-server.py] and query from webroser:
http://127.0.0.1:8000//?_ScriptTransport_id=1&nocache=1366909868006&_ScriptTransport_data={%22service%22%3A%22qooxdoo.test%22%2C%22method%22%3A%22echo%22%2C%22id%22%3A1%2C%22params%22%3A[%22Por%20fin%22]}
and the reply in webroser is:
qx.io.remote.ScriptTransport._requestFinished(1,{"error": null, "id": 1, "result": "Client said: [ Por fin ]"});
but if i query from qooxdoo like the reply is [error.png]
The code for qooxdoo:
var rpc = new qx.io.remote.Rpc( "http://127.0.0.1:8000/");
rpc.setCrossDomain( true);
rpc.setServiceName( 'qooxdoo.test');
// asynchronous call
var handler = function(result, exc) {
if (exc == null) {
alert("Result of async call: " + result);
} else {
alert("Exception during async call: " + exc+ result);
}
};
rpc.callAsync(handler, "echo", "Por fin");
I lost :((
Files in:
http://mieresdelcamin.es/owncloud/public.php?service=files&dir=%2Fjesus%2Ffiles%2FQooxdoo
Well, with Firebug this error in owncloud qx.io.remote.ScriptTransport.....is detect
¿?.............
Best Regards.

I'm guessing you confuse XML-RPC with JSON-RPC and qooxdoo only supports the latter. These protocols are similar but the data interchange format is different (XML or JSON). Instead of the SimpleXMLRPCServer you could use "RpcPython" on the server side which is a qooxdoo contrib project.
See:
http://qooxdoo.org/contrib/project/rpcpython
http://sourceforge.net/p/qooxdoo-contrib/code/HEAD/tree/trunk/qooxdoo-contrib/RpcPython/
Once you have this server up and running you should be able to test it:
http://manual.qooxdoo.org/2.1.1/pages/communication/rpc_server_writer_guide.html#testing-a-new-server
http://sourceforge.net/p/qooxdoo-contrib/code/HEAD/tree/trunk/qooxdoo-contrib/RpcPython/trunk/services/class/qooxdoo/test.py
After that your qooxdoo (client) code hopefully works also. :)

Ok,
In the file http.py of qxjsonrc module in the line 66 change
response='qx.io.remote.ScriptTransport._requestFinished(%s,%s);'%(scriptTransportID,response)
for
response='qx.io.remote.transport.Script._requestFinished(%s,%s);'%(scriptTransportID,response)
And run fine :))
This link for package modified:
http://mieresdelcamin.es/owncloud/public.php?service=files&dir=%2Fjesus%2Ffiles%2FQooxdoo
Best Regards and thanks!!!

As Richard already pointed Qooxdoo only supports its flavor of JSON-RPC.
I maintain a fork of original rpcpython called QooxdooCherrypyJsonRpc. The main goal was to hand over transport protocol to some robust framework, and leave only JSON RPC stuff. CherryPy, obviously a robust framework, allows HTTP, WSGI and FastCGI deployment. Code was refactored and covered with tests. Later I added upload/download support and consistent timezone datetime interchange.
At very minimum your Python backend may look like (call it test.py):
import cherrypy
import qxcpjsonrpc as rpc
class Test(rpc.Service):
#rpc.public
def add(self, x, y):
return x + y
config = {
'/service' : {
'tools.jsonrpc.on' : True
},
'/resource' : {
'tools.staticdir.on' : True,
'tools.staticdir.dir' : '/path/to/your/built/qooxdoo/app'
}
}
cherrypy.tools.jsonrpc = rpc.ServerTool()
if __name__ == '__main__':
cherrypy.quickstart(config = config)
Then you can do in your qooxdoo code as follows:
var rpc = new qx.io.remote.Rpc();
rpc.setServiceName('test.Test');
rpc.setUrl('http://127.0.0.1:8080/service');
rpc.setCrossDomain(true); // you need this for opening app from file://
rpc.addListener("completed", function(event)
{
console.log(event.getData());
});
rpc.callAsyncListeners(this, 'add', 5, 7);
Or open the link directly:
http://127.0.0.1:8080/service?_ScriptTransport_id=1&_ScriptTransport_data=%7B%22params%22%3A+%5B12%2C+13%5D%2C+%22id%22%3A+1%2C+%22service%22%3A+%22test.Test%22%2C+%22method%22%3A+%22add%22%7D
For more info take a look at the package page I posted above.

Richard Sternagel wrote about rpcpython. This version of rpcpython doesn't work with present version of simplejson. Becouse in json.py have incorrect import:
from simplejson.decoder import ANYTHING
from simplejson.scanner import Scanner, pattern
Improve rpcpython or use another server, for example CherryPy.