Develop Reference

Is it OK to send the whole POST as a JSON object?

I am using GAE with python, and I am using many forms. Usually, my code looks something like this:
class Handler(BaseHandler):
#...
def post(self):
name = self.request.get("name")
last_name = self.request.get("last_name")
# More variables...
n = self.request.get("n")
#Do something with the variables, validations, etc.
#Add them to a dictionary
data = dict(name=name, last_name=last_name, n=n)
info = testdb.Test(**data)
info.put()
I have noticed lately that it gets too long when there are many inputs in the form (variables), so I thought maybe I could send a stringified JSON object (which can be treated as a python dictionary using json.loads). Right now it looks like this:
class Handler(BaseHandler):
#...
def post(self):
data = validate_dict(json.loads(self.request.body))
#Use a variable like this: data['last_name']
test = testdb.Test(**data)
test.put()
Which is a lot shorter. I am inclined to do things this way (and stop using self.request.get("something")), but I am worried I may be missing some disadvantage of doing this apart from the client needing javascript for it to even work. Is it OK to do this or is there something I should consider before rearranging my code?

There is absolutely nothing wrong with your short JSON-focused code variant (few web apps today bother supporting clients w/o Javascript anyway:-).
You'll just need to adapt the client-side code preparing that POST, from being just a traditional HTML form, to a JS-richer approach, of course. But, I'm pretty sure you're aware of that -- just spelling it out!-)
BTW, there is nothing here that's App Engine - specific: the same considerations would apply no matter how you chose to deploy your server.

Django reverse routes - two optional parameters

I have a question about Django and it's routing system. I believe that it can be powerfull, but right now I am struggling with one issue I don't experience when working in other frameworks and I can't seem to get a grip on it. Worth to mention that I don't have much experience with Django at this point.
The issue is simple - I have a view which takes two optional parameters, defined like this
def test_view(id=None, grid=None):
Both parameters are optional and frequently are not passed. Id can only be an integer and grid will never be an integer (it is a special string to control datagrid when I don't want to use sessions). I have a route defined like this:
url(a(r'^test_view (\/(?P<id>\d+))? (\/(?P<grid>[^\/]+))? \/?$'), views.test_view, name='test_view'),
This works great and I am not having trouble with using one-way routes. But when I try to use the reverse function or url template tag, following error occurs:
Reverse for 'test_view' with arguments '('20~id~desc~1',)' and keyword arguments '{}' not found.
In this example I tried to find reverse without the id, just with the grid parameter. I have tried various methods of passing parameters to the reverse function:
(grid, )
(None, grid)
('', grid)
{id=None, grid=grid}
All of them result in same error or similliar one.
Is there a way to implement this in django? Maybe just disable the cool URL for the grid parameter. That is how I do it in for example Nette framework for PHP, isntead of having an url like this: 'localhost/test_view/1/20~id~desc~1' I have url like this: 'localhost/test_view/1?grid=20~id~desc~1'. This would be completely sufficient, but I have no idea how to achive this in Django.

As you note in your question, the best way to achieve this is to use standard GET query parameters, rather than doing it in the path itself. In Django you do that exclusively in the view; the URL itself is then just
url(r'^test_view$', views.test_view, name='test_view'),
and you request it via localhost/test_view?id=1&grid=20~id~desc~1. You get the params from request.GET, which is a dictionary-like object; you can use .get so that it does not raise a KeyError when the key is not provided.
def test_view(request):
id = request.GET.get('id')
grid = request.GET.get('grid')

Best option for alternate/suggested search in Django

I have a Django app that searches for data from a model Classified and displays them using simple querysets based on the input term. This works perfectly and I've got no complains with this method.
However, if someone enters a term that returns no data, I would like to provide an option with alternate/suggested search.
Eg: Someone searches 'Ambulance Services' which doesn't return data. I'd like to suggest 'Ambulance' or 'Services' as alternate search options which may return data from the model depending on the data present in the model.
I Googled suggested search in django and it gave me options of Haystack/elastic search, etc which I'm not sure are really required since the search is across just one model.
Note: SO tells me that my question may be closed as it is subjective. If thats the case, please suggest where I can move it to. Thank you!

This is just an idea, but might work for you:
The user enters the search data: "Ambulance Services"
If the query inside the view returns nothing, redirect to the same view using your selected alternative search data, lets say "Ambulance", and a flag value thats saids the view you're preforming a suggested search.
You must have two things in consideration:
What if the alternate search don't returns anything either? You have to set a recursion limit here.
How I'm going to select the data of the alternate search? Well, thats another question about a completly different topic.
This is this idea in code:
def search(request, data, alternate=False, recursion_level=3):
result = Classified.objects.filter(some_filed=data)
if 0 == result.count() and 0 != recursion_level: # Conditions needed for perform alternate search.
new_data = select_new_data(data) # The logic inside this function is up to you.
r_level = recursion_level - 1 # Decreas recursion level.
return redirect('search', alternate=True, recursion_level=r_level) # Redirecting using view name, you can use others
# forms of redirection see link below for visit
# the Djando redirect API doc.
else:
return_data = {}
if alternate:
# You can use this flag in order to reflect
# to the user you have performed an alternate search.
# Example:
return_data["alternate"] = True
# Build your return data.
# and render.
#return render_to_template()
Django redirect doc: redirect

Haystack is, indeed, a great option, here you will find how to give 'spelling suggestions', you can see an example in this OS question/answer
No matter you have only one model, this tool is really great, simple to install /set up/use, and very flexible.
Perhaps this helps as well.

Best-practice: automated web API testing

I've written a program in Python, which works with two distinct API to get the data from two different services (CKAN and MediaWiki).
In particular, there is a class Resource, which requests the data from the above mentioned services and process it.
At some point I've come to conclusion, that there is a need for tests for my app.
And the problem is that all examples I've found on web and in books do not deal with such cases.
For example, inside Resource class I've got a method:
def load_from_ckan(self):
"""
Get the resource
specified by self.id
from config.ckan_api_url
"""
data = json.dumps({'id': self.id})
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
url = config.ckan_api_url + '/action/resource_show'
r = requests.post(url, timeout=config.ckan_request_timeout, data=data, headers=headers)
assert r.ok, r
resource = json.loads(r.content)
resource = resource["result"]
for key in resource:
setattr(self, key, resource[key])
The load_from_ckan method get the data about resource from the CKAN API and assign it to the object. It is simple, but...
My question is: how to test the methods like this? OR What should I test here?
I thought about the possibility to pickle (save) results to HDD. Then I could load it in the test and compare with the object initialized with load_from_ckan(). But CKAN is community-driven platform and such behavior of such tests will be unpredictable.
If there exist any books on philosophy of automated tests (like what to test, what not to test, how to make tests meaningful etc.), please, give me a link to it.

With any testing, the key question really is - what could go wrong?
In your case, it looks like the three risks are:
The web API in question could stop working. But you check for this already, with assert r.ok.
You, or someone else, could make a mistaken change to the code in future (e.g. mistyping a variable) which breaks it.
The API could change, so that it no longer returns the fields or the format you need.
It feels like you could write a fairly simple test for the latter two, depending on what data from this API you actually rely on: for example, if you're expecting the JSON to have a field called "temperature" which is a floating-point Celsius number, you could write a test which calls your function, then checks that self.temperature is an instance of 'float' and is within a sensible range of values (-30 to 50?). That should leave you confident that both the API and your function are working as designed.

Typically if you want to test against some external service like this you will need to use a mock/dummy object to fake the api of the external service. This must be configurable at run-time either via the method's arguments or the class's constructor or another type of indirection. Another more complex option would be to monkey patch globals during testing, like "import requests; request.post = fake_post", but that can create more problems.
So for example your method could take an argument like so:
def load_from_ckan(self, post=requests.post):
# ...
r = post(url, timeout=config.ckan_request_timeout, data=data,
headers=headers)
# ...
Then during testing your would write your own post function that returned json results you'd see coming back from ckan. For example:
def mock_post(url, timeout=30, data='', headers=None):
# ... probably check input arguments
class DummyResponse:
pass
r = DummyResponse()
r.ok = True
r.content = json.dumps({'result': {'attr1': 1, 'attr2': 2}})
return r
Constructing the result in your test gives you a lot more flexibility than pickling results and returning them because you can fabricate error conditions or focus in on specific formats your code might not expect but you know could exist.
Overall you can see how complicated this could become so I would only start adding this sort of testing if you are experiencing repeated errors or other difficulties. This will just more code you have to maintain.

At this point, you can test that the response from CKAN is properly parsed. So you can pull the JSON from CKAN and ensure that it's returning data with the attributes you're interested in.

Linkedin API for python

I used this library to make API requests and got the access tokens successfully.
But the documentation does not explain how _access_token and _access_token_secret should be used afterwards future. I suppose that there should be a method like:
set_access_token(_access_token, _access_token_secret)
but I could't find a method like that in this code or documentation.
Please help me to solve this problem.

I have not used this particular library or API, but a common pattern in these cases is that you pass the token as an argument to subsequent calls. Looking at the source, I can see a function called get_user_profile in __init__.py:
def get_user_profile(self, access_token, selectors=None, headers=None, **kwargs):
"""
Get a user profile. If keyword argument "id" is not supplied, this
returns the current user's profile, else it will return the profile of
the user whose id is specificed. The "selectors" keyword argument takes
a list of LinkedIn compatible field selectors.
"""
So I'd guess you just want to pass the token as the first argument (in this instance).