Develop Reference

flask-restless use pagination or get full response

Sometimes I want to get from a resource the full response and sometimes I want it with pagination. Until now I was only able to use either the one or the other.
But isn't there a way to set flask-restless to use both depending on the paramters i pass on the GET request?
If I want to disable pagination for a specific resource I change the settings like this:
manager.create_api(someresource, methods=['GET'], results_per_page=None)
But now pagination is completly disabled and that's not the behaviour I wish.
And if pagination is enabled as default it returns only the first page.
Isn't there a way to tell flask-restless to get only the first page if I specifically pass the page 1 in the query string like so:
GET http://someaddress/resource?page=1
I was actually able to solve the problem using a loop but I don't think it is a nice solution because I have to use multiple requests.
I requested the resource and fetched the total_pages and then I ran a loop to total_pages and passed each iteration as an argument in the query string for another new request to fetch each page:
i = 1
while i <= response.total_pages:
page_response = requests.get("http://someurl/someresource?page=" + str(i))
...
But I don't think it is a nice way to solve that issue. If there is a possibility to change the settings on flask-restless to fetch only the first page if it is passed as an argument in the query string then I would be more than happy but if there is still another way to use both then it's also good.

You can get the behaviour you want by disabling pagination with:
manager.create_api(someresource, methods=['GET'], results_per_page=0)
And then query the API with the results_per_page parameter like so:
GET http://someaddress/resource?results_per_page=2
The results_per_page parameter has to be a positive integer and will be your new page size. The parameter is further documented here.
Getting the full response without pagination is straight forward with this configuration. Just omit the results_per_page parameter:
GET http://someaddress/resource

Send form data to MongoDB using Flask

I'm having troubles sending my form data to MongoDB with a Flask setup. The form looks something like this: https://jsfiddle.net/8gqLtv7e
On the client-side, I see no errors. But when I submit the form, I receive a 500 Internal Server Error and I'm having a hard time finding the solution. The problem is the last line below in my views.py file:
#app.route('/recordReport', methods=['POST'])
def recordReport():
homeReportsCollection = db["reports"]
address=request.form.get['address']
rgbImage=request.form.get['rgb']
homeReportsCollection.insert({'address':address, 'rgb':rgbImage})
Because if I replace that with return json.dumps({'status':'OK', 'address':'address', 'rgb':'rgbImage'}), I can see the correct data in my browser. Am just not able to send it to a collection in MongoDB.

This answer is a summary of the comments (where a solution was found).
Have you tried typecasting address and rgbImage to String before inserting?
Type invalidation is the root of many common bugs in DB operations.
There used to be a bug in Mongo back in 2013. The data would be inserted into the collection. But Mongo would not return a correct status response. That led to servers going 500. Have you tried verifying if the data was indeed inserted into the collection?
Additionally run your flask app in debug=True mode. That might give additional data.
Flask has very good debug traceback reporting support. This is generally a good idea. In fact this should be the first thing to do when encountering an error.
So this is weird, I turned on debug=True and I get the following error: ValueError: View function did not return a response. BUT the data did actually get sent to DB via homeReportsCollection.insert({'address':address, 'rgb':rgbImage}) line. I see it in my collection. How do I fix the error? Because the user is redirected to /recordReport.
So the data was indeed inserted into the collection. It is possibly a Flask only issue. The traceback says it all. Flask requires that something is returned by a view method.
Try returning something in your function recordReport(). Return anything you want. It could be an OK message. It could be the _id of the document you just inserted into the collection. Just don't return None. Try this.
This behaviour is documented in this SO question.
Yeah, I returned an HTML template and no error now.
This is indeed the solution. Return something other than None from your flask view methods. This also validates the behaviour observed by asker in the question:
Because if I replace that with return json.dumps({'status':'OK', 'address':'address', 'rgb':'rgbImage'}), I can see the correct data in my browser.

How do I troubleshoot in Django?

I come from a PHP background and I've been trying to learn Python but I'm having a lot of trouble when it comes to debugging as I'm not yet sure how to go about this in Django or Python in general.
I'm used to being able to print_r or var_dump everything in PHP. I could do it in the controller, In a service layer or the even the model and data would show up in my web browser.
I can't do this in Django. Depending on what I'm doing, attempting to just perform a print on an object from my view will bring the page down or output something to my console that doesn't really help me. Here's an example:
class Page(View):
def get(self, request, *args, **kwargs):
response = Data.objects.all()
# for whatever reason, I want to print something right now:
print response
# return JsonResponse({'success':response})
The above will take down my page completely with a notice saying:
The view didn't return an HttpResponse object. It returned None instead.
There are some instances while working with CBV's where I noticed I could get the data to just dump somewhere such as the console. But it wouldn't be anything that would help me. For instance if I was trying to to take a look at the contents of response from above, it would just show up like so:
[object Object] [object Object] [object Object]
A var_dump would have allowed me to actually see inside of it.
So I'm guessing I'm going about this all wrong. Do people just dump data when they're debugging in Python? If they do, how do you perform this, and does it show up in the web browser or the console? If not, how do I go handle basic troubleshooting in Django? Example scenarios:
I want to see the contents of a list or dictionary
I want to see the raw sql query being performed by the ORM
I want to see if a function is being executed by slipping some text inside to be outputted on the front end

The basic problem is that you are used to how PHP is wired into the entire request/response chain, and this is not how Python is configured when developing web applications.
In PHP-world, the server guarantees a response, which closes the request/response cycle. The PHP file is directly requested by the browser, so you are unaware of what is actually happening in the background.
A typical PHP request, is just the same as a request for any other static asset, like a plain index.html file or logo.gif. The browser requests, the web server accepts the request, and then returns the response; the only difference being if a file with .php is requested, then it goes through an intermediary step where the PHP interpreter evaluates the file, and sends the result back to the client.
In Python however, when a request is made that is mapped to a Python backend process (sometimes called upstream process); the webserver waits for a response from the process (the timeout for this can be adjusted). If a response isn't received within the defined timeout, the webserver sends a timeout error page (504 ERROR).
It is the responsibility of the Python process to send a proper response (with all headers, etc.) as is expected by the browser. In PHP, this is hidden from you (as the developer), because the PHP engine will add these extra information for you. So when you have Python code that does not send such a response (as in your case), django helps you out by printing a friendly error message.
In your case - the view is not returning a response; its just printing something. This print statement will go to the standard output (or error stream) of the application (will be printed on the console if you launched it on the shell, or written to the server's log, etc.) it will not be sent back to the client (the browser).
In order to debug django applications:
Make sure DEBUG = True is set in your settings.py
Run your application with python manage.py runserver
Now, when you do any print statement, it will be shown on the console and if there is an error in your application code - as long as you are returning a valid response - you will get a rich error page along with a stack trace to help identify the problem.
There is no more "debugging statements and printing things on the browser" as you are developing; this is just how Python is wired into the web world.
For your other questions:
I want to see the contents of a list or dictionary
Simply print it. The output will be on your console (the same place where you wrote python manage.py runserver)
I want to see the raw sql query being performed by the ORM
If you are testing things out in the django shell, you can just put .query at the end of your ORM call to see the query being sent:
>>> result = MyModel.objects.filter(foo='bar')
>>> result.query
(query printed here)
For a more rich debugging experience, install django_debug_toolbar.
I want to see if a function is being executed by slipping some text inside to be outputted on the front end
There is no "outputting to the front end". For such things, you can just print() what you need, or even better use the logging system.

First, the reason you're getting an error isn't because you're printing, it's because you commented out the return:
class Page(View):
def get(self, request, *args, **kwargs):
response = Data.objects.all()
# for whatever reason, I want to print something right now:
print response
return JsonResponse({'success':response}) # <-- dont comment this out
Secondly, printing arbitrary objects won't always provide the best amount of information. If something defines a __str__ or __unicode__ method, that's what will be printed to the console. Otherwise, the object name along with its memory id will be printed. Not the most useful. Printing an object doesn't do a "deep" print.
You can try to print out the locals and globals:
print(locals())
print(globals())
Or serialise an object to JSON and print that:
print(json.dumps(response)) # some objects may not be serialisable to JSON though.
But you may not get the amount of detail you want from that either. The alternative is to run your webserver in debug mode, and raise an exception:
# settings.py
DEBUG = True
# views.py
def my_view(request):
raise Exception('debug')
.. and rely on django to show you the debug error page, which includes a stack trace, and allows you to inspect all the variables available in each frame.

First off, Django views require some sort of HttpResponse object to be returned. From the docs-
In contrast to HttpRequest objects, which are created automatically by Django, HttpResponse objects are your responsibility. Each view you write is responsible for instantiating, populating and returning an HttpResponse.
There are many classes that that you can use to return that response (render_to_response, HttpResponseRedirect, JsonResponse and many, many more- https://docs.djangoproject.com/en/dev/ref/request-response/#httpresponse-subclasses). Your line # return JsonResponse({'success':response}) will do that if you get rid of the #.
Instead of a var_dump, you can use Python's dir function. All attributes of a class or class instance will be shown. See- Is there a function in Python to print all the current properties and values of an object?
You can print a dictionary. There are many ways to do it.
for key,value in your_dictionary.items():
print(key, ":", value)
Easy to do. print Data.objects.all().query . See - How to show the SQL Django is running
Or you could add print statements inside the function (or a decorator that states the function is executing).
These are pretty basic parts of Django and Python. Not to be rude, but your time would probably be better spent doing some tutorials in full than embarking on your own projects. Once it clicks, it will be very simple. MVC/MVT frameworks have a different structure than PHP and you'll need to get used to working within it or will be frustrated.

Capture all URL parameters in Django views.py?

There are lots of question discussing how to get 1 parameter, but how do you get all of the parameters, preserving their order?
There's this way: request.GET.get('q', '') to get 1 parameter.
I need to capture POST requests to my URL, then add a parameter to the URL, and send it right back to confirm it's validity and source. This is for PayPal IPN if you're wondering.
Thanks!

As #Daniel Roseman said you probably don't need to preserve the order, in which case you can just use the request.GET dict
Alternatively you can get the raw querystring:
request.META['QUERY_STRING']
https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.META

As Daniel Roseman mentions, order should not be significant between POST or GET request parameters; think of them like key-value pairs rather than a list.
If you want to maintain order, then perhaps pass a list as the value in your POST and grab it in Django:
myData = request.POST.get("myQuery")
Specifically, POST requests don't use the querystring* (see here). POSTs use the request body while GETs use the query string. Note that security-wise, this also means important client information isn't blatantly displayed in the URL -- which is especially important when dealing with payments.
Update: *Apparently, POSTs can use the query string, but they really shouldn't. See this SO post for more.

Yes, order is significant here. This is what I'm going to use:
newParameteres = 'cmd=_notify-validate&' + self.request.POST.urlencode()
req = urllib2.Request("http://www.paypal.com/cgi-bin/webscr", newParameteres)

Django bootstrap/middleware/enter-exit

I have following problem. I want to add to django some kind of setup/teardown for each request. For example at the beginning of per user request I want to collect start data collection and at the end of request dump all data to database (1).
What comes to my mind right now, at the start of middleware instantiate an object (like singleton), every other part of the code can import this object, use its methods and then same middleware before returning response will scrap the object. The only concern I have is to be a threadsafe, so maybe create a global dict, and register keys that are build upon url + session_id hash or maybe request object id (internal python object id, maybe is good way to go?). At the end of request key will be scrapped from dict.
Any recommendations, thoughts, ideas?
(1) Please do not ask me why I cannot access DB directly or anything like this. This is only an example. I'm looking for general idea for something like enter and exit but request-response wise that can be imported in any place in a code and safely used.

In your middleware, you can create new object for data you want to maintain and put it in request.META dict. It will be available wherever, request is available. In this case, I don't think you need to worry about thread-safety as each request will create new object.
If you want to just create data object once when request processing starts, destroy after processing the request and no other code references this data then you can look at request_started and request_finished signals.

Middleware is very certainly not thread-safe. You should not store anything per-request either on the middleware object, or in the global namespace.
The usual way to do this is sort of thing to annotate it onto the request object. Middleware and views have access to this, but to get it anywhere else (eg in the model) you'll need to pass it around.