Develop Reference

Upgrading to django 3.2 breaks API tests (response.data['detail'])

I'm upgrading our django app from 3.0.5 to 3.2.9, and I'm having some issues with API tests. The response returned has apparently changed, and I would like to know why.
self.user.is_superuser = False
self.user.save()
self.assertEqual(self.user.get_all_permissions(), set())
put_url = reverse(..., kwargs={"pk": 1})
put_data = {
...
}
response = self.client.put(put_url, json.dumps(put_data), content_type="application/json")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertEqual(response.data, {"detail": "You do not have permission to perform this action."})
This used to pass, but now response.data contains
{'detail': ErrorDetail(string='Authentication credentials were not provided.', code='not_authenticated')}
and of course the tests fail.
The odd thing is that the error code remains 403, not 401. Is there a way to just have the string returned in detail?
Edit:
Login is done in setUp like so:
def setUp(self):
self.user = User.objects.get(username="test-user")
self.client.login(username=self.user, password="test")
and login returns true. Auth backends are GoogleOAuth2 and ModelBackend, so I guess ModelBackend is used in this case
However, when I try the request on the frontend with the user without permissions I get the expected response {"detail":"You do not have permission to perform this action."}
Edit 2:
I found another case, which is probably related:
self.user.is_superuser = False
self.user.save()
self.assertEqual(self.user.get_all_permissions(), set())
retrieve_url = reverse(..., kwargs={"pk": 1})
response = self.client.get(retrieve_url)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
p = Permission.objects.get(codename="view_...")
self.user.user_permissions.add(p)
response = self.client.get(retrieve_url)
self.assertEqual(response.status_code, status.HTTP_200_OK)
The first assert (for 403) passes, but then the second one (after adding the permission) fails, because 403 is returned again.

It appears that in 3.2 after setting self.user.is_superuser = False the django backend will log the user out, thus resulting in not_authenticated, but the response returning the invalid error code 403, instead of 401.
After logging the user back in (each time after revoking superuser status) all tests pass as they did before on django 3.0.5.

Django receiving json post request from external source

I have written a view function that processes a post request containing json data from a source outside of django (labview). I'm just testing it to begin with so it looks like this
def post_entry(request):
'''Process incoming json string
'''
if request.method == 'POST':
post_data = request.body
# Return a response
return HttpResponse('data received OK')
I've written a test to test this and it passes fine:
def test_post_entry_view_good_post_data(self):
'''post_entry view should return a 200 status if valid
'''
data = {'DHTP Data': ['10', '50.296', '50.94', '50.418', '50.425', '50.431', '50.94'],
'Test String': 'My Test String'}
request_url = reverse('post_entry')
response = self.client.post(request_url, content_type='application/json',
data=dumps(data))
# Should return a 200 response indicating ok
self.assertEqual(response.status_code, 200)
But when labview posts the data post_entry returns a 403 forbidden error. I guess this is due to no csrf token being present, but why does the test pass in this case?

The test client works around the CSRF functionality. See https://docs.djangoproject.com/en/1.9/ref/csrf/#testing

If you are going to have a view that accepts post data from a source external to your app you need to make your view exempt from CSRF protection by using csrf_exempt:
#csrf_exempt
def post_entry(request):
'''Process incoming json string
'''
If you are going to do this, you should use some other method of validating the request

If your view is supposed to accept POST from external sources it is upon you to validate the request as every POST request is required to have a CSRF token (Refer: CSRF). Hence, for your purpose, you'll have to exempt the view from CSRF validation using #csrf_exempt decorator and write your own validation for the request using something like Token Authentication

Use this line to get the decorator needed to bypass CSRF protection:
from django.views.decorators.csrf import csrf_exempt
then put the #csrf_exempt decorator on the line before your function.

Django input request.META into requests headers?

I have a Django application in which I want to get a page from my own website, and mimic a client side request by passing the request's headers and cookies.
middleware.py
r = requests.get('https://google.com', None, headers=request.META, cookies=request.COOKIES)
However, I noticed that in Django the request.META is capitalized and uses underscores instead of hyphens. While request.COOKIES returns empty, which leads me to believe they don't show up in middleware?
Am I missing something, this seems to be quite a downer since requests is such a popular library and Django is such a popular framework.

Are you sure you have set the cookie properly? I have tested this with a small piece of custom middleware and request.COOKIES had the cookie present. Here was my view:
def test(request):
response = HttpResponse('test')
response.set_cookie('foo', 'bar')
return response
Then, after setting a breakpoint with ipdb in my custom middleware:
class MyMiddleware(object):
def process_request(self, request):
import ipdb; ipdb.set_trace()
I could see the set cookie just fine:
ipdb> request.COOKIES
{'foo': 'bar'}

Testing Flask login and authentication?

I'm developing a Flask application and using Flask-security for user authentication (which in turn uses Flask-login underneath).
I have a route which requires authentication, /user. I'm trying to write a unit test which tests that, for an authenticated user, this returns the appropriate response.
In my unittest I'm creating a user and logging as that user like so:
from unittest import TestCase
from app import app, db
from models import User
from flask_security.utils import login_user
class UserTest(TestCase):
def setUp(self):
self.app = app
self.client = self.app.test_client()
self._ctx = self.app.test_request_context()
self._ctx.push()
db.create_all()
def tearDown(self):
if self._ctx is not None:
self._ctx.pop()
db.session.remove()
db.drop_all()
def test_user_authentication():
# (the test case is within a test request context)
user = User(active=True)
db.session.add(user)
db.session.commit()
login_user(user)
# current_user here is the user
print(current_user)
# current_user within this request is an anonymous user
r = test_client.get('/user')
Within the test current_user returns the correct user. However, the requested view always returns an AnonymousUser as the current_user.
The /user route is defined as:
class CurrentUser(Resource):
def get(self):
return current_user # returns an AnonymousUser
I'm fairly certain I'm just not fully understanding how testing Flask request contexts work. I've read this Flask Request Context documentation a bunch but am still not understanding how to approach this particular unit test.

The problem is different request contexts.
In your normal Flask application, each request creates a new context which will be reused through the whole chain until creating the final response and sending it back to the browser.
When you create and run Flask tests and execute a request (e.g. self.client.post(...)) the context is discarded after receiving the response. Therefore, the current_user is always an AnonymousUser.
To fix this, we have to tell Flask to reuse the same context for the whole test. You can do that by simply wrapping your code with:
with self.client:
You can read more about this topic in the following wonderful article:
https://realpython.com/blog/python/python-web-applications-with-flask-part-iii/
Example
Before:
def test_that_something_works():
response = self.client.post('login', { username: 'James', password: '007' })
# this will fail, because current_user is an AnonymousUser
assertEquals(current_user.username, 'James')
After:
def test_that_something_works():
with self.client:
response = self.client.post('login', { username: 'James', password: '007' })
# success
assertEquals(current_user.username, 'James')

The problem is that the test_client.get() call causes a new request context to be pushed, so the one you pushed in your the setUp() method of your test case is not the one that the /user handler sees.
I think the approach shown in the Logging In and Out and Test Adding Messages sections of the documentation is the best approach for testing logins. The idea is to send the login request through the application, like a regular client would. This will take care of registering the logged in user in the user session of the test client.

I didn't much like the other solution shown, mainly because you have to keep your password in a unit test file (and I'm using Flask-LDAP-Login, so it's nonobvious to add a dummy user, etc.), so I hacked around it:
In the place where I set up my test app, I added:
#app.route('/auto_login')
def auto_login():
user = ( models.User
.query
.filter_by(username="Test User")
.first() )
login_user(user, remember=True)
return "ok"
However, I am making quite a lot of changes to the test instance of the flask app, like using a different DB, where I construct it, so adding a route doesn't make the code noticeably messier. Obv this route doesn't exist in the real app.
Then I do:
def login(self):
response = self.app.test_client.get("/auto_login")
Anything done after that with test_client should be logged in.

From the docs: https://flask-login.readthedocs.io/en/latest/
It can be convenient to globally turn off authentication when unit testing. To enable this, if the application configuration variable LOGIN_DISABLED is set to True, this decorator will be ignored.

Using Requests python library to connect Django app failed on authentication

Maybe a stupid question here:
Is Requests(A python HTTP lib) support Django 1.4 ?
I use Requests follow the Official Quick Start like below:
requests.get('http://127.0.0.1:8000/getAllTracks', auth=('myUser', 'myPass'))
but i never get authentication right.(Of course i've checked the url, username, password again and again.)
The above url 'http://127.0.0.1:8000/getAllTracks' matches an url pattern of the url.py of a Django project, and that url pattern's callback is the 'getAllTracks' view of a Django app.
If i comment out the authentication code of the 'getAllTracks' view, then the above code works OK, but if i add those authentication code back for the view, then the above code never get authenticated right.
The authentication code of the view is actually very simple, exactly like below (The second line):
def getAllTracks(request):
if request.user.is_authenticated():
tracks = Tracks.objects.all()
if tracks:
# Do sth. here
Which means if i delete the above second line(with some indents adjustments of course), then the requests.get() operation do the right thing for me, but if not(keep the second line), then it never get it right.
Any help would be appreciated.

In Django authentication works in following way:
There is a SessionMiddleware and AuthenticationMiddleware. The process_request() of both these classes is called before any view is called.
SessionMiddleware uses cookies at a lower level. It checks for a cookie named sessionid and try to associate this cookie with a user.
AuthenticationMiddleware checks if this cookie is associated with an user then sets request.user as that corresponding user. If the cookie sessionid is not found or can't be associated with any user, then request.user is set to an instance of AnonymousUser().
Since Http is a stateless protocol, django maintains session for a particular user using these two middlewares and using cookies at a lower level.
Coming to the code, so that requests can work with django.
You must first call the view where you authenticate and login the user. The response from this view will contain sessionid in cookies.
You should use this cookie and send it in the next request so that django can authenticate this particular user and so that your request.user.is_authenticated() passes.
from django.contrib.auth import authenticate, login
def login_user(request):
user = authenticate(username=request.POST.get('username'), password=request.POST.get('password'))
if user:
login(request, user)
return HttpResponse("Logged In")
return HttpResponse("Not Logged In")
def getAllTracks(request):
if request.user.is_authenticated():
return HttpResponse("Authenticated user")
return HttpResponse("Non Authenticated user")
Making the requests:
import requests
resp = requests.post('http://127.0.0.1:8000/login/', {'username': 'akshar', 'password': 'abc'})
print resp.status_code
200 #output
print resp.content
'Logged In' #output
cookies = dict(sessionid=resp.cookies.get('sessionid'))
print cookies
{'sessionid': '1fe38ea7b22b4d4f8d1b391e1ea816c0'} #output
response_two = requests.get('http://127.0.0.1:8000/getAllTracks/', cookies=cookies)
Notice that we pass cookies using cookies keyword argument
print response_two.status_code
200 #output
print response_two.content
'Authenticated user' #output
So, our request.user.is_authenticated() worked properly.
response_three = requests.get('http://127.0.0.1:8000/hogwarts/getAllTracks/')
Notice we do not pass the cookies here.
print response_three.content
'Non Authenticated user' #output

I guess, auth keyword for Requests enables HTTP Basic authentication which is not what is used in Django. You should make a POST request to login url of your project with username and password provided in POST data, after that your Requests instance will receive a session cookie with saved authentication data and will be able to do successful requests to auth-protected views.

Might be easier for you to just set a cookie on initial authentication, pass that back to the client, and then for future requests expect the client to send back that token in the headers, like so:
r = requests.post('http://127.0.0.1:8000', auth=(UN, PW))
self.token = r.cookies['token']
self.headers = {'token': token}
and then in further calls you could, assuming you're in the same class, just do:
r = requests.post('http://127.0.0.1:8000/getAllTracks', headers=self.headers)