I am writing a Django app, and am wondering if any client side validation is necessary. Django handles all validation through forms in python on the backend. If something validates wrong, the user is returned to the screen with all their information still there.
I can't see any reason I need to implement client side validation in Django? Is this true? The only reason I can think of is it would save a few hits to the server, but this seems negligible.
If you have a web application that faces the public internet client side, validation is pretty much a user expectation. You might be able to ignore this if volume is low and people are motivated to use your website.
For an company intranet site, the additional development cost may weigh against client side validation. However, if you use an available client framework (e.g. jquery or django-parsley) the additional cost for client side validation is actually fairly small and likely worth the effort in intraweb applications.
ADDED
Yes, as others had already stated client-side only validation is very bad as it is the same as no validation -- you can coerce the browser to send whatever you want back to the server.
You can do also do lots of nice things client side that you cannot server side. Sometimes these are closely related to client side validation.
E.g., limiting a comment to 500 characters. With client side code you can display a characters remaining count on screen -- with a little planning this can be integrated with the validation code.
Client side validation may improve user experience (less page reloads). It may decrease number of hits to he server (but sometimes this number is increased :). But it is not necessary.
Anyway server side validation is a must. You can't trust data from user input.
This is largely a matter of opinion, but I would have to say no - you don't need to implement client-side validation. Especially when you can get all of the errors from your Django form returned as JSON via a simple Ajax POST.
Django forms already do an excellent job of validating input, so why add yet more code you have to maintain in two places that does the same thing? You absolutely MUST do server-side validation anyway, so why not just do it all in one place?
Additionally, if you don't implement the same validations on the server as on the client, or worse - only do client-side validation, someone can always turn JavaScript off in the browser and possibly bypass your validation(s) or allow junk data to get into your database if you're not careful.
Related
I built a simple Flask application that receives a POST request and performs some actions after receiving it. Here is my simple code:
#app.route('/<user>/', methods=['POST'])
def Receiver(user):
Query = User.query.filter_by(token=user)
Content = request.data.decode('UTF-8')
Data = {'Content': Content, 'Username': Query.Username, 'UserID': Query.UserID}
return jsonify(Data)
I would like to make this code as safe as possible, but i'm just getting started to Flask and security in general. What dangers can i run using this code? I'm using the variable user to make a query to my database, can it be harmful if that variable gets set to an SQL query, for example? What other threats should i consider in this case?
Here some of my thoughts to your question:
Why is this a POST request and not a GET request? POST requests are meant to change data, GETs are for queries.
You don't validate input data. What happens, if the user sends you e.g. a 100kB long user name? How will the database handle it? Will it have impact on performance? Will it allow a DOS attack on server/database?
Yes, SQL injection too. Everywhere where relational databases are concerned.
What if the user ID does not exist? Should we not return 404?
What is actually security? What is safety? The two terms are not interchangeable. Safety is when the code does not harm the world. Security is when the world does not harm the code.
There is a wide variety of things to consider that could impact your code security (meaning providing confidentiality, integrity and accessibility of the data the code touches), that are unrelated to your code like: communication channel protection, server misconfigurations, DDOS attacks... Even if your code is perfect, the system holding it might still be insecure.
Just to add to what Marek said, would also recommend changing to a GET... As long as there's no sensitive information being passed along in the URL. This link nicely explains the differences. It might be a good idea to look at encrypting the URL token string too, so that any parameters aren't passed over in plain text, as this leaves room for vulnerability.
Alongside this, if the site is to be made live- definitely ensure to use SSL encryption.
In terms of SQL validation, you'll need to sanitize the input before it ever reaches the database. You can do this in Flask, simply by using the HTML escape special chars... But Flask provides their own function. This link might help in that regard.
In terms of error handling, I found this tutorial mighty useful. That whole series of blog posts walks you right the way through.
Developing a simple chat system via Django - I've implemented some methods for sending/receiving messages.
For example ( this isn't my implementation but just an example one from another chat application ) :
def post(request):
time.sleep(2)
if not request.is_ajax():
HttpResponse (" Not an AJAX request ")
if request.method == 'POST':
if request.POST['message']:
message = request.POST['message']
to_user = request.POST['to_user']
ChatMessage.objects.create(sender = request.user, receiver = User.objects.get(username = to_user), message = message, session = Session.objects.get(session_key = request.session.session_key))
return HttpResponse (" Not an POST request ")
Now that I have the method written - I need to test to see if the message is added. At this point I have not written any JavaScript for I.e. intervals to refresh and wait for messages ect. Should I go straight into writing JS or test this method first and see if it works correctly then write the JS for it? sounds like an idiotic question but I'm finding it hard to understand how I'd go about testing the method...
In my opinion it doesn't matter. You will find many developers who are test-before, and many who are test-after. Implement it in whatever order you prefer. Sometimes when in haste it is easier to write a working prototype without any tests first (especially if the interfaces aren't finalized and still change a lot during development, in this case you don't waste time by adjusting a lot of tests every time you have to change some interfaces.). Just don't be lazy to write the tests at least when you already have the finalized interfaces.
What really matters is to have well defined interfaces. From a backend perspective you will have an interface implementation (the view in this case) and one or more users for the interface: both the frontend and the test are users of the backend implementation but there could be even more. After having the interface it really doesn't matter which one you implement first. By mocking one side of the interface you can implement the other side without having the original version of the mocked side. For example by mocking/faking server responses with javascript code (with pre-baked possibly constant data) you can write only a frontend without any backend code. You can also write the backend first, or a the tests for the backend... You decide.
In teams where you have specialized developers (frontend/backend) you can agree on an interface and then both on the frontend and the backend side you can "mock" (fake) the other side of the interface: The frontend guys write some code that emulates the server responses with some fake data, and the backend guys write some tests that emulate the client with some fake requests. This way the frontend and backend team doesn't have to wait for each other to finish with the code and both the frontend and the backend are testable alone. Of course later it is recommended to add end-to-end (e2e) testing that tests the whole stack connected together.
Again, what really matters most is usually having well defined interfaces and not the code that is written around the interfaces. In crappy systems the problem is usually that you have only code without interfaces... If a system is architecturally well built and the interfaces are well defined then quite a lot of crappy code written around the interfaces can be manageable.
In case of django views that have a well defined interface I usually develop the backend first along with the tests. In your case the django test is super simple: You just create a django test client (https://docs.djangoproject.com/en/1.8/topics/testing/tools/#test-client), post some fake requests with it to simulate the client and then you check whether the db contains the expected objects as a result.
Some additional advices:
Decorate your view with #require_POST
I think you shouldn't use request.is_ajax() to deny responding to the client. request.is_ajax() is usually used to find out what kind of response is needed by the client. If the POST request was sent by a form of an html page then you want to generate another html page as a response. If the request was sent using ajax then you usually want to respond with processable data (json, xml, etc...) instead of html.
Since your method has to have a request made by the client and then served a HttpResponse (on-the-whole), you can only check that via an AJAX query from the front-end. I would recommend building a quick AJAX query, also because, the ChatMessage creation involves a session_key, which can only be found when a request is made in the first place.
I'm making a new RESTful API in Flask that should accept both GET (for requesting the resource) and PATCH (for performing various incremental, non-idempotent updates) for a given object. The thing is that some of the data that's patched in must be authenticated, and some shouldn't be.
To clarify this with an example, let's say I'm building an app that let's everyone query how many times a resource has been clicked on and how many times its page has been viewed. It also let's people do an update on the resource in javascript saying the resource was clicked again (unauthenticated, since it's coming from the front-end). It additionally let's an authenticated backend increment the number of times the page has been viewed.
So, following RESTful principles, I'm thinking all three actions should be done on the same path-- something like /pages/some_page_name which should accept both GET and PATCH and should accept two different kinds of data with PATCH. The problem is that in Flask, it looks like authentication is always done with a decorator around a method, so if I have a method like #app.route('/pages/<page_id>', methods=['GET', 'PATCH']), my authentication would be done with a decorator like #auth.login_required for that whole method, which would force even the methods that don't require authentication to be authenticated.
So, my question is three-fold:
Am I right in structuring all three actions mentioned under the same path/ is this important?
If I am right, and this is important, how do I require authentication only for the one type of PATCH?
If this is not important, what's a better or simpler way to structure this API?
I see several problems with your design.
let's say I'm building an app that let's everyone query how many times a resource has been clicked on and how many times its page has been viewed
Hmm. This isn't really a good REST design. You can't have clients query select "properties" of resources, only the resources themselves. If your resource is a "page", then a GET request to /pages/some_page_name should return something like this (in JSON):
{
'url': 'http://example.com/api/pages/some_page_name',
'clicks': 35,
'page_views': 102,
<any other properties of a page resource here>
}
It also let's people do an update on the resource in javascript saying the resource was clicked again
"clicking something" is an action, so it isn't a good REST model. I don't know enough about your project so I can be wrong, but I think the best solution for this is to let the user click the thing, then the server will receive some sort of a request (maybe a GET to obtain the resource that was clicked?). The server is then in a position to increment the clicks property of the resource on its own.
(unauthenticated, since it's coming from the front-end).
This can be dangerous. If you allow changes to your resources from anybody, then you are open to attacks, which may be a problem. Nothing will prevent me from looking at your Javascript and reverse engineering your API, and then send bogus requests to artificially change the counters. This may be an acceptable risk, but make sure you understand this may happen.
It additionally let's an authenticated backend increment the number of times the page has been viewed.
Backend? Is this a client or a server? Sounds like it should be a client. Once again, "incrementing" is not a good match for REST type APIs. Let the server manage the counters based on the requests it receives from clients.
Assuming I understand what you are saying, it seems to me you only need to support GET. The server can update these counters on its own as it receives requests, clients do not need to bother with that.
UPDATE: After some additional info provided in the comments below, what I think you can do to be RESTful is to also implement a PUT request (or PATCH if you are into partial resource updates).
If you do a PUT, then the client will send the same JSON representation above, but it will increment the corresponding counter. You could add validation in the server to ensure that the counters are incremented sequentially, and return a 400 status code if it finds that they are not (maybe this validation is skipped for certain authenticated users, up to you). For example, starting from the above example, if you need to increment the clicks (but not the page views), then send a PUT request with:
{
'url': 'http://example.com/api/pages/some_page_name',
'clicks': 36,
'page_views': 102
}
If you are using PATCH, then you can remove the items that don't change:
{
'clicks': 36
}
I honestly feel this is not the best design for your problem. You have very specific client and server here, that are designed to work with each other. REST is a good design for decoupled clients and servers, but if you are on both sides of the line then REST doesn't really give you a lot.
Now regarding your authentication question, if your PUT/PATCH needs to selectively authenticate, then you can issue the HTTP Basic authentication exchange only when necessary. I wrote the Flask-HTTPAuth extension, you can look at how I implemented this exchange and copy the code into your view function, so that you can issue it only when necessary.
I hope this clarifies things a bit.
It seems like the security model fits very small projects, but that it is probably not feasible to write all possible registered users' hashed passwords in security.py. Do you know any examples of scaling up Pyramid's authentication, or are there any benefits to calling through Pyramid's security scheme into my own database of security information?
I dont think the size of the project is related to the security model. Either you want a simple or a complex security model. Both can be applied to projects of any size. One of Pyramid's strong points is its extensibility.
Why would you store hashed passwords in security.py? (cmiiw here, I probably misunderstood) If you read this on someone's code, that's probably just an example. In real apps, you save them in a storage/persistence system of your choice.
Again, I don't understand what you mean by "scaling up authentication". My guess is you want some working examples:
tutorial from the docs
shootout application: small and good example with forms
pyramid auth demo: complex/granular/row-level permission
pyramid apex: 3rd party auth (google, twitter, etc) with velruse, forms etc
pyramid registration: unfinished library; you can steal some ideas from it
No idea what your needs are or what you mean by "scaling up security", but pyramids authentication policy is very flexible. You need to understand though that it doesn't maintain users and passwords it merely provides a mechanism for obtaining a user identifier from the incoming request. For example, the AuthTktAuthenticationPolicy keeps track of the user id by cookie that you set using the remember method.
What meaningful information you derive from that user id is totally up to you and is application specific.
So really the question you may want to ask is can your application "scale up security".
I can't show you code because it's proprietary but I've needed to support openid, http auth and your typical db backed user store on the same application, with the extra added complication that users are stored in different database shards and the shard can't be immediately determined. It takes very little code to support this.
I ended up building something for myself that makes authentication a little easier if you happen to be using MongoDB.
https://github.com/mosesn/mongauth
It isn't built into pyramid, but hooks in easily enough. Everything is pretty transparent.
What would be the best way to implement a simple crash / error reporting mechanism?
Details: my app is cross-platform (mac/windows/linux) and written in Python, so I just need something that will send me a small amount of text, e.g. just a timestamp and a traceback (which I already generate and show in my error dialog).
It would be fine if it could simply email it, but I can't think of a way to do this without including a username and password for the smtp server in the application...
Should I implement a simple web service on the server side and have my app send it an HTTP request with the info? Any better ideas?
The web service is the best way, but there are some caveats:
You should always ask the user if it is ok to send error feedback information.
You should be prepared to fail gracefully if there are network errors. Don't let a failure to report a crash impede recovery!
You should avoid including user identifying or sensitive information unless the user knows (see #1) and you should either use SSL or otherwise protect it. Some jurisdictions impose burdens on you that you might not want to deal with, so it's best to simply not save such information.
Like any web service, make sure your service is not exploitable by miscreants.
I can't think of a way to do this without including a username and password for the smtp server in the application...
You only need a username and password for authenticating yourself to a smarthost. You don't need it to send mail directly, you need it to send mail through a relay, e.g. your ISP's mail server. It's perfectly possible to send email without authentication - that's why spam is so hard to stop.
Having said that, some ISPs block outbound traffic on port 25, so the most robust alternative is an HTTP POST, which is unlikely to be blocked by anything. Be sure to pick a URL that you won't feel restricted by later on, or better yet, have the application periodically check for updates, so if you decide to change domains or something, you can push an update in advance.
Security isn't really an issue. You can fairly easily discard junk data, so all that really concerns you is whether or not somebody would go to the trouble of constructing fake tracebacks to mess with you, and that's a very unlikely situation.
As for the payload, PyCrash can help you with that.
The web hit is the way to go, but make sure you pick a good URL - your app will be hitting it for years to come.
PyCrash?
Whether you use SMTP or HTTP to send the data, you need to have a username/password in the application to prevent just anyone from sending random data to you.
With that in mind, I suspect it would be easier to use SMTP rather than HTTP to send the data.
Some kind of simple web service would suffice. You would have to consider security so not just anyone could make requests to your service..
On a larger scale we considered a JMS messaging system. Put a serialized object of data containing the traceback/error message into a queue and consume it every x minutes generating reports/alerts from that data.