So I'm trying to port some old Pylons code to Pyramid, and I'd like to be able to improve on the Auth setup - specifically support better RBAC, and Pyramid has good support for this.
However, I'd like to offer unauthorised users better info when they try illegal pages:
"Sorry, in order to view [page] you ([user]) need [group] privileges - please contact [admin]"
However I don't see how that's practical in Pyramid - I can do stuff in the forbidden_view_config page, however I can't easily find all the info needed from the page which was attempted - is it possible to get the exception or similar with the actual reason why permission was not granted?
The request object itself should have all the bits you need.
Specifically, security-related pieces lists some of the request attributes that you can retrieve. Also the request.exception attribute will be available when an exception is raised. There are several URL-related pieces available to get the "page", including application_url.
Related
Is there any way to detect if a link is invalid using webbot?
I need to tell the user that the link they provided was unreachable.
The only way to be completely sure that a url sends you to a valid page is to fetch that page and check it works. You could try making a request other than GET to try to avoid the wasted bandwith downloading the page, but not all servers will respond: the only way to be absolutely sure is to GET and see what happens. Something like:
import requests
from requests.exceptions import ConnectionError
def check_url(url):
try:
r = requests.get(url, timeout=1)
return r.status_code == 200
except ConnectionError:
return False
Is this a good idea? It's only a GET request, and get is supposed to idempotent, so you shouldn't cause anybody any harm. On the other hand, what if a user sets up a script to add a new link every second pointing to the same website? Then you're DDOSing that website. So when you allow users to cause your server to do things like this, you need to think how you might protect it. (In this case: you could keep a cache of valid links expiring every n seconds, and only look up if the cache doesn't hold the link.)
Note that if you just want to check the link points to a valid domain it's a bit easier: you can just do a dns query. (The same point about caching and avoiding abuse probably applies.)
Note that I used requests, because it is easy, but you likely want to do this in the bacground, either with requests in a thread, or with one of the asyncio http libraries and an asyncio event loop. Otherwise your code will block for at least timeout seconds.
(Another attack: this gets the whole page. What if a user links to a massive page? See this question for a discussion of protecting from oversize responses. For your use case you likely just want to get a few bytes. I've deliberately not complicated the example code here because I wanted to illustrate the principle.)
Note that this just checks that something is available on that page. What if it's one of the many dead links which redirects to a domain-name website? You could enforce 'no redirects'---but then some redirects are valid. (Likewise, you could try to detect redirects up to the main domain or to a blacklist of venders' domains, but this will always be imperfect.) There is a tradeoff here to consider, which depends on your concrete use case, but it's worth being aware of.
You could try sending an HTTP request, opening the result, and have a list of known error codes, 404, etc. You can easily implement this in Python and is efficient and quick. Be warned that SOMETIMES (quite rarely) a website might detect your scraper and artificially return an Error Code to confuse you.
Hello I am interested in one thing.I know it can be silly question but I can not understand one thing here:
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<LocationConstraint>eu-central-1</LocationConstraint>
</CreateBucketConfiguration>
is "http://s3.amazonaws.com/doc/2006-03-01/" URL or URI?
cause when I type it in browser it shows me this:
screenshot
It's both. A uniform resource identifier (URI) provides a name for a resource. A URL (uniform resource locator) is a kind of URI that describes where (and how) the resource can be accessed.
The URL you provide, http://s3.amazonaws.com/doc/2006-03-01/, doesn't tell you anything about what is there. It simply says that if you use the HTTP protocol to connect to s3.amazonaws.com and request /doc/2006-03-01/, you'll get something back. What that something is is only implied by the name of the XML attribute that has the URL as its value.
(In practice, the server may not actually provide a resource at that location, but it could. The error message you see indicates there might be something there, but you don't have permission to access it.)
Let's say I have a try and catch and there is an exception ... What is the proper way to deal with those exceptions/errors on a live production (django) site?
So I have
try:
create_response = wepay.call('/account/create',
{'name': name, 'description': desc})
self.wepay_account_id = create_response['account_id']
self.save()
except WePay.WePayError as e:
..... (what do I put here?
You can set up e-mail error reporting through Django: https://docs.djangoproject.com/en/dev/howto/error-reporting/
Or you can use a service like Rollbar (has a free account) to track error occurances.
Or you could use self-hosted Greylog (like suggested in comments), here's a good guide for django: http://www.caktusgroup.com/blog/2013/09/18/central-logging-django-graylog2-and-graypy/
Respond with (optionally a redirect to) a appropriate page explaining the problem to the user and if possible, provide a solution. Serving a 500 to your users in production is something you want to avoid, so catching the exception is a good idea.
So:
except WePay.WePayError as e:
return render_to_response('wepay_error_page.html')
or:
except WePay.WePayError as e:
return HttpResponseRedirect('/errors/wepay/') # Note: better use urlresolvers
(note this particular code will only work if it's in a view)
Then (optionally), make sure you get a copy of the error, by for example sending yourself an email.
A suggestion for this particular case (if I interpret the code succesfully) may be to notify yourself, and repond with a page explaining to the user their payment went wrong. Tell them this might occur because of their actions (maybe they cancelled their payment), and provide contact details for when users think it was not their fault.
Django by default mails (when mail is properly configured) all 500 errors to settings.ADMINS, but these only occur on uncaught exceptions, so in this particular question services like Rollbar or a central logging solution will only work if you re-raise the exception (will result in a 500) or send the error to one of these manually in the catch block.
I would recommend the above solution of redirecting over to a page that explains WePay error, combined with using django-wepay app available on pypi that features logging of all errors, and optionally all calls.
I started playing around with web2py the other day for a new project. I really like the structure and the whole concept which feels like a breath of fresh air after spending a few years with PHP frameworks.
The only thing (currently) that is bothering me is the ticketing system. Each time I make a misstake a page with a link to a ticket is presented. I guess I could live with that if the link worked. It currently points to an admin page with http as protocol instead of https. I've done a bit of reading and the forced https for admin seems to be a security measure, but this makes debugging a pain.
Whats the standard solution here? Alter the error page, allow http for admin och use logs for debugging?
Best regards
Fredrik
I was in the same boat as you, I did not like the default mechanism. Luckily, customized exception handling with web2py is very straightforward. Take a look at routes.py in the root of your web2py directory. I've added the following to mine:
routes_onerror = [('application_name/*','/application_name/error/index')]
This routes any exceptions to my error handler controller (application_name/controllers/error.py) in which I defined my def index as:
def index():
if request.vars.code == '400':
return(dict(app=request.application,
ticket=None,
traceback="A 400 error was raised, this is controller/method path not found",
code=None,
layer=None,
wasEmailed=False))
elif request.vars.code == '404':
return(dict(app=request.application,
ticket=None,
traceback="A 404 error was raised, this is bad.",
code=None,
layer=None,
wasEmailed=False))
else:
fH = file('applications/%s/errors/%s' % (request.application,request.vars.ticket.split("/")[1]))
e = cPickle.load(fH)
fH.close()
__sendEmail(request.application,e['layer'],e['traceback'],e['code'])
return(dict(app=request.application,
ticket=request.vars.ticket,
traceback=e['traceback'],
code=e['code'],
layer=e['layer'],
wasEmailed=True))
As you can see for non-400 and 404 errors, I'm emailing the traceback to myself and then invoking the corresponding views/error/index.html. In production, this view gives a generic "I'm sorry an error has occurred, developers have been emailed". On my development server, it displays the formatted traceback.
Normally, I just use http://127.0.0.1/ (if you are local or over ssh) or edit/navigate using https://...
So, you will logon the admin app the first time, but always will the show the tickets after.
Is there some way or using the bulkloader.py dump and restore functionality without authentication?
I have tried using:
- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
without the login-parameter, but login still seems to be required.
I still get
[ERROR ] Exception during authentication
I struggled with this for 6 hours yesterday, without any solution.
And yes, I have tried GAEBAR. It failed, however when it got to entities that contain up to 1MB (the maximum pr. entity) Blobs.
So, I am looking to dump (and restore) for backup-purposes mainly.
remote_api, which the bulkloader uses, is written to deliberately require authentication, even if you omit the relevant clause in app.yaml. You can override it if you really want, but it's an incredibly bad idea - it would allow any anonymous user to do practically anything they liked to your app!