I'm looking for a good Django custom storage backend for use with Amazon S3.
I've been googling around and found a lot of blog posts with code snippets or half-baked gist.github.com one-off jobs. But I can't seem to find a solid, well-tested one.
Is there a widely accepted standard Amazon S3 Django custom storage backend out there? It doesn't particularly matter to me what Python backend library it uses--i.e., either S3.py or boto are fine.
Have you checked out django-storages? I would lean towards the boto library as I have had good experiences with boto.
Related
I have a Django app hosted on AWS Lambda (deployed using Zappa tool) an using RDS for hosting the Postgres database. The app is having performance issues with some pages loading very slowly but not others, making me think that the issue is with poorly written data interactions.
I remember at work, the devs were using Azure Application Insights which was monitoring apps at a code level. I think the devs had to insert code throughout their apps to get the monitoring, and I think it monitored on a page by page and code line by line basis.
Is there any similar tool on AWS which I could use with my App?
I appreciate there are much simpler ways of solving my problem but I thought an AWS tool would be something neat to learn.
Yes, there are a few monitoring and troubleshooting tools that work on the AWS environment.
These tools will help you both dive-in into a specific invocation, thus let you understand what wasted the time, but also understand higher-level insights, such as the average time to a DB call or any 3rd party.
Disclosure: I'm working for Lumigo, which is a company that does exactly that. Here's a screenshot that shows this behavior:
What is an alternative to firebase for user management/auth for python apps. I know I can use node.js w/ firebase but, I would rather authenticate users through a managed 3rd party API in python using HTTPS requests,if possible. Appery.io has this feature but, I do not need all that comes with appery.io
Check out Amazon Cognito . They offer a quite nice package for small projects. Backendless is also a fantastic service, providing authentication and database with very helpful documentation and also SDK for different platforms including iOS, Android, Javascript, Rest API, Angular, React and React Native. I have been using Backendless for a couple of months and I highly recommend you use it, too.
If you're looking for a self-hosted solution, Keycloak is a pretty robust option. If you want a service, Auth0 and Okta have quite a lot of features. They also offer a free tier with reasonable limits.
You could try using Auth0 for pure authentication management. The Auth0 python package can be found here.
I'm trying to learn the boto API and I noticed that there are two major versions/packages for Python: boto and boto3.
What is the difference between the AWS boto and boto3 libraries?
The boto package is the hand-coded Python library that has been around since 2006. It is very popular and is fully supported by AWS but because it is hand-coded and there are so many services available (with more appearing all the time) it is difficult to maintain.
So, boto3 is a new version of the boto library based on botocore. All of the low-level interfaces to AWS are driven from JSON service descriptions that are generated automatically from the canonical descriptions of the services. So, the interfaces are always correct and always up to date. There is a resource layer on top of the client-layer that provides a nicer, more Pythonic interface.
The boto3 library is being actively developed by AWS and is the one I would recommend people use if they are starting new development.
I've done a lot of research for Google App engine. I ended up with webapp2. Now we realized that AWS has many advantages over GAE for our project.
I feel a little bit overwhelmed because there is so much information about AWS.
I ended up with the nosql dynamoDB solution.
Now I really love lightweight frameworks like webapp2 , its really simple and easy to use. It also don't hide post and get from you.
Now does it makes sense to use webapp2 for AWS ? Mabye there other frameworks that would fit better for our project.
(We want to go in a filehosting direction)
I mean there are literally 50 frameworks for python. And I didn't find a chart with pro/cons.
The last frameworks that I investigated are : web2py , pylon(now pyramids?) and tornado.
To be honest I am really confused.
So if I want to have a lightweight framework for AWS would you recommend me to stay with webapp2?
Resources:
Amazon AWS web framework for Python
Edit #1
Now it's a decision between tornado and webapp2. considering that I want to use dynamoDB.
Tornado is a fast non-blocking web server itself. It's easy to write simple apps in a few minutes, but it has nothing AFAIK like templates, views, etcetera. If you want to serve files it would be very quick to connect a tornado server together with boto (https://github.com/boto/boto) and use dynamoDB or S3.
On the other hand, pylons (now pyrmaid) and we2py are full web application frameworks with no web server. In fact, they come with a lightweight development server (at least pylons), but if you want to put up a site in AWS you should use something like nginx (http://nginx.org/), apache or some other wsgi server (http://wsgi.readthedocs.org/).
If you want to go for a lightweight framework and use python, I'd go for web2py, easy to configure and build apps. (A curious note: it used to be a single .py file) you can try it online: http://www.web2py.com/demo_admin/default/site
Regarding webapp2 I've never used it, but I've heard it's similar to Appengine web framework. So if you're comfortable with it stay there.
Either direction, boto is the python interface for AWS so if you choose python, you'll have to check it out. It is actively mantained.
I'm planning an application running on Google App Engine. The only worry I would have is portability. Or just the option to have the app run on a local, private cluster.
I expected an option for Google App Engine applications to run on other systems, a compatibility layer, to spring up. I could imagine a GAE compatible framework utilizing Amazon SimpleDB or CouchDB to offer near 100% compatibility, if needs be through an abstraction layer. I prefer Python though Java would be acceptable.
However, as far as I know, none such facility exists today. Am I mistaken and if so where could I find this Googe App Engine compatibility layer. If I'm not, the questions is "why"? Are there unforetold technical issues or is there just no demand from the market (which would potentially hint at low rates of GAE adoption).
Regards,
Iwan
The appscale project is designed to do exactly this. See https://github.com/AppScale/appscale/wiki
I could imagine a GAE compatible framework utilizing Amazon SimpleDB or CouchDB
to offer near 100% compatibility
GAE/J uses DataNucleus for persistence. DataNucleus also has plugins for RDBMS, LDAP, XML, Excel, ODF, OODBMS, HBase (HADOOP), and Amazon S3. Consequently the persistence layer (using JDO or JPA) could, in principle, be used across any of those. To write a DataNucleus plugin for Amazon SimpleDB shouldn't be too hard either, or CouchDB.
--Andy (DataNucleus)
Typhoonae Might be interesting to you, it is a new project to implement a full production server stack using exisiting technologies, capable of hosting AppEngine instances. It also aims to do this while staying compatable with the AppEngine API, to allow easy portability. I'm not sure what stage they have reached with the integration, but it should definatley be worth a look.
Another taken from this question:
Waxy
If you develop with web2py your code will run GAE other architectures wihtout changes using any of the 10 supported relational databases. The compatibility layer covers database api (including blobs and listproperty), email, and fetching).