I have a Django app hosted on AWS Lambda (deployed using Zappa tool) an using RDS for hosting the Postgres database. The app is having performance issues with some pages loading very slowly but not others, making me think that the issue is with poorly written data interactions.
I remember at work, the devs were using Azure Application Insights which was monitoring apps at a code level. I think the devs had to insert code throughout their apps to get the monitoring, and I think it monitored on a page by page and code line by line basis.
Is there any similar tool on AWS which I could use with my App?
I appreciate there are much simpler ways of solving my problem but I thought an AWS tool would be something neat to learn.
Yes, there are a few monitoring and troubleshooting tools that work on the AWS environment.
These tools will help you both dive-in into a specific invocation, thus let you understand what wasted the time, but also understand higher-level insights, such as the average time to a DB call or any 3rd party.
Disclosure: I'm working for Lumigo, which is a company that does exactly that. Here's a screenshot that shows this behavior:
Related
So I recently finished my python project, grabbing values from an API and put it into my website.
Now I have no clue how I actually start the website (finding a host) and making it accessible to other people, I thought turning to here might find the solution.
I have done a good amount of research, tried "pythonanywhere" and "google app engine" but seem to not really find a solution.
I was hoping to be able to use "hostinger" as a host, as they have a good price and a good host. Contacted them but they said that they couldn't, though I could upload it to a VPS (which they have). Would it work for me to upload my files to this VPS and therefor get it to a website? or should I use another host?
I just deployed my first app and used Heroku. I'm very happy with it so far and it was fairly straight forward to set up. It does require a basic git understanding but was not hard to learn
A VPS would work, but you'll need to understand basic linux server admin to get things setup properly.
Sounds like you don't have any experience with server admin, so something like App Engine would be great for you. There are a ton of tutorials on the internet for deploying flask to GAE.
I have a backend server running on heroku. Right now for going through logs all I have been using is the 'heroku logs' command. I have been using that command also to track how long different requests to each endpoint are taking.
Is there a better way to see a list of how long requests to different endpoints are taking and a good way to track bottlenecks for what is slowing down these endpoints? Also is there any good add ons for heroku that can point out bad responses that are not status =200?
I am using python with django if that is relevant.
The best tool I found is newrelic.com It hooks nicely into django apps and heroku. It can even show you the bottlenecks due to queries and functions inside your views.
So I've been looking into ways to using Heroku for a small-scale personal project (Python Flask + MongoDB), however I can't seem to find much information on how to do simple continuos integration testing or simple unit testing on a Heroku staging instance. I feel that this would be necessary to make sure that everything will work in production, before actually making it public.
There doesn't seem to be much information on as to how I could achieve this. There are a couple of CI addons that would help, but they currently work only with Ruby/RoR (tddium, Rails on Fire) and proper testing on Heroku seems like a problem that should already be solved by a number of people. Buildpacks seem like a potential way to achieve what I need, but I'd rather use existing tools than re-invent the wheel myself.
So the question is, what are my options?
I wouldn't advise on running your tests on Heroku, as the platform isn't designed to do this. It will probably take you much longer to get the Platform to work than simply using another hosted service. There are lots of other alternatives (e.g. Codeship where I am one of the founders).
At Codeship we are currently working on Python support which will be released soon. MongoDB (as well as lots of other tools) is integrated nicely and works out of the box. We are also focusing very strongly on helping you deploy often and integrate that nicely, so you can work on your app and not your infrastructure.
CircleCi has Python support! It also directly supports MongoDB. You'll be able to set it up very easily.
None of the hosted CI solutions, Circle included, run directly on Heroku. We (Circle - I'm a founder) have looked into it, but the way people write tests make this awkward (they're really designed to be run on the same machine). Heroku is also very slow and memory constrained, while the main goal of a CI system is to get results to you quickly.
I've done a lot of research for Google App engine. I ended up with webapp2. Now we realized that AWS has many advantages over GAE for our project.
I feel a little bit overwhelmed because there is so much information about AWS.
I ended up with the nosql dynamoDB solution.
Now I really love lightweight frameworks like webapp2 , its really simple and easy to use. It also don't hide post and get from you.
Now does it makes sense to use webapp2 for AWS ? Mabye there other frameworks that would fit better for our project.
(We want to go in a filehosting direction)
I mean there are literally 50 frameworks for python. And I didn't find a chart with pro/cons.
The last frameworks that I investigated are : web2py , pylon(now pyramids?) and tornado.
To be honest I am really confused.
So if I want to have a lightweight framework for AWS would you recommend me to stay with webapp2?
Resources:
Amazon AWS web framework for Python
Edit #1
Now it's a decision between tornado and webapp2. considering that I want to use dynamoDB.
Tornado is a fast non-blocking web server itself. It's easy to write simple apps in a few minutes, but it has nothing AFAIK like templates, views, etcetera. If you want to serve files it would be very quick to connect a tornado server together with boto (https://github.com/boto/boto) and use dynamoDB or S3.
On the other hand, pylons (now pyrmaid) and we2py are full web application frameworks with no web server. In fact, they come with a lightweight development server (at least pylons), but if you want to put up a site in AWS you should use something like nginx (http://nginx.org/), apache or some other wsgi server (http://wsgi.readthedocs.org/).
If you want to go for a lightweight framework and use python, I'd go for web2py, easy to configure and build apps. (A curious note: it used to be a single .py file) you can try it online: http://www.web2py.com/demo_admin/default/site
Regarding webapp2 I've never used it, but I've heard it's similar to Appengine web framework. So if you're comfortable with it stay there.
Either direction, boto is the python interface for AWS so if you choose python, you'll have to check it out. It is actively mantained.
I dealt with GAE before and I like simplicity of its webapp. Now I am trying to learn how to work with Amazon EC2. My question - where do I start to make a simple web application that I will be able to access form browser? I suppose I should use WSGI for this purpose. I don't want to use Django as I want to keep the application small and lightweight without unnecessary features. What can you recommend? Thanks.
AWS and GAE are very different. GAE very strictly defines what you can and can't do in terms of development environment. AWS gives you a server to do whatever you want with.
GAE is good when you don't want to have to figure out how all the pieces fit together to scale well. AWS is good when you need flexibility to do whatever you want in your environment.
So to answer your question -- you can use any framework / environment you like.
Personally, I like Django/Pinax for anything requiring a user system. You want a lighter weight system, I've heard good things about Pylons.
Here's a listing of a few others:
http://wiki.python.org/moin/WebFrameworks
Since we're talking AWS, it doesn't have to be python. Ruby on Rails is great.
As already mentioned, you have a lot more flexibility with Amazon than with GAE. If you want to stick with Python and would like to be able to access your app through your browser, you might consider web2py, which enables you to edit and manage your apps through a web-based IDE and admin interface (see demo).
web2py was designed for ease of use and developer productivity, so it's very easy to learn and get going quickly, and you can get a lot done with very little code. Although it's easy to do simple (and even not-so-simple) things quickly and easily, if your app gets more complex, web2py can handle that too, as it is a well-integrated full-stack framework with lots of built-in functionality, including a database abstraction layer, form handling and validation, access control, web services, and easy Ajax integration.
web2py runs on GAE out of the box, and web2py apps are portable from GAE to other platforms without requiring any code changes. However, if you're looking for a simple, scalable cloud hosting option with more flexibility than GAE, you might take a look at the new DotCloud (still in beta), which actually runs on EC2. There's a demo web2py app running there now, and a tutorial explaining the simple deployment process.
If you have any questions about web2py, you'll get lots of help from the friendly and responsive mailing list. I know some of the users have hosted web2py apps on EC2. For example, this demo Q&A site powered by web2py is hosted on EC2.