So I've been looking into ways to using Heroku for a small-scale personal project (Python Flask + MongoDB), however I can't seem to find much information on how to do simple continuos integration testing or simple unit testing on a Heroku staging instance. I feel that this would be necessary to make sure that everything will work in production, before actually making it public.
There doesn't seem to be much information on as to how I could achieve this. There are a couple of CI addons that would help, but they currently work only with Ruby/RoR (tddium, Rails on Fire) and proper testing on Heroku seems like a problem that should already be solved by a number of people. Buildpacks seem like a potential way to achieve what I need, but I'd rather use existing tools than re-invent the wheel myself.
So the question is, what are my options?
I wouldn't advise on running your tests on Heroku, as the platform isn't designed to do this. It will probably take you much longer to get the Platform to work than simply using another hosted service. There are lots of other alternatives (e.g. Codeship where I am one of the founders).
At Codeship we are currently working on Python support which will be released soon. MongoDB (as well as lots of other tools) is integrated nicely and works out of the box. We are also focusing very strongly on helping you deploy often and integrate that nicely, so you can work on your app and not your infrastructure.
CircleCi has Python support! It also directly supports MongoDB. You'll be able to set it up very easily.
None of the hosted CI solutions, Circle included, run directly on Heroku. We (Circle - I'm a founder) have looked into it, but the way people write tests make this awkward (they're really designed to be run on the same machine). Heroku is also very slow and memory constrained, while the main goal of a CI system is to get results to you quickly.
Related
I'm working in a small CG animation company that does need small tools for production and CG artists. I'm coming from an animator's background, so my knowledge of code is extremely basic, but I'm learning.
Here is the situation: I write python tools, most are using PySide for GUI. Now I need to regularly and easily deploy tools and updates to the machines in our studio (about 20 computers).
We do want to do things the right way and are willing to learn, but the amount of information is quite overwhelming for I am a neophyte.
I keep reading about various scary keywords that seem to be the answer I'm looking for, such as DevOps, CI/CD, Docker, Kubernetes...
We are setting up a Gitlab instance on our local server, and now I'm asking what would be the most suitable way to easily deploy my scripts and their requested environment easily onto the company's computers (MacOs, Windows, and Linux), and how should I package them.
I hope my question isn't vague but if so, please tell me and I'll try to be more precise.
Thanks a lot
pyupdater seems to do just what you're trying to do.
have a look at pyupdater.org and this demo.
As part of an effort to make the scikit-image examples gallery interactive, I would like to build a web service that receives a Python code snippet, executes it, and provides me with the generated output image.
For safety, the Python instances launched should be sandboxed and resource controlled, so I was thinking of using LXC containers.
Is this a good way to approach the problem? If so, what is the recommended way of launching one Python VM per request?
Stefan, perhaps "Docker" could be of use? I get the impression that you could constrain the VM that the application is run in -- an example web service:
http://docs.docker.io/en/latest/examples/python_web_app/
You could try running the application on Digital Ocean, like so:
https://www.digitalocean.com/community/articles/how-to-install-and-use-docker-getting-started
[disclaimer: I'm an engineer at Continuum working on Wakari]
Wakari Enterprise (http://enterprise.wakari.io) is aiming to do exactly this, and we're hoping to back-port the functionality into Wakari Cloud (http://wakari.io) so "published" IPython Notebooks can have some knobs on them for variable input control, then they can be "invoked" in a sandboxed state, and then the output given back to the user.
However for things that exist now, you should look at Sage Notebook. A few years ago several people worked hard on a Sage Notebook Cell Server that could do exactly what you were asking for: execute small code snippets. I haven't followed it since then, but it seems it is still alive and well from a quick search:
http://sagecell.sagemath.org/?q=ejwwif
http://sagecell.sagemath.org
http://www.sagemath.org/eval.html
For the last URL, check out Graphics->Mandelbrot and you can see that Sage already has some great capabilities for UI widgets that are tied to the "cell execution".
I think docker is the way to go for this. The instances are very light weight, and docker is designed to spawn 100s of instances at a time (Spin up time is fractions of a second vs traditional VMs couple of seconds). Configured correctly I believe it also gives you a complete sandboxed environment. Then it matters not about trying to sandbox python :-D
I'm not sure if you really have to go as far as setting up LXC containers:
There is seccomp-nurse, a Python sandbox that leverages the seccomp feature of the Linux kernel.
Another option would be to use PyPy, which has explicit support for sandboxing out of the box.
In any case, do not use pysandbox, it is broken by design and has severe security risks.
I need to write a very light database (sqlite is fine) app that will initially be run locally on a clients windows PC but could, should it ever be necessary, be upgraded to work over the public interwebs without a complete rewrite.
My end user is not very technically inclined and I'd like to keep things as simple as posible. To that end I really want to avoid having to install a local webserver, however "easy" that may seem to you or I. Django specifically warns not to use it's inbuilt webserver in production so my two options seem to be...
a) Use django's built in server anyway while the app is running locally on windows and, if it ever needs to be upgraded to work over the net just stick it behind apache on a linux box somewhere in the cloud.
b) Use a framework that has a more robust built in web server from the start.
My understanding is that the only two disadvantages of django's built in server are a lack of security testing (moot if running only locally) and it's single threaded nature (not likely to be a big deal either for a low/zero concurrency single user app running locally). Am I way off base?
If so, then can I get some other "full stack" framework recommendations please - I strongly prefer python but I'm open to PHP and ruby based solutions too if there's no clear python winner. I'm probably going to have to support this app for a decade or more so I'd rather not use anything too new or esoteric unless it's from developers with some serious pedigree.
Thanks for your advice :)
Roger
I find Django's admin very easy to use for non-technical clients. In fact, that is the major consideration for my using Django as of late. Once set up properly, non-technical people can very easily update information, which can reflected on the front end immediately.
The client feels empowered.
Use Django. It's very simple for you to get started. Also, they have the best documentation. Follow the step by step app creating tutorial. Django supports all the databases that exist. Also, the built in server is very simple to use for the development and production server. I would highly recommend Django.
How can i create a safe environment to create new and play with my web applications?
My ingredients so far: python,flask,github,heroku --- I have effectively created a app using flask and heroku (it just says hello world)
I have some knowledge of python. Have read through but hardly understand the flaskr tutorial for Flask. I do however envision that i'm going to be very frustrated if i can't have some kind of version history of my stumblings.
Without being sure i feel like the recipe might include: Github and virtualenv and some coping pasting of directories...
whats a recommended way to do this?
Just setup a GitHub repository and start hacking away with Flask.
Some pointers:
Use SQLAlchemy for database operations. It's secure and easy to use.
Either dive right into your project or make some simple test ones with Flask. It's a really beautiful framework to work with and is easy to learn, so just try to accomplish some simple tasks and get used to how it works.
I suggest you to use virtualenv and virtualenvwrapper, this way you can work on isolated pythons without filling your system's python with many eggs and switch between them easily.
I am planning to do a small web application that will be distributed as a single installable. I have plans to develop this application in either Python/Django or Ruby On Rails. (I am a Java/C++ programmer, hence both these languages are new to me).
My main concern is about the size and simplicity of final installable (say setup.exe). I want it to be small in size and also should be able to pack all required components in it.
Which one among Python/Django and Ruby On Rails is suitable for me?
I personally prefer Python/django. Size is small given u have necessary things installed.
With disk space at the current price, size shouldn't matter. Give both a try and figure out which will be easier for you to learn and maintain. Despite the fact that people believe that when you know one language, you know all, that's only true as long as you write code on the "hello world" level.
One option with Ruby on Rails is to go with a JRuby deployment which would allow you to pack it all into a single .war file. This would require the person deploying the web application to have a java web application server (Jetty is probably the smallest and easiest to bundle).
With Rails, you are generally going to have to install Ruby and any required ruby gems. The Ruby install is going to be machine specific- different for Windows/Linux. Everything else should be easily scripted. If you go with an Apache Passenger (mod_ruby) solution, you will need to get that installed as well.
In reality, I haven't run into many server applications with simple, compact installs.
I just used heroku to deploy a blog written in Rails, and it was a fantastically easy experience. If you're interested in simplicity, it's probably the most simple deploy I've ever experienced.
I don't think you can get them both. I'm sorry to say this but you have to choose which one is more important to you.
Django application is smaller in size because many things is already provided out of the box, but deployment is not as easy.
On the other hand, RoR apps deployment is easier (both Ruby MRI or JRuby) but the application's size is naturally larger given you have to install other gems and Ruby On Rails plugins.
If you are experienced with Java and concerned about deploying Django and Rails apps, I'd recommend you give JRuby a try. This will give you several benefits from a Java-perpective:
You can call Java-classes and components from your Ruby/Rails app
You can use a familiar IDE such as Netbeans
You can package and deploy our entire Rails app as a single WAR-file with all dependencies included
With the cheeseshop, any python application can be made installable with a single command. I'm a big fan of Django, but it will require you to hook into an external webserver, as the built in server is for development only. You might look for something that has a more robust builtin web server if you want something you can just plunk down and start running. Twisted might meet your needs, though there's a bit more of a learning curve on that. I'm not sure how other python or ruby apps stand up on this front.