I'm working in a small CG animation company that does need small tools for production and CG artists. I'm coming from an animator's background, so my knowledge of code is extremely basic, but I'm learning.
Here is the situation: I write python tools, most are using PySide for GUI. Now I need to regularly and easily deploy tools and updates to the machines in our studio (about 20 computers).
We do want to do things the right way and are willing to learn, but the amount of information is quite overwhelming for I am a neophyte.
I keep reading about various scary keywords that seem to be the answer I'm looking for, such as DevOps, CI/CD, Docker, Kubernetes...
We are setting up a Gitlab instance on our local server, and now I'm asking what would be the most suitable way to easily deploy my scripts and their requested environment easily onto the company's computers (MacOs, Windows, and Linux), and how should I package them.
I hope my question isn't vague but if so, please tell me and I'll try to be more precise.
Thanks a lot
pyupdater seems to do just what you're trying to do.
have a look at pyupdater.org and this demo.
Related
I was studying about DApps and I found most of them are written using Solidity and JS. Also, I found that there's a web3.py library for people who love python (and I didn't search for the same thing in ruby programming language). But, I have a question, is there any decentralized system like heroku? I mean, writing a piece of code (e.g. Sinatra/Flask based API) and then deploy on that network?
Regards.
I think that you're asking: is there an application that will let me build applications and deploy them to the network?
You might take a look at Truffle. You can develop and deploy applications in Javascript.
However, if you like Python (like I do), I just released an open source tool called Ezo. It's in alpha, so patience please. :) It's a command-line tool for building off-chain event responders (or oracles) for Ethereum. Here's an article on how to use it.
You'll also want to download and install Ganache to simulate a working Ethereum EVM. You'll likely lean on it a lot during any development you may do. Good luck.
As part of an effort to make the scikit-image examples gallery interactive, I would like to build a web service that receives a Python code snippet, executes it, and provides me with the generated output image.
For safety, the Python instances launched should be sandboxed and resource controlled, so I was thinking of using LXC containers.
Is this a good way to approach the problem? If so, what is the recommended way of launching one Python VM per request?
Stefan, perhaps "Docker" could be of use? I get the impression that you could constrain the VM that the application is run in -- an example web service:
http://docs.docker.io/en/latest/examples/python_web_app/
You could try running the application on Digital Ocean, like so:
https://www.digitalocean.com/community/articles/how-to-install-and-use-docker-getting-started
[disclaimer: I'm an engineer at Continuum working on Wakari]
Wakari Enterprise (http://enterprise.wakari.io) is aiming to do exactly this, and we're hoping to back-port the functionality into Wakari Cloud (http://wakari.io) so "published" IPython Notebooks can have some knobs on them for variable input control, then they can be "invoked" in a sandboxed state, and then the output given back to the user.
However for things that exist now, you should look at Sage Notebook. A few years ago several people worked hard on a Sage Notebook Cell Server that could do exactly what you were asking for: execute small code snippets. I haven't followed it since then, but it seems it is still alive and well from a quick search:
http://sagecell.sagemath.org/?q=ejwwif
http://sagecell.sagemath.org
http://www.sagemath.org/eval.html
For the last URL, check out Graphics->Mandelbrot and you can see that Sage already has some great capabilities for UI widgets that are tied to the "cell execution".
I think docker is the way to go for this. The instances are very light weight, and docker is designed to spawn 100s of instances at a time (Spin up time is fractions of a second vs traditional VMs couple of seconds). Configured correctly I believe it also gives you a complete sandboxed environment. Then it matters not about trying to sandbox python :-D
I'm not sure if you really have to go as far as setting up LXC containers:
There is seccomp-nurse, a Python sandbox that leverages the seccomp feature of the Linux kernel.
Another option would be to use PyPy, which has explicit support for sandboxing out of the box.
In any case, do not use pysandbox, it is broken by design and has severe security risks.
So I've been looking into ways to using Heroku for a small-scale personal project (Python Flask + MongoDB), however I can't seem to find much information on how to do simple continuos integration testing or simple unit testing on a Heroku staging instance. I feel that this would be necessary to make sure that everything will work in production, before actually making it public.
There doesn't seem to be much information on as to how I could achieve this. There are a couple of CI addons that would help, but they currently work only with Ruby/RoR (tddium, Rails on Fire) and proper testing on Heroku seems like a problem that should already be solved by a number of people. Buildpacks seem like a potential way to achieve what I need, but I'd rather use existing tools than re-invent the wheel myself.
So the question is, what are my options?
I wouldn't advise on running your tests on Heroku, as the platform isn't designed to do this. It will probably take you much longer to get the Platform to work than simply using another hosted service. There are lots of other alternatives (e.g. Codeship where I am one of the founders).
At Codeship we are currently working on Python support which will be released soon. MongoDB (as well as lots of other tools) is integrated nicely and works out of the box. We are also focusing very strongly on helping you deploy often and integrate that nicely, so you can work on your app and not your infrastructure.
CircleCi has Python support! It also directly supports MongoDB. You'll be able to set it up very easily.
None of the hosted CI solutions, Circle included, run directly on Heroku. We (Circle - I'm a founder) have looked into it, but the way people write tests make this awkward (they're really designed to be run on the same machine). Heroku is also very slow and memory constrained, while the main goal of a CI system is to get results to you quickly.
I want to remove as much complexity as I can from administering Python in on Amazon EC2 following some truly awful experiences with hosting providers who claim support for Python. I am looking for some guidance on which AMI to choose so that I have a stable and easily managed environment which already included Python and ideally an Apache web server and a database.
I am agnostic to Python version, web server, DB and OS as I am still early enough in my development cycle that I can influence those choices. Cost is not a consideration (within bounds) so Windows will work fine if it means easy administration.
Anyone have any practical experience or recommendations they can share?
Try the Ubuntu EC2 images. Python 2.7 is installed by default. The rest you just apt-get install and optionally create an image when the baseline is the way you want it (or just maintain a script that installs all the pieces and run after you create the base Ubuntu instance).
If you can get by with using the Amazon provided ones, I'd recommend it. I tend to use ami-84db39ed.
Honestly though, if you plan on leaving this running all the time, you would probably save a bit of money by just going with a VPS. Amazon tends to be cheaper if you are turning the service on and off over time.
I am planning to do a small web application that will be distributed as a single installable. I have plans to develop this application in either Python/Django or Ruby On Rails. (I am a Java/C++ programmer, hence both these languages are new to me).
My main concern is about the size and simplicity of final installable (say setup.exe). I want it to be small in size and also should be able to pack all required components in it.
Which one among Python/Django and Ruby On Rails is suitable for me?
I personally prefer Python/django. Size is small given u have necessary things installed.
With disk space at the current price, size shouldn't matter. Give both a try and figure out which will be easier for you to learn and maintain. Despite the fact that people believe that when you know one language, you know all, that's only true as long as you write code on the "hello world" level.
One option with Ruby on Rails is to go with a JRuby deployment which would allow you to pack it all into a single .war file. This would require the person deploying the web application to have a java web application server (Jetty is probably the smallest and easiest to bundle).
With Rails, you are generally going to have to install Ruby and any required ruby gems. The Ruby install is going to be machine specific- different for Windows/Linux. Everything else should be easily scripted. If you go with an Apache Passenger (mod_ruby) solution, you will need to get that installed as well.
In reality, I haven't run into many server applications with simple, compact installs.
I just used heroku to deploy a blog written in Rails, and it was a fantastically easy experience. If you're interested in simplicity, it's probably the most simple deploy I've ever experienced.
I don't think you can get them both. I'm sorry to say this but you have to choose which one is more important to you.
Django application is smaller in size because many things is already provided out of the box, but deployment is not as easy.
On the other hand, RoR apps deployment is easier (both Ruby MRI or JRuby) but the application's size is naturally larger given you have to install other gems and Ruby On Rails plugins.
If you are experienced with Java and concerned about deploying Django and Rails apps, I'd recommend you give JRuby a try. This will give you several benefits from a Java-perpective:
You can call Java-classes and components from your Ruby/Rails app
You can use a familiar IDE such as Netbeans
You can package and deploy our entire Rails app as a single WAR-file with all dependencies included
With the cheeseshop, any python application can be made installable with a single command. I'm a big fan of Django, but it will require you to hook into an external webserver, as the built in server is for development only. You might look for something that has a more robust builtin web server if you want something you can just plunk down and start running. Twisted might meet your needs, though there's a bit more of a learning curve on that. I'm not sure how other python or ruby apps stand up on this front.