This could be a dumb question... but are there any PaaS platforms that would allow me to host python code which could control my local app via sockets?
We automate a lot of application tasks via python, but ideally we'd like to host the code.
The Cloudify Open Source PaaS stack allows you to run whatever code you want. So as part of your Cloudify 'recipe' you can install python and run your code. Cloudify will handle the provisioning of machines, logging, etc.
You can run Cloudify in one of the supported clouds, or using your local available machines using the Bring-Your-Own-Node cloud driver: http://www.cloudifysource.org/guide/2.2/setup/configuring_byon.html.
Disclaimer: I work for GigaSpaces, developer of the Cloudify open-source PaaS stack.
Related
I have an existing .NET Core / asp.net app service hosted on Azure. I need to call (on demand) a python script to return data based on custom user input.
It does not appear that I can use IronPython, since I need python modules that are built in CPython, which unfortunately aren't supported by IronPython.
The two options I see are:
I might be able to install the right python version and libraries on the app service and call it from the .NET code. This seems like it might be deprecated: https://learn.microsoft.com/en-us/visualstudio/python/publishing-python-web-applications-to-azure-from-visual-studio?view=vs-2017
I can create a whole new and separate app service for just the python script and call it as a REST API on demand from the .NET app service. This seems like overkill, and introduces the problem of opening up a whole new service publicly, which I don't want to do. This also appears to have the limitation that Flask isn't mean for production, so hosting many calls at once is not really workable. https://learn.microsoft.com/en-us/visualstudio/python/publish-to-app-service-windows?view=vs-2017
What is the best way to call a python script on demand from .NET app service on Azure?
Per my experience, there are two ways to call a Python script in C# without IronPython.
Directly use System.Diagnostics.Process in C# to run a command as same as the SO thread Run Command Prompt Commands to get the result via parse the content of the process standard output. Simply to do it, you can use py2exe to wrap a Python script as a .exe file to avoid for installing Python modules and setting environment variables on Azure App Service. However, considering for concurrency, it's not a good idea for performance.
The second option as you said is to deploy a Python script as a REST API in the same instance of Azure Web App. You can follow the blog Deploying multiple virtual directories to a single Azure Website to deploy a flask app with your Python script as a child project via Visual Studio with PTVS to expose an API url like https://<your web app name>.azurewebsites.net/pyapi which can be called from your ASP.NET via HttpClient. I tried this solution, it works.
Note: Due to the restriction of Azure Web App sandbox for Local Address Requests, you have to use <your web app name>.azurewebsites.net as hostname, neither localhost or 127.0.0.1.
I'm getting my feet wet with GCP and GAE, also nodejs and python and networking (I know).
[+] What I have:
Basically I have some nodejs code that takes in some input and is supposed to then send that input to some python code that will do more stuff to it. My first idea was to deploy the nodejs code via GAE, then host the python code in a python server, then make post requests from the nodejs front-end to the python server backend.
[+] What I would like to be able to do:
just deploy both my nodejs code and my python code in the same project and instance of GAE so that the nodejs is the frontend that people see but so that the python server is also running in the same environment and can just communicate with the nodejs without sending anything online.
[+] What I have read
https://www.netguru.co/blog/use-node-js-backend
Google App Engine - Front and Backend Web Development
and countless other google searches for this type of setup but to no avail.
If anyone can point me in the right direction I would really appreciate it.
You can't have both python and nodejs running in the same instance, but they can run as separate services, each with their own instance(s) inside the same GAE app/project. See Service isolation and maybe Deploying different languages services to the same Application [Google App Engine]
Using post requests can work pretty well, but will likely take some effort to ensure no outside access.
Since you intend to use as frontend the nodejs service you're limited to using only the flexible environment for it, which limits the inter-service communication options - you can't use push queues (properly supported only in the standard environment) which IMHO would be a better/more secure solution than post requests.
Another secure communication option would be for the nodejs service to place the data into the datastore and have the python service pick it up from there - the datastore is shared by all instances/versions/services inside the same GAE app. Also more loosely coupled IMHO - each service can function (at least for a while) without the other being alive (not possible if using the post requests).
Maybe of interest: How to tell if a Google App Engine documentation page applies to the standard or the flexible environment
UPDATE:
Node.JS is currently available in the standard environment as well, so you can use those features, see:
Now, you can deploy your Node.js app to App Engine standard environment
Google App Engine Node.js Standard Environment Documentation
So my team and I have bought into Docker - it is fantastic for deployment and testing. My real question is how to set up a great developer experience, specifically around writing Python apps, but this question could be generalized to nodejs, Java, etc.
The problem: When writing a Python app, I really like having decent linting/autocomplete functionality, there are some really good editors out there (Atom, VSCode, PyCharm) that provide these, but most really want a Python install on the local disk. The real advantage of Docker is that all of the core language and any project libraries can all be in the container, so reproducing all of that on the host machine just for developing is a pain.
I know that PyCharm pro does support Docker and docker-compose, but I found it quite sluggish and a lot of the test running capabilities were busted. On top of that, I really would like something that I can commit to version control so that the team can share dev setup and people don't have to repeat all of the steps for their own system.
A few Ideas that I had were:
Install an editor (like Atom) in a sidecar Docker container and use X11 forwarding
Use a browser based editor such as https://c9.io/ in a container - this seems most promising
Install some agent in a dev container that could handle autocomplete/linting, etc. and connect to it from a locally running editor - I think this would be the best solution, but I also think that right now it actually doesn't exist.
Has anyone had luck setting up a more productive development environment besides just mounting volumes and editing text?
You should use an 'advanced' IDE like IntelliJ (Pycharm) and configure a remote Python SDK using SSH-Access to your App-Docker-Container (using a shared ssh-key to auth against the app-container with a preinstalled openssh server and preconfigured authorized_keys file).
You can share this SDK information in your project file with all devs, so they wlll have this setup out of the box
1) This will ensure, your IDE knows about all the python libs/symbols available/installed in your docker-container during runtime. It will also enable you to properly debug remotely at the same time
2) This ensures, you have an IDE at your hand including a lot of important additional features like the inspector, 3way duff, search in path.. . hardly any of the Browser-Based IDEs will catch up with Pycharm at this point IMHO
Of course, as already mentioned in the comments, you need to share aka mount your code into the container. On linux, you plainly use host-volume-mounts from your local src folder to the container.
On OSX, you will run into performance issues when using host mounts. You might use something like http://docker-sync.io ( i am biased - there are also a lot of other similar tools )
I know this is an old question, but as I stumbled across it while trying to see what other editors might offer in this space, I would like to point out Visual Studio Code's notion of a Dev Container, which seems to provide the best level of integration I've seen for this so far. I'm hoping to see this turn into an industry trend myself.
Could use x11docker
x11docker allows to run graphical desktop applications (and entire desktops) in Docker Linux containers.
Docker allows to run applications in an isolated container environment. Containers need much less resources than virtual machines for similar tasks.
Docker does not provide a display server that would allow to run applications with a graphical user interface.
x11docker fills the gap. It runs an X display server on the host system and provides it to Docker containers.
Additionally x11docker does some security setup to enhance container isolation and to avoid X security leaks. This allows a sandbox environment that fairly well protects the host system from possibly malicious or buggy software.
https://github.com/mviereck/x11docker
https://github.com/mviereck/x11docker/wiki (extensive! knowledge)
https://dev.to/brickpop/my-dream-come-true-launching-gui-docker-sessions-with-dx11-in-seconds-1a53
How can we call the CLI executables commands using Python
For example i have 3 linux servers which are at the remote location and i want to execute some commands on those servers like finding the version of the operating system or executing any other commands. So how can we do this in Python. I know this is done through some sort of web service (SOAP or REST) or API but i am not sure....... So could you all please guide me.
Depends on how you want to design your software.
You could do stand-alone scripts as servers listening for requests on specific ports,
or you could use a webserver which runs python scripts so you just have to access a URL.
REST is one option to implement the latter.
You should then look for frameworks for REST development with python, or if it’s simple logic with not so many possible requests can do it on your own as a web-script.
Maybe you should take a look at Pushy, which allows to connect to remote machines through SSH and make them execute various Python functions. I like using it because there are no server-side dependencies except the SSH server and a Python interpreter, and is therefore really easy to deploy.
Edit: But if you wish to code this by yourself, i think SOAP is a nice solution, the SOAPpy module is great and very easy to use.
You can use Twisted,
It is easy create ssh clients or servers.
Examples:
http://twistedmatrix.com/documents/current/conch/examples/
I want to provide my colleagues with an interface (using Windows Forms or WPF) to control the states of virtual machines (KVM based) on a linux host. On the command line of this server, I'm using a tool, called libvirt, which provides python bindings to access its functionality.
What whould be the best pratice to remotely access several function like libvirt or reading logfiles on the server. I thought about a REST Full Webservice generated by Python. Are there other viable options to consider?
Thanks,
Henrik
I'd develop an intranet web application, using any python web framework of choice.
That way you don't have to develop/install software on your client. They just point the browser and it works.
Because you are using a server-side tool that has Python bindings, you should give a serious look at PYRO which is a Python RPC library.
http://pyro.sourceforge.net/
To use this you would also have to use Python on the client, but that shouldn't be a problem. If you haven't start writing your client, then you could do it all in IronPython. Or, if you need to add this to an already existing client, then you could still bind in either IronPython or CPython as an embedded scripting engine.
For more on PYRO and Ironpython, see this wiki page http://www.razorvine.net/python/PyroAndIronpython
Proxmox VE is a complete solution to manage KVM (and OpenVZ) based virtual machines, including a comprehensive web console, so maybe you can get a full solution without developing anything?