Is there a guide or tutorial on how to use the quartz Scheduler with Python.
Is there an existing API for Python?
Given that Quartz is a Java application/library, the simplest thing to do may be to run it within Jython.
Failing that, and if you simply want to control the configuration of jobs from Python, perhaps the JDBC-JobStore is of use, and you could write jobs into the database via Python. You'll still need an instance of Quartz running in a JVM. However, I'm not sure how well supported that is as a scenario.
Finally, you may be able to talk to a running Quartz instance via JMX. You'll need a JMX client, and consequently you'll need to spawn off a Java JMX client from your Python process.
I think (regrettably) there's a non-trivial mismatch between a Quartz instance (in Java) and your Python code.
Related
Basically, I have a use case where I'm trying to set up a python process to run in containers on EKS to consume data from Kafka and process the same. So far, I've gathered that I could consider using the following three frameworks for this:
Apache Spark(Pyspark)
kafka-python Module
Confluent-kafka-python Module.
I'm looking for a solution that has the following features:
Seamless scalability
If a container dies, and it gets spun up again, it should know where to pick up the offset where it left off from.
Any advice on which of the above mentioned libraries would be most suitable for this? Thanks in advance!
Since you're looking for a mere library, I would avoid PySpark since it's a whole framework with it's own deployment model that is going to have a major impact on how you structure your application. Unless you explicitly know that you need the specific features of Spark, I would skip this one. Similarly, Apache Flink and Beam frameworks exist with Python support.
Confluent Kafka is a Python wrapper around the librdkafka C library, so you have the guarantee it's going to be the most performant and compatible option. Also, since it's maintained by Confluent it should continue being updated by very knowledgeable professional in the long term: it's a dependable solution.
You also have Kafka python which is implemented fully in python. It might "feel" more pythonic, but may also be less performant. At the time of writing this the repo does not show a lot of recent activity.
There's also faust that aimed to be Kafka Streams-like, and aiokafka which uses AsyncIO support
Are there any command-line options or configurations that forbids Python from writing to disk?
I know I can hack open but it doesn't sound very safe.
I've hosted some Python tutorials I wrote myself on my website for friends who want to learn Python, and I want them to have access to a Python console so they can try as they learn. This is done by creating a Python subprocess from the http server.
However, I do not want them to accidentally or intentionally damage my server, so I need to forbid the Python process from writing anything to disk.
Also I'm running the server on Ubuntu Linux so doing it Python-wise or system-wise are both OK.
I doubt there's a way to do this in the interpreter itself: there are way too many things to patch (open, subprocess, os.system, file, and probably others). I'd suggest looking into a way of containerizing the python runtime via something like Docker. The containerization gives some guarantees restricting access, though not as much as virtualization. See here for more discussion about the security implications.
Running a jupyter/ipython notebook in the docker container would probably be the easiest way to expose a web-frontend. jupyter provides a collection of docker containers for this purpose: see https://github.com/jupyter/tmpnb and https://github.com/jupyter/docker-stacks
I have a python server that I need to run in both a Linux and Windows environment, and my question is about deployment. What is the best method for deploying the solution instead of just double clicking on the file and running it?
Since I use the server_forever() on the server, I can just run the script from command line, but this keeps the python window open. If I log off the machine, naturally the process will stop. So what is the best method for deploying a python script that needs to keep running if the user is logged in or off a machine.
Since I am going to be using multiple environment, Linux and Windows, can you please be specific in what OS you are talking about?
For windows, I was thinking of running the script 'At Startup' using the Windows scheduler. But I wanted to see if anyone had a better option. For linux, I really don't know what to create. I am assuming a CRON job?
Deployment does refer to coding, so using serve_forever() on a multiprocessing job manager keeps the python window open upon execution. Is there a way to hide this window through code? Would you recommend using a conversion tool like py2exe instead?
This is the subject matter of a whole library of books, so I will just give an introduction here :-)
You can basically start scripts directly and then have multiple options to do this in a way that they keep running in the background.
If you have certain functionality that needs to run on regular moments, you would do this by scheduling it:
Windows: Windows Scheduler or specific scheduling tools
Linux: Cron
If your problem is that you want to start a script without it closing on you while SSH'ing into Linux, you want to look into the "screen" or "tmux" tools.
If you want to have it started automatically this could be done by using the "At Startup" as you point out and Linux has similar functionalities, but the preferred and more robust way would be to set up a service that is better integrated with the OS.
Windows: Windows Service
Linux: Daemon
Even more capabilities can be yielded by using an application server such a Django
Tomcat (see comment) is an option, but definitely not the standard one; you'll have a hard time finding support both from Tomcat people running Python or Python people running their stuff on Tomcat. That being said, I imagine you could enable CGI and have it run a Python command with your script.
Yet, instead of just starting a Python script I would strongly encourage you to have a look at different Python options that are probably available for your specific use case. From lightweight web solutions like Flask over a versatile networking engine like Twisted to a full blown web framework like Django.
They all have rather well-thought-out deployment solutions available. Look up WSGI for more background.
what is the advantage of using a python virtualbox API instead of using XPCOM?
The advantage is that pyvb is lot easier to work with.
On the contrary the documentation for the python API of XPCOM doesn't exist, and the API is not pythonic at all. You can't do introspection to find methods/attributes of an object, etc. So you have to check the C++ source to find how it works or some python scripts already written (like vboxshell.py and VBoxWebSrv.py).
On the other hand pyvb is really just python wrapper that call VirtuaBoxManager on the command line. I don't know if it's a real disadvantage or not?
I would generally recommend against either one. If you need to use virtualization programmatically, take a look at libvirt, which gives you cross platform and cross hypervisor support; which lets you do kvm/xen/vz/vmware later on.
That said, the SOAP api is using two extra abstraction layers (the client and server side of the HTTP transaction), which is pretty clearly then just calling the XPCOM interface.
If you need local host only support, use XPCOM. The extra indirection of libvirt/SOAP doesn't help you.
If you need to access virtualbox on a various hosts across multiple client machines, use SOAP or libvirt
If you want cross platform support, or to run your code on Linux, use libvirt.
From sun's site on VirtualBox python APIs:
SOAP allows to control remote VMs over
HTTP, while XPCOM is much more
high-performing and exposes certain
functionality not available with SOAP.
They use very different technologies
(SOAP is procedural, while XPCOM is
OOP), but as it is ultimately API to
the same functionality of the
VirtualBox, we kept in bindings
original semantics, so other that
connection establishment, code could
be written in such a way that people
may not care what communication
channel with VirtualBox instance is
used.
From that article, I'm having trouble seeing the difference between "python virtualbox API" and "XPCOM". Could you provide a link to the API you're thinking of?
I want to provide my colleagues with an interface (using Windows Forms or WPF) to control the states of virtual machines (KVM based) on a linux host. On the command line of this server, I'm using a tool, called libvirt, which provides python bindings to access its functionality.
What whould be the best pratice to remotely access several function like libvirt or reading logfiles on the server. I thought about a REST Full Webservice generated by Python. Are there other viable options to consider?
Thanks,
Henrik
I'd develop an intranet web application, using any python web framework of choice.
That way you don't have to develop/install software on your client. They just point the browser and it works.
Because you are using a server-side tool that has Python bindings, you should give a serious look at PYRO which is a Python RPC library.
http://pyro.sourceforge.net/
To use this you would also have to use Python on the client, but that shouldn't be a problem. If you haven't start writing your client, then you could do it all in IronPython. Or, if you need to add this to an already existing client, then you could still bind in either IronPython or CPython as an embedded scripting engine.
For more on PYRO and Ironpython, see this wiki page http://www.razorvine.net/python/PyroAndIronpython
Proxmox VE is a complete solution to manage KVM (and OpenVZ) based virtual machines, including a comprehensive web console, so maybe you can get a full solution without developing anything?