This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Can select() be used with files in Python under Windows?
On UNIX I am able to pass sys.stdin to select.select in Python. I am attempting to do this on Windows, but select.select in Python on Windows will not allow it.
To more accurately describe what I am doing see https://github.com/eldarion/gondor-client/blob/ccbbf9d4b61ecbc2f66f510b993eb5fba0d81c09/gondor/run.py.
The unix_run_poll function is what I am trying to accomplish on Windows. The basic idea is that I have a socket connection to a server which has hooked up streaming stdin, stdout, stderr to a process running remotely and I am interacting with it from the local client and making it appear as if the local client is running the process.
The win32_run_poll is my attempt at porting it to Windows and it does work, sort of. It is a little wonky and the approach, IMO, is very bad.
Does anyone have suggestions on how this can be improved? The dependency on win32api is less than ideal, but I am okay with keeping it.
On Windows select is only defined for sockets, and will not work for arbitrary file handles (windows has no concept of file descriptors). For more information about this issue, see the msdn documentation, it is also mentioned in the python documentation for the select module.
If you want to use polling for arbitary files, you should look into something that abstracts polling sockets and file handles. This might be the twisted reactor referred to in a comment to your post, or it might be a python binding to libuv, or some other event library of your choice.
Related
I have a CLI application which is executed via Wine on Linux as it needs some closed source DLLs which are only available for Windows. However I also have another tool which is much easier to compile/run on Linux. That Linux application communicates via STDIN/STDOUT.
So I want to spawn a native Linux process from Wine, pass some data (ideally via stdin), wait for the process to complete and read its result (ideally via stdout). This is trivial if both processes would be run in the same OS environment (pure Linux/Posix/Windows) but more complicated in my case.
I can spawn a Linux process using popen but I can't get its stdout (always getting an empty string).
I understand that Wine itself won't/can't provide blocking process creation (probably this creates a lot of edge cases when trying to maintain Windows semantics) as detailed in Wine bug 18335, stackoverflow answer "Execute Shell Commands from Program running in WINE".
However the Wine process is still running under Linux so I think it should be possible to somehow tap into Linux's (= kernel) functionality and do a blocking read.
Does anyone have some pointers on how to launch a Linux process and get its stdout from Wine?
Any other ideas on how to do IPC without complicated server installs?
Theoretically I could use to file system and wait for a result file to appear or run a TCP/HTTP server for communication. Ideally the input is only accessible for the launched application without a server port which every application on the same host can access.
I read about "winelib" as a way to access native Unix functionality from "Windows" programs but I'm not sure I fully grasp how to use it and if it helps me (I can adapt the Wine program but as I mentioned earlier I need to access some closed source DLLs which I can not modify).
Edit: I just noticed the zugbruecke library which allows to communicate with a Windows DLL from (Unix) Python (via a custom wine+TCP connection from Python's multiprocesing). I can not use that as-is (my DLL library uses a lot of pointers so I have wrapped it via pybind11) and it would mean I have to rework my application a bit. However it might result in an elegant solution where the Windows bits are more isolated and I can have more Linux fun. :-)
I have an interactive console application and I need to work with it using Python (send commands and receive output). The application is started by another one, I can't start it from Python script.
Is it possible to connect to already running console application and get access to its stdin/stdout?
Ideally the solution should work both in Windows and Unix, but just Windows version would also be helpful. Currently I am using the solution found here
http://code.activestate.com/recipes/440554/
but it doesn't allow connecting to existing process.
Thanks for any input,
Have you considered using sockets since they are straight forward for simple/streaming. They are also platform independent.
The most critical point is thread safety where having to pass IO streams between threads/processes tends to be hectic.
If on the other hand you use a socket, a lot can be communicated without adding too much complexity to how the processes work(coding an error prone RPC for instance).
try Documentation or
example
I'd like to write a python script to perform some very simple "agentless" monitoring of remote processes running on linux servers.
It would perform the following tasks, in psuedocode:
for each remoteIPAddress in listOfIPAddresses:
log into server#remoteIPAddress via ssh
execute the equivalent of a 'ps -ef' command
grep the result to make sure a particular process (by name) is still running
One way to do this is to have python call shell scripts in a subprocess and parse their output.
That seems pretty inefficient. Is there a better way to do this via python libraries?
All I could find via research here and elsewhere was:
psutil - looks like it doesn't do remote monitoring, so I'd have to run agents on the remote machines to report stats back via RPC.
pymeter - I would have to write my own plugin for monitoring a specific remote service.
stackoverflow #4546492 - Some helpful links but the poster was looking for a different solution.
Thanks, and please go easy on me, it's my first question :-)
The Fabric library may be of interest to you.
Check out paramiko. You can use it to ssh into the server and run commands. You can then parse the results and do what you'd like with them.
Taking cues from the answers above, I investigated Fabric and found the following presentation particularly interesting/helpful. It is an overview of three libraries -- Fabric, Cuisine, and Watchdog -- for server monitoring and administration.
For posterity:
Using Fabric, Cuisine, and Watchdog for server administration in Python
It might be heavier than what you're looking for, but Zenoss supports agentless monitoring.
paramiko and Fabric, suggested in the other answers, are great options too.
Why don't you use a dedicated monitoring tool like Nagios ?
Nagios has agent and agent less monitoring through NRPE plugins and SSH plugins etc.
Try it out.
I've written a specialized JSON-RPC server and just started working my way up into the application logic and finding it is a tad annoying to constantly having to stop/restart the server to make certain changes.
Previously I had a handler that ran in intervals to compare module modified time stamps with the past check then reload the module as needed. Unfortunately I don't trust it to work correctly now.
Is there a way for a reactor to stop and restart itself in a manner similar to Paster's Reloadable HTTPServer?
Shipped with Twisted is the twisted.python.rebuild module, so that is probably a good place to start.
Also see this SO question: Checking for code changes in all imported python modules
You could write something similar to paster's reloader, that would work like this:
start your main function, and before importing / using any twisted code, fork/spawn a subprocess.
In the subprocess, run your twisted application.
In the main process, run your code which checks for changed files. If code has changed, reload the subprocess.
However, the issue here is that unlike a development webserver, most twisted apps have a lot more state and just flat out killing / restarting the process is a bad idea, you may lose some state.
There is a way to do it cleanly:
When you spawn the twisted app, use subprocess.Popen() or similar, to get stdin/stdout pipes. Now in your subprocess, use the twisted reactor to listen on stdin (there is code for this in twisted, see twisted.internet.stdio which allows you to have a Protocol which talks to a stdio transport, in the usual twisted non-blocking manner).
Finally, when you decide it's time to reload, write something to the stdin of the subprocess telling it to shutdown. Now your twisted code can respond and shut down gracefully. Once it's cleanly quit, your master process can just spawn it again.
(Alternately you can use signals to achieve this, but this may not be OS portable)
I've been searching for a library that can access multiple ssh connections at once, Ruby has a Net::SSH::Multi module that allows multiple ssh connections at once. However I rather prefer coding this in Python, are there any similar SSH module for python?
Paramiko is Python's SSH library.
I've never tried concurrent connections with Paramiko, but this answer says it's possible, and this little script seems to make multiple connections in different threads.
The Paramiko mailing list also confirms it's possible to make multiple connections by forking -- there was a security issue regarding that, and it was patched in early 2008.