Is there a way to use Python's socket module to replicate some of the functionality of the Linux ss utility?
ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state informations than other tools.
Pretty much all of the socket documentation revolves around creating new sockets, but I can't find any information on getting statistics from the system's sockets.
I don't see how such functionality could be implemented purely with the socket module. The socket module is for working directly with sockets: opening/closing, sending/receiving, etc. It's simply a thin wrapper over the standard BSD socket interface.
On the other hand, getting metadata about existing sockets already allocated on the system requires knowledge of the other processes running on a system. This has little to do with actually manipulating a socket, and much more to do with monitoring other processes and/or their file descriptors.
For example, it seems that both ss and netstat are actually implemented (at least on Linux) by reading and parsing the /proc pseudo-filesystem. (See here and here, for examples.) The kernel manages the processes and their opened sockets, and exposes (some of) the information about them to other processes via procfs. This provides a simple and safe way of exporting some of the information about processes to userspace, obviating the need for lots of system calls or reading kernel data structures directly.
Note that it pretty much has to work this way. Strong process isolation necessitates that information about another process's open files, including sockets, has to come through the kernel in some way. That could be either via procfs on Linux, or some kernel-provided API (e.g. libproc on macOS). Anything else would be a massive security hole.
As an alternative to the socket module, you could try the psutil package or something similar. The psutil.net_connections() function seems appropriate.
Related
Is it possible, with Python, to communicate directly on the data link layer, prior-to or outside of, an IP address? Similarly to communicating with USB?
I have a client interested in trying this. As far I can know, there's no way. But I never want to underestimate the power of Python.
There's nothing intrinsic about Python which prevents you from writing your own user-level networking stack. However, if you want to say, access the raw ethernet driver to send raw ethernet packets, that has to be supported by the operating system.
I'll try to paint a vague picture of what's going on. Some of this you may know already (or not). A conventional operating system provides an abstraction called the system call layer to allow programs to interact with hardware. This abstraction is typically somewhat "high level" in that it abstracts away some of the details of the hardware. In an operating system which implements Unix abstractions, one of the network abstraction system calls is socket(int domain, int type, int proto), which creates a new socket endpoint. What's getting abstracted away here? Well, for most protocols, dealing with data-link layer details becomes unnecessary. Obviously you lose some flexibility here so that you can gain safety (you don't have to worry about clobbering other OS data structures if you have raw hardware access) and convenience (most people don't need to implement a user-level networking stack).
So, whether it "can" be done without modifying your kernel depends on what abstractions are provided by the OS. Linux provides the packet(7) interface which allows you to use AF_PACKET as your socket domain. According to the man page, "Packet sockets are used to receive or send raw packets at the device driver (OSI Layer 2) level."
So can this be accessed in Python? You bet!
import socket
s = socket(socket.AF_PACKET, socket.SOCK_RAW)
s.bind(("eth1", 0))
s should now be a socket which you can use to send raw packets. See the other Stack Overflow post for more information about how to do this -- they do a better job than I can. It looks like this technique should work on Windows as well, as I suspect they provide a similar abstraction.
I am using twisted to run a rather complicated server that allows for data collection, communication, and commanding of a hardware device remotely. On the client-side there are a number of data retrieval and command operations available. Typically I use the wxpython reactor to interface with the client reactor, but I would also like to setup a simpler command-line style interface.
Is there a reactor that I can use to setup a local non-blocking python-like or raw_input-style interface for the client? After successful access to the server, the server will occasionally send data down without being requested as a result of server-side events.
I have considered manhole, but I am not interested in accessing the server as an interface, I am strictly interested in accessing the client-side data and commands. This is mostly for debugging, but it can also come in handy for creating a much more rudimentary client interface when needed.
See the stdin.py and stdiodemo.py examples, I think that's similar to what you're aiming for. They demonstrate connecting a protocol (like a LineReceiver) to StandardIO.
I think you could also use a StandardIOEndpoint (and maybe we should update the examples for that), but that doesn't change the way you'd write your protocol.
Can someone please tell how to write a Non-Blocking server code using the socket library alone.Thanks
Frankly, just don't (unless it's for an exercise). The Twisted Framework will do everything network-related for you, so you have to write only your protocol without caring about the transport layer. Writing socket code is not easy, so why not use code somebody else wrote and tested.
Why socket alone? It's so much simpler to use another standard library module, asyncore -- and if you can't, at the very least select!
If you're constrained by your homework's condition to only use socket, then I hope you can at least add threading (or multiprocessing), otherwise you're seriously out of luck -- you can make sockets with timeout, but juggling timing-out sockets without the needed help from any of the other obvious standard library modules (to support either async or threaded serving) is a serious mess indeed-y...;-).
Not sure what you mean by "socket library alone" - you surely will need other modules from the standard Python library.
The lowest level of non-blocking code is the select module. This allows you to have many simultaneous client connections, and reports which of them have input pending to process. So you select both the server (accept) socket, plus any client connections that you have already accepted. A thin layer on top of that is the asyncore module.
Use eventlets or gevent. It monkey patches existing libraries. socket module can be used without any changes. Though code appears synchronous, it executes asynchronously.
Example:
http://eventlet.net/doc/examples.html#socket-connect
In short: Is there an existing open-source Python interface for the Apache scoreboard IPC facility? I need to collect statistics from a running server WITHOUT using the "mod_status" HTTP interface, and I'd like to avoid Perl if possible.
Some background: As I understand it, the Apache web server uses a functionality called the "scoreboard" to handle inter-process coordination. This may be purely in-memory, or it may be file-backed shared memory. (PLEASE correct me if that's a mis-statement!)
Among other uses, "mod_status" lets you query a special path on a properly-configured server, receiving back a dynamically-generated page with a human-readable breakdown of Apache's overall functioning: uptime, request count, aggregate data transfer size, and process/thread status. (VERY useful information for monitoring perforamnce, or troubleshooting running servers that can't be shut down for debugging.)
But what if you need the Apache status, but you can't open an HTTP connection to the server? (My organization sees this case from time to time. For example, the Slowloris attack.) What are some different ways that we can get the scoreboard statistics, and is there a Python interface for any of those methods?
Note that a Perl module, Apache::Scoreboard, appears to be able to do this. But I'm not at all sure whether it can reach a local server's stats directly (shared memory, with or without a backing file), or whether it has to make a TCP connection to the localhost interface. So I'm not even sure whether this Perl module can do what we're asking. Beyond that, I'd like to avoid Perl in the solution for independent organizational reasons (no offense, Perl guys!).
Also, if people are using a completely different method to grab these statistics, I would be interested in learning about it.
Apache::Scoreboard can both fetch the scoreboard over HTTP, or, if it is loaded into the same server, access the scoreboard memory directly. This is done via a XS extension (i.e. native C). See httpd/include/scoreboard.h for how to access the in-memory scoreboard from C.
If you're running in mod_python, you should be able to use the same trick as Apache::Scoreboard: write a C extension to access the scoreboard directly.
Problem:
I have a python script that I have running as a service. It's a subclass of the win32 class win32serviceutil.ServiceFramework. I want a simple straightforward way of sending arbitrary commands to it via the command line.
What I've looked at:
It looks like the standard way of controlling the service once it's started is by using another program and sending it command signals, but I need to be able to send a short string to it as well as an argument. It looks like using a NamedPipe might be a good idea, but it's really too complex for what I wanted to do, is there any other simpler way?
Not really.
You have many, many ways to do "Interprocess Communication" (IPC) in Python.
Sockets
Named Pipes (see http://developers.sun.com/solaris/articles/named_pipes.html) -- it involves a little bit of OS magic to create, but then it's just a file that you read and write.
Shared Memory (see http://en.wikipedia.org/wiki/Shared_memory) -- this also involves a fair amount of OS-level magic.
Semphores and Locks; files with locks can work well for IPC.
Higher-level protocols built on sockets...
HTTP; this is what WSGI is all about.
FTP
etc.
A common solution is to use HTTP and define "RESTful" commands. Your service listens on port 80 for HTTP requests that contain arguments and parameters. Look at wsgiref for more information on this.