Develop Reference

Traffic redirector using Python, socket() and maybe proxy type functionality

I would like to use Python to redirect (ie route) internet bound traffic via a specific interface based on some logic (eg based on source IP address). The traffic can be anything (not just http(s) traffic).
The server that I would like the Python program to run can be the default gateway. ie Some internet traffic will be redirected towards this Python server. I will enable promiscuous mode on the interface. I am looking to capture this traffic using potentially python sockets for servers.
I do not want to change any IP addresses. So it does not seem to me that it is a pure client/server model, where I am in control of the destinations nodes.
My challenge here is using the socket() (regardless of Python, C++ ...) which seems to be designed for client-server models.
How can I simply "route" traffic through like this using Python?
Any suggestions would be great, including about my approach of using sockets. Maybe my approach is wrong, and should incorporate open-source proxy servers instead of Python's socket() functionality.See diagram here

ZeroRPC auto-assign free port number

I am using ZeroRPC for a project, where there may be multiple instances running on the same machine. For this reason, I need to abe able to auto-assign unused port numbers. I know how to accomplish this with regular sockets using socket.bind(('', 0)) or with ZeroMQ using the bind_to_random_port method, but I cannot figure out how to do this with ZeroRPC.
Since ZeroRPC is based on ZeroMQ, it must be possible.
Any ideas?

Having read details about ZeroRPC-python current state, the safest option to solve the task would be to create a central LotterySINGLETON, that would receive <-REQ/REP-> send a next free port# upon an instance's request.
This approach is isolated from ZeroRPC-dev/mod(s) modifications of use of the otherwise stable ZeroMQ API and gives you the full control over the port#-s pre-configured/included-in/excluded-from LotterySINGLETON's draws.
The other way aroung would be to try to by-pass the ZeroRPC layer and ask ZeroMQ directly about the next random port, but the ZeroRPC documentation discourages from bypassing their own controls imposed on (otherwise pure) ZeroMQ framework elements ( which is quite a reasonable to be emphasised, as it erodes the ZeroRPC-layer consistency of it's add-on operations & services, so it shall rather be "obeyed" than "challenged" in trial/error ... )

The following will let ZMQ choose a free port:
s = zerorpc.Server(SomeClass())
s.bind('tcp://127.0.0.1:0')
The problem with this is that now you don't know which port it bound to. I managed to find the port with netstat and successfully connected to it, but that's probably not what you want to do. I made a seperate question out of this: Find out bound ports of zerorpc server

How to assign IP address to interface in python?

I have python script that set the IP4 address for my wireless and wired interfaces. So far, I use subprocess command like :
subprocess.call(["ip addr add local 192.168.1.2/24 broadcast 192.168.1.255 dev wlan0"])
How can I set the IP4 address of an interface using python libraries?
and if there is any way to get an already existing IP configurations using python libraries ?

With pyroute2.IPRoute:
from pyroute2 import IPRoute
ip = IPRoute()
index = ip.link_lookup(ifname='em1')[0]
ip.addr('add', index, address='192.168.0.1', mask=24)
ip.close()
With pyroute2.IPDB:
from pyroute2 import IPDB
ip = IPDB()
with ip.interfaces.em1 as em1:
em1.add_ip('192.168.0.1/24')
ip.release()

Set an address via the older ioctl interface:
import socket, struct, fcntl
SIOCSIFADDR = 0x8916
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def setIpAddr(iface, ip):
bin_ip = socket.inet_aton(ip)
ifreq = struct.pack('16sH2s4s8s', iface, socket.AF_INET, '\x00' * 2, bin_ip, '\x00' * 8)
fcntl.ioctl(sock, SIOCSIFADDR, ifreq)
setIpAddr('em1', '192.168.0.1')
(setting the netmask is done with SIOCSIFNETMASK = 0x891C)
Ip addresses can be retrieved in the same way: Finding local IP addresses using Python's stdlib
I believe there is a python implementation of Netlink should you want to use that over ioctl

You have multiple options to do it from your python program.
One could use the ip tool like you showed. While this is not the best option at all this usualy does the job while being a little bit slow and arkward to program.
Another way would be to do the things ip does on your own by using the kernel netlink interface directly. I know that libnl has some experimental (?) python bindings. This may work but you will have to deal with a lot of low level stuff. I wouldn't recommend this way for simple "set and get" ips but it's the most "correct" and reliable way to do so.
The best way in my opinion (if you only want to set and get ips) would be to use the NetworkManagers dbus interface. While this is very limited and may have its own problems (it might behave not the way you would like it to) this is the most straight forward way if the NetworkManager is running anyway.
So, choose the libnl approach if you want to get your hands dirty, it's clearly superior but also way more work. You may also get away with the NetworkManager dbus interface, depending on your needs and general system setup. Otherwise you can just leave it that way.

How do I communicate between 2 Python programs using sockets that are on separate NATs?

I want to send and receive messages between two Python programs using sockets. I can do this using the private IPs when the computers are connected to the same router, but how do I do it when there are 2 NATs separating them?
Thanks (my first SO question)

You can't do it in general without a relay server on a publicly addressable IP address. There are so-called "punchthrough" techniques that can, in some cases, let you switch to a direct connection after using a public server to figure out port mappings - this article describes how the RakNet game networking library does it - but even these tend to be finicky and unreliable.

Zeromq, is the way to go. It is blazing fast and very easy. http://www.zeromq.org/

Redis, could work but not the exact same functionality.

How can I perform a ping or traceroute using native python?

I would like to be able to perform a ping and traceroute from within Python without having to execute the corresponding shell commands so I'd prefer a native python solution.

If you don't mind using an external module and not using UDP or TCP, scapy is an easy solution:
from scapy.all import *
target = ["192.168.1.254"]
result, unans = traceroute(target,l4=UDP(sport=RandShort())/DNS(qd=DNSQR(qname="www.google.com")))
Or you can use the tcp version
from scapy.all import *
target = ["192.168.1.254"]
result, unans = traceroute(target,maxttl=32)
Please note you will have to run scapy as root in order to be able to perform these tasks or you will get:
socket.error: [Errno 1] Operation not permitted

Running interpreters as root is often frowned upon on security grounds (and of course you DO need to have root permission to access the "raw" socked as needed by the ICMP specs of ping and traceroute!), but if you have no problems with that it's not hard -- e.g., this post(dead?) or this post give a workable ping, and Jeremy Hylton's old page has still-usable underlying code for ICMP (both ping and traceroute) though it's written for very old Python versions and needs a litte facelift to shine with modern ones -- but, the concepts ARE all there, in both the URLs I gave you!

The Webb Library is very handy in performing all kinds of web related extracts...and ping and traceroute can be done easily through it. Just include the URL you want to traceroute to:
import webb
webb.traceroute("your-web-page-url")
If you wish to store the traceroute log to a text file automatically, use the following command:
webb.traceroute("your-web-page-url",'file-name.txt')
Similarly a IP address of a URl (server) can be obtained with the following lines of code:
print(webb.get_ip("your-web-page-url"))
Hope it helps!

The mtrpacket package can be used to send network probes, which can perform either a ping or a traceroute. Since it uses the back-end to the mtr commandline tool, it doesn't require that your script run as root.
It also uses asyncio's event loop, so you can have multiple ongoing traceroutes or pings simultaneously, and deal with their results as they complete.
Here is a Python script to traceroute to 'example.com':
import asyncio
import mtrpacket
async def trace():
async with mtrpacket.MtrPacket() as mtr:
for ttl in range(1, 256):
result = await mtr.probe('example.com', ttl=ttl)
print(result)
if result.success:
break
asyncio.get_event_loop().run_until_complete(trace())
The loop with 'ttl' is used because the 'time-to-live' of an outgoing packet determines the number of network hops the packet will travel before expiring and sending an error back to the original source.

you might want to check out the scapy package. it's the swiss army knife of network tools for python.

ICMP Ping is standard as part of the ICMP protocol.
Traceroute uses features of ICMP and IP to determine a path via Time To Live values. Using TTL values, you can do traceroutes in a variety of protocols as long as IP/ICMP work because it is the ICMP TTL EXceeded messages that tell you about the hop in the path.
If you attempt to access a port where no listener is available, by ICMP protocol rules, the host is supposed to send an ICMP Port Unreachable message.

I wrote a simple tcptraceroute in python which does not need root privileges http://www.thomas-guettler.de/scripts/tcptraceroute.py.txt
But it can't display the IP addresses of the intermediate hops. But sometimes it is useful, since you can guess where the blocking firewall is: Either at the beginning or at the end of the route.