I am working on a basic IPS written in Python, which should protect a webserver. This is done for a school project, so it's mostly just a "proof of concept" kind of thing. The thought is that the IPS shall block any IP addresses that sends requests that does not fall within the model for normal behaviour for a number of minutes. My initial thought was to use scapy to do this, but I've come to realize that while it is possible to read the incoming data with scapy it will probably not be possible to block the traffic from reaching the web server. One idea is to use iptables to block the traffic, but that solution seems a bit clumpsy. We have also had a look at mitmproxy, but it seems this has to run on a separate computer, so that won't be an option. My question is if there is an easier way to do this than adding and removing iptables rules every 15 minutes?
Related
I am attempting to create a basic chatroom in python, and I would like to know how I could transmit data from one script to another, preferably without using google drive. If needed to, I could create a webserver on Replit, but I don't do well with HTML or PHP.
Side note: I can't port forward, as my google wifi doesn't accept any level of port forwarding.
I would send messages of about 50 characters every couple seconds
Since you mention port forwarding, I assume you want two chat clients that run on different local networks to talk to each other, for example your own and the chat client of a friend in a remote location, over the internet.
If you (or your counterpart) cannot set up port forwarding, then direct communication between the script on your computer and theirs is hard, if not impossible. The solution is to set up a third computer or service on the internet that can be reached by both clients and use it for relaying messages between them.
A network is typically protected by a firewall of sorts and will typically be behind a router that performs network address translation (NAT) to help multiple devices on a network to simultaneously access services on the internet, whilst all using the same IP address on the internet. Port forwarding fits into that by connecting a specific port from the outside directly to a port on a machine on the inside - without that, an outside computer might be able to reach your IP address, but they could never connect to a computer or program on the inside of the network, as the router wouldn't know what computer to contact, also the firewall might disallow the connection to begin with.
But if your computer on the inside establishes a connection with an accessible server on the internet, expecting a response, that creates a temporary conduit through the router and firewall that can be used by the server to send messages (look up 'hole punching' for more information). And if both computers do this, the server can relay message between both clients. Only the server then needs to run in an environment that doesn't have firewall restrictions or NAT that prevent this.
You could write a simple Python server, that accepts incoming connections and can send several responses and a simple client that connects to it, identifying itself and joining a chatroom, or having a direct conversation with another connected client. There are many techniques that would allow you to do this, but I think web sockets might be a good starting point, as long as you don't plan to do advanced fast or high volume stuff that would require something like a UDP connection.
A library like websockets could be a good starting point, but you may want to start out by figuring out where you would have this service hosted first, since there may be limitations on what you're able and allowed to do.
Also, if all you're looking to do is send simple messages, you may want to stay away from writing your own server an protocols at all - have a look around for open source message servers written in a language you are comfortable with, or that just work out of the box without any development, in which case the language doesn't even really matter, as long as you can connect to it and exchange messages from Python.
I guess it's socket programming. But I have never done socket programming expect for running the tutorial examples while learning Python. I need some more ideas to implement this.
What I specifically need is to run a monitoring program of a server which will poll or listen to traffic being exchange from different IPs across different popular ports. For example, how do I get data received and sent through port 80 of 192.168.1.10 and 192.168.1.1 ( which is the gateway).
I checked out a number of ready made tools like MRTG, Bwmon, Ntop etc but since we are looking at doing some specific pattern studies, we need to do data capturing within the program.
Idea is to monitor some popular ports and do a study of network traffic across some periods and compare them with some other data.
We would like to figure a way to do all this with Python....
You probably want to use scapy for that. Just sniff all ethernet traffic on a particular interface, drop everything that is not TCP and doesn't match the port.
Not sure if scapy can already track TCP connections (stuff like recognizing duplicate sequence numbers, extracting just the payload stream) but I would guess it probably can, and if not it's not too hard to hack together a good-enough TCP connection tracker that works for 95% of the traffic.
Alternatives would be to use sockets directly (look for raw sockets) or libpcap, which can both be done from Python. You may also want to check out the filter experssion syntax of the 'tcpdump' commandline tool, maybe it can do what you want already.
I bet there are more specialized high-level tools for this, but I don't know them.
PS: if you don't know wireshark yet, go check it out and play around with it first. It can follow TCP streams and will teach you what TCP connection tracking means. Maybe its commandline binary, tshark, can be used to extract TCP streams for what you want.
IPTraf is an ncurses based IP LAN monitoring tool. Has a capability to generate network statistics including TCP,UDP,ICMP and some more.
Since you're thinking to execute it from python, you may consider to use screen (screen manager with VT100/ANSI terminal emulation) to overcome ncurses issues and you may want to pass logging and interval parameters to IPTraf which forces iptraf to log to a file in a given interval. Little bit tricky but eventually you can have what you are looking for by basically parsing the log file.
I made some python script and ran it both on my computer and some distant shell (some website that provides shell access).
I'm using threads, pipes and UDP sockets to transfer data in a P2P fashion, so each script can both receive and send through the same socket. To test if this works, I open one terminal on my computer and one other terminal with ssh, connected to my shell. I make sure the script is the same on both machines, and feed it with an ip address.
Here is the script: http://codepad.org/V9Q1KcDT
(I don't know if I should paste it directly here or not)
My problem is this: strings I send seems to land something 20% of the time, sometimes often, sometimes not, and it seems to be random...
What am I doing wrong ?
Are UDP so unreliable ?
Are python thread+pipe+socket too slow ?
Could it be some kind of network problem with my shell provider ?
Is my program flawed ?
Are pipes a good solution to communicate with threads ?
I have no problem not using a shell and I have not tried, but it's useful for testing purposes.
BTW if i'm behing a router, how does the router knows where to send the packet if I'm not the only computer connected ? (I tried when I was the only one, it behaved identically).
User Datagram Protocol(UDP) could easily stand for Unreliable Datagram Protocol. UDP provides no guarantees regarding delivery so if you need reliability you have to implement it yourself by resending messages.
That said, threading and multiprocessing and remote machines and networks are a lot of variables to diagnose at once. I'd advise you to step back and try to boil the problem down to something simpler.
For instance:
first try TCP instead of UDP
instead of threads, run 2 processes (one doing nit_send, one doing nit_recv)
run them both on your local machine (use localhost: 127.0.0.1)
Once you have this basic test working then add features back in. For instance, switch to UDP. Once you've got your UDP issues sorted out locally try introducing remote machines or threads, etc.
Regarding the router question all machines connected to your router are most likely NAT'd so they all appear to have the same IP address (the IP address of your router) to machines on the internet. Look into NAT and port-forwarding if you want to route traffic to a specific machine on your local subnet.
I want to write a Python script that will check the users local network for other instances of the script currently running.
For the purposes of this question, let's say that I'm writing an application that runs solely via the command line, and will just update the screen when another instance of the application is "found" on the local network. Sample output below:
$ python question.py
Thanks for running ThisApp! You are 192.168.1.101.
Found 192.168.1.102 running this application.
Found 192.168.1.104 running this application.
What libraries/projects exist to help facilitate something like this?
One of the ways to do this would be the Application under question is broadcasting UDP packets and your application is receiving that from different nodes and then displaying it. Twisted Networking Framework provides facilities for doing such a job. The documentation provides some simple examples too.
Well, you could write something using the socket module. You would have to have two programs though, a server on the users local computer, and then a client program that would interface with the server. The server would also use the select module to listen for multiple connections. You would then have a client program that sends something to the server when it is run, or whenever you want it to. The server could then print out which connections it is maintaining, including the details such as IP address.
This is documented extremely well at this link, more so than you need but it will explain it to you as it did to me. http://ilab.cs.byu.edu/python/
You can try broadcast UDP, I found some example here: http://vizible.wordpress.com/2009/01/31/python-broadcast-udp/
You can have a server-based solution: a central server where clients register themselves, and query for other clients being registered. A server framework like Twisted can help here.
In a peer-to-peer setting, push technologies like UDP broadcasts can be used, where each client is putting out a heartbeat packet ever so often on the network, for others to receive. Basic modules like socket would help with that.
Alternatively, you could go for a pull approach, where the interesting peer would need to discover the others actively. This is probably the least straight-forward. For one, you need to scan the network, i.e. find out which IPs belong to the local network and go through them. Then you would need to contact each IP in turn. If your program opens a TCP port, you could try to connect to this and find out your program is running there. If you want your program to be completely ignorant of these queries, you might need to open an ssh connection to the remote IP and scan the process list for your program. All this might involve various modules and libraries. One you might want to look at is execnet.
Is it possible to write a peer-to-peer chat application in Python?
I am thinking of this from a hobbyist project point-of-view. Can two machines connect to each other directly without involving a server? I have always wondered this, but never actually seen it implemented anywhere so I am thinking there must be a catch somewhere.
PS: I intend to learn Twisted, so if that is involved, it would be an added advantage!
Yes. You can do this pretty easily with Twisted. Just have one of the peers act like a server and the other one act like a client. In fact, the twisted tutorial will get you most of the way there.
The only problem you're likely to run into is firewalls. Most people run their home machines behind SNAT routers, which make it tougher to connect directly to them from outside. You can get around it with port forwarding though.
Yes, each computer (as long as their on the same network) can establish a server instance with inbound and outbound POST/GET.
I think i am way too late in putting my two bits here, i accidentally stumbled upon here as i was also searching on similar lines. I think you can do this fairly easily using just sockets only, however as mentioned above one of the machines would have to act like a server, to whome the other will connect.
I am not familiar with twisted, but i did achieved this using just sockets. But yes even i am curious to know how would you achieve peer2peer chat communication if there are multiple clients connected to a server. Creating a chat room kind of app is easy but i am having hard time in thinking how to handle peer to peer connections.