Develop Reference

Python network scanner (host discovery tool) is telling me that my source and destination are the same when sending UDP datagrams

I'm following Black Hat Python (2ed.), in which I'm writing a network scanning tool. The tool is in theory supposed to send UDP packets out to a given subnet, and if a host is up on that subnet, the response packet is decoded, found to contain the message in the original datagram, and used to indicate the host is up. This seems to generally be working well to capture packets; I can go to a website, or ping another host, and the tool reliably provides the correct source and destination addresses for those cases.
Here is the meat of the code (I have not included the class creation, or the passing of the host argument for brevity, but the host is 192.168.10.85).
class IP:
"""layer 3 (IP) packet header decoder"""
def __init__(self, buff=None):
header = struct.unpack('<BBHHHBBH4s4s', buff)
self.ver = header[0] >> 4
self.ihl = header[0] & 0xF
self.tos = header[1]
self.len = header[2]
self.id = header[3]
self.offset = header[4]
self.ttl = header[5]
self.protocol_num = header[6]
self.sum = header[7]
self.src = header[8]
self.dst = header[9]
# make IP addrs human readable
self.src_address = ipaddress.ip_address(self.src)
self.dst_address = ipaddress.ip_address(self.dst)
# the protocol_num is actually a code for the protocol name
self.protocol_name = {1: 'ICMP', 6: 'TCP', 17: 'UDP'}
# try to provide the human version of the protocol, otherwise just give the code
try:
self.protocol = self.protocol_name[self.protocol_num]
except KeyError as error:
self.protocol = self.protocol_num
print(f'Protocol is unrecognized, try googling "IP protocol {self.protocol_num}"')
class ICMP:
"""layer 4 (ICMP) packet header decoder"""
def __init__(self, buff):
header = struct.unpack('<BBHHH', buff)
self.type = header[0]
self.code = header[1]
self.checksum = header[2]
self.ident = header[3]
self.seq_num = header[4]
def udp_sender():
# blasts udp packets into the network to solicit responses
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sender:
for ip in ipaddress.ip_network(SUBNET).hosts():
# time.sleep(1)
print(f'sending a test message to {ip}')
# send our test message out to port 65212 on the destination
sender.sendto(bytes(MESSAGE, 'utf8'), (str(ip), 65212))
class Scanner:
def __init__(self, host):
self.host = host
# create raw socket, bind to public interface
# if windows:
if os.name == 'nt':
socket_protocol = socket.IPPROTO_IP
# if linux/mac:
else:
socket_protocol = socket.IPPROTO_ICMP
self.socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)
self.socket.bind((host, 0))
# socket options, include header
self.socket.setsockopt(socket_protocol, socket.IP_HDRINCL, 1)
# enable promiscuous mode for windows
if os.name == 'nt':
self.socket.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
def sniff(self):
# set of all hosts that are up (respond to our ICMP message)
hosts_up = {f'{str(self.host)} *'}
try:
while True:
# read a packet, and parse the IP header
raw_buffer = self.socket.recvfrom(65535)[0]
# create IP header from the first 20 bytes
ip_header = IP(raw_buffer[0:20])
# if the protocol is ICMP, do some additional things
# print(f'src={ip_header.src_address}, dst={ip_header.dst_address}, prot_name={ip_header.protocol}')
if ip_header.protocol == 'ICMP':
# calculate where the ICMP packet starts
offset = ip_header.ihl * 4
buf = raw_buffer[offset:offset + 8]
# create ICMP structure
icmp_header = ICMP(buf)
print(f'type: {icmp_header.type}, code: {icmp_header.code}')
print(f'src={ip_header.src_address}, dst={ip_header.dst_address}, prot_name={ip_header.protocol}')
if icmp_header.type == 3 and icmp_header.code == 3:
print(f'type: {icmp_header.type}, code: {icmp_header.code}')
print(f'src={ip_header.src_address}, dst={ip_header.dst_address}, prot_name={ip_header.protocol}')
if ipaddress.ip_address(ip_header.src_address) in ipaddress.IPv4Network(SUBNET):
# make sure the packet has our test message
if raw_buffer[len(raw_buffer) - len(MESSAGE):] == bytes(MESSAGE, 'utf8'):
tgt = str(ip_header.src_address)
if tgt != self.host and tgt not in hosts_up:
hosts_up.add(str(ip_header.src_address))
print(f'Host Up: {tgt}')
However, when receiving the ICMP responses as a result of my datagram, the tool reports that the source and destination addresses are the same (my host, 192.168.10.85). Furthermore, while I should be receiving responses with Type 3 and Code 3 (destination unreachable, and port unreachable), but I am receiving (in my program) Type 3 and Code 1.
Here is an example of the output when I issue a ping command while the scanner is running, which seems correct:
src=192.168.10.85, dst=192.168.10.200, prot_name=ICMP type: 0, code: 0 src=192.168.10.200, dst=192.168.10.85, prot_name=ICMP type: 8, code: 0
Here is an example of the output to what I am assuming is the UDP packet response, which seems incorrect):
src=192.168.10.85, dst=192.168.10.85, prot_name=ICMP type: 3, code: 1
If I open wireshark while I'm running my code, I can correctly see the ICMP Type 3/Code 3 responses, so I know they are going through, here is a screen grab of one host on the target subnet as an example:
Why is my scanner not seeing these responses that are in wireshark?
I've tried running wireshark alongside my program, to see if the packets are being correctly decoded, and that the message in the UDP packet is properly in place. All signs indicate that the packets are going out to the hosts I'm trying to detect, and the correct responses are coming back, but my scanner refuses to find them.

How to depickle the object same as it was sent from socket in python?

I am trying to serialize the object (using pickle) of type scapy.layers.l2.Ether (please refer scapy import statement of code). But while deserializing that object, it is not being retrieved properly (also shown in output). How can I fix this ? and get the object as same as it was serialized ?
import socket
import time
import pickle #serialize the object
import sys # for command line argument
from scapy.all import * # for packet sniffing
HEADERSIZE = 10
clientsocket = 1
address = 1
#for serialization
def pkt_callback(pkt):
pkt.show()
msg = pickle.dumps(pkt)
print(12)
clientsocket.send(msg)
#check whether ip and port number provided or not
if len(sys.argv) != 3:
print ("Correct usage: script, IP address, port number", flush = True)
exit(0)
IP_address = str(sys.argv[1])
Port = int(sys.argv[2])
#socket creation ipv6
s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
s.bind((IP_address, Port))
s.listen(5)
while True:
# now our endpoint knows about the OTHER endpoint.
clientsocket, address = s.accept()
print(f"Connection from {address} has been established.",flush=True)
sniff(iface="wlp3s0", prn=pkt_callback, filter="tcp", store=0)
clientsocket.close()
print('Data sent to server')
The above code serializes the object to be sent at pkt_callback
Now below code is from receiving side which deserializes the object
import socket
import pickle
import sys
from scapy.all import * # for packet sniffing
HEADERSIZE = 10
#check whether address and port number given or not
if len(sys.argv) != 3:
print ("Correct usage: script, IP address, port number",flush = True)
exit(0)
IP_address = str(sys.argv[1])
Port = int(sys.argv[2])
#create ipv6 socket
s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
s.connect((IP_address, Port))
while True:
data = s.recv(4096)
data_recved = pickle.loads(data)
(scapy.layers.l2.Ether(data_recved)).show()
s.close()
The 1st code which serializes the object works fine and shows output as expected.
The 2nd code which deserializes the object works but gives wrong output as below
###[ Raw ]###
load = '\xb0\xfc6`\xe5\xe3\xe4\xa7\xc5I,\x1c\x08\x00E(\x004%\xd4#\x00h\x06s!o\xdd\x1d\xfe\xc0\xa8+#\x01\xbb\xda\xb4\xf5 \x05\xfbv\xac\xccj\x80\x10\x04\x02\xb6z\x00\x00\x01\x01\x08\n!\x84\xf3\xa3\x066\x0c\x99'
###[ Raw ]###
load = "\xb0\xfc6`\xe5\xe3\xe4\xa7\xc5I,\x1c\x08\x00E(\x01o%\xd5#\x00h\x06q\xe5o\xdd\x1d\xfe\xc0\xa8+#\x01\xbb\xda\xb4\xf5 \x05\xfbv\xac\xccj\x80\x19\x04\x02B\xbf\x00\x00\x01\x01\x08\n!\x84\xf3\xfd\x066\x0c\x99\x17\x03\x03\x016\x00\x00\x00\x00\x00\x00\x00\x01n\xb3\x18\x93\xd2xT\x00\xe3Xh\xa7\xc0?|\xb3n\xf6?g\x00\xdf\xcf\x97wx\xccw\x04\x08\xbfM\xf2L1^\x84\xa1\xb0\x06\xaas\xdb\xc9\xb1\xed\x99\x8cz\x9a5P\xa3{\x04y\xbb\xc38\xcfT\\\x98f\xf1z/F\x1e\xd6tD\x90pN\x92\xe1\xf9\xee#\xbc\x1a(\x89{x\xf2\\\xd1\x14\x1d?\x02\x07\x16N\xe4\x017\x140\xc5\x81\xb4\xcd\t\x8ai\x1a\xe7PZ\x01\xbek\xba \x84[\\\xcc\xf0\xecs\xc8^\xff!\xce\x19z,\x13\xbd\xa3\x0c\x0fq\xca=\xbc\x1a834\xee\xf9hC\x0b\xb4\x85\x114'5W\xee\xed&\x93Y0\xd1\x85\xc3(\xb3\x0e0u\\\x9d\xa7\xb5g\xb0\xb4C_S\xe7\xf8\xbaH\t\x94\x84\xaf\xa9\tS\xb6\xf4XYD\tM\xf8=`\xbd\x04\xeeo\x1f\xd3\x7f*\xef\x8e\xd2\x00\x8f\xfb\x8c\xfemB\x02\xc0\x12\x8a\r\x14\xdco\\K\xfd\x97%\xbc\x15}\xb1\xe2<\xdei\x15\x17\xeec>=\xa0\xc0]\xfaa\xeal`\x10c\xe8\x86B\xff'w\xe3\xe4\xce\x9d\xbf0\xdc\xb0F\x12\x8b{;9\xa1\xce\xfdf\x1c\x8b\x8f\xc4\xc0F\xf3\\\x86\xbd\xcdu\xdb\xb8>t\xf6\xff.~h"
###[ Raw ]###
load = "\xe4\xa7\xc5I,\x1c\xb0\xfc6`\xe5\xe3\x08\x00E\x00\x00SP\xcb#\x00#\x06p3\xc0\xa8+#o\xdd\x1d\xfe\xda\xb4\x01\xbbv\xac\xccj\xf5 \x077\x80\x18\x01>\x93\x10\x00\x00\x01\x01\x08\n\x066\r\xb7!\x84\xf3\xfd\x15\x03\x03\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x02Nkk'T\xcdv$\xb7e\xe00\xc9\x9f\xf2\xffw4"
###[ Raw ]###
load = '\xe4\xa7\xc5I,\x1c\xb0\xfc6`\xe5\xe3\x08\x00E\x00\x004P\xcc#\x00#\x06pQ\xc0\xa8+#o\xdd\x1d\xfe\xda\xb4\x01\xbbv\xac\xcc\x89\xf5 \x077\x80\x11\x01>\xb6j\x00\x00\x01\x01\x08\n\x066\r\xb7!\x84\xf3\xfd'
but it should be like
###[ Ethernet ]###
dst = e4:a7:c5:49:2c:1c
src = b0:fc:36:60:e5:e3
type = IPv6
###[ IPv6 ]###
version = 6
tc = 0
fl = 630855
plen = 32
nh = TCP
hlim = 255
src = 2409:4042:2ea4:752f:cd15:9fe5:6dc1:1575
dst = 2404:6800:4009:810::200e
###[ TCP ]###
sport = 36318
dport = https
seq = 3279176249
ack = 3388920021
dataofs = 8
reserved = 0
flags = A
window = 254
chksum = 0xf7e7
urgptr = 0
options = [('NOP', None), ('NOP', None), ('Timestamp', (427468701, 3715875499))]
How can I fix it ?

I have recently found the solution. Actually in second code snippet, the object show function on object was called by doing unnecessary typecasting. The line (scapy.layers.l2.Ether(data_recved)).show() should be simply like data_recved.show(). And then we can get desired output

Fake DNS response

I wanted to create a fake dns response with scapy and it's just doesn't work... When i sniff the packets in Wireshark it shows me that the packets are correct but Windows just takes the genuine response packet althought...
Can someone tell me how to fix it please?
Thanks
import sys
i, o, e = sys.stdin, sys.stdout, sys.stderr
from scapy.all import *
sys.stdin, sys.stdout, sys.stderr = i, o, e
def f(packet):
if DNS in packet and DNSQR in packet :
return True
return False
while True:
a=sniff(lfilter=f,count=1)
ip = a[0].getlayer(IP)
dns = a[0].getlayer(DNS)
pkt = Ether(dst = a[0][Ether].src, src = a[0][Ether].dst)/IP(dst=ip.src, src=ip.dst)/UDP(chksum=None, dport=ip.sport,sport=ip.dport)/DNS(qd=a[0][DNS].qd, qdcount=1, ancount=0, nscount=0, arcount=1, ra = 1, qr = 1, id=dns.id, an = (DNSRR(rrname=dns.qd.qname, type= "A" , ttl=3600, rdata="192.168.1.12")))
pkt.show()
for i in range(10):
sendp(pkt)

You're just sniffing packets, if you want to manipulate packets you should send them to a function then forward them to the destination. use prn attribute in Sniff :
packets = sniff(filter="port 53" , prn=func , count=1)
def func(packet):
if packet.haslayer(UDP) and packet.haslayer(DNS):
manipulate your DNS packet here then forward it

scapy 3-way hand shake not work

I want to use scapy to do tcp 3-way hand shake, my code is below:
from scapy.all import *
# VARIABLES
src = sys.argv[1]
dst = sys.argv[2]
sport = random.randint(20000,65535)
dport = int(sys.argv[3])
print src
print dst
print sport
print dport
# SYN
ip=IP(src=src,dst=dst)
SYN=TCP(sport=sport,dport=dport,flags='S',seq=1000)
SYNACK=sr1(ip/SYN)
SYN.show()
SYNACK.show()
# ACK
seq=SYNACK.ack + 1
ack=SYNACK.seq + 1
print seq
print ack
ACK=TCP(sport=sport, dport=dport, flags='A', seq=seq, ack=ack)
sr1(ip/ACK)
ACK.show()
exec command:
python 3WSK.py 10.16.11.21 10.11.157.130 26789
And the SYNACK result is below:
###[ IP ]###
version = 4L
ihl = 5L
tos = 0x0
len = 44
id = 0
flags = DF
frag = 0L
ttl = 60
proto = tcp
chksum = 0x821a
src = 10.11.157.130
dst = 10.16.11.21
\options \
###[ TCP ]###
sport = 26789
dport = 35067
seq = 1918207620
ack = 1001
dataofs = 6L
reserved = 0L
flags = SA
window = 5840
chksum = 0xd630
urgptr = 0
options = [('MSS', 1460)]
###[ Padding ]###
load = '\x00\x00'
but my tcp server not log any thing.
for test:
I use telnet 10.11.157.130 26789 to connect my tcp server, then my tcp server can log something like: "INFO l.t.TcpServer - in, /10.16.11.21:34906"
So, I`m not sure my scapy code is connect successful or not, I also want to know
how to validate whether tcp connect is work or not. thanks

Check if you see a RST being sent from your os when the SYN+ACK arrives. If so have a look at this answer and drop the RST with iptables:
iptables -A OUTPUT -p tcp --tcp-flags RST RST -s <your_ip> -j DROP
If thats not the case have a look at this gist which works fine for me.

Socket Programming; File corrupts when transferring over multiple devices

The problem I'm having is to get a file from the server to client across devices. Everything works fine on localhost.
Lets say I want to "get ./testing.pdf" which sends the pdf from the server to the client. It sends but it is always missing bytes. Is there any problems with how I am sending the data. If so how can I fix it? I left out the code for my other functionalities since they are not used for this function.
sending a txt file with "hello" in it works perfectly
server.py
import socket, os, subprocess # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
#host = ''
port = 5000 # Reserve a port for your service.
bufsize = 4096
s.bind((host, port)) # Bind to the port
s.listen(5) # Now wait for client connection.
while True:
c, addr = s.accept() # Establish connection with client.
print 'Got connection from', addr
while True:
userInput = c.recv(1024)
.... CODE ABOUT OTHER FUNCTIONALITY
elif userInput.split(" ")[0] == "get":
print "inputed get"
somefile = userInput.split(" ")[1]
size = os.stat(somefile).st_size
print size
c.send(str(size))
bytes = open(somefile).read()
c.send(bytes)
print c.recv(1024)
c.close()
client.py
import socket, os # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
#host = '192.168.0.18'
port = 5000 # Reserve a port for your service.
bufsize = 1
s.connect((host, port))
print s.recv(1024)
print "Welcome to the server :)"
while 1 < 2:
userInput = raw_input()
.... CODE ABOUT OTHER FUNCTIONALITY
elif userInput.split(" ")[0] == "get":
print "inputed get"
s.send(userInput)
fName = os.path.basename(userInput.split(" ")[1])
myfile = open(fName, 'w')
size = s.recv(1024)
size = int(size)
data = ""
while True:
data += s.recv(bufsize)
size -= bufsize
if size < 0: break
print 'writing file .... %d' % size
myfile = open('Testing.pdf', 'w')
myfile.write(data)
myfile.close()
s.send('success')
s.close

I can see two problems right away. I don't know if these are the problems you are having, but they are problems. Both of them relate to the fact that TCP is a byte stream, not a packet stream. That is, recv calls do not necessarily match one-for-one with the send calls.
size = s.recv(1024) It is possible that this recv could return only some of the size digits. It is also possible that this recv could return all of the size digits plus some of the data. I'll leave it for you to fix this case.
data += s.recv(bufsize) / size -= bufsize There is no guarantee that that the recv call returns bufsize bytes. It may return a buffer much smaller than bufsize. The fix for this case is simple: datum = s.recv(bufsize) / size -= len(datum) / data += datum.