I have an application which read live SIP Packets and decode information in real time.
when packet is small UDP/TCP is able to get the information, but when packet is large, it arrives in different segments:
The following is an extract from Wireshark:
3 Reassembled TCP Segments (3331 bytes): #1(1448), #3(1448), #5(435)
Frame: 1, payload: 0-1447 (1448 bytes)
Frame: 3, payload: 1448-2895 (1448 bytes)
Frame: 5, payload: 2896-3330 (435 bytes)
Segment count: 3
Reassembled TCP length: 3331
My application believes there is a new SIP Packet for each fragment and fails to decode info.
How can I do this? I need to read the packet, assemble all sip message if fragmented and pass the info to my control module. This is my current code:
s = socket.socket( socket.AF_PACKET , socket.SOCK_RAW , socket.ntohs(0x0003))
while (True):
packet = s.recvfrom(65565)
#packet string from tuple
packet = packet[0]
#parse ethernet header
eth_length = 14
eth_header = packet[:eth_length]
eth = unpack('!6s6sH' , eth_header)
eth_protocol = socket.ntohs(eth[2])
if eth_protocol == 8 :
#Parse IP header
#take first 20 characters for the ip header
ip_header = packet[eth_length:20+eth_length]
#now unpack them :)
iph = unpack('!BBHHHBBH4s4s' , ip_header)
version_ihl = iph[0]
version = version_ihl >> 4
ihl = version_ihl & 0xF
iph_length = ihl * 4
ttl = iph[5]
protocol = iph[6]
s_addr = socket.inet_ntoa(iph[8]);
d_addr = socket.inet_ntoa(iph[9]);
#TCP protocol
if protocol == 6 :
t = iph_length + eth_length
tcp_header = packet[t:t+20]
#now unpack them :)
tcph = unpack('!HHLLBBHHH' , tcp_header)
source_port = tcph[0]
dest_port = tcph[1]
sequence = tcph[2]
acknowledgement = tcph[3]
doff_reserved = tcph[4]
tcph_length = doff_reserved >> 4
if dest_port == sipLocatorConfig.SIP_PORT:
print
logging.info("------------------------------------------------------SIP Packet detected------------------------------------------------------")
h_size = eth_length + iph_length + tcph_length * 4
data_size = len(packet) - h_size
#get data from the packet
data = packet[h_size:]
ipInfo = {}
ipInfo['protocol'] = protocol
ipInfo['s_addr'] = str(s_addr)
ipInfo['source_port'] = source_port
ipInfo['d_addr'] = str(d_addr)
ipInfo['dest_port'] = dest_port
processSipPacket(data,ipInfo)
I believe this is what I wrote bufsock for:
http://stromberg.dnsalias.org/~strombrg/bufsock.html
It allows you to say "give me all the data until the next null" or "give me the next 64 bytes" and similar things. It deals intelligently with fragmented and aggregated packets.
Unlike many such tools, it does not require that you have bufsock at both the producer and the consumer - you can use it fine on one end and not the other. It is a little bit like stdio for sockets, in python.
It works on CPython 2.x, CPython 3.x, Pypy, Pypy3 (which is still beta at this time) and Jython.
Related
I'm trying to parse question field in a DNS packet where I can read domain and DNS response from a DNS server. I can extract a DNS header, but I'm having trouble to parse the question field because the size of the data is unknown.
I follow this example, but the part of extracting the question field is not working.
What I need is someone to show me the way to do it properly.
I have this code where everything is right...
This is my code:
#!/usr/bin/env python3
from socket import *
import struct
import binascii
def ethernet_frame(raw_data):
mac_dest, mac_src, protocol = struct.unpack('! 6s 6s H',
raw_data[:14])
return byte_to_hex_mac(mac_dest), byte_to_hex_mac(mac_src),
htons(protocol), raw_data[14:]
def byte_to_hex_mac(mac_bytes):
addr = binascii.hexlify(mac_bytes).decode("ascii")
return ":".join([addr[i:i+2] for i in range(0,12,2)])
def data_packet_udp(data):
tuple_data_udp = struct.unpack('! H H H H', data[:8])
port_src = tuple_data_udp[0]
port_dest = tuple_data_udp[1]
udp_len = tuple_data_udp[2]
udp_checksum = tuple_data_udp[3]
return port_src, port_dest, udp_len, udp_checksum, data[8:]
def data_packet_ipv4(data):
tuple_data_ipv4 = struct.unpack("!BBHHHBBH4s4s", data[:20])
version = tuple_data_ipv4[0]
header_len = version >> 4
type_service = tuple_data_ipv4[1]
length_total = tuple_data_ipv4[2]
identification = tuple_data_ipv4[3]
offset_fragment = tuple_data_ipv4[4]
ttl = tuple_data_ipv4[5]
protocols = tuple_data_ipv4[6]
checksum_header = tuple_data_ipv4[7]
ip_src = inet_ntoa(tuple_data_ipv4[8])
ip_dest = inet_ntoa(tuple_data_ipv4[9])
length_header_bytes = (version & 15) * 4
return version, header_len, type_service, + \
length_total, identification, offset_fragment, + \
ttl, protocols, checksum_header, ip_src, ip_dest,
data[length_header_bytes:]
def data_packet_dns(data):
tuple_data_dns = struct.unpack('!HHHHHH', data[:12])
identification = tuple_data_dns[0]
flags = tuple_data_dns[1]
number_queries = tuple_data_dns[2]
number_response = tuple_data_dns[3]
number_authority = tuple_data_dns[4]
number_additional = tuple_data_dns[5]
qr = (flags & 32768) != 0
opcode = (flags & 30720 ) >> 11
aa = (flags & 1024) != 0
tc = (flags & 512) != 0
rd = (flags & 256) != 0
ra = (flags & 128) != 0
z = (flags & 112) >> 4
rcode = flags & 15
return identification, flags, number_queries, number_response, + \
number_authority, number_additional, qr, opcode, aa, tc, + \
rd, ra, z, rcode
sock = socket(AF_PACKET, SOCK_RAW, ntohs(0x0003))
while True:
raw_dados, addr = sock.recvfrom(65536)
mac_dest, mac_src, protocol, payload = ethernet_frame(raw_dados)
if protocol == 8:
( version, header_len, type_service,
length_total, identification, offset_fragment,
ttl, protocols, checksum_header,
ip_src, ip_dest, data ) = data_packet_ipv4(payload)
if protocols == 17:
port_src, port_dest, udp_len, udp_checksum, data =
data_packet_udp(data)
print("--------- HEADER UDP ----------")
print("Port Source : {}".format(port_src))
print("Port Dest : {}".format(port_dest))
print("UDP Length : {}".format(udp_len))
print("UDP Checksum : {}\n".format(udp_checksum))
if port_src == 53 or port_dest == 53:
(identification, flags, number_queries, \
number_response,number_authority,number_additional, \
qr, opcode, aa, tc, rd, ra, z, rcode) = data_packet_dns(data)
print("\t--------- HEADER DNS ----------")
print("\tidentification : {}".format(identification))
print("\tFlags : {}".format(flags))
print("\tnumber_queries : {}".format(number_queries))
print("\tnumber_response : {}".format(number_response))
print("\tnumber_authority : {}".format(number_authority))
print("\tnumber_additional : {}".format(number_additional))
print("\tQr : {}".format(qr))
print("\tOpcode : {}".format(opcode))
print("\tAA : {}".format(aa))
print("\tTC : {}".format(tc))
print("\tRD : {}".format(rd))
print("\tRA : {}".format(ra))
print("\tZ : {}".format(z))
print("\tRCODE : {}".format(rcode))
I want to send packet over Modbus TCP. I want to use:
But I can not send this way how can I send this packet? (I don't know something will be)
req = struct.pack(
'Something', transaction, identifier, length, unitid, func_code, reg_addr
)
These are my variables:
transaction=0x01
identifier=0x00
length=[0x00,0x06]
unitid=0x01
func_code=0x03
reg_addr=[0x13,0x14,0x15]
At first you can use pymodbus library with very features.
Also struct.pack() not support a list as argument.
0001 0000 0006 11 03 006B 0003 is a standard example of Modbus-TCP packet which contained:
0001: Transaction Identifier
0000: Protocol Identifier
0006: Message Length (6 bytes to follow)
11: The Unit Identifier (17 = 11 hex)
03: The Function Code (read Analog Output Holding Registers)
006B: The Data Address of the first register requested. (40108-40001 = 107 =6B hex)
0003: The total number of registers requested. (read 3 registers 40108 to 40110)
Reference
Thus, you can create a Modbus-TCP packet with the above example:
import struct
transaction = 0x0001
identifier = 0x0000
length = 0x0006
unitid = 0x11
fcode = 0x03 # Holding register fcode.
reg_addr = 0x006B # Register address.
count = 0x0003 # Read three register.
total_pack_string = '0x{:04x}{:04x}{:04x}{:02x}{:02x}{:04x}{:04x}'.format(
transaction, identifier, length, unitid, fcode, reg_addr, count
)
total_pack_hex = hex(int(total_pack_string, 16))
'''Or with using pack method.'''
pack_ = struct.pack(
'>HHHBBHH', transaction, identifier, length, unitid, fcode, reg_addr, count
)
# Then send the pack_ or total_pack_hex using a TCP-Socket.
[NOTE]:
transaction is 2Byte == Short == H
identifier is 2Byte == Short == H
length is 2Byte == Short == H
unitid is 1Byte == B
fcode is 1Byte == B
reg_addr is 2Byte == Short == H
count is 2Byte == Short == H
B is unsigned byte
H is unsigned short
Thus, the format will be like this >HHHBBHH
Using pymodbus equivalent:
from pymodbus.client.sync import ModbusTcpClient
unitid = 0x11
fcode = 0x03 # Holding register fcode.
reg_addr = 0x006B # Register address.
count = 0x0003 # Read three register.
cli = ModbusTcpClient('127.0.0.1', port=502)
if cli.connect():
res = cli.read_holding_registers(reg_addr, count=count, unit=unitid)
if not res.isError():
print(res.registers)
else:
print('There is an error.')
cli.close()
else:
print('Error in connection.')
I am trying to understand how to add a custom dissector in Scapy. I am using Python 3.4 and Scapy3 if that has any bearing on the result.
I have a stupid class, and the packet.show2() command correctly renders the nested packet. But I can not access the new Layers field values.
Scary Class and bind_layer follows...
from scapy.all import *
#Create simple Class
class DUMBO(Packet):
fields_desc = [
ShortField('ears',0),
ShortField('legs',0),
ShortField('trunk',0)
]
#Inform TCP that ports 9898 are this protocol
bind_layers(TCP, DUMBO, sport=9898, dport=9898)
I make a packet like this
#Make a Packet
pack=IP()/TCP(sport=9898, dport=9898)/Raw(load=b'\x00\x02\x00\x04\x00\x01')
Looking at the Packet I have created using ls yields
version : BitField = 4 (4)
ihl : BitField = None (None)
tos : XByteField = 0 (0)
len : ShortField = None (None)
id : ShortField = 1 (1)
flags : FlagsField = 0 (0)
frag : BitField = 0 (0)
ttl : ByteField = 64 (64)
proto : ByteEnumField = 6 (0)
chksum : XShortField = None (None)
src : Emph = '127.0.0.1' (None)
dst : Emph = '127.0.0.1' ('127.0.0.1')
options : PacketListField = [] ([])
--
sport : ShortEnumField = 9898 (20)
dport : ShortEnumField = 9898 (80)
seq : IntField = 0 (0)
ack : IntField = 0 (0)
dataofs : BitField = None (None)
reserved : BitField = 0 (0)
flags : FlagsField = 2 (2)
window : ShortField = 8192 (8192)
chksum : XShortField = None (None)
urgptr : ShortField = 0 (0)
options : TCPOptionsField = {} ({})
--
load : StrField = b'\x00\x02\x00\x04\x00\x01' (b'')
And display it using Show2 it all looks good
pack.show2()
###[ IP ]###
version = 4
ihl = 5
tos = 0x0
len = 46
id = 1
flags =
frag = 0
ttl = 64
proto = tcp
chksum = 0x7cc7
src = 127.0.0.1
dst = 127.0.0.1
\options \
###[ TCP ]###
sport = monkeycom
dport = monkeycom
seq = 0
ack = 0
dataofs = 5
reserved = 0
flags = S
window = 8192
chksum = 0x447f
urgptr = 0
options = []
###[ DUMBO ]###
ears = 2
legs = 4
trunk = 1
I now want to access the DUMBO Layer fields
But
PACK[DUMBO].ears
Is not correct - as the packet when displayed as pack.show() still has the Payload as Raw....
What am I missing ??
Ok - This is my solution....
pack=IP()/TCP(sport=19898, dport=19898)/Raw(load=b'\x00\x02\x00\x04\x00\x01')
#Cast this packet back
pack=IP(bytes(pack))
pack.show2()
pack.show()
if DUMBO in pack:
print('Elephant in the house')
print('Ears -> {}'.format(pack[DUMBO].ears))
If anyone else can improve on this I would be happy on seeing the solution.
Note: I'm just getting started with Scapy, so I can't promise this is the correct/only way to go.
As per Documentation: Add new protocols to Scapy, put the code with the protocol definition in a seperate python file. Make sure you also set the required headers. Then place that file either in scapy/layers or scapy/contrib.
After that, the protocol can be loaded with load_layer(...) or load_contrib(...) where you plan on using it.
For DUMBO we'll go with contrib.
dumbo.py:
# scapy.contrib.description = Dumbo the elephant
# scapy.contrib.status = loads
from scapy.packet import Packet, bind_layers
from scapy.fields import ShortField
from scapy.layers.inet import TCP
#Create simple Class
class DUMBO(Packet):
fields_desc = [
ShortField('ears',0),
ShortField('legs',0),
ShortField('trunk',0)
]
#Inform TCP that ports 9898 are this protocol
bind_layers(TCP, DUMBO, sport=9898, dport=9898)
Now let's use it:
$ scapy
>>> load_contrib("dumbo")
>>> pack1=IP()/TCP(sport=9898, dport=9898)/DUMBO(b'\x00\x02\x00\x04\x00\x01')
>>> pack2=IP()/TCP(sport=9898, dport=9898)/DUMBO(ears=2, legs=4, trunk=1)
>>> pack1
<IP frag=0 proto=tcp |<TCP sport=9898 dport=9898 |<DUMBO ears=2 legs=4 trunk=1 |>>>
>>> pack2
<IP frag=0 proto=tcp |<TCP sport=9898 dport=9898 |<DUMBO ears=2 legs=4 trunk=1 |>>>
>>> pack1[DUMBO].ears
2
>>> pack2[DUMBO].ears
2
Hope this helps somebody who stumbles upon this question.
Versions used: Python v3.8.5 ; Scapy v2.4.5
I've written a function for scapy which helps dissecting a packet. The function receives a raw packet and returns the class needed to dissect the contained data. The function should work, yet it somehow returns the wrong classes. Now I tested something. I just added the line
struct.unpack("!B", packet)
at the beginning of the function, which actually does nothing, but - and that's what's strange to me - somehow this makes the function do what it's supposed to. How can this be? I've tested it multiple times. Without this one line at the beginning of the function it does not work properly, with this line, however, it behaves like it should, although the output seems a bit distorted and I don't know why either.
Output with the line. It's distorted, some lines are at the wrong place and some are even duplicated.
###[ Ethernet ]###
src = 00:30:de:09:c7:76
dst = ff:ff:ff:ff:ff:ff
type = 0x800
src = 00:30:de:09:c7:83
###[ IP ]###
type = 0x800 version = 4L
ihl = 5L
###[ IP ]###
tos = 0x0
version = 4L
len = 52
ihl = 5L
id = 312
tos = 0x0 flags = DF
frag = 0L
len = 52
ttl = 64
id = 91
proto = udp
flags = DF
frag = 0L
ttl = 64
proto = udp
chksum = 0xb52d
chksum = 0xb60b
src = 192.168.1.4
src = 192.168.1.3
dst = 192.168.1.255 dst = 192.168.1.255
\options \
\options \
###[ UDP ]###
###[ UDP ]### sport = 47808
dport = 47808
sport = 47808
len = 32
dport = 47808
chksum = 0x6cec
len = 32
###[ BVLC ]###
chksum = 0x79eb
type = 0x81
###[ BVLC ]###
function = ORIGINAL_BROADCAST_NPDU
type = 0x81
length = 24
function = ORIGINAL_BROADCAST_NPDU###[ NPDU ]###
version = 1
length = 24
control = 32L
###[ NPDU ]###
dnet = 65535
version = 1
dlen = 0 control = 32L
hopCount = 255
dnet = 65535
###[ APDU ]###
dlen = 0
pduType = UNCONFIRMED_SERVICE_REQUEST
hopCount = 255
reserved = None
###[ APDU ]###
serviceChoice= I_AM
pduType = UNCONFIRMED_SERVICE_REQUEST
\tagsField \
reserved = None
|###[ Raw ]###
serviceChoice= I_AM
| load = '\xc4\x02\t\xc7\x83"\x01\xe0\x91\x00!\xde'
\tagsField \
|###[ Raw ]###
| load = '\xc4\x02\t\xc7v"\x01\xe0\x91\x00!\xde'
Without the line the output looks like this, but the data is actually wrong. The Raw layer should actually be an APDU layer.
###[ Ethernet ]###
dst = ff:ff:ff:ff:ff:ff
src = 00:30:de:09:c7:83
type = 0x800
###[ IP ]###
version = 4L
ihl = 5L
tos = 0x0
len = 52
id = 105
flags = DF
frag = 0L
ttl = 64
proto = udp
chksum = 0xb5fd
src = 192.168.1.3
dst = 192.168.1.255
\options \
###[ UDP ]###
sport = 47808
dport = 47808
len = 32
chksum = 0x6cec
###[ BVLC ]###
type = 0x81
function = ORIGINAL_BROADCAST_NPDU
length = 24
###[ NPDU ]###
version = 1
control = 32L
dnet = 65535
dlen = 0
hopCount = 255
###[ Raw ]###
load = '\x10\x00\xc4\x02\t\xc7\x83"\x01\xe0\x91\x00!\xde'
The function looks like this
def guessBACnetTagClass(packet, **kargs):
""" Returns the correct BACnetTag Class needed to dissect
the current tag
#type packet: binary string
#param packet: the current packet
#type cls: class
#return cls: the correct class for dissection
"""
struct.unpack("!B", packet) # <------ with this line it works
# Convert main tag Byte to binary format
tagByteBinary = "{0:08b}".format(int(struct.unpack("!B", packet[0])[0]))
# Extract information from main tag Byte
tagNumber = int(tagByteBinary[0:4], 2)
tagClass = BACnetTagClass.revDict()[int(tagByteBinary[4:5], 2)]
lengthValueType = int(tagByteBinary[5:8], 2)
# Tag is Application Tag
if tagClass == BACnetTagClass.APPLICATION:
clsName = BACnetApplicationTagClasses[tagNumber]
cls = globals()[clsName]
print cls
return cls(packet, **kargs)
# Tag is Context Specific Tag
if tagClass == BACnetTagClass.CONTEXT_SPECIFIC:
if lengthValueType == BACnetConstructedLVT.OPENING_TAG:
cls = BACnetTag_Opening
if lengthValueType == BACnetConstructedLVT.CLOSING_TAG:
cls = BACnetTag_Closing
# Check if a class was selected
if cls is not None:
print cls
return cls(packet, **kargs)
Does this make any sense? How can executing this one function change output this much?
I've struggled the past 2 hours with the new Websocket version. I've managed to get the handshake and receiving these new frames, but I'm having problems sending them now.
I'm encoding my text like this:
def encode_hybi(buf, opcode, base64=False):
""" Encode a HyBi style WebSocket frame.
Optional opcode:
0x0 - continuation
0x1 - text frame (base64 encode buf)
0x2 - binary frame (use raw buf)
0x8 - connection close
0x9 - ping
0xA - pong
"""
if base64:
buf = b64encode(buf)
b1 = 0x80 | (opcode & 0x0f) # FIN + opcode
payload_len = len(buf)
if payload_len <= 125:
header = struct.pack('>BB', b1, payload_len)
elif payload_len > 125 and payload_len < 65536:
header = struct.pack('>BBH', b1, 126, payload_len)
elif payload_len >= 65536:
header = struct.pack('>BBQ', b1, 127, payload_len)
#print("Encoded: %s" % repr(header + buf))
#return header + buf, len(header), 0
return header+buf
But I don't know in what form I have to pour it to send it over the socket.
By the way: isn't there some easy python websocket module somewhere? My code has now seen 3 websocket versions and it's an utter mess.