I have written a basic network packet sniffer with Python. When the packets comes, the program prints broken characters in my gnome-terminal.
Program codes:
#!/usr/bin/python
import socket
import struct
import binascii
s = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, 8)
i = 1
while True:
pkt = s.recvfrom(2048)
ethhead = pkt[0][0:14]
eth = struct.unpack("!6s6s2s",ethhead)
print "--------Ethernet Frame %d------" % i
print "Destination MAC: ", binascii.hexlify(eth[0])
print "Source MAC: ", binascii.hexlify(eth[1])
binascii.hexlify(eth[2])
ipheader = pkt[0][14:34] #next 20 bytes
ip_hdr = struct.unpack("!8sB3s4s4s",ipheader)
print "-----------IP------------------"
print "TTL :", ip_hdr[1]
print "Source IP", socket.inet_ntoa(ip_hdr[3])
print "Destination IP", socket.inet_ntoa(ip_hdr[4])
tcpheader = pkt[0][34:54] #extracts next 20 bytes
tcp_hdr = struct.unpack("!HH9ss6s",tcpheader)
print "---------TCP----------"
print "Source Port ", tcp_hdr[0]
print "Destination port ", tcp_hdr[1]
print "Flag ",binascii.hexlify(tcp_hdr[3])
print "\n\n"
i += 1
print pkt[0][54:]
The sample output:
Sample Output Picture
I had set Terminal>Set character Encoding>Unicode(UTF-8) but it did not work either.
I am using Kali Linux 1.1.0, Gnome Terminal v 3.4.1.1.
Unfotunately, many protocols such as SSH and HTTPS use encryption and show characters that don't exist in ASCII or UTF-8. Your terminal will not be able to show these. This is because things like passwords and data should be hidden so that people can't do MITM nearly as easily. Try Wireshark, a graphical packet sniffer, it might work.
Bytes in the range 128-255 do not generally form valid UTF-8 strings. Even if the payload is encoded in UTF-8, there is no guarantee that packet boundaries will coincide with code point boundaries.
Perhaps as a workaround you would like to render the packets as e.g. ISO-8859-1 strings, or maybe even (gasp) offer an option to specify an encoding. The only change you need is pkt[0][54:].decode(encoding) - if there ever is a packet containing only complete, valid UTF-8 sequences, mentally decoding it from this output should be possible with a bit of training.
Related
I tried TCP/IP communication between the same machine and TCP/IP communication between different machines.
First of all, I tried communication in the same Windows machine.The server and client code used is:
TCP_server.py
import socket
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(('', 50001))
s.listen(1)
while True:
conn, addr = s.accept()
with conn:
while True:
data = conn.recv(30000)
if not data:
break
if len(data.decode('utf-8')) < 35:
print("error")
break
print(data.decode('utf-8')+"\n")
TCP_client.py
# -*- coding : UTF-8 -*-
import socket
target_ip = "192.168.1.5"
target_port = 50001
buffer_size = 4096
tcp_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcp_client.connect((target_ip,target_port))
message = b'123456789101112131415161718192021222324252627282930\n'
while True:
tcp_client.send(message)
The IP address of my Windows machine is 192.168.1.5, so the above code works. And it executed successfully without any error. The printed string is shown in the image below.
But when I tried to communicate with Mac and Windows using the exact same code, I had a problem. I used a Mac for the client and Windows for the server.The character string output on the server side is as follows.
As you can see from the image above, it is normally printed normally, but sometimes a line break is made and the character string is divided.
And my server-side code says that if the number of characters is less than 35, it will print error. However, error is not printed in this execution result.In other words, communication is not performed twice, but line breaks are inserted in one communication.
Is it possible to avoid this problem? Do I always have to be aware of line breaks when sending from another machine over TCP/IP?
I'm only using Python in this sample, but I had a similar problem using iOS's Swift for client-side code. So I would like to know a general solution.
There is no line break added by transmission of the data. This line break is instead added by the server code:
print(data.decode('utf-8')+"\n")
Both the print itself causes a line break and then you also add another one.
In general you are assuming that each send has a matching recv. This assumption is wrong. TCP is a byte stream and not a message stream and the payloads from multiple send might be merged together to reduce the overhead of sending and it might also cause a "split" into a single "message".
This is especially true when sending traffic between machines since the bandwidth between the machines is less than the local bandwidth and the MTU of the data layer is also much smaller.
Given that you have to first collect your "messages" at the server side. Only after you've got a complete "message" (whatever this is in your case) you should decode('utf-8'). Otherwise your code might crash when trying to decode a character which has a multi-byte UTF-8 encoding but where not all bytes were received yet.
I am receiving UDP packets over wifi running a simple python script on a PC. The server and the PC are in the same subnet.
The server is sending 15 uint_8 (4 bytes each) every 20 ms or so. The data received seems to be corrupted (non Hex values). Any feedback why this could be happening greatly appreciated.
For example I get something like this,
'\xb3}fC\xb7v\t>\xc8X\xd2=g\x8e1\xbf\xe6D3\xbf\x00\x00\x13\xc3\xc8g\x1b#\xc2\x12\xb2B\x01\x000=\x02\xc0~?\x01\x00\x94<\x00\x00\x00\x00\x00\x00\x00\x00\x00
#\x9c\xbe\xac\xc9V#', ('192.168.4.1', 4097))
The script is attached here.
from socket import *
import time
HOST = '192.168.4.10'
PORT = 9048
address = (HOST, PORT)
client_socket = socket(AF_INET, SOCK_DGRAM) #Set Up the Socket
client_socket.bind((HOST, PORT))
client_socket.settimeout(5) #only wait 5 second for a response, otherwise timeout
while(1): #Main Loop
single_var = client_socket.recvfrom(1024)
print single_var #Print the response from Arduino
time.sleep(10/1000000) # sleep 10 microseconds
The print statement doesn't know that you want hex output, so it interprets hex values that have valid character representations as characters. If you want to print it as hex bytes, see the solution in Print a string as hex bytes.
i.e. do:
print ":".join("{:02x}".format(ord(c)) for c in single_var)
I am not a Python programmer but am rather electronic circuit designer, however this time I must process some raw data sent by a microcontroller via RS232 port towards Python script (which is called by PHP script).
I've spent quite a few hours trying to determine the best ways of reading raw bytes from serial (RS232) port using Python and I did get the results - but I would like if someone could clarify certain inconsistencies I noticed during researching and here they are:
1:
I can see a lot of people who asked similar question had been asked whether they are using serial or pySerial module and how did they install the serial library. I can only say I don't really know which module I am using as the module worked out-of-the-box. Somewhere I read serial and pySerial is the same thing but I cannot find if that is true. All I know is I am using Python 2.7.9 with Raspbian OS.
2:
I've read there are read() and readline() methods for reading from the serial port but in the pySerial API docs there is no mention of the readline() method. Futhermore, I discovered the 'number of bytes to read' argument can be passed to readline() method as well as to the read() method (and works the same way, limiting the number of bytes to be read) but I cannot find that to be documented.
3:
When searching for how to determine if all of the data from the RS232 buffer has been read I have here found the following code:
read_byte = ser.read()
while read_byte is not None:
read_byte = ser.read()
print '%x' % ord(read_byte)
but that results with the:
Traceback (most recent call last):
File "./testread.py", line 53, in <module>
read_all()
File "./testread.py", line 32, in read_all
print '%x' % ord(read_byte)
TypeError: ord() expected a character, but string of length 0 found
upon reading the last byte from the buffer and I was able to detect the empty buffer only with the following code:
while True:
c = rs232.read()
if len(c) == 0:
break
print int(c.encode("hex"), 16), " ",
so I am not sure if the code that didn't work for me is for some serial library that is other than mine. My code for openinig port is BTW:
rs232 = serial.Serial(
port = '/dev/ttyUSB0',
baudrate = 2400,
parity = serial.PARITY_NONE,
stopbits = serial.STOPBITS_ONE,
bytesize = serial.EIGHTBITS,
timeout = 1
)
4:
The data I am receiving from µC is in the format:
0x16 0x02 0x0b 0xc9 ... 0x0d 0x0a
That is some raw bytes + \r\n. Since 'raw bytes' can contain 0x00, can someone confirm that is not a problem regarding reading the bytes into the Python string variable? As I understand that should work well but am not 100% sure.
PySerial works for me although haven't used it on a Pi.
3: Read() returns a string - this will be zero length if no data is read, so your later version is correct. As a string is not a character, you should use e.g. ord(read_byte[0]) to print the number corresponding to the first character (if the length of the string >0)
Your function:
while True:
c = rs232.read()
if len(c) == 0:
break
print int(c.encode("hex"), 16), " ",
Needs something adding to accumulate the data read, otherwise it is thrown away
rcvd = ""
while True:
c = rs232.read()
if len(c) == 0:
break
rcvd += c
for ch in c:
print ord(ch), " ",
4:
Yes you can receive and put nul (0x00) bytes in a string. For example:
a="\x00"
print len(a)
will print length 1
I have a Python CGI script on Raspberry Pi which converts the contents of a text box to 1's and 0's. I'm looking to have a string of 1000 ish max.
The binary string is formatted slightly and sent to an arduino via the serial port... the arduino uses the data in the string to do stuff with LED's. If I cut and paste the binary string into the arduino serial monitor, all works fine, when I try to automote this via the following python code, everything starts to work but stops soon into the cycle.
I can control how long this takes to work by altering the time.sleep(3) after the ser.write command..... but I don't want to set an unecessary long delay I would like to ensure the code waits for the string to send before moving on to printing the HTML stuff back, (and stopping the light display on the arduino).
Having said this, the entire string MUST be getting through to the arduino as the arduino waits for '\n' at the end of the string to process it.
Guess this must be a school boy error.... tips and pointers much appreciated. Here is the CGI code I'm using.
#!/usr/bin/python
# Import modules for CGI handling and serial
import cgi, cgitb, serial, time, binascii
# Create instance of FieldStorage
form = cgi.FieldStorage()
#define serial port
ser = serial.Serial('/dev/ttyUSB0', 9600, timeout=1)
#Wait while serial connects
time.sleep(3)
# Get data from fields
prayer = form.getvalue('prayer')
# Convert to binary
binprayer = bin(int(binascii.hexlify(prayer), 16))
# remove the '0b' from the front end of the string
bintrimint = binprayer[2:]
# add a \n to the end
bintrim = bintrimint + '\n'
ser.write(bintrim)
time.sleep(3)
print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "<title>Binary Prayer - a test script</title>"
print "</head>"
print "<body>"
print "<h2>You entered the following text: %s</h2>" % prayer
print "<h2>%s</h2>" % binprayer
print "<h2>%s</h2>" % bintrim
print "</body>"
print "</html>"
I had a look at http://pyserial.sourceforge.net/pyserial_api.html
write(data)
Parameters: data – Data to send.
Returns: Number of bytes written.
I think you should make sure you write everything.
I'm trying to create a simple thingermajigger in python at the moment just to test out sending UDP packets over a socket. I think that I'm getting my script perfectly fine other than using the socket.sendto command. I keep getting errors regarding the portion where "bytes" would go... either TypeError: an interget is required, or when I make it an interget TypeError: a string is required. Could someone provide me with an example of how to send a byte?
the point in my script where I'm getting the error is as follows... please fill out an example as well as a possible explanation / documentation for a newbie.
#full script as requested
import socket
import random
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
butes = random._urandom(1024)
#originally found this as a way to generate bytes to send, but it didn't work out
print("target IP: ")
ip = input()
print("port: ")
port = input()
while 1:
sock.sendto(butes, (ip, port))
print("Sent %s amount of packets to %s at port %s." % (sent,ip,port))
sent += 1
In your posted code, port is a str, you should use port = int(input())
Aside: b'0x2E' which you had in the original question is 4 characters. If you mean chr(0x2E) you can also write '\x2E'