collective internet,
I am a very new programmer that has given myself a specific project to teach myself coding. I work with a lot of equipment that can take TCP commands so I set out to build a system of buttons that will send different commands per each button. I got myself a Raspberry Pi 3b and took online classes on Python. I've got reasonably far on my own (I've got the buttons working how I want!) but where I've been stuck is sending TCP commands.
To be more specific: I am sending data and it is being received but the string command is not being encoded properly. The commands are functional when I execute them in a telnet session, but obviously I want them executed as part of my script. The commands don't specify that they need to be received over a telnet session and, by other means, I've had these commands work as TCP commands exterior to a telnet session. I read about a telnet module for Python but I don't think I should need it.
I verified packet delivery with wireshark. I captured the packets sent by my script and the packets sent by the telnet session and they're similar but not the same. Horseshoes and hand grenades, right? My current method has been to just preface the string (within ') with a lower case b. I also tried putting .encode() after the string (omitting the b in that situation).
The string command has the format:
setInput "InputName" Value
So for my use case, I'm setting the input named "One" to a value of 1:
setInput "One" 1
So as you can see in my script (inserted below) I ended up using:
s.sendall(b'setInput "One" 1')
But it's not quite sending the right information because it is not working and it doesn't look the same in wireshark.
TL;DR: I'm trying to send packets via TCP but they're not being encoded properly.
Ultimately, my question is if I am even headed in the right direction using these commands and just need a different means to encode the string or if I need to explore another direction entirely (perhaps the telnet module?)
Here is the script I've been using to test and the wireshark output of my script:
import socket
import time
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('192.168.100.40', 3040))
print('connected')
time.sleep(2)
s.sendall(b'setInput "One" 1')
print('sent increase')
time.sleep(2)
s.sendall(b'setInput "One" 0')
print('sent decrease')
Wireshark log of my script
Here is the wireshark output of the telnet session that was successful:
Wireshark log of the telnet session
Any and all help is appreciated. I looked far and wide and can't seem to find any cases similar to mine.
EDITS: Sorry for the poor formatting. I appreciate the advice on how better to present posts. This is my first post here and I'm just getting the hang of it. My photos are still links due to my lack of privileges here. Sorry if I was too wordy, I just wanted to supply as much information as possible so as to help people understand my problem and, if a solution is found, to help people with a similar issue find this.
The telnet tcp data includes a carriage return and a linefeed and the end of the data. Apparently the receiving part needs this to be included to make things work. So change your Python string to
b'setInput "One" 1\r\n'
Related
Ok, I realize this situation is somewhat unusual, but I need to establish a TCP connection (the 3-way handshake) using only raw sockets (in C, in linux) -- i.e. I need to construct the IP headers and TCP headers myself. I'm writing a server (so I have to first respond to the incoming SYN packet), and for whatever reason I can't seem to get it right. Yes, I realize that a SOCK_STREAM will handle this for me, but for reasons I don't want to go into that isn't an option.
The tutorials I've found online on using raw sockets all describe how to build a SYN flooder, but this is somewhat easier than actually establishing a TCP connection, since you don't have to construct a response based on the original packet. I've gotten the SYN flooder examples working, and I can read the incoming SYN packet just fine from the raw socket, but I'm still having trouble creating a valid SYN/ACK response to an incoming SYN from the client.
So, does anyone know a good tutorial on using raw sockets that goes beyond creating a SYN flooder, or does anyone have some code that could do this (using SOCK_RAW, and not SOCK_STREAM)? I would be very grateful.
MarkR is absolutely right -- the problem is that the kernel is sending reset packets in response to the initial packet because it thinks the port is closed. The kernel is beating me to the response and the connection dies. I was using tcpdump to monitor the connection already -- I should have been more observant and noticed that there were TWO replies one of which was a reset that was screwing things up, as well as the response my program created. D'OH!
The solution that seems to work best is to use an iptables rule, as suggested by MarkR, to block the outbound packets. However, there's an easier way to do it than using the mark option, as suggested. I just match whether the reset TCP flag is set. During the course of a normal connection this is unlikely to be needed, and it doesn't really matter to my application if I block all outbound reset packets from the port being used. This effectively blocks the kernel's unwanted response, but not my own packets. If the port my program is listening on is 9999 then the iptables rule looks like this:
iptables -t filter -I OUTPUT -p tcp --sport 9999 --tcp-flags RST RST -j DROP
You want to implement part of a TCP stack in userspace... this is ok, some other apps do this.
One problem you will come across is that the kernel will be sending out (generally negative, unhelpful) replies to incoming packets. This is going to screw up any communication you attempt to initiate.
One way to avoid this is to use an IP address and interface that the kernel does not have its own IP stack using- which is fine but you will need to deal with link-layer stuff (specifically, arp) yourself. That would require a socket lower than IPPROTO_IP, SOCK_RAW - you need a packet socket (I think).
It may also be possible to block the kernel's responses using an iptables rule- but I rather suspect that the rules will apply to your own packets as well somehow, unless you can manage to get them treated differently (perhaps applying a netfilter "mark" to your own packets?)
Read the man pages
socket(7)
ip(7)
packet(7)
Which explain about various options and ioctls which apply to types of sockets.
Of course you'll need a tool like Wireshark to inspect what's going on. You will need several machines to test this, I recommend using vmware (or similar) to reduce the amount of hardware required.
Sorry I can't recommend a specific tutorial.
Good luck.
I realise that this is an old thread, but here's a tutorial that goes beyond the normal SYN flooders: http://www.enderunix.org/docs/en/rawipspoof/
Hope it might be of help to someone.
I can't help you out on any tutorials.
But I can give you some advice on the tools that you could use to assist in debugging.
First off, as bmdhacks has suggested, get yourself a copy of wireshark (or tcpdump - but wireshark is easier to use). Capture a good handshake. Make sure that you save this.
Capture one of your handshakes that fails. Wireshark has quite good packet parsing and error checking, so if there's a straightforward error it will probably tell you.
Next, get yourself a copy of tcpreplay. This should also include a tool called "tcprewrite".
tcprewrite will allow you to split your previously saved capture files into two - one for each side of the handshake.
You can then use tcpreplay to play back one side of the handshake so you have a consistent set of packets to play with.
Then you use wireshark (again) to check your responses.
I don't have a tutorial, but I recently used Wireshark to good effect to debug some raw sockets programming I was doing. If you capture the packets you're sending, wireshark will do a good job of showing you if they're malformed or not. It's useful for comparing to a normal connection too.
There are structures for IP and TCP headers declared in netinet/ip.h & netinet/tcp.h respectively. You may want to look at the other headers in this directory for extra macros & stuff that may be of use.
You send a packet with the SYN flag set and a random sequence number (x). You should receive a SYN+ACK from the other side. This packet will have an acknowledgement number (y) that indicates the next sequence number the other side is expecting to receive as well as another sequence number (z). You send back an ACK packet that has sequence number x+1 and ack number z+1 to complete the connection.
You also need to make sure you calculate appropriate TCP/IP checksums & fill out the remainder of the header for the packets you send. Also, don't forget about things like host & network byte order.
TCP is defined in RFC 793, available here: http://www.faqs.org/rfcs/rfc793.html
Depending on what you're trying to do it may be easier to get existing software to handle the TCP handshaking for you.
One open source IP stack is lwIP (http://savannah.nongnu.org/projects/lwip/) which provides a full tcp/ip stack. It is very possible to get it running in user mode using either SOCK_RAW or pcap.
if you are using raw sockets, if you send using different source mac address to the actual one, linux will ignore the response packet and not send an rst.
I have a little problem receiving data correctly via pySerial: it often does not read the full data or too much of it. Sometimes and sometimes more often, there are additional characters or some characters/parts of the sended data are missing. It seems, PC and the emitter of the data are not synchronised correctly.
In the current example I use a arduino, sending 'Hello World' to the serial port of my PC (os is Ubuntu 14.04), with the following simple code:
void setup(){
Serial.begin(9600);
Serial.print("Programme initiated\n");
}
I use the following python3 code to receive the data:
import serial
import time
arduino = serial.Serial(port, baudrate=9600, timeout=2)
print(arduino.isOpen)
print(arduino)
time.sleep(1)
while True:
print(arduino.readline())
This is pretty much a simple tutorial example, and here is what I receive (apart from the correct stuff):
b'PrProgramme initiated\n'
or
b'PProgramme initiated\n'
or
b'ProgProgramme initiated\n'
or
b'ogramme initiated\n'
I moved on with more complex problems in my code, but still I didn't solved that problem. When sending a message in a loop from the arduino (the standard hello world code), it often needs time to stabilise (while that, it again does only show the middle fragment of the data) and after that running quite stable, but even then it sometimes breaks down single lines.
I faced the same difficulties when communicating with a multimeter device. There, it often does not read the first characters or mixes up with previous data.
Did anyone faced that problem before? I think it is a question of synchronisation, but I don't know how to solve it. And what puzzles me, is that I really only used tutorial stuff and it doesn't seem to work properly. Or is it a configuration problem of my PC?
What you are looking at is happening because some different things are going on.
First of all every time you open the serial port, you are causing what is called and "autoreset" and the Arduino reboot. That can be avoided in hardware or even in software by explicitly disabling RST signal on open. How to do that may vary and is out of scope of the question.
Then we have to understand that serial does NOT wait for the other part to be listening to send data; so if you disable the autoreset and connecting to the Arduino you should see random part of output of the program, depending of its current state.
Finally we have some buffer capability on pc (and sometimes even on the UART to USB side), so its not true that if you are not listening that data get lost, it may be still in the buffer.
We could say the first 3 artifact may be given by buffered data + reboot (this happen a lot when you send a lot of data, and that break the autoupload of code and you have to do a manual procedure), while the last one may be something that prevented the buffer to fill, maybe it was disabled by you, maybe some weird timing opening the serial, maybe you disabled the autoreset, maybe time that the arduino got enumerated part of the message was gone.
I'm making a cmd IRC client in Python. I want to receive data at the same time I can write message, in the previous code I did I could only write 2 messages and then it bugs and I can't write until it receives some kind of data.
The question is, can I have one cmd window running the received data and other one with a constant input waiting for me to write something to send?, maybe with threads?
I've looked through the subprocess library but I don't really know how to code it.
CMD1:
while Connected:
print socket.recv(1024)
CMD2:
while Connected:
text = raw_input("Text to send>> ")
socket.send(text)
(This is a pseudocode not a real one)
This approach you are proposing could be done by making a server like application, and 2 client applications that connect via localhost to send and receive events. So that way you could have 2 terminals open , connected to the same session of the server.
On the other side you should consider a different design approach that include ncurses which allow you to make a terminal ui with input and output at the same terminal (two sections up and down). You can reference: http://gnosis.cx/publish/programming/charming_python_6.html
I'm trying to write a proof of concept code, which will automatically spoof packets to a thick-client application.
I've chosen python as the language of choice. I have found resources to help me monitor for packets using scapy or other similar libraries.
How do I go about spoofing the packet.
Eg. Scenario :
Client C, Server S
C sends get request R(HTTP) to S
Proxy_python intercepts request R
Proxy_python crafts a HTTP response (r)
Proxy_python sends r to C
Essentially a MiTm on C. Is this possible. One condition to note is that, the proxy i'm writing should not need any configuration to get redirected to. It should ubiquotously listen for all packets.
Which makes me ask me another question : Can I make the python proxy listen to a particular PID ?
At least to answer the question regarding whether you can tie a PID to the packets being sent, this is not something that is explicitly included within the packet data. However, you can determine which port the process is sending traffic on and associate packets on that port to the process. I would reference this question for some information on how to get that port info. Hope this helps a little, not sure exactly what else you are looking for at the moment.
I guess it's socket programming. But I have never done socket programming expect for running the tutorial examples while learning Python. I need some more ideas to implement this.
What I specifically need is to run a monitoring program of a server which will poll or listen to traffic being exchange from different IPs across different popular ports. For example, how do I get data received and sent through port 80 of 192.168.1.10 and 192.168.1.1 ( which is the gateway).
I checked out a number of ready made tools like MRTG, Bwmon, Ntop etc but since we are looking at doing some specific pattern studies, we need to do data capturing within the program.
Idea is to monitor some popular ports and do a study of network traffic across some periods and compare them with some other data.
We would like to figure a way to do all this with Python....
You probably want to use scapy for that. Just sniff all ethernet traffic on a particular interface, drop everything that is not TCP and doesn't match the port.
Not sure if scapy can already track TCP connections (stuff like recognizing duplicate sequence numbers, extracting just the payload stream) but I would guess it probably can, and if not it's not too hard to hack together a good-enough TCP connection tracker that works for 95% of the traffic.
Alternatives would be to use sockets directly (look for raw sockets) or libpcap, which can both be done from Python. You may also want to check out the filter experssion syntax of the 'tcpdump' commandline tool, maybe it can do what you want already.
I bet there are more specialized high-level tools for this, but I don't know them.
PS: if you don't know wireshark yet, go check it out and play around with it first. It can follow TCP streams and will teach you what TCP connection tracking means. Maybe its commandline binary, tshark, can be used to extract TCP streams for what you want.
IPTraf is an ncurses based IP LAN monitoring tool. Has a capability to generate network statistics including TCP,UDP,ICMP and some more.
Since you're thinking to execute it from python, you may consider to use screen (screen manager with VT100/ANSI terminal emulation) to overcome ncurses issues and you may want to pass logging and interval parameters to IPTraf which forces iptraf to log to a file in a given interval. Little bit tricky but eventually you can have what you are looking for by basically parsing the log file.