I am trying to create a port scanner (using SYN packets) with the sockets library (yes I know scapy would make this much easier, but I'm mostly doing this for a learning exercise.) I have crafted the packet and successfully sent it, however I'm having troubled receiving and parsing the subsequent response.
So far I've tried the s.recv(1024) and 4096, as well as recvfrom().
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
s.sendto(packet, (dstip, 80))
r = s.recv(1024)
print(r)
However, I am having trouble receiving the response, I can see that the packet is being sent correctly via Wireshark, and the SYN-ACK is sent to my machine, however I am unable to properly receive and print it. Is there a better way I can use the s.recv() function for this sort of input? Or am I using the wrong function?
Any help is appreciated, I'm new to the sockets library. Thanks.
The book Black Hat Python has en example using the socket library to create a scanner, unfortunately not a port scanner. They check if a host is up, and they use a raw socket to receive data. The code is available here.
They are sending SYN-packets with one socket object in a new thread, and sniffing the replies using another socket object.
In the example they use socket.IPPROTO_IP or socket.IPPROTO_ICMP instead of socket.IPPROTO_RAW depending on if it is Windows or not.
For the sniffer they use the function setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1) for sniffing, where IPPROTO_IP is a dummy-protocol for TCP, IP_HDRINCL is to include headers in the IP packets, and 1 is mapped to the ICMP-protocol in the code.
Good luck!
Below is a recent module I wrote with the help from various sources for socket IO, take what you would like from it.
import socket
import threading
import time
import pygogo as gogo
from icentralsimulator.bridgeio.read_packets import PacketFactory
from icentralsimulator.bridgeio.write_packets import WritePacket
from icentralsimulator.configurations.interfaces import IServerInfoProvider
logger = gogo.Gogo(__name__).logger
send_lock = threading.Lock()
class BridgeConnection:
def __init__(self, bridge_info_provider: IServerInfoProvider):
info = bridge_info_provider.get_bridge_server_info()
self.callback = None
self.bridge_ip = info.IpAddress
self.bridge_port = info.Port
self._connection = None
self._terminate_wait_for_incoming = False
#property
def is_connected(self):
return self._connection is not None
def connect(self, callback):
"""
The purpose of this method is to create (and hold) a connection to the server. At the same time,
it creates a new thread for the purpose of waiting on incoming packets.
"""
if self._connection is not None: return
self._connection = socket.create_connection((self.bridge_ip, self.bridge_port))
self._connection.settimeout(0.5)
self.callback = callback
t = threading.Thread(target=self._wait_for_incoming)
t.start()
time.sleep(5)
def disconnect(self):
"""
Breaks existing connection to the server if one is currently made and cancels the thread that is waiting
for incoming packets. If the connection is not currently open, simply returns silently -- thus it is safe
to call this method repeatedly.
"""
self._terminate_wait_for_incoming = True
while self._terminate_wait_for_incoming:
time.sleep(0.1)
self._connection.close()
self._connection = None
def send_packet(self, packet: WritePacket):
"""
Sends an arbitrary packet to the server.
"""
with send_lock:
logger.debug(f"Sending packet: {packet.payload_plain_text}")
payload = packet.payload
self._connection.sendall(payload)
def _wait_for_incoming(self):
"""
Continually runs a loop to wait for incoming data on the open socket. If data is received, it is converted
to a receive packet and forwarded to the consumer as part of a callback.
"""
self._terminate_wait_for_incoming = False
buf_len = 4096
try:
while not self._terminate_wait_for_incoming:
data = None
try:
_cnx = self._connection
if _cnx is None: break
data = _cnx.recv(buf_len)
if data is not None and len(data) > 0:
while True:
new_data = _cnx.recv(buf_len)
if new_data is None or len(new_data) == 0:
break
data = data + new_data
except socket.timeout:
if data is not None and self.callback is not None:
packet = PacketFactory.get_packet(data)
self.callback(packet)
logger.debug(f"Received packet: {data}")
time.sleep(0.5)
except OSError: # Happens when stopping the application
logger.info("Application aborted")
return
finally:
self._terminate_wait_for_incoming = False
Note that I don't include IServerInfoProvider, or the PacketFactory here. Those are pretty custom to my application. You will need to interpret the packet according to the packet data that arrives in your specific use case.
Related
I wrote a python script that connects to an TCP socket (plaintext) and wait for requests. The TCP connection is persistent, alls requests and response are handle over the same one connection. See https://openvpn.net/community-resources/management-interface/ for technical docs.
My current script works fine, but it's single threaded. I'm using the python select api for wait until new data on the socket is available, then do something and write a response back. While do some stuff (it could be time.sleep(30) as example) the whole application is blocked a new request wont be answer.
Since the requests have identifier, the responses must not send in the same order as the requests send.
Minimalistic code example of the current solution:
import select
import socket
import time
def _socket_recv(_sock) -> str:
"""Receive bytes from socket and convert to string.
"""
buffer_size = 4096 # 4 KiB
data = b""
while True:
part = _sock.recv(buffer_size)
data += part
if len(part) < buffer_size:
# either 0 or end of data
break
return data.decode("utf-8")
def do_work(data: str) -> None:
print(data)
# Do something
time.sleep(10)
ret = "status 1\n"
s.send(bytes(ret, "utf-8"))
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("{}".format('127.0.0.1'), 8081))
# Password
s.send(bytes("pass\n", "utf-8"))
while True:
# wait until data is available
_ = select.select([s], [], [])[0]
# CLIENT notifications may be multi-line, and
# the sequentiality of a given CLIENT notification, its associated environmental
# variables, and the terminating ">CLIENT:ENV,END" line are guaranteed to be
# atomic.
notifications = _socket_recv(s)
if notifications.startswith(">CLIENT:"):
do_work(notifications)
Maybe a ThreadPool/WorkerPool should be a good approach, but how to manage concurrents writes to the TCP socket? Should the thread write to the socket? There are known frameworks?
You may introduce a buffer between socket and process engine, as below.
import select
import socket
import time
def _socket_recv(_sock) -> str:
"""Receive bytes from socket and convert to string.
"""
buffer_size = 4096 # 4 KiB
data = b""
while True:
part = _sock.recv(buffer_size)
data += part
if len(part) < buffer_size:
# either 0 or end of data
break
return data.decode("utf-8")
buffer=[]
def do_work() -> None:
while True:
if len(buffer)>0:
data=buffer.pop(0)
print(data)
# Do something
time.sleep(10)
ret = "status 1\n"
s.send(bytes(ret, "utf-8"))
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("{}".format('127.0.0.1'), 8081))
threading.Thread(target=do_work).start()
# Password
s.send(bytes("pass\n", "utf-8"))
while True:
# wait until data is available
_ = select.select([s], [], [])[0]
# CLIENT notifications may be multi-line, and
# the sequentiality of a given CLIENT notification, its associated environmental
# variables, and the terminating ">CLIENT:ENV,END" line are guaranteed to be
# atomic.
notifications = _socket_recv(s)
if notifications.startswith(">CLIENT:"):
buffer.append(notifications)
I am creating a socket client and trying to obtain some data. In order to do so, I need to connect to a web server via socket and the server actually creates another socket which listens and awaits for the data after which sends back to the client.
The problem I have with the code below is that my socket client does not wait for the incoming data from the server and just accepts empty data.
How can I wait for a non-empty data from the server using Python sockets?
My code:
import sys
import json
import socketIO_client
import time
host = 'https://SOME_URL'
socketIO = socketIO_client.SocketIO(host, params={"email" : "edmund#gmail.com"})
def on_connect(*args):
print "socket.io connected"
def on_disconnect(*args):
print "socketIO diconnected"
socketIO.on('connect', on_connect)
socketIO.on('disconnect', on_disconnect)
def on_response_state(*args):
print args # Prints ()
socketIO.emit('receive_state',on_response_state)
socketIO.wait_for_callbacks(seconds=3)
Here's an example using socket. Using s.accept(), the client will wait till a client accepts the connection before starting the while loop to receive data. This should help with your problem.
def receiver():
PORT = 123
CHUNK_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('0.0.0.0', PORT))
s.listen(1)
conn,address=s.accept() # accept an incoming connection using accept() method which will block until a new client connects
while True:
datachunk = conn.recv(CHUNK_SIZE) # reads data chunk from the socket in batches using method recv() until it returns an empty string
if not datachunk:
break # no more data coming in, so break out of the while loop
data.append(datachunk) # add chunk to your already collected data
conn.close()
print(data)
return
receiver()
put the recv socket in a while thread.
like this:
def rec(self):
while 1:
sleep 0.01
rdata = self.clientsocket.recv(self.buffsize)
print("rec from server: ", rdata.decode('utf8'),'\n','press enter to continue')
....
t2 = threading.Thread(target=y.rec, name="rec")
t2.start()
Since you're using the SocketIO library to include parameters (achieved using requests), and want to emit a message, you can wait indefinitely for a response by not specifying a wait time.
with SocketIO(host, params={"email" : "edmund#gmail.com"}) as socketIO:
def on_response_state(*args):
print args # Prints ()
socketIO.emit('receive_state', on_response_state)
socketIO.wait()
I need to implement a server that receives a request specifying a set of topics via control channel (req-rep), and then as response sends a URL pointing to a publisher socket that will be opened for this specific client or a rejection message (because of insufficient privileges).
I managed to implement a version that can handle only one client at the time (with two endless loops), but I do not know which pattern to use to handle multiple clients concurrently.
It's important for me that the sockets for different clients stay separate.
Here's simplified code:
import zmq
context = zmq.Context()
upstream_port = 10000
upstream_host = 'localhost'
control_channel_port = 11000
upstream_addr = 'tcp://{}:{}'.format(upstream_host, upstream_port)
def should_grant(request):
'''
Permission checking - irrelevant to the question
'''
return True
def bind_downstream():
downstream = context.socket(zmq.PUB)
addr = 'tcp://*'
port = downstream.bind_to_random_port(addr)
return downstream, port
def bind_control_channel():
control_channel_sock = context.socket(zmq.REP)
control_channel_sock.bind('tcp://*:{}'.format(control_channel_port))
return control_channel_sock
def connect_upstream(topics):
raw_data = context.socket(zmq.SUB)
for t in topics:
raw_data.setsockopt_unicode(zmq.SUBSCRIBE, unicode(t))
raw_data.connect(upstream_addr)
return raw_data
if __name__ == '__main__':
print("Binding control channel socket on {}".format('tcp://*:{}'.format(control_channel_port)))
control_channel = bind_control_channel()
while True:
request = control_channel.recv_json()
print("Received request {}".format(request))
if should_grant(request):
(downstream_sock, downstream_port) = bind_downstream()
print("Downstream socket open on {}".format('tcp://*:{}'.format(downstream_port)))
print("Connecting to upstream on {}".format(upstream_addr))
upstream_sock = connect_upstream(request['topics'])
control_channel.send_json({'status': 'ok', 'port': downstream_port})
while True:
parts = upstream_sock.recv_multipart() # Simple forwarding
downstream_sock.send_multipart(parts)
else:
control_channel.send_json({'status': 'rejected'})
The correct way to do this would be to use threads.
Your main program or thread would handle the control channel loop. As soon as a connection appears, you would create the upstream and downstream sockets but handle the actual transfer in a thread. I am not sure if the code below works as I do not have a client that would work with it, but give it a go and see what happens. You will get the idea nevertheless.
from threading import Thread
....
....
class ClientManager(Thread):
def __init__(self, ups, downs):
super(ClientManager, self).__init__(self)
self.upstream_socket = ups
self.downstream_socket = downs
def run(self):
while True:
_parts = self.upstream_socket.recv_multipart()
self.downstream_socket.send_multipart(_parts)
if __name__ == '__main__':
print("Binding control channel socket on {}".format('tcp://*:{}'.format(control_channel_port)))
control_channel = bind_control_channel()
while True:
request = control_channel.recv_json()
print("Received request {}".format(request))
if should_grant(request):
(downstream_sock, downstream_port) = bind_downstream()
print("Downstream socket open on {}".format('tcp://*:{}'.format(downstream_port)))
print("Connecting to upstream on {}".format(upstream_addr))
upstream_sock = connect_upstream(request['topics'])
control_channel.send_json({'status': 'ok', 'port': downstream_port})
_nct = ClientManager(upstream_sock, downstream_sock)
_nct.daemon = True
_nct.start()
else:
control_channel.send_json({'status': 'rejected'})
I have python script with only one socket object that is connect to a java server.
I started a thread for sending heart beat message to server per 5 secs.
And another thread for receiving message from server.
BTW, all the data send/recv is in protobuffer format.
# socket_client.py
def recv_handler():
global client_socket
while True:
try:
# read 4 bytes first
pack_len = client_socket.recv(4)
pack_len = struct.unpack('!i', pack_len)[0]
# read the rest
recv_data = client_socket.recv(pack_len)
# decode
decompressed_data = data_util.decompressMessage(recv_data)
sc_pb_message = data_util.decodePBMessage(decompressed_data)
sc_head = data_util.parseHead(sc_pb_message)
except:
print 'error'
def heart_handler():
global client_socket
while True:
if client_socket:
message = data_util.makeMessage('MSG_HEART_BEAT')
compressed_data = data_util.compressMessage(message)
send_data = data_util.makeSendData(compressed_data)
try:
client_socket.send(send_data)
except:
print 'except'
pass
time.sleep(5)
def connect(address, port):
global client_socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect((address, port))
# thread sending heart beat message
th = threading.Thread(target = heart_handler)
th.start()
# thread recving message
tr = threading.Thread(target = recv_handler)
tr.start()
The code above works just fine. The script will send a heart beat message per 5 secs and receive the message from server and the message can be decoded successfully.
And here comes the trigger part than I do not know how to implement.
My python script need to receive input from the browser at the same time, so I started a BaseHTTPServer, to handle the POST request from the browser.
When a request come, I would like to call the client_socket.send method to send a specific message to the server and of course I need to return the data from server back to the browser.
# http_server.py
def do_POST(self):
# ...
result = socket_client.request(message)
self.send_response(200)
self.end_headers()
self.wfile.write(...)
And here is what I tried to do in request:
def request(message):
global client_socket
client_socket.send(message)
pack_len = client_socket.recv(4)
pack_len = struct.unpack('!i', pack_len)[0]
recv_data = client_socket.recv(pack_len)
return recv_data
The problem I am having is the data I received in the request method after calling the send method seems to be disturbed by the data of heart beat in the thread.
If I comment out the heart beat thread and the receive thread, than the request method will work just fine. The data from server can decoded with no error and it can be sent back to the browser successfully.
My solution now might be wrong and I really do not know how to get this work.
Any advice will be appreciated, thanks :)
socket object in Python is not thread-safe, you need to access the shared resources (in this case the client_socket object) with the help of some synchronization primitives, such as threading.Lock in Python 2. Check here for a similar problem: Python: Socket and threads?
I send data to socket on one side every second, but I can read that data on another side in any moment. Here's the writer:
from settings import Config
filename = Config.NAVIGATION_SOCKET_FILE
client = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
client.settimeout(None)
while True:
try:
client.connect(filename)
break
except Exception:
continue
messages = ["$GPRMC,125504.049,A,5542.2389,N,03741.6063,E,0.06,25.82,200906,,,*17",
"$GPRMC,155604.049,A,5542.2389,N,03741.6063,E,0.06,25.82,200906,,,*19",]
while True:
msg = random.choice(messages)
client.send(msg)
print msg
time.sleep(1)
And here's reader:
navigation_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
if os.path.exists(app.config['NAVIGATION_SOCKET_FILE']):
os.remove(app.config['NAVIGATION_SOCKET_FILE'])
navigation_socket.bind(app.config['NAVIGATION_SOCKET_FILE'])
class NavigationInfo(restful.Resource):
def get(self):
msg = navigation_socket.recv(1024)
regex = re.compile(r"^\$GPRMC,(?P<time>\d{6}\.\d{3}),(?P<status>A|V),"
r"(?P<latitude>\d{4}\.\d{4}),(?P<lat_n_s>N|S),"
r"(?P<longitude>\d{5}\.\d{4}),(?P<long_e_w>E|W),"
r"(?P<hor_speed>\d+.\d+),(?P<track_angle>\d+.\d+),"
r"(?P<date>\d{6}),(?P<magnetic_declination>\d+\.\d+)?,"
r"(?P<magnetic_decl_direction>\d)?,"
r"(?P<mode>A|D|E|N)?\*(?P<checksum>\d\d)")
result = regex.match(msg)
navigation_info = result.groupdict()
return navigation_info
So the first problem is that writer just stops writing data to socket when buffer is full (at least that's what I see) and when I request data on the other side, it's too old.
Can just I store one value in buffer and then rewrite it? Or maybe I'm getting it all wrong?
I think that you are using the solution in reverse.
Instead of pushing messaging, while not pulling messages ?
Your server may look like:
Wait for a connection
Give a random message
go to step 1
And your client may just connect to the server when he needs a message.
In your case, the connection is "opened all the time", in my solution the socket is opened only when needed. and closed right after the message is delivered.