I use python program to do traffic generator in Linux Ubuntu, and the code like below:
import socket, sys
host = sys.argv[1] #Server IP Address
textport = sys.argv[2] #Server Binding Port
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) #socket
try:
port = int(textport)
except ValueError:
port = socket.getservbyname(textport, 'udp')
while 1:
try:
data = open('auth3.log')#read file
for each_line in data: #each rows
try:
(role,line_spoken) = each_line.split(': ',1)#split two parts
role = role.strip()
s.sendto(role, (host, port))
print('Send: ' + str(role) + "\n" )
except:
pass
except IOError as err:
print('file isn\'t exist!!~'+str(err))
finally:
if 'data' in locals(): #If data have no object, Don't use data to close!!~
data.close()
print "\n"
The size of auth3.log is about 1.8M.
When I send data to the destination server, I use snmp which OID is ''ifInOctets'' to get the traffic information.
But I the traffic recalculate to unit of ''Kbits'' is about 128.
How can I use this program to fill the bandwidth up to 1Gbits?(In other words, I want to fill out the bandwidth)
Thanks for your helping.
This version of your code implements the first two optimizations suggested by Chris Merck.
import socket, sys, itertools
host = sys.argv[1] #Server IP Address
textport = sys.argv[2] #Server Binding Port
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
port = int(textport)
except ValueError:
port = socket.getservbyname(textport, 'udp')
# preprocess file data
with open('auth3.log') as data:
roles = [role for (role, line_spoken) in line.split(': ', 1) for line in data]
# preprocess everything we can
hp = (host, port)
send = s.sendto
for role in itertools.cycle(roles):
try:
send(role, hp)
except:
pass
For further optimizations, you might want to process it using Cython, which might further speed up the loop. If the code still doesn't generate enough traffic, you'll probably need to launch several processes in parallel.
Your program is not running fast enough to generate 1Gbps on the wire.
To make it run faster, you can:
Remove the call to print after sendto. (Print is slow by nature.)
Preprocess your auth3.log file so that you do not need to process it within your inner loop. (Right now you are looping on .split and .strip, both of which are wasting CPU time.
Rewrite your program to send larger chunks.
But, I fear the result will still not reach 1Gbps. To really max out your line, try using a traffic generation program such as Colasoft Packet Builder (although I'm not sure even that program will do it. 1Gbps is a lot of traffic.)
Related
I am trying multiple clients to send files to the server simultaneously on one port(i.e. server is running different ports and multiple clients are connected to each port and sending files). I have looked several answers such as this, but they are using different approaches and I just want somebody to pinpoint what I am doing wrong here, so I can use same code which I understand better. Please help me with:
Why my code is not working with multiple file transfer?
I am also calculating the throughput(i.e. actual file transfer), is it the correct method?
Thanks for help.
----- server.py ---
import socket,time
import sys, optparse,datetime
#def client(net,src,dst):
#def server(net,src):
print("we are in server ...")
parser = optparse.OptionParser()
parser.add_option('-i',dest='ip',default='')
parser.add_option('-p',dest='port',type='int',default=5001)
parser.add_option('-t',dest='ftype',type='str',default='.txt')
(options,args) = parser.parse_args()
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
host = socket.gethostname()
server_socket.bind((options.ip, options.port))
server_socket.listen(100)
s = datetime.datetime.now().strftime("%d%m%y_%H%M%S")
f = open('.recfile_%s_%s'%(s,options.port)+options.ftype,'wb')
count = 0
while 1:
client_socket, addr = server_socket.accept()
start_time = datetime.datetime.now()
cl_addr = addr[0]
print 'Got connection from', addr
print("Receiving ...")
data = client_socket.recv(1024)
while(data):
f.write(data)
data = client_socket.recv(1024)
count+=len(data)
continue
f.close()
client_socket.close()
end_time = datetime.datetime.now()
total_time = end_time - start_time
total_time = total_time.total_seconds()
average_speed = round((1024*count*0.001)/(total_time),3)
fd = open('server_data.csv','a+')
fd.write(str(cl_addr)+','+str(start_time)+','+str(end_time)+','+str(total_time)+','+str(average_speed)+','+str(options.port)+'\n\r')
fd.close()
server_socket.close()
client side
----- client.py -----
import socket
import sys, optparse
#def client(net,src,dst):
print("we are in client ..")
parser = optparse.OptionParser()
parser.add_option('-i',dest='ip',default='')
parser.add_option('-p',dest='port',type='int',default=5001)
parser.add_option('-f',dest='fname',type='str',default='hugefile.txt')
(options,args) = parser.parse_args()
client_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
client_socket.connect((options.ip,options.port))
img_file = options.fname
f = open(img_file,'rb')
data = f.read(1024)
while(data):
client_socket.send(data)
data = f.read(1024)
f.close()
client_socket.shutdown(socket.SHUT_WR)
client_socket.close()
print "Data Sent successfully!"
There is at least one problem: the recfile file is opened before starting the loop, and closed inside the loop. That means that beginning from the second iteration, you will try to write on a closed file and get an exception.
How to avoid: with open(...) as ...: blocks are great because not only they guarantee proper closure in case of errors, but they also ensure a correct bloc structure in your program.
BTW, count should also be reset to 0 inside the loop, and the closer to the loop the better for future readers and maintainers of the code
I found a solution by improvising this .
Multiple connection to same socket is not possible without either multiprocessing or multithreading. Since I am using Python 2.7 multithreading is not an option for me.
Hi i have an exercise to build with sockets select and msvcrt, server and clients of mltiplie chat(the server and the clients need to be built non-blocking) that every client will send message and the server will send the message to all the clients except the one who sent it, the server:
import socket
import select
IP = "192.168.1.154"
port = 123
default_buffer_size = 1024
open_client_sockets = []
messages_to_send = []
def send_waiting_messages(wlist):
for message in messages_to_send:
(client_sock, data) = message
if client_sock in wlist:
for sock in open_client_sockets:
if sock is not client_sock:
sock.send(data)
messages_to_send.remove(message)
def main():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((IP, port))
sock.listen(5)
print("The server is on and waiting for client...")
while True:
rlist, wlist, xlist = select.select([sock] + open_client_sockets, open_client_sockets, [])
for current_socket in rlist:
if current_socket is sock:
(new_socket, addr) = sock.accept()
open_client_sockets.append(new_socket)
else:
data = current_socket.recv(default_buffer_size)
if data == "":
open_client_sockets.remove(current_socket)
print("Connection with client closed")
else:
messages_to_send.append((current_socket, 'Hello ' + data))
send_waiting_messages(wlist)
if __name__ == '__main__':
main()
Building the server wasnt hard because it was guided(if it was not guided i would never got this code working) by the book but i have problem building the client and the main reason is that i dont understand how select.select works, couldn't find answer that will simplify enough this module for me.
this is what i did with the client:
import socket
import select
import msvcrt
IP = "192.168.1.154"
port = 123
sockets = []
def write():
pass
def main():
sock = socket.socket()
sock.connect((IP, port))
while True:
rlist, wlist, xlist = select.select(sockets, sockets, [])
for current_socket in rlist:
if current_socket is sock:
data = current_socket.recv(1024)
print(data)
else:
sockets.append(current_socket)
write()
if __name__ == '__main__':
main()
This probably shows you that I have low understanding of the module select and the exercise actually. I saw some threads that has similar question but I understand nothing from them so I realy need good explantion.
In conclusion I realy am lost...
select takes as parameters a list of sockets to wait for readablity, a list of sockets to wait for writability, and a list of sockets to wait for errors. It returns lists of ready to read, ready to write, and error sockets. From help:
>>> help(select.select)
Help on built-in function select in module select:
select(...)
select(rlist, wlist, xlist[, timeout]) -> (rlist, wlist, xlist)
Wait until one or more file descriptors are ready for some kind of I/O.
The first three arguments are sequences of file descriptors to be waited for:
rlist -- wait until ready for reading
wlist -- wait until ready for writing
xlist -- wait for an ``exceptional condition''
If only one kind of condition is required, pass [] for the other lists.
A file descriptor is either a socket or file object, or a small integer
gotten from a fileno() method call on one of those.
The optional 4th argument specifies a timeout in seconds; it may be
a floating point number to specify fractions of seconds. If it is absent
or None, the call will never time out.
The return value is a tuple of three lists corresponding to the first three
arguments; each contains the subset of the corresponding file descriptors
that are ready.
*** IMPORTANT NOTICE ***
On Windows, only sockets are supported; on Unix, all file
descriptors can be used.
So to fix your client, you need to add the socket you opened (sock) to the sockets list. Your write function can then be called if your socket is ready to be written.
In write, use msvcrt.kbhit() to test for characters typed. You can't just use input because it will block. Then read the character if one has been typed. Collect up the characters until you hit enter, then build a message and write it to the socket. Something like:
message = []
def write(sock):
if msvcrt.kbhit():
c = msvcrt.getche()
if c == '\r':
data = ''.join(message)
print 'sending:',data
sock.sendall(data)
message.clear()
else:
message.append(c)
I have a program that scans for open ports on remote host.
It will take long time to complete the scan.I want to make it work fast.
Here's my code:
Port Scan
import socket
import subprocess
host = input("Enter a remote host to scan: ")
hostIP = socket.gethostbyname(host)
print("Please wait, scanning remote host", hostIP)
try:
for port in range(1,1024):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex((hostIP, port))
if result == 0:
print("Port: \t Open".format(port))
sock.close()
Could one of you Python wizards help me with this.
Advance Thanks.
You can set a timeout on the socket so it wont spend to much time on a closed port. I would also use threads and allow the user to specify how many threads they want to run. here is a link to some code you could adapt to implement threading with the threading module Python Network Programming.
#!/usr/bin/env python
'''
A simple port scanner.
'''
import socket
def scan_host(host, **options):
'''
Scan a host for open ports.
'''
options.setdefault('timeout', 0.30)
options.setdefault('port_range', (1, 1024))
timeout = options.get('timeout')
port_range = options.get('port_range')
host_ip = socket.gethostbyname(host)
print("Please wait, scanning remote host {} : {}".format(host, host_ip))
for port in xrange(*port_range):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(timeout)
result = sock.connect_ex((host_ip, port))
if result == 0:
print "Port: {} Open".format(port)
sock.close()
if __name__ == '__main__':
scan_host('www.google.com', timeout=0.30, port_range=(1, 8000))
This program became too simple. It monitors only one port at once and it takes long time on one port to see if it is listening. So try reducing the time to listen, if it can't connect, deem it to be closed by setting a recursion limit for that number under the "expect:" in run().
As in like this,
try:
# connect to the given host:port
result = sock.connect_ex((hostIP, port))
if result == 0:
print "%s:%d Open" % (hostIP, port)
sock.close()
except: #pass
sock.recurse += 1
if sock.recurse < sock.limit:
sock.run()
else:
print "%s:%d Closed" % (hostIP, port)
There is other way to make it much more efficient by importing threading() module which can be used to keep an eye on a large number of sockets at once.
Here's the document on threading.
Refer this,
https://docs.python.org/2/library/threading.html#
Hope that helped you.
All the best.
In Python how do I fill a buffer with lines of data (strings) and consume it with a second process? There are ample of examples here adding and reading lines from a string, but I need to remove the consumed line from the string for the string to work as a buffer.
Example: read sporadic data from a serial port and send it via TCP/IP to a server. Line-by-line within one loop and no buffering = no problem, but in case the destination is unreachable the data should be stored in the buffer and then sent once connection is available.
#!/usr/bin/python
import serial
import socket
from multiprocessing import Process
ip = "someURL"
port = 12345
ser = serial.Serial("/dev/ttyUSB0", 57600, timeout=0)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def serial_reader():
while True:
for line in ser.read():
try:
response = ser.readlines(None)
response = str(response)
message = response[7:]
except:
print datetime.datetime.now(), " No data from serial connection."
##
def data_sender():
s.connect((ip, port))
while True:
for line in queue():
try:
s.send(message)
except:
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip, port))
continue
except:
s.close()
##
if __name__ == '__main__':
Process(target=serial_reader).start()
Process(target=data_sender).start()
I think the best way to achieve what you want is to use a queue:
from multiprocessing import Queue
specifically use queue.put() to put a string on the queue, queue.get() to retrieve it, and queue.task_done() to indicate that the task is complete.
https://docs.python.org/2/library/queue.html#Queue.Queue
if you need a bigger gun take a look at RabbitMQ and python libraries that implement the AMPQ protocol such as rabbitpy. This is the defacto standard for inter process/inter service communication and has a lot of usefyl stuff already baked in, such as persisting messages in case the processes shut down, load balancing tasks across multiple processes, etc.
I am totally new to socket programming.
I have a product and trying to connect.
I can send the data and see its result on product, but I cannot receive anything.
this is my script:
import socket
def ScktConn():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 5006))
# our local IP is 192.168.2.1, but it works even with 127.0.0.1, I don't know from where #it is coming
Freq=raw_input('Frequency(450-2500): ')
CmdF='0 ace_set_frequency C1 '+str(Freq)+' \r\n'
s.send(CmdF)
# so far I sent a tcl command to product to set the frequency and it works
s.send('0 ace_azplayer_remove_player XXX \r\n')
# sending another tcl command and works
s.send('0 ace_azplayer_add_player \r\n')
# here it is working too
s.send('0 ace_azplayer_add_ace XXX C1\r\n')
Path='C:/Users/AM_RJ/Desktop/gridview_script/PBF/4x4U_wocorr_SNR.csv'
s.send('0 ace_azplayer_load_csvfile AzPlayer1 '+Path+' \r\n')
# here I should receive some numbers, but always returning me 0!
#even if I send ('hello!') and use recv(1024), it returns 0!
csvid=s.recv(4096)
print csvid
Path2='0 ace_azplayer_edit_playback_file AzPlayer1 '+str(csvid)+' -linkConfiguration "4x4" \r\n'
print Path2
s.send(Path2)
After using recv(4096), I should receive some numbers, but it always returning me 0!
even if I send ('hello!') and use recv(1024), it returns 0!
I'm using python 2.7.
I am not even sure whether or not the server and client sides are correct in my script!
Please help me out about it.
You need more than one socket, here is a minimal example (which would need more work to be made robust). ScktConn spawns a new thread which creates a server socket that listens for the connection from s.
import socket
import threading
import time
address = ('127.0.0.1', 5007)
def ScktRecv():
r = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
r.bind(address)
r.listen(1)
conn, _ = r.accept()
csvid = conn.recv(4096)
print "recv: %s" % csvid
def ScktConn():
recv_thread = threading.Thread(target=ScktRecv)
recv_thread.start()
time.sleep(1)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(address)
# our local IP is 192.168.2.1, but it works even with 127.0.0.1, I don't know from where #it is coming
Freq=raw_input('Frequency(450-2500): ')
CmdF='0 ace_set_frequency C1 '+str(Freq)+' \r\n'
s.send(CmdF)