Develop Reference

How to implement simple client-server interactions in pure python?

I want to implement an IoT application. I will give here a toy version of what I want to do.
Say I have two clients : 'client1' and 'client2' on REMOTE COMPUTERS, and a server 'server', that regulates the computations. The hard thing for me is the fact that the computations can't be made at the same place.
We have : clients_list = ['client1', 'client2']
I want to simulate an algorithm that looks like this:
The server starts with an initial value server_value
for round in range(R):
client_values_dict = {}
for client_id in clients_list:
server broadcasts server_value to the client 'client_id' # via http
client_value = action(server_value) # executed on clients computer
client broadcasts its value to the server # via http
at the meantime, server waits for the response
server fills dictionary with keys clients_list, values client values obtained with 'action' :
client_values_dict[client_id]
server_value = aggregate(client_values_dict) # executed on server computer
On the client side (in client.py), I have a function:
import time
def action(server_value):
time.sleep(10*random.random())
return server_value + random.random()-0.5
On the server side (in server.py), I have a function:
def aggregate(client_values_dict):
return sum(client_values_dict.values())/len(client_values_dict.values())
I want to implement that : I want to write a loop at server level that performs this. I think what I need is an API to handle client-server interactions and parallel computing.
I thought of using Flask for this but I'm afraid that the loop at server level will be blocked by the app.run(debug=True) loop, and that my code won't run until I break the app with CTRL+C.
I want the computations to be made in parallel by the two clients.
I am not familiar with web developpement, my problem might seem trivial and help is probably to be found everywhere on internet, but I don't know where to look at. Any help is cheerfully welcomed.
Here is an example ofa script that simulates what I want, but online.
# -*- coding: utf-8 -*-
import time
import random
server_value = 0
R = 10
clients_list = ['client1', 'client2']
def action(server_value):
time.sleep(3*random.random())
return server_value + random.random()-0.5
def aggregate(client_values_dict):
return sum(client_values_dict.values())/len(client_values_dict.values())
for round in range(R):
client_values_dict = {}
for client_id in clients_list:
client_value = action(server_value) # executed on clients computer
client_values_dict[client_id] = client_value
server_value = aggregate(client_values_dict)
print(server_value)

Have you tried network zero? It's an amazing networking library that I use all the time.
Install:
pip install networkzero
PyPI link: https://pypi.org/project/networkzero/
Docs: https://networkzero.readthedocs.io/en/latest/
Code sample (from their doc page):
Machine/process A:
import networkzero as nw0
address = nw0.advertise("hello")
while True:
name = nw0.wait_for_message_from(address)
nw0.send_reply_to(address, "Hello " + name)
Machine/process B:
import networkzero as nw0
hello = nw0.discover("hello")
reply = nw0.send_message_to(hello, "World!")
print(reply)
reply = nw0.send_message_to(hello, "Tim")
print(reply)
This library also supports more than just 2 connections on the local WiFi, read the docs for more info.
NOTE: I've used this answer before. You can see it here: How to set up a server for a local wifi multiplayer game for python

How to configure in Google Core IoT?

I've made a python program in Raspberry Pi 3 Model B that's supposed to connect to Google Cloud IoT Core with MQTT-protocol and get configurations. Sending data to Core has worked so far, but I just can't figure out how configuring works! Here's a code that's just supposed to get a config:
import time
import datetime
import jwt
import ssl
import json
import paho.mqtt.client as mqtt
time_now = datetime.datetime.utcnow()
#make key
token = {'iat' : time_now ,
'exp' : time_now + datetime.timedelta(minutes=60),
'aud' : "[PROJECT]"}
r = open("[PRIVATE KEY]", 'r')
pub_key = r.read()
jwt_key = jwt.encode(token, pub_key, algorithm='RS256')
#connect to iot-core
client = mqtt.Client(client_id='projects/[PROJECT]/locations/[LOCATION]/registries/[REGISTER]/devices/[DEVICE]')
client.username_pw_set(username='unused', password=jwt_key)
client.tls_set(ca_certs='/home/pi/.local/lib/python2.7/site-packages/grpc/_cython/_credentials/roots.pem', tls_version=ssl.PROTOCOL_TLSv1_2)
client.connect("mqtt.googleapis.com", 8883)
#configure and change state
state = 0
print state #naturally shows 0
print client.subscribe([DEVICE]/config, qos=1) #doesn't print custom config either, just (0,1)
print state #still shows 0
configuration in Iot Core device id is:
{ "state": 1 }
Even after running the program, the device's "state"-variable stays at 0 and the Core's Configuration & State History state that the CONFIG is "Not yet acknowledged by the device"
How do I get the device's "state" variable change from 0 to 1 from Core?

You have done the half of the job. 2 remarks.
It's maybe a detail, but you name your private key pub_key. And you don't close the file.
Based on this tutorial, you only subscribe to the MQTT. The return tuple (0,1) means MQTT_ERR_SUCCESS on QOS = 1. Thus your are connected. Great! Now, do the second part: consume the messages in the channel, and make your logic (according with the received message, change the state in your app if it's your use case)

SMTP - Fast and reliable connection probing without auth?

Briefing
I am currently building a python SMTP Mail sender program.
I added a feature so that the user would not be able to log in if there was no active internet connection, I tried many solutions/variations to make the real time connection checking as swift as possible, there were many problems such as:
The thread where the connection handler was running suddenly lagged when I pulled out the ethernet cable ( to test how it would handle the sudden disconnect )
The whole program crashed
It took several seconds for the program to detect the change
My current solution
I set up a data handling class which would contain all the necessary info ( the modules needed to share info effectively )
import smtplib
from socket import gaierror, timeout
class DataHandler:
is_logged_in = None
is_connected = None
server_conn = None
user_address = ''
user_passwd = ''
#staticmethod
def try_connect():
try:
DataHandler.server_conn = smtplib.SMTP('smtp.gmail.com', 587, timeout=1) # The place where the connection is checked
DataHandler.is_connected = True
except (smtplib.SMTPException, gaierror, timeout):
DataHandler.is_connected = False # Connection status changed upon a connection error
I put a connection handler class on a second thread, the server connection process slowed down the gui when it was all on one thread.
from root_gui import Root
import threading
from time import sleep
from data_handler import DataHandler
def handle_conn():
DataHandler.try_connect()
smtp_client.refresh() # Refreshes the gui according to the current status
def conn_manager(): # Working pretty well
while 'smtp_client' in globals():
sleep(0.6)
try:
handle_conn() # Calls the connection
except NameError: # If the user quits the tkinter gui
break
smtp_client = Root()
handle_conn()
MyConnManager = threading.Thread(target=conn_manager)
MyConnManager.start()
smtp_client.mainloop()
del smtp_client # The connection manager will detect this and stop running
My question is:
Is this a good practice or a terrible waste of resources? Is there a better way to do this because no matter what I tried, this was the only solution that worked.
From what I know the try_connect() function creates a completely new smtp object each time it is run ( which is once in 0.6 seconds! )
Resources/observations
The project on git: https://github.com/cernyd/smtp_client
Observation: the timeout parameter when creating the smtp object improved response times drastically, why is that so?

pcscd protocol changes but atr does not

I've been working on hid omnikey 5325 cl and trying to get the ATR value. I'm using pcscd and pyscard library on Ubuntu. System automatically gets the ATR using T0 protocol but I need RAW return. On windows using helloprox or omnikey's official software I can get what I need, but on ubuntu even though I've change the protocol but ATR does not change.
here is the code I use:
class PrintObserver(CardObserver):
"""A simple card observer that is notified
when cards are inserted/removed from the system and
prints the list of cards
"""
def update(self, observable, actions):
(addedcards, removedcards) = actions
for card in addedcards:
card.connection=card.createConnection()
card.connection.connect(protocol=CardConnection.RAW_protocol)
binary_atr=bin(int(toHexString(card.connection.getATR()).replace(" ",""), 16))
binary_atr=binary_atr[len(binary_atr)-24:len(binary_atr)-1]
print int(binary_atr,2)
for card in removedcards:
print "-Removed: ", toHexString(card.atr)
if __name__ == '__main__':
cardmonitor = CardMonitor()
cardobserver = PrintObserver()
cardmonitor.addObserver(cardobserver)
#Get Ports
config=ConfigParser.ConfigParser()
config.read('../config.ini')
Read_Port=int(config.get('CardReaderSocketPorts','Read',0))
Write_Port=int(config.get('CardReaderSocketPorts','Write',0))
# Initialise socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_address = ('localhost', Read_Port)
return_address = ('localhost', Write_Port)
sock.bind(server_address)
while 1:
message, address = sock.recvfrom(4096)
if message:
parsed_json=json.loads(message)
#Single Line
if parsed_json['set'].upper()=="Stop":
sock.sendto('{"success":"True"}',return_address)
break
# don't forget to remove observer, or the
# monitor will poll forever...
cardmonitor.deleteObserver(cardobserver)

Some considerations, which may be helpful, even if they don't add to a full answer:
You are working with contactless cards or tags
These tags have no ATR, but depending on the protocol something like ATQB or ATS, which is completely differently structured.
PCSC is a dated software concept, which strictly requires an ATR
So contactless reader manufacturers have to invent an ATR, based on the information they get from the card
What your software receives as "ATR" is the result of this invention.
Omnikey apparently uses the Windows registry to control the invention process, which is obviously a non-portable approach.
So my advice would be, to contact the Omnikey support, how to proceed from Linux. According to my experience they are cooperative.

After contacting the Omnikey support, the problem is solved.
As in developer guide I was looking for the config file "/etc/cmrfid.ini" to change the prox format. The file name was changed into "/etc/Omnikey.ini"

How I can identify a packet from the Ryu-controller to OF switch?

I need to identify a control packet from Python RYU-controller.
In other words: How I can to do the following instruction?
If (I receive a OFPT_PACKET_OUT msg from ryu-controller)
do something (for example all control traffic must mirroring to an output port)
and How can I match this rule?
I saw in OpenFlow v1.3 specification that there is a ofproto.OFPP_CONTROLLER reserved port: How can I use it as an ingress port?
From OFv1.3 spec.:
"OFPP_CONTROLLER: Represents the control channel with the OpenFlow controller. Can be used as an ingress port or as an output port.
When used as an output port, encapsulate the packet in a packet-in message and send it using the OpenFlow protocol.
When used as an ingress port, identify a packet originating from the controller."
Thanks for the help.

Regarding the first part of your question, let's see a basic Layer 2 Switch that simply floods the incoming packets to all output ports:
from ryu.base import app_manager
from ryu.controller import ofp_event
from ryu.controller.handler import MAIN_DISPATCHER
from ryu.controller.handler import set_ev_cls
class L2Switch(app_manager.RyuApp):
def __init__(self, *args, **kwargs):
super(L2Switch, self).__init__(*args, **kwargs)
#set_ev_cls(ofp_event.EventOFPPacketIn, MAIN_DISPATCHER)
def packet_in_handler(self, ev):
msg = ev.msg
dp = msg.datapath
ofp = dp.ofproto
ofp_parser = dp.ofproto_parser
actions = [ofp_parser.OFPActionOutput(ofp.OFPP_FLOOD)]
out = ofp_parser.OFPPacketOut(
datapath=dp, buffer_id=msg.buffer_id, in_port=msg.in_port,
actions=actions)
dp.send_msg(out)
The last two statements are
out = ofp_parser.OFPPacketOut(
datapath=dp, buffer_id=msg.buffer_id, in_port=msg.in_port,
actions=actions)
dp.send_msg(out)
These statements generate a packet_out message, however, I don't think there's a corresponding event that is raised for a packet_out message (Like a packet_in message generates the EventOFPPacketIn event which can be detected in code, and a method can be attached to it). I haven't used Ryu API much, but I think the reason is simple. A packet_out message is sent via the code itself, and you can simply add a few more lines of code after the lines generating this event. These few lines can execute whatever you want to do upon the generation of a packet_out message. For example, in the above code, you can just add the lines mirroring control traffic to a specific port after the dp.send_msg(out) line. Correct/edit my answer if I'm wrong.