Develop Reference

how to use INPUT_PULLUP using pyfirmata2

In Arduino IDE (C++ programming language) we can use pinMode(button, INPUT_PULLUP) to avoid using resistor, but here I'm using Arduino microcontroller and running StandardFirmata inside it, then I use python and pyfirmata library but I don't know how to call INPUT_PULLUP, and if I run the program the output will be true and false.
from pyfirmata2 import Arduino, util
import time
board = Arduino('COM6')
iterator = util.Iterator(board)
iterator.start()
button = board.get_pin('d:2:i')
while True:
print(button.read())
time.sleep(1)

I don't believe that pyfirmata supports pullup. You can try Telemetrix Telemetrix is similar to Firmata, but does not use 7bit bytes. You can view its API here. If you really need to use Firmata, then also look at pymata4.

How to send a Python dictionary consisting of ndarray and None using ZeroMQ PUB/SUB?

I'm trying to pass a python dictionary of 3 images (stored as ndarray) using ZeroMQ to pass it to another program as a consumer and parse back the data to original form. Followed three ways, but couldn't achieve success in anyone of the ways.
Below is the sample minimal reproduced code:
import pickle
import zmq
# Adding ZMQ Context
def zmq_context():
# Creating ZMQ context starts
context = zmq.Context()
footage_socket = context.socket(zmq.PUB)
footage_socket.connect('tcp://localhost:5002')
return footage_socket
wagon_dtl, ctr1_dtl, ctr2_dtl = NdArrays of images
socket_ctx = zmq_context()
# Trying two different ways of formatting the image before creating the dict, the below approach works for all three ndarrays
# 1st way
wagon_dtl = image_np_save # image_np_save is the original image
# 2nd way (this I tried because an ndarray isn't JSON serializable)
encoded, buffer = cv2.imencode('.jpg', image_np_save)
wagon_dtl = base64.b64encode(buffer)
if cond == "fulfilled":
full_wgn_dict = {"wagon_dtl": wagon_dtl, "ctr1_dtl": ctr1_dtl, "ctr2_dtl": ctr2_dtl}
# 1st way
dict_as_b64text = base64.b64encode(full_wgn_dict)
socket_ctx.send(dict_as_b64text)
# 2nd way
myjson = json.dumps(full_wgn_dict)
socket_ctx.send(myjson)
# 3rd way
dict_as_text = pickle.dumps(full_wgn_dict).encode('base64', 'strict')
socket_ctx.send(dict_as_text)
How to solve this?
I've followed these Q/As while working on this solution: 1, 2, 3, 4, 5

Q : "How to send a Python dictionary consisting of ndarray and None using ZeroMQ PUB/SUB?"
Easy, one may best use the ready-made .send_pyobj()-method for doing right this.
The sending side,the PUB shall be doing a call to the socket.send_pyobj( full_wgn_dict )-method, and that's basically all on this side.
A receiving side,each of the potential SUB-s shall reuse the .recv_pyobj()-method.
Yet all the SUB-s have to also do one more step, to actively subscribe to receive any message at all.
For details on socket.setsockopt( zmq.SUBSCRIBE, "" ) see the ZeroMQ documented API, or do not hesitate to sip from many examples here.
Some additional tricks (not needed for trivial dict-s) may help with the pickle-phase of the SER/DES stage. Yet these go way beyond of the scope of this Question, and may introduce advantages in controlled environments but problems in open, uncontrolled environments, where you lack zero chances to meet the required prerequisites - in my apps, I prefer to use import dill as pickle for having way higher robustness of the pickle.dumps()-SER/DES processing of objects and many more advances, like storing a full-session snapshot. Credits go to #MikeMcKearns
Feel free to re-read the documentation present for all syntax-related details in the __doc__ strings:
>>> print zmq.Socket.send_pyobj.__doc__
Send a Python object as a message using pickle to serialize.
Parameters
----------
obj : Python object
The Python object to send.
flags : int
Any valid flags for :func:`Socket.send`.
protocol : int
The pickle protocol number to use. The default is pickle.DEFAULT_PROTOCOL
where defined, and pickle.HIGHEST_PROTOCOL elsewhere.
>>> print zmq.Socket.recv_pyobj.__doc__
Receive a Python object as a message using pickle to serialize.
Parameters
----------
flags : int
Any valid flags for :func:`Socket.recv`.
Returns
-------
obj : Python object
The Python object that arrives as a message.
Raises
------
ZMQError
for any of the reasons :func:`~Socket.recv` might fail

Adafruit BLE python library can't list descriptors

I'm trying to use BLE library for python to communicate with one Nordic nrf51844 chipset. Because one characteristic is notification enabled, I need to enable notification from client side by setting the descriptor Client Characteristic Configuration to 0x0001. But I failed to get the descriptor with the call "characteristic.find_descriptor()" to get it. I also tried to print out all of descriptors discovered, but looks like there is no luck to get it work.
Below is the code I'm using to discover characteristics and its descriptor referred to the example of Adafruit BLE library:
def enable_notification(characteristic):
_enableDesc = characteristic.find_descriptor(CLIENT_CHARACTERISTIC_CONFIG_DESCRIPTOR_UUID)
_cmd = bytearray([0x01, 0x00])
_enableDesc.write_value(_cmd)
def handle_device_message(device):
global _status
# Once connected do everything else in a try/finally to make sure the device
# is disconnected when done.
# Wait for service discovery to complete for at least the specified
# service and characteristic UUID lists. Will time out after 60 seconds
# (specify timeout_sec parameter to override).
_logger.info('Discovering services...')
device.discover([HES_SERVICE_UUID], [DATA_CHAR_UUID, STATUS_CHAR_UUID, FACTOR_CHAR_UUID])
# Find the HES service and its characteristics.
hes = device.find_service(HES_SERVICE_UUID)
dataC = hes.find_characteristic(DATA_CHAR_UUID)
statusC = hes.find_characteristic(STATUS_CHAR_UUID)
#factorC = hes.find_characteristic(FACTOR_CHAR_UUID)
dataC.list_descriptors()
statusC.list_descriptors()
enable_notification(dataC)
enable_notification(statusC)
But it always failed at "characteristic.find_descriptor()" with below error:
_enableDesc =
characteristic.find_descriptor(CLIENT_CHARACTERISTIC_CONFIG_DESCRIPTOR_UUID)
File "build/bdist.macosx-10.11-x86_64/egg/Adafruit_BluefruitLE/interfaces/gatt.py", line 98, in find_descriptor
File "build/bdist.macosx-10.11-x86_64/egg/Adafruit_BluefruitLE/corebluetooth/gatt.py", line 124, in list_descriptors
File "build/bdist.macosx-10.11-x86_64/egg/Adafruit_BluefruitLE/corebluetooth/metadata.py", line 63, in get_all
TypeError: 'NoneType' object is not iterable
I looked into the source code of library, but can't find the interface to get descriptors explicitly. Can anyone help me on this?
Thanks!

Finally I figured out by checking the API of IOS to set notification. It should be set by calling setNotify for characteristic instead of writeValue for descriptor. And for the descriptor stuff, it shows we need to wait for some time before all descriptors are discovered and returned. Might be the issue implemented with Python. Not really verified with IOS native program.
BTW, after setting the notification, we need to wait for some time as well before the device sends notification to client.
Will get a Linux box to verify the implementation with blueZ is working well.

Heavily confused by win32api + COM and an answer from SO

From my other question here on SO, I asked how to retrieve the current playing song from Windows Media Player and Zune, I got an answer from a c++ dev who gave me an explanation of how I would do this for WMP.
However, I am no C++ dev, nor am I very experienced with the pywin32 library. And on-top of all that, the documentation on all this (especially concerning WMP) is horrible.
Therefor, I need your help understanding how I would do the following in Python.
Source
I have working code in C++ to print the name of media currently
playing in WMP. It's a simple console application (78 lines of code).
Steps:
1) implements a basic COM object implementing IUnknown, IOleClientSite, IServiceProvider and IWMPRemoteMediaServices. This is
straightforward (sort of, your mileage may vary) using the ATL
template CComObjectRootEx. The only methods needing (simple) code are
IServiceProvider::QueryService and
IWMPRemoteMediaServices::GetServiceType. All other methods may return
E_NOTIMPL
2) Instantiate the "WMPlayer.OCX" COM object (in my case, via CoCreateInstance)
3) Retrieve from the object an IOleObject interface pointer via QueryInterface
4) Instanciate an object from the class seen in 1) (I use the CComObject<>::CreateInstance template)
5) Use the SetClientSite method from the interface you got at 3), passing a pointer to your OleClientSite implementation.
6) During the SetClientSite call, WMP will callback you: fisrt asking for an IServiceProvider interface pointer, second calling the
QueryService method, asking for an IWMPRemoteMediaServices interface
pointer. Return your implementation of IWMPRemoteMediaServices and,
third, you will be called again via GetServiceType. You must then
return "Remote". You are now connected to the WMP running instance
7) Query the COM object for an IWMPMedia interface pointer
8) If 7) didn't gave NULL, read the the IWMPMedia::name property.
9) DONE
All the above was tested with VS2010 / Windows Seven, and with WMP
running (if there is no Media Player process running, just do
nothing).
I don't know if yoy can/want to implement COM interface and object in
Python. If you are interested by my C++ code, let me know. You could
use that code in a C++ DLL, and then call it from python.
I know a little bit about the win32api.
At the first step, I really don't know what to do, googling IOleClientSite results in the msdn documentation, it's an interface. However, that's where I get stuck already. I can't find anything (might just be my horrendous googling skills) on working with these things in Python.
The second step:
WMP = win32com.client.Dispatch("WMPlayer.OCX")
Alright, that's doable.
On to the third step. QueryInterface -
"regardless of the object you have, you can always call its QueryInterface() method to obtain a new interface, such as IStream."
source
However, not for me. As I understand his explanation, I think it means that every com object sort of "inherits" three methods from IUnknown, one of which is QueryInterface, however this does not seem the case since calling QueryInterface on my WMP object fails miserably. (Object has no attribute 'QueryInterface')
I could ramble on, but I believe you got the point, I have no idea how to work with this. Can anyone help me out with this one? Preferably with code examples, but resources/documentation is welcome too.

Almost final answser but CAN'T finish.
I seems that pythoncom can't be used to implement custom Interface without the help of a C++ module.
Here is an answser from Mark Hammon (Mon, 13 Jan 2003): How to create COM Servers with IID_IDTExtensibility2 interface
Sorry - you are SOL. To support arbitary interfaces, you need C++
support, in the form of an extension module. There is a new "Univgw"
that may help you out, but I dont know much about this
I am not able to find anything about that "Univgw" thing...
The comtypes python module is intended to resolve the problem, and I found links saying it does, but I can't make it works with my fresh new Python 3.3. It's Python 2.x code. comtypes seems outdated and unmaintained.
Step 1 OK for IOleClientSite and IServiceProvider, KO for IWMPRemoteMediaServices
Step 2, 3, 4 and 5 OK
Step 6, 7 and 8 can't be implemented without IWMPRemoteMediaServices :-(
disclaimer: complete newbie in Python, please don't yell
import pythoncom
import win32com.client as wc
from win32com.axcontrol import axcontrol
import win32com.server as ws
from win32com.server import util
from win32com.server.exception import COMException
import winerror
import pywintypes
# Windows Media Player Custom Interface IWMPRemoteMediaServices
IWMPRemoteMediaServices = pywintypes.IID("{CBB92747-741F-44FE-AB5B-F1A48F3B2A59}")
class OleClientSite:
_public_methods_ = [ 'SaveObject', 'GetMoniker', 'GetContainer', 'ShowObject', 'OnShowWindow', 'RequestNewObjectLayout', 'QueryService' ]
_com_interfaces_ = [ axcontrol.IID_IOleClientSite, pythoncom.IID_IServiceProvider ]
def SaveObject(self):
print("SaveObject")
raise COMException(hresult=winerror.E_NOTIMPL)
def GetMoniker(self, dwAssign, dwWhichMoniker):
print("GetMoniker ")
raise COMException(hresult=winerror.E_NOTIMPL)
def GetContainer(self):
print("GetContainer")
raise COMException(hresult=winerror.E_NOTIMPL)
def ShowObject(self):
print("ShowObject")
raise COMException(hresult=winerror.E_NOTIMPL)
def OnShowWindow(self, fShow):
print("ShowObject" + str(fShow))
raise COMException(hresult=winerror.E_NOTIMPL)
def RequestNewObjectLayout(self):
print("RequestNewObjectLayout")
raise COMException(hresult=winerror.E_NOTIMPL)
def QueryService(self, guidService, riid):
print("QueryService",guidService,riid)
if riid == IWMPRemoteMediaServices:
print("Known Requested IID, but can't implement!")
raise COMException(hresult=winerror.E_NOINTERFACE)
else:
print("Requested IID is not IWMPRemoteMediaServices" )
raise COMException(hresult=winerror.E_NOINTERFACE)
if __name__=='__main__':
wmp = wc.Dispatch("WMPlayer.OCX")
IOO = wmp._oleobj_.QueryInterface(axcontrol.IID_IOleObject)
pyOCS = OleClientSite()
comOCS = ws.util.wrap(pyOCS, axcontrol.IID_IOleClientSite)
IOO.SetClientSite(comOCS)

How do you call Python code from C code?

I want to extend a large C project with some new functionality, but I really want to write it in Python. Basically, I want to call Python code from C code. However, Python->C wrappers like SWIG allow for the OPPOSITE, that is writing C modules and calling C from Python.
I'm considering an approach involving IPC or RPC (I don't mind having multiple processes); that is, having my pure-Python component run in a separate process (on the same machine) and having my C project communicate with it by writing/reading from a socket (or unix pipe). my python component can read/write to socket to communicate. Is that a reasonable approach? Is there something better? Like some special RPC mechanism?
Thanks for the answer so far - however, i'd like to focus on IPC-based approaches since I want to have my Python program in a separate process as my C program. I don't want to embed a Python interpreter. Thanks!

I recommend the approaches detailed here. It starts by explaining how to execute strings of Python code, then from there details how to set up a Python environment to interact with your C program, call Python functions from your C code, manipulate Python objects from your C code, etc.
EDIT: If you really want to go the route of IPC, then you'll want to use the struct module or better yet, protlib. Most communication between a Python and C process revolves around passing structs back and forth, either over a socket or through shared memory.
I recommend creating a Command struct with fields and codes to represent commands and their arguments. I can't give much more specific advice without knowing more about what you want to accomplish, but in general I recommend the protlib library, since it's what I use to communicate between C and Python programs (disclaimer: I am the author of protlib).

Have you considered just wrapping your python application in a shell script and invoking it from within your C application?
Not the most elegant solution, but it is very simple.

See the relevant chapter in the manual: http://docs.python.org/extending/
Essentially you'll have to embed the python interpreter into your program.

I haven't used an IPC approach for Python<->C communication but it should work pretty well. I would have the C program do a standard fork-exec and use redirected stdin and stdout in the child process for the communication. A nice text-based communication will make it very easy to develop and test the Python program.

If I had decided to go with IPC, I'd probably splurge with XML-RPC -- cross-platform, lets you easily put the Python server project on a different node later if you want, has many excellent implementations (see here for many, including C and Python ones, and here for the simple XML-RPC server that's part the Python standard library -- not as highly scalable as other approaches but probably fine and convenient for your use case).
It may not be a perfect IPC approach for all cases (or even a perfect RPC one, by all means!), but the convenience, flexibility, robustness, and broad range of implementations outweigh a lot of minor defects, in my opinion.

This seems quite nice http://thrift.apache.org/, there is even a book about it.
Details:
The Apache Thrift software framework, for scalable cross-language
services development, combines a software stack with a code generation
engine to build services that work efficiently and seamlessly between
C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa,
JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.

I've used the "standard" approach of Embedding Python in Another Application. But it's complicated/tedious. Each new function in Python is painful to implement.
I saw an example of Calling PyPy from C. It uses CFFI to simplify the interface but it requires PyPy, not Python. Read and understand this example first, at least at a high level.
I modified the C/PyPy example to work with Python. Here's how to call Python from C using CFFI.
My example is more complicated because I implemented three functions in Python instead of one. I wanted to cover additional aspects of passing data back and forth.
The complicated part is now isolated to passing the address of api to Python. That only has to be implemented once. After that it's easy to add new functions in Python.
interface.h
// These are the three functions that I implemented in Python.
// Any additional function would be added here.
struct API {
double (*add_numbers)(double x, double y);
char* (*dump_buffer)(char *buffer, int buffer_size);
int (*release_object)(char *obj);
};
test_cffi.c
//
// Calling Python from C.
// Based on Calling PyPy from C:
// http://doc.pypy.org/en/latest/embedding.html#more-complete-example
//
#include <stdio.h>
#include <assert.h>
#include "Python.h"
#include "interface.h"
struct API api; /* global var */
int main(int argc, char *argv[])
{
int rc;
// Start Python interpreter and initialize "api" in interface.py using
// old style "Embedding Python in Another Application":
// https://docs.python.org/2/extending/embedding.html#embedding-python-in-another-application
PyObject *pName, *pModule, *py_results;
PyObject *fill_api;
#define PYVERIFY(exp) if ((exp) == 0) { fprintf(stderr, "%s[%d]: ", __FILE__, __LINE__); PyErr_Print(); exit(1); }
Py_SetProgramName(argv[0]); /* optional but recommended */
Py_Initialize();
PyRun_SimpleString(
"import sys;"
"sys.path.insert(0, '.')" );
PYVERIFY( pName = PyString_FromString("interface") )
PYVERIFY( pModule = PyImport_Import(pName) )
Py_DECREF(pName);
PYVERIFY( fill_api = PyObject_GetAttrString(pModule, "fill_api") )
// "k" = [unsigned long],
// see https://docs.python.org/2/c-api/arg.html#c.Py_BuildValue
PYVERIFY( py_results = PyObject_CallFunction(fill_api, "k", &api) )
assert(py_results == Py_None);
// Call Python function from C using cffi.
printf("sum: %f\n", api.add_numbers(12.3, 45.6));
// More complex example.
char buffer[20];
char * result = api.dump_buffer(buffer, sizeof buffer);
assert(result != 0);
printf("buffer: %s\n", result);
// Let Python perform garbage collection on result now.
rc = api.release_object(result);
assert(rc == 0);
// Close Python interpreter.
Py_Finalize();
return 0;
}
interface.py
import cffi
import sys
import traceback
ffi = cffi.FFI()
ffi.cdef(file('interface.h').read())
# Hold references to objects to prevent garbage collection.
noGCDict = {}
# Add two numbers.
# This function was copied from the PyPy example.
#ffi.callback("double (double, double)")
def add_numbers(x, y):
return x + y
# Convert input buffer to repr(buffer).
#ffi.callback("char *(char*, int)")
def dump_buffer(buffer, buffer_len):
try:
# First attempt to access data in buffer.
# Using the ffi/lib objects:
# http://cffi.readthedocs.org/en/latest/using.html#using-the-ffi-lib-objects
# One char at time, Looks inefficient.
#data = ''.join([buffer[i] for i in xrange(buffer_len)])
# Second attempt.
# FFI Interface:
# http://cffi.readthedocs.org/en/latest/using.html#ffi-interface
# Works but doc says "str() gives inconsistent results".
#data = str( ffi.buffer(buffer, buffer_len) )
# Convert C buffer to Python str.
# Doc says [:] is recommended instead of str().
data = ffi.buffer(buffer, buffer_len)[:]
# The goal is to return repr(data)
# but it has to be converted to a C buffer.
result = ffi.new('char []', repr(data))
# Save reference to data so it's not freed until released by C program.
noGCDict[ffi.addressof(result)] = result
return result
except:
print >>sys.stderr, traceback.format_exc()
return ffi.NULL
# Release object so that Python can reclaim the memory.
#ffi.callback("int (char*)")
def release_object(ptr):
try:
del noGCDict[ptr]
return 0
except:
print >>sys.stderr, traceback.format_exc()
return 1
def fill_api(ptr):
global api
api = ffi.cast("struct API*", ptr)
api.add_numbers = add_numbers
api.dump_buffer = dump_buffer
api.release_object = release_object
Compile:
gcc -o test_cffi test_cffi.c -I/home/jmudd/pgsql-native/Python-2.7.10.install/include/python2.7 -L/home/jmudd/pgsql-native/Python-2.7.10.install/lib -lpython2.7
Execute:
$ test_cffi
sum: 57.900000
buffer: 'T\x9e\x04\x08\xa8\x93\xff\xbf]\x86\x04\x08\x00\x00\x00\x00\x00\x00\x00\x00'
$

Few tips for binding it with Python 3
file() not supported, use open()
ffi.cdef(open('interface.h').read())
PyObject* PyStr_FromString(const char *u)
Create a PyStr from a UTF-8 encoded null-terminated character buffer.
Python 2: PyString_FromString
Python 3: PyUnicode_FromString
Change to: PYVERIFY( pName = PyUnicode_FromString("interface") )
Program name
wchar_t *name = Py_DecodeLocale(argv[0], NULL);
Py_SetProgramName(name);
for compiling
gcc cc.c -o cc -I/usr/include/python3.6m -I/usr/include/x86_64-linux-gnu/python3.6m -lpython3.6m
I butchered dump def .. maybe it will give some ideas
def get_prediction(buffer, buffer_len):
try:
data = ffi.buffer(buffer, buffer_len)[:]
result = ffi.new('char []', data)
print('\n I am doing something here here........',data )
resultA = ffi.new('char []', b"Failed") ### New message
##noGCDict[ffi.addressof(resultA)] = resultA
return resultA
except:
print >>sys.stderr, traceback.format_exc()
return ffi.NULL
}
Hopefully it will help and save you some time

apparently Python need to be able to compile to win32 dll, it will solve the problem
In such a way that converting c# code to win32 dlls will make it usable by any development tool