Develop Reference

Creating a new timer_t object after deleting a previous one doesn't work

I'll try to keep it as short as possible.
I'm making a python application which uses a C Extension (hw_timer).
This extension is in charge of creating a timer.
The Python application starts by calling hw_timer.StartTimer which instantiates a timer object.
After that, inside an infinite loop, the Python keeps calling hw_timer.ResetTimer and hw_timer.StartTimer. Reset timer de facto destroys the timer and then a new one is created with StartTimer and so on.
The code went through several changes since, no matter what, I kept getting segmentation faults whenever I tried to create a new timer.
However the situation now is VERY strange.
Here's the Python code: you'll notice I've removed the infinite loop I described before because I wanted to create a very simple scenario to showcase my issue
if __name__ == "__main__":
timerPtr = hw_timer.StartTimer(os.getpid()) #start timer returns a tuple
print("timer,tid,pid -> {}".format(timerPtr))
print("Result of reset timer: {}".format(hw_timer.ResetTimer(timerPtr[0])))
print("---------")
time.sleep(10)
timerPtr=hw_timer.StartTimer(os.getpid())
print("timer,tid,pid -> {}".format(timerPtr))
print("Result of reset timer: {}".format(hw_timer.ResetTimer(timerPtr[0])))
print("---------")
time.sleep(10)
And this is the output
timer,tid,pid -> (16985424, 16598768, 45975)
Result of reset timer: (16985424, 16598768, 45975)
---------
timer,tid,pid -> (16598768, 15553760, 45975)
timer_delete: Invalid argument
As you can see, despite replicating the same code twice, the second time, system method time_delete fails during the computation of ResetTimer.
Now the following is the C code (I've removed as much Python.h related content as possible to make it more readable)
#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <time.h>
struct tmr_struct
{
timer_t* tid;
int pid;
};
typedef struct sigevent signal_event;
typedef struct tmr_struct tmr_data;
typedef struct itimerspec tmr_spec;
static const int TIMER_SECOND = 5;
//Method called after timer expiration
void signal_handler (int sv)
{
exit(1);
}
//Creation of a new timer_t object stored inside a tmr_data struct
static void _StartTimer(tmr_data* timer)
{
timer->tid = new timer_t();
signal_event sev;
signal(SIGUSR1, signal_handler);
sev.sigev_notify = SIGEV_THREAD_ID;
sev.sigev_notify_function = NULL;
sev._sigev_un._tid = timer->pid;
sev.sigev_signo = SIGUSR1;
sev.sigev_notify_attributes = NULL;
tmr_spec ts;
if (timer_create(CLOCK_REALTIME, &sev, timer->tid) == -1) {
perror("timer_create");
exit(1);
}
memset(&ts,0,sizeof(ts));
ts.it_value.tv_nsec = 0;
ts.it_value.tv_sec = TIMER_SECOND;
ts.it_interval.tv_nsec = 0;
ts.it_interval.tv_sec = 0;
if (timer_settime(*(timer->tid), 0, &ts, NULL) == -1) {
perror("timer_settime");
exit(1);
}
return;
}
//Start timer method called from the Python application
//Accepts the PID of the Python App and later passes it to _StartTimer using the tmr_data struct
static PyObject* StartTimer(PyObject* self, PyObject* args) {
int pid;
tmr_data* timer = new tmr_data();
if (!PyArg_ParseTuple(args, "i", &pid)) {
return NULL;
}
timer->pid = pid;
_StartTimer(timer);
return Py_BuildValue("iii", timer,timer->tid,timer->pid);
}
//Receives a pointer to a tmr_data struct object, deletes the timer contained inside it
static PyObject* ResetTimer(PyObject* self, PyObject* args)
{
tmr_data* timer;
long ptr_timer;
if (!PyArg_ParseTuple(args, "i", &ptr_timer)) {
return NULL;
}
timer = (tmr_data*)ptr_timer;
if(timer_delete(timer->tid) != 0)
{
perror("timer_delete");
exit(1);
}
return Py_BuildValue("iii", timer,timer->tid,timer->pid);
}
Now consider that the entire code of _StartTimer is correct: it's already been used in other parts of the project for pure C applications and, indeed, the timer does work here as well, at least the first time.
But still, this stuff is all over the place: originally I had ResetTimer calling _StartTimer, but whenever I'd call "timer = new tmr_data()" I would get segmentation fault, so I'm starting to think that the entire timer implementation might be somewhat tricky or prone to errors.

How to print the file path from an open syscall using ebpf python?

I use bpf from the python bcc module, and I want that my probe function will print the file path of the current file (kind of a custom simplified opensnoop).
How can I do that?
This is what I have so far:
b = BPF(text="""
#include <linux/ptrace.h>
#include<linux/sched.h>
BPF_HASH(last);
int trace_entry(struct pt_regs *ctx)
{
char fileName[200] = {0};
bpf_probe_read(fileName, sizeof(fileName), &PT_REGS_PARM1(ctx));
bpf_trace_printk("File Opened<%s>\\n", fileName);
return 0;
}
""")
print("Tracing for open... Ctrl-C to end")
b.attach_kprobe(event="do_sys_open", fn_name="trace_entry")
#b.attach_kprobe(event=b.get_syscall_fnname("open"), fn_name='funcky')
b.trace_print()

Easy:
Insert this in kernel code:
// Nicer way to call bpf_trace_printk()
#define bpf_custom_printk(fmt, ...) \
({ \
char ____fmt[] = fmt; \
bpf_trace_printk(____fmt, sizeof(____fmt), \
##__VA_ARGS__); \
})
struct pair {
uint32_t lip; // local IP
uint32_t rip; // remote IP
};
and in kernel code insert print out code. This function call works exactly like printf with all format and placeholder ...
bpf_custom_printk("This year is %d\n", 2020);
Print output:
sudo cat /sys/kernel/debug/tracing/trace_pipe

Handler Issues Applying cdll in Python

I'm trying to porting some C dll(FANUC FOCAS Library - for CNC) code via Python using ctypes, so I wrote porting code. (as belows), but get a very strange result when loading the DLL and calling the function. In my case, I don't understand using handler in python.
I would like to apply the following c code in python.
Declaration(for c)
#include "fwlib64.h"
FWLIBAPI short WINAPI cnc_allclibhndl3(const char *ipaddr,unsigned short port,
long timeout, unsigned short *FlibHndl);
Example Code(in focas library manual for c)
#include "fwlib64.h"
void example( void )
{
unsigned short h;
short ret;
ODBST buf;
ret = cnc_allclibhndl3( "192.168.0.100", 8193, 1, &h ) ;
//
if ( !ret ) {
cnc_statinfo( h, &buf ) ;
cnc_freelibhndl( h ) ;
} else {
printf( "ERROR!(%d)\n", ret ) ;
}
}
Testfocas.py
from ctypes import *
mylib = cdll.LoadLibrary('./Fwlib64.dll')
class ODBSYS(Structure):
pass
_fields_ =[
("dummy", c_ushort),
("max_axis", c_char*2),
("cnc_type", c_char*2),
("mt_type",c_char*2),
("series",c_char*4),
("version",c_char*4),
("axes",c_char*2),]
h=c_ushort()
pt=pointer(h)
ret=c_short()
buf=ODBSYS()
ret=mylib.cnc_allclibhndl3('192.168.0.100',8193,1,pt)
mylib.cnc_statinfo(h,buf)
mylib.cnc_freelibhndl(h)
I want the function to return 0 or -16 but, in my case the function return is
cnc_allclibhndl3 = 65520 (i guess open port)
cnc_statinfo = -8
cnc_freelibhndl -8
Return Status of Data Window Functions
EW_OK(0) Normal termination
EW_SOCKET(-16) Socket communication error Check the power supply of CNC, Ethernet I/F board, Ethernet connection cable.
EW_HANDLE(-8) Allocation of handle number is failed.
I do not know what I was wrong with.

CDLL is for __cdecl calling convention. cdll is not recommended for use because it is a shared instance across modules.
WINAPI is defined as __stdcall, so use WinDLL:
mylib = WinDLL.LoadLibrary('./Fwlib64.dll')
Next, define argtypes and restype for your argument and result types for your function:
mylib.cnc_allclibhndl3.argtypes = c_char_p,c_ushort,c_long,POINTER(c_ushort)
mylib.cnc_allclibhndl3.restype = c_short
Finally, pass the output parameter by reference. It is more efficient than creating a pointer:
h = c_ushort()
ret = mylib.cnc_allclibhndl3('192.168.0.100',8193,1,byref(h))
Prototypes for cnc_statinfo and cnc_freelibhndl were not provided. Define argtypes and restype for them as well.

How do yield works in Python C code, good & bad part

Recently I've been looking into the code of Python. I know how to use generators (next, send and etc..), but it's fun to understand it by reading the Python C code.
I found the code in Object/genobject.c, and it's not that hard (but still not easy) to understand. So I want to know how it really works, and make sure I do not have a misunderstanding about generators in Python.
I know everything calls
static PyObject *
gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
and the result is returned from PyEval_EvalFrameEx which looks like it's a dynamic frame struct, could I understand it as stack or something?
Ok, It looks like Python stores some context in memory (am I right?). It looks like every time we use yield it creates a generator, and stores the context in memory, although not all of the functions and vars.
I know if I have big loop or big data to parse, yield is amazing, it saves a lot of memory and make it simple. But some of my workmates like to use yield everywhere, just like return. It's not easy to read and understand the code, and Python stores context for most of the function that may never be called again. Is it a bad practice?
So, the questions are:
How does PyEval_EvalFrameEx work.
Memory use of yield.
Is it bad practice for using yield everywhere.
And I found if I have a generator, function gen_send_ex will be called twice, why?
def test():
while 1:
yield 'test here'
test().next()
It will call gen_send_ex twice, first time with no args, with args the second time, and get the result.
Thanks for your patience.

I saw these articles:
This article tell me how does PyEval_EvalFrameEx work.
http://tech.blog.aknin.name/2010/09/02/pythons-innards-hello-ceval-c-2/
This article tell me the frame struct in Python.
http://tech.blog.aknin.name/2010/07/22/pythons-innards-interpreter-stacks/
These two stuff are very important for us.
So let me answer my question myself. I don't know if am I right.
If I have misunderstanding or completely wrong, Please let me know.
If I have code:
def gen():
count = 0
while count < 10:
count += 1
print 'call here'
yield count
That is a very simple generator.
f = gen()
And every time we call it, Python create a generator object.
PyObject *
PyGen_New(PyFrameObject *f)
{
PyGenObject *gen = PyObject_GC_New(PyGenObject, &PyGen_Type);
if (gen == NULL) {
Py_DECREF(f);
return NULL;
}
gen->gi_frame = f;
Py_INCREF(f->f_code);
gen->gi_code = (PyObject *)(f->f_code);
gen->gi_running = 0;
gen->gi_weakreflist = NULL;
_PyObject_GC_TRACK(gen);
return (PyObject *)gen;
}
We could see it init a generator object. And Init a Frame.
Anything we do like f.send() or f.next(), It will call gen_send_ex, and the code below:
static PyObject *
gen_iternext(PyGenObject *gen)
{
return gen_send_ex(gen, NULL, 0);
}
static PyObject *
gen_send(PyGenObject *gen, PyObject *arg)
{
return gen_send_ex(gen, arg, 0);
}
Only difference between two function is arg, send is send an arg, next send NULL.
gen_send_ex code below:
static PyObject *
gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
{
PyThreadState *tstate = PyThreadState_GET();
PyFrameObject *f = gen->gi_frame;
PyObject *result;
if (gen->gi_running) {
fprintf(stderr, "gi init\n");
PyErr_SetString(PyExc_ValueError,
"generator already executing");
return NULL;
}
if (f==NULL || f->f_stacktop == NULL) {
fprintf(stderr, "check stack\n");
/* Only set exception if called from send() */
if (arg && !exc)
PyErr_SetNone(PyExc_StopIteration);
return NULL;
}
if (f->f_lasti == -1) {
fprintf(stderr, "f->f_lasti\n");
if (arg && arg != Py_None) {
fprintf(stderr, "something here\n");
PyErr_SetString(PyExc_TypeError,
"can't send non-None value to a "
"just-started generator");
return NULL;
}
} else {
/* Push arg onto the frame's value stack */
fprintf(stderr, "frame\n");
if(arg) {
/* fprintf arg */
}
result = arg ? arg : Py_None;
Py_INCREF(result);
*(f->f_stacktop++) = result;
}
fprintf(stderr, "here\n");
/* Generators always return to their most recent caller, not
* necessarily their creator. */
Py_XINCREF(tstate->frame);
assert(f->f_back == NULL);
f->f_back = tstate->frame;
gen->gi_running = 1;
result = PyEval_EvalFrameEx(f, exc);
gen->gi_running = 0;
/* Don't keep the reference to f_back any longer than necessary. It
* may keep a chain of frames alive or it could create a reference
* cycle. */
assert(f->f_back == tstate->frame);
Py_CLEAR(f->f_back);
/* If the generator just returned (as opposed to yielding), signal
* that the generator is exhausted. */
if (result == Py_None && f->f_stacktop == NULL) {
fprintf(stderr, "here2\n");
Py_DECREF(result);
result = NULL;
/* Set exception if not called by gen_iternext() */
if (arg)
PyErr_SetNone(PyExc_StopIteration);
}
if (!result || f->f_stacktop == NULL) {
fprintf(stderr, "here3\n");
/* generator can't be rerun, so release the frame */
Py_DECREF(f);
gen->gi_frame = NULL;
}
fprintf(stderr, "return result\n");
return result;
}
Looks like Generator Object is a controller of it's own Frame which called gi_frame.
I add some fprintf (...), so let's run code.
f.next()
f->f_lasti
here
call here
return result
1
So, first it goes to f_lasti(This is a integer offset into the byte code of the last instructions executed, initialized to -1), and yes it's -1, but with no args, then function goes on.
Then goto here, the most important thing now is PyEval_EvalFrameEx. PyEval_EvalFrameEx implements CPython’s evaluation loop, we could thing it runs every code (in fact is Python opcode), and run the line print 'call here', it print text.
When code goes to yield, Python stores context by using frame object (we could search Call Stack). Give value back and give up control of code.
After everything done, then return result, and showing value 1 in terminal.
Next time we run next(), it will not go to f_lasti scope. It shows:
frame
here
call here
return result
2
We did not send arg so still get result from PyEval_EvalFrameEx and result is 2.

C to Python via SWIG: can't get void** parameters to hold their value

I have a C interface that looks like this (simplified):
extern bool Operation(void ** ppData);
extern float GetFieldValue(void* pData);
extern void Cleanup(p);
which is used as follows:
void * p = NULL;
float theAnswer = 0.0f;
if (Operation(&p))
{
theAnswer = GetFieldValue(p);
Cleanup(p);
}
You'll note that Operation() allocates the buffer p, that GetFieldValue queries p, and that Cleanup frees p. I don't have any control over the C interface -- that code is widely used elsewhere.
I'd like to call this code from Python via SWIG, but I was unable to find any good examples of how to pass a pointer to a pointer -- and retrieve its value.
I think the correct way to do this is by use of typemaps, so I defined an interface that would automatically dereference p for me on the C side:
%typemap(in) void** {
$1 = (void**)&($input);
}
However, I was unable to get the following python code to work:
import test
p = None
theAnswer = 0.0f
if test.Operation(p):
theAnswer = test.GetFieldValue(p)
test.Cleanup(p)
After calling test.Operation(), p always kept its initial value of None.
Any help with figuring out the correct way to do this in SWIG would be much appreciated. Otherwise, I'm likely to just write a C++ wrapper around the C code that stops Python from having to deal with the pointer. And then wrap that wrapper with SWIG. Somebody stop me!
Edit:
Thanks to Jorenko, I now have the following SWIG interface:
% module Test
%typemap (in,numinputs=0) void** (void *temp)
{
$1 = &temp;
}
%typemap (argout) void**
{
PyObject *obj = PyCObject_FromVoidPtr(*$1, Cleanup);
$result = PyTuple_Pack(2, $result, obj);
}
%{
extern bool Operation(void ** ppData);
extern float GetFieldValue(void *p);
extern void Cleanup(void *p);
%}
%inline
%{
float gfv(void *p){ return GetFieldValue(p);}
%}
%typemap (in) void*
{
if (PyCObject_Check($input))
{
$1 = PyCObject_AsVoidPtr($input);
}
}
The python code that uses this SWIG interface is as follows:
import test
success, p = test.Operation()
if success:
f = test.GetFieldValue(p) # This doesn't work
f = test.gvp(p) # This works!
test.Cleanup(p)
Oddly, in the python code, test.GetFieldValue(p) returns gibberish, but test.gfv(p) returns the correct value. I've inserting debugging code into the typemap for void*, and both have the same value of p! The call Any ideas about that?
Update: I've decided to use ctypes. MUCH easier.

I agree with theller, you should use ctypes instead. It's always easier than thinking about typemaps.
But, if you're dead set on using swig, what you need to do is make a typemap for void** that RETURNS the newly allocated void*:
%typemap (in,numinputs=0) void** (void *temp)
{
$1 = &temp;
}
%typemap (argout) void**
{
PyObject *obj = PyCObject_FromVoidPtr(*$1);
$result = PyTuple_Pack(2, $result, obj);
}
Then your python looks like:
import test
success, p = test.Operation()
theAnswer = 0.0f
if success:
theAnswer = test.GetFieldValue(p)
test.Cleanup(p)
Edit:
I'd expect swig to handle a simple by-value void* arg gracefully on its own, but just in case, here's swig code to wrap the void* for GetFieldValue() and Cleanup():
%typemap (in) void*
{
$1 = PyCObject_AsVoidPtr($input);
}

Would you be willing to use ctypes? Here is sample code that should work (although it is untested):
from ctypes import *
test = cdll("mydll")
test.Operation.restype = c_bool
test.Operation.argtypes = [POINTER(c_void_p)]
test.GetFieldValue.restype = c_float
test.GetFieldValue.argtypes = [c_void_p]
test.Cleanup.restype = None
test.Cleanup.argtypes = [c_void_p]
if __name__ == "__main__":
p = c_void_p()
if test.Operation(byref(p)):
theAnswer = test.GetFieldValue(p)
test.Cleanup(p)