Develop Reference

Receiving data in python callback function from dll

I am writing a program in Python that communicates with a spectrometer from Avantes. There are some proprietary dlls available whose code I don't access to, but they have some decent documentation. I am having some trouble to find a good way to store the data received via callbacks.
The proprietary shared library
Basically, the dll contains a function that I have to call to start measuring and that receives a callback function that will be called whenever the spectrometer has finished a measurement. The function is the following:
int AVS_MeasureCallback(AvsHandle a_hDevice,void (*__Done)(AvsHandle*, int*),short a_Nmsr)
The first argument is a handle object that identifies the spectrometer, the second is the actual callback function and the third is the amount of measurements to be made.
The callback function will receive then receive another type of handle identifying the spetrometer and information about the amount of data available after a measurement.
Python library
I am using a library that has Python wrappers for many equipments, including my spectrometer.
def measure_callback(self, num_measurements, callback=None):
self.sdk.AVS_MeasureCallback(self._handle, callback, num_measurements)
And they also have defined the following decorator:
MeasureCallback = FUNCTYPE(None, POINTER(c_int32), POINTER(c_int32))
The idea is that when the callback function is finally called, this will trigger the get_data() function that will retrieve data from the equipment.
The recommended example is
#MeasureCallback
def callback_fcn(handle, info):
print('The DLL handle is:', handle.contents.value)
if info.contents.value == 0: # equals 0 if everything is okay (see manual)
print(' callback data:', ava.get_data())
ava.measure_callback(-1, callback_fcn)
My problem
I have to store the received data in a 2D numpy array that I have created somewhere else in my main code, but I can't figure out what is the best way to update this array with the new data available inside the callback function.
I wondered if I could pass this numpy array as an argument for the callback function, but even in this case I cannot find a good way to do this since it is expected that the callback function will have only those 2 arguments.
Edit 1
I found a possible solution here but I am not sure it is the best way to do it. I'd rather not create a new class just to hold a single numpy array inside.
Edit 2
I actually changed my mind about my approach, because inside my callback I'd like to do many operations with the received data and save the results in many different variables. So, I went back to the class approach mentioned here, where I would basically have a class with all the variables that will somehow be used in the callback function and that would also inherit or have an object of the class ava.
However, as shown in this other question, the self parameter is a problem in this case.

If you don't want to create a new class, you can use a function closure:
# Initialize it however you want
numpy_array = ...
def callback_fcn(handle, info):
# Do what you want with the value of the variable
store_data(numpy_array, ...)
# After the callback is called, you can access the changes made to the object
print(get_data(numpy_array))
How this works is that when the callback_fcn is defined, it keeps a reference to the value of the variable numpy_array, so when it's called, it can manipulate it, as if it were passed as an argument to the function. So you get the effect of passing it in, without the callback caller having to worry about it.

I finally managed to solve my problem with a solution envolving a new class and also a closure function to deal with the self parameter that is described here. Besides that, another problem would appear by garbage collection of the new created method.
My final solution is:
class spectrometer():
def measurement_callback(self,handle,info):
if info.contents.value >= 0:
timestamp,spectrum = self.ava.get_data()
self.spectral_data[self.spectrum_index,:] = np.ctypeslib.as_array(spectrum[0:pixel_amount])
self.timestamps[self.spectrum_index] = timestamp
self.spectrum_index += 1
def __init__(self,ava):
self.ava = ava
self.measurement_callback = MeasureCallback(self.measurement_callback)
def register_callback(self,scans,pattern_amount,pixel_amount):
self.spectrum_index = 0
self.timestamps = np.empty((pattern_amount),dtype=np.uint32)
self.spectral_data = np.empty((pattern_amount,pixel_amount),dtype=np.float64)
self.ava.measure_callback(scans, self.measurement_callback)

Scopes and Customizing Open (learning Python p. 539)

I'm a bit confused about a code in the book "Learning Python", p. 539.
As far as I know assignments within a function are only in this local scope. So if I want to change a global one I first have to declare it global. But why does the following code change the builtin.open() to custom completely once called?
import builtins
def makeopen(id):
original = builtins.open
def custom(*pargs, **kargs):
print('Custom open call %r: ' % id, pargs, kargs)
return original(*pargs, **kargs)
builtins.open = custom
If I call makeopen('spam') and a F = open('text.txt') afterwards I get the custom call. So the builtin.open() has been changed in the whole script after the makeopen('spam'). Why?
And if I would make some more makeopen('xx') one builtin.open('text.txt') would print the custom call for every created makeopen. Why?
Comparing this code to
x = 99
def changing():
x = 88
changing()
print(x)
doesnt even help me. Isn't it the same but with an x instead of builtin.open()?

A variable is considered local if you assign to it anywhere in the function, unless you declare it global.
In your first piece of code, you never assign anything to builtins, so it's not considered local. You just change one of its attributes, open.
The rule is respected!
In your second piece of code, you assign something to x in x = 88, so it is considered local.
When you call makeopen, you replace the original, global open with custom. custom, when executed, prints its name and calls the original open.
If you call makeopen a second time, it will create a second, different custom function, and make the name builtins.open refer to it. When you call this function, it will print its name, then call original, which is what builtins.open referred to when it was created - and that is your first custom function, which will print its name and call the original open.
So, successive calls to makeopen create a chain of functions, and calling open will make each of them run and call its predecessor.

Python use input to select local variable inside another function

All, I have this request but first I will explain what I'm trying to achieve. I coded a python script with many global variables but also many methods defined inside different modules (.py files).
The script sometimes moves to a method and inside this method I call another method defined in another module. The script is quite complex.
Most of my code is inside Try/Except so that every time an exception is triggered my code runs a method called "check_issue()" in which I print to console the traceback and then I ask myself if there's any variable's value I want to double check. Now, I read many stackoverflow useful pages in which users show how to use/select globals(), locals() and eval() to see current global variables and local variables.
What I would specifically need though is the ability to input inside method "check_issue()" the name of a variable that may be defined not as global and not inside the method check_issue() either.
Using classes is not a solution since I would need to change hundreds of lines of code.
These are the links I already read:
Viewing all defined variables
Calling variable defined inside one function from another function
How to get value of variable entered from user input?
This is a sample code that doesn't work:
a = 4
b = "apple"
def func_a():
c = "orange"
...
check_issue()
def check_issue():
print("Something went wrong")
var_to_review = input("Input name of var you want to review")
# I need to be able to enter "c" and print the its value "orange"
print(func_a.locals()[var_to_review ]) # this doesn't work
Could somebody suggest how to fix it?
Many thanks

When you call locals() inside check_issue(), you can only access to the locals of this function, which would be : ['var_to_review'].
You can add a parameter to the check_issue function and pass locals whenever you call it.
a = 4
b = "apple"
def func_a():
c = "orange"
check_issue(locals())
def check_issue(local_vars):
print("Something went wrong")
var_to_review = input("Input name of var you want to review")
print(local_vars[var_to_review])

Python Pass function parameter as list identifier

I've created a function which can take a parameter which defines another call to manipulate a list. For example if I call sliprotor(Rotorid1, 1) directly, then the Rotorid1 list is manipulated as I want. Function below:
def sliprotor(rotorid,offset_qty):
for movers in range(26,0,-1):
rotorid[movers-1+offset_qty]=rotorid[movers-1]
for movers_refill in range(offset_qty):
rotorid[movers_refill]=rotorid[movers_refill+26]
However, if I try to call this 'indirectly' by building the list name and then executing it, 'rotorid' is not translated to the value, as it is when called directly.
The way I am doing this is
def set_curr_rotor(XX):
rotorid = "Rotorid"+str(XX)
return rotorid
rid1 = input("First rotor slip : ")
if(rid1):
sliprotor(set_curr_rotor(rid1),1)
So the 'indirect' call doesn't pass the value created by the set_curr_rotor function into the sliprotor function. The direct call does use the passed in value.
If I look in debug, you can see that it is directly calling rotorid[] as the list, not Rotorid1 or other Rotoridx and hence I get an index error.
....
File "", line 3, in sliprotor
rotorid[movers-1+offset_qty]=rotorid[movers-1]
IndexError: string index out of range
I could restructure the way I have the code, but I would prefer not to. Is there some method / scope issue I am missing? Is this just an intrinsic attribute of Python? I'm very new to Python so I'm just doing an exercise to model an Enigma machine.
Any help appreciated.
Ed

I'll assume that you have defined your rotors already, something like this:
Rotorid1 = list('abcdefghijklmnopqrstuvwxyz')
Rotorid2 = list('abcdefghijklmnopqrstuvwxyz')
And now you're reluctant to change this, because ... reasons.
That's fine. But you're still wrong. What you need to do is to create a larger data structure. You can do it like this:
Rotors = [ Rotorid1, Rotorid2, ... ]
Now you have a list-of-lists. The Rotors variable now contains all the various Rotorid variables. (Well, it references them. But that'll do.)
Instead of passing in the variable name as a handle to the rotor, you can simply pass in an index number:
def set_rotor(id):
global Current_rotor
Current_rotor = id
def slip_rotor(amount):
global Current_rotor
global Rotors
rotor = Rotors[Current_rotor]
for movers in range(26,0,-1):
rotor[movers-1+offset_qty]=rotor[movers-1]
# etc...
Also, be sure an look up slicings in Python - you can do a lot by manipulating sublists and substrings using slices.

Is there any way to access a variable from outside a function, explicitly as a variable and not via assignment to a function?

I'm wanting to replace keywords with values from an associated dictionary.
file1.py
import file2
file2.replacefunction('Some text','a_unique_key', string_variable1)
file2.replacefunction('Other text','another_unique_key', string_variable2)
file2.replacefunction('More text','unique_key_3', string_variable2)
stringvariable1, used in the first function call, is a local variable in file1.py and therefore is accessible as a parameter in the function. It is intentionally a different variable than the one later used in that parameter position.
file2.py
import re
keywords = {
"a_unique_key":"<b>Some text</b>",
"another_unique_key":"<b>Other text</b>",
"unique_key_3":"<b>More text</b>",
}
def replacefunction(str_to_replace, replacement_key, dynamic_source):
string_variable2 = re.sub(str_to_replace, keywords[replacement_key], dynamic_source)
return string_variable2 <-- this variable needs to be accessible
The replacement values in the keywords dictionary are more complicated than shown above, and just demonstrated like this for brevity.
The problem occurs at the second call to replacefunction in file1.py - it cannot access stringvariable2 which is the result of the first function that is run.
I have seen that the way to access a variable produced in a function outside of that function is to do something like:
def helloworld()
a = 5
return a
mynewvariable = helloworld()
print mynewvariable
5 <-- is printed
But this approach won't work in this situation because the function needs to work on a string that is updated after each function call ie:
do this to string 2 # changes occur to string 2
do this to string 2 # changes occur to string 2
do this to string 2 # changes occur to string 2
I can achieve the required functionality without a function but was just trying to minimise code.
Is there any way to access a variable from outside a function, explicitly as a variable and not via assignment to a function?

Don't confuse variables with values. The name string_variable2 references a value, and you just return that from your function.
Where you call the function, you assign the returned value to a local variable, and use that reference to pass it into the next function call:
string_variable2 = file2.replacefunction('Some text','a_unique_key', string_variable1)
string_variable2 = file2.replacefunction('Other text','another_unique_key', string_variable2)
file2.replacefunction('More text','unique_key_3', string_variable2)
Here the replacefunction returns something, that is stored in string_variable2, and then passed to the second call. The return value of the second function call is again stored (using the same name here), and passed to the third call. And so on.