The top Embedded Python Block in this GRC application is the stock block. The bottom one is a LPF the code for which is below.
Without the Throttle Block the application works. With the Throttle Block the error gr::log :ERROR: thread_body_wrapper - ERROR thread[thread-per-block[2]: <block Embedded Python Block 2(3)>]: ValueError: could not broadcast input array from shape (17,) into shape (16,) is generated. There is no such problem when the Throttle Block is in the path of the stock block.
I learning GRC and would like to understand why the introduction of the Throttle Block is causing the problem.
import numpy as np
from gnuradio import gr
class blk(gr.sync_block): # other base classes are basic_block, decim_block, interp_block
"""Embedded Python Block example - a simple multiply const"""
h = np.array([0.0397989, -0.0488053, -0.0345932, 0.0659844, 0.0315417, -0.1074744,
-0.0299212, 0.3187633, 0.5294118, 0.3187633, -0.0299212, -0.1074744, 0.0315417,
0.0659844, -0.0345932, -0.0488053, 0.0397989], dtype=np.float32)
a = True;
def __init__(self, example_param=2.0): # only default arguments here
"""arguments to this function show up as parameters in GRC"""
gr.sync_block.__init__(
self,
name='Embedded Python Block 2', # will show up in GRC
in_sig=[np.float32],
out_sig=[np.float32]
)
# if an attribute with the same name as a parameter is found,
# a callback is registered (properties work, too).
self.example_param = example_param
def work(self, input_items, output_items):
"""Convolution of input with LPF coefficients"""
output_items[0][:] = np.convolve(input_items[0], self.h, 'same')
return len(output_items[0])
First of all, this won't work – convolution has memory, and your convolver doesn't save that. This makes the numbers that come out of your blk to depend on in what kind of "chunks" your input signal is delivered. That's not OK, because GNU Radio can (and will) use different lengths at any time.
Your algorithm must be written in a way that always yields the same result, no matter how your (infinite or finite) input stream is being partitioned for individual calls to work.
That's basically it – the problem here seems to be that the result of your np.convolve call doesn't have the same length as output_itmes[0]. Read the documentation of numpy.convolve; it specifies exactly why and when this happens:
Mode ‘same’ returns output of length max(M, N). Boundary effects are still visible.
So, if the chunk of samples you've gotten is shorter than h, then you're trying to save len(h) items in a len(output_items[0]) sized array, which is too short.
This can always happen – as said, GNU Radio needs to "cut" the signal into pieces and hand them to your work, and the length of these chunks isn't fixed. Your Throttle just makes it more likely to happen.
So, you first need to conceptually solve the fact that this is not proper mathematically (the convolution of a long signal is not the same as a sequence of truncated convolutions of short sub-segments, simple as that). Most likely the answer to this is to just use one of the filter blocks GNU Radio comes with anyway.
Related
I'm trying to simulate a system of ODEs. While doing so, I need to increase the current value of certain variables by some factor at specific time points when the odeint runs?
I tried doing the following. But what i could notice is that the time values are in floating point. This makes it difficult for me to specify an if-condition for adding a certain value to the inputs that are going to be integrated further in the process.
Below is the problem case. Please help me out with this.
def myfunc(s,t):
# whenever the time is an even day, increase the variable by 2
if t%2==0:
addition = 2
else:
addition = 0
dsdt = (2s+8)+addition
return dsdt
Problem: The incoming time(t) in the function is a floating point number. This prevents me from applying a if condition for specific discrete even values of 't'
Detailed description:
(a)I define a timespan vector , Tspan = np.linspace(1,100,100), and a initial condition s0 = [3].
(b) When I run the " odeint(myfunc, s0, Tspan) ", I need to update the incoming 's' variable by some factor, only at certain timepoints ( Say, for t = 25,50,75).
(c) But for me to this, if I place print(t) inside the "myfunc(s,t)", I could watch out that the incoming 't' is in float type.
(d) And one important note is that the # myfunc is called > #Timesteps in the Tspan vector. This is why the runtime 't' is in floating points.
(e) with this said if i try to perform "if ceil(t)%25==0 or round" the same int is returned for next 4 to 5 function calls ( this is because the there are few number of function calls happening between two subsequent timepoints), as a result, if I try to update the incoming 's' with an if condition on the ceiled(t), the update on 's' is performed for 4 to 5 subsequent function calls instead of once at a specific time point, and this should be avoided.
I hope my problem is clear. Please help me out if you could, in someway. Thanks folks!
All "professional" solvers use an internal adaptive step size. The step size for the output has no or minimal influence on this. Each method step uses multiple evaluations of the ODE function. Depending on the output sampling frequency, you can have multiple internal steps per output step, or multiple output steps get interpolated from the same internal step.
What you describe as desired mechanism is different from your example code. Your example code changes a parameter of the ODE. The description amounts to a state change. While the first can be done with deleterious but recoverable effects on the step size controller, the second requires an event-action mechanism with a state-changing action. Such is not implemented in any of the scipy solvers.
The best way is to solve for the segments between the changes, perform the state change at the end of each segment and restart the integration with the new state. Use array concatenation on the time and value segments to get the large solution function table.
t1=np.linspace(0,25,25+1);
u10=u0
u1=odeint(fun,u10,t1);
t2=t1+25; # or a more specific construction for non-equal interval lengths
u20 = 3*u1[-1] # or some other change of the last state u1[-1]
u2=odeint(fun,u20,t2);
t3=t2+25;
u30 = u2[-1].copy();
u30[0] -=5; # or what the actual state change was supposed to be
u3=odeint(fun,u30,t3);
# combine the segments for plotting, gives vertical lines at the event locations
t=np.concatenate([t1,t2,t3]);
u=np.concatenate([u1,u2,u3]);
For more segments it is better do organize this in a loop, especially if the state change at the event locations via an uniform procedure depending on a few parameters.
I'm looking for a way to replace or delete the last message wrote by python's logging module. The goal is to log a change in variables once it occurs. If the variable changes again, the old log message should be deleted and the new one printed instead.
Hi,
I am using pythons's logging module for a deep learning project I'm currently working on. As some GPUs just don't have enough memory to support the default batch size during training and there is no apparent connection between batch size and actual memory usage that could be used for calculations beforehand, I'm catching the runtime error once it occurs and decrease the batch size by one.
This process can be repeated quite a bit and I'm always logging which batch size did not work and which will be the next one tried. Instead of having 10-30 of these messages (or more) I'd like to simply delete the last one and replace it with the newer one instead.
I've already checked out the python logging documentation, stumbled upon the LogRecord object, but upon trying to deal with it, it seems this object does not actually keep a record of all logs, but rather saves some more info on one specific log instead.
If there is simply no way to do this, I will look into some kind of bundling solution as described here: Python logging: bundle reoccurring messages
The code below shows the log message I'm looking to replace.
Any help is greatly apreciated.
training_not_successful = True
while training_not_successful:
try:
model.run_training(global_settings['epochs'],
train_loader,
test_loader,
global_settings['checkpoint_output_path'],
model_name,
global_settings['best_net_criterion'])
training_not_successful = False
except MemoryError:
logging.warning("Ran out of CUDA memory using batch size " + str(batch_size) +
". Trying again with batch size " + str(batch_size-1))
batch_size -= 1
train_loader, test_loader = get_train_test_loaders(
train_dataset_list,
test_dataset_list,
value_counts,
batch_size
)
I believe (correct me if I'm wrong) that the logging module does not allow to supress newlines, meaning that it's simply not possible to do something like that.
It is possible to do it with a print though:
import shutil
def display(variable, rewritable=False):
columns, lines = shutil.get_terminal_size(fallback=(80, 20))
text = str(variable)
filled = text + ((columns - len(text)) * ' ')
print(filled, end='\r' if rewritable else '\n')
if __name__ == "__main__":
from random import random
from time import sleep
for i in range(10):
display(f"x = {random()}", True)
sleep(1)
display(f"x = 0.0") # test if old value is overwritten completely
display("Done!")
Tested this on linux, but it should work everywhere. (the shutil.get_terminal_size function)
It's not mandatory, but very nice when entire line is overwritten as opposed to only the part that's changed.
The key is character \r - it returns the cursor to the end of the line, that's it. Now you can start writing from the front again, overwriting the line if it has anything else, which is exactly what you want.
Display function is simple, but I'll explain it anyway:
First line gets terminal size, what we need is the width of the line, so we can pad the text with spaces and fill entire line with spaces, to completely overwrite previous line no matter what it had.
Then we convert our variable to a string.
After that, it's just simple math, our string takes n characters, so the rest should be spaces, so we add width - n spaces to final string, and then we print it - entire line is overwritten.
rewritable flag allows to control when the variable should be rewritten next time you call display.
While this is not what you want, as it does not use logging module, since there's no way (that I know of) to make logging module to print \r instead of \n, I think this is a good enough substitute, that could be used if it turns out that you can indeed do this with logging module.
I am working with Baxter robot and what I am trying to do is get the position of an object using augmented reality markers and move the hand to that position in order to grasp it.
I am using the ar_tools package to get the position/orientation of the object, but that with respect to the head_camera which I am using. The last couple of days I've been experimenting with how to change that reference frame (head_camera) to the base frame as this frame is used by moveit to make the plans. I have tried to set the frame_id of the header of the message I receive from the ar_tools manually to 'base':
pose_target.header.frame_id = 'base'
but the position and orientation I am getting are still WRT the head_camera. I also tried to do:
self.tl.lookupTransform("/base", "/head_camera", self.t)
where self.t = self.tl.getLatestCommonTime("/head_camera", "/base"), but I was getting an extrapolation error. It was something like
the transformation requires to extrapolate in the past
(I can't really remember now and I'm not in the lab.) Then I thought I might need to run the lookupTransform from the head_cam to head, from head to torso and from torso to Baxter's base.
Could someone guide me on how to change the frame of the marker of the ar_tools from head_camera to base?
Also, for the extrapolation error, is there a way to do this in a static way?
There is a slightly more straightforwards way to do this, presuming you're reviving a PoseStamped message from ar_tools:
on_received_pose(pose):
'''
Handler for your poses from ar_tools
'''
if self.tl.waitForTransform(pose.header.frame_id, "/base", pose.header.stamp, rospy.Duration(0.1)): # this line prevents your extrapolation error, it waits up to 0.1 seconds for the transform to become valid
transd_pose = self.tl.transformPose("/base",pose)
# do whatever with the transformed pose here
else:
rospy.logwarn('Couldn\'t Transform from "{}" to "/base" before timeout. Are you updating TF tree sufficiently?'.format(pose.header.frame_id))
You're getting that extrapolation error likely because the transform network wasn't fully formed at the time you got your first message; tf refuses to extrapolate, it will only interpolate, so if you haven't received at least one transform message for every frame both before and after (or exactly at) the timestamp you're trying to transform to, it will throw an exception. That added if statement checks to see if it can actually perform the transform before trying to do so. You could of course also just surround the transformPose() call in a try/catch block instead, but I feel that for tf this makes it more explicit what you're trying to do.
In general, check out the simple ROS tf Python Tutorial for more examples/modes of operation.
I have a piece of code that is supposed to calculate a simple
matrix product, in python (using theano). The matrix that I intend to multiply with is a shared variable.
The example is the smallest example that demonstrates my problem.
I have made use of two helper-functions. floatX converts its input to something of type theano.config.floatX
init_weights generates a random matrix (in type floatX), of given dimensions.
The last line causes the code to crash. In fact, this forces so much output on the commandline that I can't even scroll to the top of it anymore.
So, can anyone tell me what I'm doing wrong?
def floatX(x):
return numpy.asarray(x,dtype=theano.config.floatX)
def init_weights(shape):
return floatX(numpy.random.randn(*shape))
a = init_weights([3,3])
b = theano.shared(value=a,name="b")
x = T.matrix()
y = T.dot(x,b)
f = theano.function([x],y)
This work for me. So my guess is that you have a problem with your blas installation. Make sure to use Theano development version:
http://deeplearning.net/software/theano/install.html#bleeding-edge-install-instructions
It have better default for some configuration. If that do not fix the problem, look at the error message. There is main part that is after the code dump. After the stack trace. This is what is the most useful normally.
You can disable direct linking by Theano to blas with this Theano flag: blas.ldflags=
This can cause slowdown. But it is a quick check to confirm the problem is blas.
If you want more help, dump the error message to a text file and put it on the web and link to it from here.
I use a Raspberry Pi to collect sensor data and set digital outputs, to make it easy for other applications to set and get values I'm using a socket server. But I am having some problems finding an elegant way of making all the data available on the socket server without having to write a function for each data type.
Some examples of values and methods I have that I would like to make available on the socket server:
do[2].set_low() # set digital output 2 low
do[2].value=0 # set digital output 2 low
do[2].toggle() # toggle digital output 2
di[0].value # read value for digital input 0
ai[0].value # read value for analog input 0
ai[0].average # get the average calculated value for analog input 0
ao[4].value=255 # set analog output 4 to byte value 255
ao[4].percent=100 # set analog output 4 to 100%
I've tried eval() and exec():
self.request.sendall(str.encode(str(eval('item.' + recv_string)) + '\n'))
eval() works unless I am using equal sign (=), but I'm not to happy about the solution because of dangers involved. exec() does the work but does not return any value, also dangerous.
I've also tried getattr():
recv_string = bytes.decode(self.data).lower().split(';')
values = getattr(item, recv_string[0])
self.request.sendall(str.encode(str(values[int(recv_string[1])].value) + '\n'))
^^^^^
This works for getting my attributes, and the above example works for getting the value of the attribute I am getting with getattr(). But I can not figure out how to use getattr() on the value attribute as well.
The semi-colon (;) is used to split the incoming command, I've experimented with multiple ways of formatting the commands:
# unit means that I want to talk to a I/O interface module,
# and the name specified which one
unit;unit_name;get;do;1
unit;unit_name;get;do[1]
unit;unit_name;do[1].value
I am free to choose the format since I am also writing the software that uses these commands. I have not yet found a good format which covers all my needs.
Any suggestions how I can write an elegant way of accessing and returning the data above? Preferably with having to add new methods to the socket server every time a new value type or method is added to my I/O ports.
Edit: This is not public, it's only available on my LAN.
Suggestions
Make your API all methods so that eval can always be used:
def value_m(self, newValue=None):
if newValue is not None:
self.value = newValue
return self.value
Then you can always do
result = str(eval(message))
self.request.sendall(str.encode(result + '\n'))
For your message, I would suggest that your messages are formatted to include the exact syntax of the command exactly so that it can be evaled as-is, e.g.
message = 'do[1].value_m()' # read a value, alternatively...
message = 'do[1].value_m(None)'
or to write
message = 'do[1].value_m(0)' # write a value
This will make it easy to keep your messages up-to-date with your API, because they must match exactly, you won't have a second DSL to deal with. You really don't want to have to maintain a second API, on top of your IO one.
This is a very simple scheme, suitable for a home project. I would suggest some error handling in evaluation, like so:
import traceback
try:
result = str(eval(message))
except Exception:
result = traceback.format_exc()
self.request.sendall(str.encode(result + '\n'))
This way your caller will receive a printout of the exception traceback in the returned message. This will make it much, much easier to debug bad calls.
NOTE If this is public-facing, you cannot do this. All input must be sanitised. You will have to parse each instruction and compare it to the list of available (and desirable) commands, and verify input validity and validity ranges for everything. For such a scenario you are better off simply using one of the input validation systems used for web services, where this problem receives a great deal of attention.