I try to use copy.deepcopy(graph) but got an error:
my code:
new_graph = TNEANet.New()
....
# some define for new_graph
....
copy_graph = copy.deepcopy(new_graph)
Execution error:
TypeError: object.__new__(SwigPyObject) is not safe, use SwigPyObject.__new__()
Also I found the API has a definition API Document. So I tried to use TNEANet(Graph) for that task:
new_graph = TNEANet.New()
....
# some define for new_graph
....
copy_graph = TNEANet(new_graph)
Execution error:
TypeError: in method 'new_TNEANet', argument 1 of type 'TSIn &'
You can use ConvertGraph method for copying a graph. Just use the same type of the graph as targeted type:
graph = snap.TNEANet.New()
graph.AddNode(1)
graph.AddNode(2)
graph.AddEdge(1,2)
copy_graph = snap.ConvertGraph(type(graph), graph)
Short answer
The best thing to do is to copy the nodes, edges, and attributes yourself. I found out that dumping to a file and reloading is also working quite well, although it won't copy your attributes if you are working with a PNEANet. Here is a handy function I used to copy the graphs (faster than iterating):
def copy_graph(graph):
tmpfile = '.copy.bin'
# Saving to tmp file
FOut = snap.TFOut(tmpfile)
graph.Save(FOut)
FOut.Flush()
# Loading to new graph
FIn = snap.TFIn(tmpfile)
graphtype = type(graph)
new_graph = graphtype.New()
new_graph = new_graph.Load(FIn)
return new_graph
Long answer
There is a method to perform a deep copy your graph:
import snap
new_graph = snap.TNEANet.New()
....
# some define for new_graph
....
copy_graph = snap.TNEANet(new_graph())
(This works for any type of graph)
The problem is that it returns a snap.TNEANet when we need a snap.PNEANet. The latter is just a pointer to the former in the C++ implementation, but we have no way to get create this pointer in Python. As a result, copy_graph will have less functionalities than new_graph.
Explainations:
SNAP is using a proxy representation for python that somehow complicates the deep copy process. The copy constructor is expecting the non-proxy type, and you are giving it the python proxy type.
A TypeError is raised displaying the signature of the first constructor with 1 argument that it knows (which is completely irrelevant here).
To solve this, we need to give the constructor the underlying C++ type which is obtained by using the self.__call__() method.
But because of the C++ (bad?) implementation, snap.TNEANet(new_graph()) outputs a snap.TNEANet. That means that the output of this function will be a non proxy type. As a result it won't work with most SNAP functionalities...
Related
I am trying something like the following (using drake#00cdef03)
Note: This is not proper idiomatic Drake; you should declare a separate subclass when making a novel LeafSystem. This post just shows a compact, but hacky, form.
from pydrake.all import (
AbstractValue,
DiagramBuilder,
LeafSystem,
SpatialForce,
Value,
)
def fake_calc(context, output):
raise NotImplemented()
producer = LeafSystem()
producer.DeclareAbstractOutputPort("output", alloc=SpatialForce, calc=fake_calc)
consumer = LeafSystem()
consumer.DeclareAbstractInputPort("input", model_value=AbstractValue.Make(SpatialForce()))
builder = DiagramBuilder()
builder.AddSystem(producer)
builder.AddSystem(consumer)
builder.Connect(producer.get_output_port(), consumer.get_input_port())
For some reason, it fails on builder.Connect(...) with the following error:
Unable to cast Python instance to C++ type (compile in debug mode for details)
If I print the value of the ports, they seem like the match the signature of the function:
>>> producer.get_output_port()
<pydrake.systems.framework.LeafOutputPort_[float] object at 0x7febb5dba930>
>>> consumer.get_input_port()
<pydrake.systems.framework.InputPort_[float] object at 0x7febb5ff0930>
What's going on?
The issue lies in the fact that alloc=SpatialForce is being used, but it should be should be alloc=Value[SpatialForce].
The following shows the difference between good and bad:
system = LeafSystem()
# Good!
good = system.DeclareAbstractOutputPort("good", alloc=Value[SpatialForce], calc=fake_calc)
good.Allocate()
# BAD!
bad = system.DeclareAbstractOutputPort("bad", alloc=SpatialForce, calc=fake_calc)
bad.Allocate() # Fails here
This happens because alloc must produce an instance of AbstractValue, which is used by OutputPort::Alocate() (in C++), which uses this std::function<> type:
drake/.../value_producer.h
It happens during .Connect() because that will ultimately call Allocate():
drake/.../diagram_builder.cc
Because Python returns something not of an AbstractValue type, it is unable to cast, hence the error message
I have programmend a framework that concatenate different ( quite complicated ) linear operators in an abstract manner. It overrides the operators, "+,*,#,-" and chooses a path through the graph of compositions of functions. It isn't easy to debug to say the least, however the control flow isn't depending on the data itself and of course any operation is done with tensorflow. I was hoping to use tf.function to compile it and get an ( hopefully much faster) tf.function by XLA. However I get the following error:
TypeError: An op outside of the function building code is being passed
a "Graph" tensor. It is possible to have Graph tensors
leak out of the function building context by including a
tf.init_scope in your function building code.
For example, the following function will fail:
#tf.function
def has_init_scope():
my_constant = tf.constant(1.)
with tf.init_scope():
added = my_constant * 2
The graph tensor has name: Reshape_2:0
I do not use the tf.init_scope anywhere and there are 8 (!) google results regarding to this error - while none of them provides me any clue how to debug it.
# initilize linear operators, these are python objects that override __matmul__ etc.
P = ...
A = ...
# initilize vectors, these are compatible python objects to P and A
x = ...
y = ...
# This function recreates the python object from its raw tensorflow data.
# Since it might be dependend on the spaces and
# they also need to be set up for deserializaton the method is returned by a function of x.
# But since many vectors share the same spaces I was hoping to reuse it.
deserialize = x.deserialize()
# We want to compile the action on x to a function now
def meth( data ):
result = P # ( A.T # A # deserialize( data ) )
# we return only the raw data
return result.serialize()
meth = tf.function( meth,
#experimental_compile = True ,
input_signature = (x.serialize_signature,),
).get_concrete_function()
# we want to use meth now for many vectors
# executing this line throws the error
meth(x1)
meth(x2)
meth(x3)
Needless to say that works without the tf.function.
Did anyone stumble across the error and can help me to understand it better ? Or is the hole setup I'm trying not suitable for tensorflow ?
Edit:
The Error was caused by implicitly capturing a constant tensor in the linear operator class by a local lambda. To be honest, the error message suggest something like that, however it was difficult to understand which line in the code caused it and it wasn't easy to find the bug in the end.
Using the Maya Python API 2.0, I'm trying to make a callback that changes the value of a plug. However, none of the methods I've tried are working.
I've tried using the MPlug.setFloat() method, but this didn't lead to expected results; I found no change in the plug's value. I figured this hadn't worked because I needed to clean the plug after changing its value. So, I then tried getting the plug's data handle using the MPlug.asDataHandle() method, then using the data handle's datablock() method in order to use the data handle and datablock to set the plug's value and clean it. However, I got an error saying "RuntimeError: (kFailure): Unexpected Internal Failure" upon using MDataHandle.datablock().
Now I'm trying the following, which uses the data handle to set the plug's value and clean it:
def setPlugFloatValue(node, plugName, val):
fnSet = OpenMaya.MFnDependencyNode(node)
plug = fnSet.findPlug(plugName,True)
handle = plug.asMDataHandle()
handle.setFloat(val)
handle.setClean()
The above function is intended to find a certain plug in a node, then use its data handle to set its value and clean it. In my program, the callback uses this function to change the translateX, translateY and translateZ plugs of a node's child nodes. The callback runs when the translate value of the node it's applied to changes. In a scene I'm using to test this callback, I apply the callback to a polygon mesh object, with one child which is also a polygon mesh object. So, as I translate the parent object, I expect the translate values of its child to change. But when I select the child object after translating its parent, its translate values haven't changed.
Tried your example and used setFloat() on the plug, which appears to work fine.
import maya.api.OpenMaya as OpenMaya
def setPlugFloatValue(node, plugName, val):
fnSet = OpenMaya.MFnDependencyNode(node)
plug = fnSet.findPlug(plugName,True)
plug.setFloat(val)
def applyToSelectedObjects():
sl_list = OpenMaya.MGlobal.getActiveSelectionList()
iterator = OpenMaya.MItSelectionList(sl_list)
while not iterator.isDone():
obj = iterator.getDependNode()
setPlugFloatValue(obj, "translateX", -2.0)
iterator.next()
applyToSelectedObjects()
Perhaps your issue is something else? You can also try to use setMDistance() instead, but it didn't make any difference in my testing.
distance = OpenMaya.MDistance(val)
plug.setMDistance(distance)
I was wondering if it is possible to access native Qt-types in PyQt with an easy constructor?
Why is this an issue?
Recently, I had an unexpected error for setting a Flag:
opts = 0 # in this case, no flag was set
myQMainWindow.setDockOptions( opts )
QMainWindow.setDockOptions() expects a QMainWindow.DockOption: unexpected type 'int'
I expected the argument to have type int, because when you look at PyQt5\QtWidgets\QMainWindow.py, you will see these dockOptions defined in the same pattern:
class QMainWidget(QWidget):
AllowNestedDocks = 2
AllowTabbedDocks = 4
AnimatedDocks = 1
[...]
How I solved this:
I abused the type-function to get the underlaying class, from one of the allready known Flags, using:
dockOptionFlag = type(QMainWindow.AnimatedDocks) # not 'int'
opts = dockOptionFlag() # in Debug-Mode this is still shown as 'int':0
myQMainWindow.setDockOptions( opts )
Why would you need something like this in general?
When I recently was translating qt-code from c++ to python I realised that some Qt-classes, that are simular to python-classes, have additional functionality.
For example:
a QMap is essentialy a dict in python ... but further than that, it also provieds a functionality to add entries with an existing key, without overwriting (insertMulti()).
a QString is essentialy a str in python... but the function of left(n), which is simular to 'someText'[n:] is returning the entire string, if n is negative, which wouldn't be the case in my python translation.
I know these are simple examples and their functionality could be reimplemented in more or less time, but this is not needed since it's allready existing in the c++-wrapped pyqt.
Is their a pythonic way to access these native-types in python with an easy-constructor? something like: temp = qtType("QMap", *args, **kwargs)
I want to search surfs in all images in a given directory and save their keypoints and descriptors for future use. I decided to use pickle as shown below:
#!/usr/bin/env python
import os
import pickle
import cv2
class Frame:
def __init__(self, filename):
surf = cv2.SURF(500, 4, 2, True)
self.filename = filename
self.keypoints, self.descriptors = surf.detect(cv2.imread(filename, cv2.CV_LOAD_IMAGE_GRAYSCALE), None, False)
if __name__ == '__main__':
Fdb = open('db.dat', 'wb')
base_path = "img/"
frame_base = []
for filename in os.listdir(base_path):
frame_base.append(Frame(base_path+filename))
print filename
pickle.dump(frame_base,Fdb,-1)
Fdb.close()
When I try to execute, I get a following error:
File "src/pickle_test.py", line 23, in <module>
pickle.dump(frame_base,Fdb,-1)
...
pickle.PicklingError: Can't pickle <type 'cv2.KeyPoint'>: it's not the same object as cv2.KeyPoint
Does anybody know, what does it mean and how to fix it? I am using Python 2.6 and Opencv 2.3.1
Thank you a lot
The problem is that you cannot dump cv2.KeyPoint to a pickle file. I had the same issue, and managed to work around it by essentially serializing and deserializing the keypoints myself before dumping them with Pickle.
So represent every keypoint and its descriptor with a tuple:
temp = (point.pt, point.size, point.angle, point.response, point.octave,
point.class_id, desc)
Append all these points to some list that you then dump with Pickle.
Then when you want to retrieve the data again, load all the data with Pickle:
temp_feature = cv2.KeyPoint(x=point[0][0],y=point[0][1],_size=point[1], _angle=point[2],
_response=point[3], _octave=point[4], _class_id=point[5])
temp_descriptor = point[6]
Create a cv2.KeyPoint from this data using the above code, and you can then use these points to construct a list of features.
I suspect there is a neater way to do this, but the above works fine (and fast) for me. You might have to play around with your data format a bit, as my features are stored in format-specific lists. I tried to present the above using my idea at its generic base. I hope that this may help you.
Part of the issue is cv2.KeyPoint is a function in python that returns a cv2.KeyPoint object. Pickle is getting confused because, literally, "<type 'cv2.KeyPoint'> [is] not the same object as cv2.KeyPoint". That is, cv2.KeyPoint is a function object, while the type was cv2.KeyPoint. Why OpenCV is like that, I can only make guesses at unless I go digging. I have a feeling it has something to do with it being a wrapper around a C/C++ library.
Python does give you the ability to fix this yourself. I found the inspiration on this post about pickling methods of classes.
I actually use this clip of code, highly modified from the original in the post
import copyreg
import cv2
def _pickle_keypoints(point):
return cv2.KeyPoint, (*point.pt, point.size, point.angle,
point.response, point.octave, point.class_id)
copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoints)
Key points of note:
In Python 2, you need to use copy_reg instead of copyreg and point.pt[0], point.pt[1] instead of *point.pt.
You can't directly access the cv2.KeyPoint class for some reason, so you make a temporary object and use that.
The copyreg patching will use the otherwise problematic cv2.KeyPoint function as I have specified in the output of _pickle_keypoints when unpickling, so we don't need to implement an unpickling routine.
And to be nauseatingly complete, cv2::KeyPoint::KeyPoint is an overloaded function in C++, but in Python, this isn't exactly a thing. Whereas in the C++, there's a function that takes the point for the first argument, in Python, it would try to interpret that as an int instead. The * unrolls the point into two arguments, x and y to match the only int argument constructor.
I had been using casper's excellent solution until I realized this was possible.
A similar solution to the one provided by Poik. Just call this once before pickling.
def patch_Keypoint_pickiling(self):
# Create the bundling between class and arguments to save for Keypoint class
# See : https://stackoverflow.com/questions/50337569/pickle-exception-for-cv2-boost-when-using-multiprocessing/50394788#50394788
def _pickle_keypoint(keypoint): # : cv2.KeyPoint
return cv2.KeyPoint, (
keypoint.pt[0],
keypoint.pt[1],
keypoint.size,
keypoint.angle,
keypoint.response,
keypoint.octave,
keypoint.class_id,
)
# C++ Constructor, notice order of arguments :
# KeyPoint (float x, float y, float _size, float _angle=-1, float _response=0, int _octave=0, int _class_id=-1)
# Apply the bundling to pickle
copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoint)
More than for the code, this is for the incredibly clear explanation available there : https://stackoverflow.com/a/50394788/11094914
Please note that if you want to expand this idea to other "unpickable" class of openCV, you only need to build a similar function to "_pickle_keypoint". Be sure that you store attributes in the same order as the constructor. You can consider copying the C++ constructor, even in Python, as I did. Mostly C++ and Python constructors seems not to differ too much.
I has issue with the "pt" tuple. However, a C++ constructor exists for X and Y separated coordinates, and thus, allow this fix/workaround.