Develop Reference

Pymel: How do I close For loop after deleting null groups?

As I continue to study For Loops: I've run into some annoying errors. The problem is the script does exactly what I want it to. It deletes the null groups under the demo joints: but unlike other loops I've made for renaming which can be closed with a transform flag in the cmds.ls command: cmds.listRelatives doesn't allow a transform flag to close out the loop. You run the script by simply clicking Build Examples then hitting Delete Waste Groups
I've tried every flag according to the Maya documentation: but nothing seems to be closing the loop. I dont know if I need another variable, or a combination of some flags: or if I am using the wrong type of wording: but ideally what I would like this script to do is simply close out the loop so I dont get the error Error: No object matches name: curve
'''
import DS_wasteGroup_cleanerDemo
reload (DS_wasteGroup_cleanerDemo)
DS_wasteGroup_cleanerDemo.gui()
'''
import re
import maya.cmds as cmds
import maya.mel as mel
if cmds.window("renameWin", exists =True):
cmds.deleteUI("renameWin", window = True)
myWindow = cmds.window("renameWin",t='DS_wasteGroup_cleanerDemo',w=200, h=500, toolbox=True)
column = cmds.columnLayout(adj=True)
def gui():
cmds.button( label="Build Examples", c = buildExamples)
cmds.separator( w=200, h=3)
cmds.button( label="Delete Waste Groups", c = deleteWasteGrp)
cmds.separator( w=200, h=9)
cmds.setParent('..')
cmds.showWindow(myWindow)
def buildExamples(*args):
cmds.group(n='exampleGroup1',world=True,empty=True)
cmds.joint(n='demoJoint1')
cmds.group(n='curve1',world=True,empty=True)
cmds.parent('curve1','demoJoint1')
cmds.joint(n='demoJoint2')
cmds.parent('demoJoint2','exampleGroup1')
cmds.group(n='curve2',world=True,empty=True)
cmds.parent('curve2','demoJoint2')
cmds.joint(n='demoJoint3')
cmds.parent('demoJoint3','exampleGroup1')
cmds.group(n='curve3',world=True,empty=True)
cmds.parent('curve3','demoJoint3')
cmds.joint(n='demoJoint4')
cmds.parent('demoJoint4','exampleGroup1')
cmds.group(n='curve4',world=True,empty=True)
cmds.parent('curve4','demoJoint4')
cmds.joint(n='demoJoint5')
cmds.parent('demoJoint5','exampleGroup1')
cmds.group(n='curve5',world=True,empty=True)
cmds.parent('curve5','demoJoint5')
def deleteWasteGrp(*args):
grpList = cmds.listRelatives('demoJoint*',p=True,f=True)
for name in grpList:
print(grpList)
cmds.delete('curve*')
My apologies if I'm posting simple questions. I do write Python scripts to automate the most tedious tasks in rigging: but my knowledge is only intermediate. I want to learn more python so my scripts arent so clunky and brute forced: as well as the fact that I need them to be more adaptable to various types of characters: so any resources that dumb all this down would also be appreciated. Thank you for your help.

The error is correct, because the very first time the for loop executes, all "curve" obects are deleted, then in the next iteration, the same command does not find any curve objects because they are already deleted. If you place the delete command outside the for loop, the error should disappear.

Honestly I would take a whole different approach as you're hard-coding everything which could easily lead to disaster. When I mean hard-code, I mean you're trying to parent, let's say, "demoJoint2" to an object. This is bad because why are you assuming that "demoJoint2" even exists? If you create an object with a specific name that already exists, Maya will auto-rename the new object, and now you're referencing the wrong one right off the bat! Instead when you create your objects, capture their names in a variable then work with that, otherwise you'll be constantly shooting yourself in the foot.
Here's the same script with an approach I would try instead:
import maya.cmds as cmds
def gui():
if cmds.window("renameWin", exists=True):
cmds.deleteUI("renameWin", window=True)
myWindow = cmds.window("renameWin", t="DS_wasteGroup_cleanerDemo", w=200, h=500, toolbox=True)
column = cmds.columnLayout(adj=True)
cmds.button(label="Build Examples", c=buildExamples)
cmds.separator(w=200, h=3)
cmds.button(label="Delete Waste Groups", c=deleteWasteGrp)
cmds.separator(w=200, h=9)
cmds.setParent("..")
cmds.showWindow(myWindow)
def buildExamples(*args):
root = cmds.group(n="exampleGroup1", world=True, empty=True)
for i in range(5): # Loop to the amount of joints you want to create.
jnt = cmds.createNode("joint", name="demoJoint" + str(i + 1)) # Use `i` to help name the object.
jnt = cmds.parent(jnt, root)[0] # Parenting changes its long name, so recapture the joint in a variable.
crv = cmds.group(n="curve" + str(i + 1), world=True, empty=True) # Create empty group.
cmds.parent(crv, jnt) # Parent curve to joint.
def deleteWasteGrp(*args):
jnts = cmds.ls("demoJoint*", long=True, type="joint") # Get all `demoJoints`.
children = cmds.listRelatives(jnts, f=True, children=True, type="transform") or [] # Get all of their children, and only get transform types.
curves = [obj for obj in children if obj.split("|")[-1].startswith("curve")] # Don't assume we got the right objects. Run a final loop to collect any object that starts with `curve`. Need to use split as we're looping through long names but need to check its short name.
if curves: # `cmds.delete` will error if this list is empty, so don't assume.
cmds.delete(curves) # Delete all curves at once.
gui()
Now I can hit the build button as much as I want with no issues, and delete all the curves when pressing the delete button.
A few more notes:
Notice in buildExamples I'm using a loop to create all the objects instead of reusing redundant code that does the same thing. You could even have a spinbox in your gui that defines how many joints it creates now, where as before it wasn't possible because the count was hard-coded.
cmds.listRelatives does have a way to filter objects by transforms by setting parameter type="transform". In fact you'll see many commands have this same parameter (again start checking docs).
cmds.listRelatives('demoJoint*',p=True,f=True) was grabbing the joint's parent, not its children. The docs clearly state this.
Running cmds.delete('curve*') is going to delete ALL objects with names that start with curve, and since you're running this in a loop it's trying to do this multiple times.
maya.cmds is not pymel. There's a whole separate module called pymel.
If you're unsure with any parts of the code try adding in a print statement to see what it's doing.

I feel like you're going about this whole process a bit wrong, and I would love to elaborate if you're interested, but for now here is a fix for your loop situation:
def deleteWasteGrp(*args):
curveList = cmds.ls('curve*',transforms=True)
try:
cmds.delete(curveList)
print('Deleted the following objects: {}'.format(curveList))
except Exception as e:
cmds.warning('Cleanup failed: {}'.format(e))
The cmds.delete method accepts a list parameter, which in your case is the easiest way to get the job done. Keep in mind that when you delete a parent object, you also delete its children, so depending on your circumstances deleting objects can be order-specific.
Throwing any "likely to fail" calls in a try/except clause is generally a good idea, as it lets you handle the error gracefully. Be careful, however, to not suppress it and just move on -- you at the very least need to alert the user adequately.
Lastly, your buildExamples method will most likely fail if you run it more than once. Because you are addressing objects by string literals (hard coded names) instead of keeping track of their actual names (and full path). You will likely see this error eventually:
# Error: ValueError: file <maya console> line ??: More than one object matches name: demoJoint1 #
Edit: Some elaborations as requested
The commands cmds.group and cmds.joint return a string value indicating the actual name of the object created (in create mode). It's usually a good idea of storing this value in case Maya decides to name your object slightly differently than what you are expecting, usually when there is a naming clash. Eg:
print cmds.group(name='test', world=True, empty=True)
# Returns: test
print cmds.group(name='test', world=True, empty=True)
# Returns: test1
Example of how to capture object names as you create them. I've concatenated your five identical(ish) calls to create joints and curves in this loop:
import maya.cmds as cmds
topGroupName = 'exampleGroup'
actualTopGroupName = None
# Create top level group
actualTopGroupName = cmds.group(n=topGroupName, world=True, empty=True)
# Loop through 5 times and do the following:
for i in range(5):
# PS: hash character in name indicates "next available number"
cmds.select(clear=True)
jnt = cmds.joint(n='demoJoint#')
crv = cmds.group(n='curve#',world=True,empty=True)
cmds.parent(crv, jnt)
cmds.parent(jnt, actualTopGroupName)
Example of how to narrow down which objects to search for with cmds.ls:
topGroupName = 'exampleGroup'
print cmds.ls('|{}*|*|curve*'.format(topGroupName))
# Returns: [u'curve1', u'curve2', u'curve3', u'curve4', u'curve5']
# The string .format() expression is just a dynamic way of writing this:
# |exampleGroup*|*|curve*
Vertical pipes (|) indicate levels in a hierarchy, similar to how slashes (/) work in URLs. And asterisks/wildcards (*) indicate "any character, or none".
Hope this helps you along your way a little bit.

Search for all the if conditions in a python file and adding a print statement in the next line

I have to edit a python file such that after every if condition, i need to add a line which says
if condition_check:
if self.debug == 1: print "COVERAGE CONDITION #8.3 True (condition_check)"
#some other code
else:
if self.debug == 1: print "COVERAGE CONDITION #8.4 False (condition_check)"
#some other code
The number 8.4(generally y.x) refer to the fact that this if condition is in function number 8(y) (the functions are just sequentially numbers, nothing special about 8) and x is xth if condition in yth function.
and of course, the line that will be added will have to be added with proper indentation. The condition_check is the condition being checked.
For example:
if (self.order_in_cb):
self.ccu_process_crossing_buffer_order()
becomes:
if (self.order_in_cb):
if self.debug == 1: print "COVERAGE CONDITION #8.2 TRUE (self.order_in_cb)"
self.ccu_process_crossing_buffer_order()
How do i achieve this?
EXTRA BACKGROUND:
I have about 1200 lines of python code with about 180 if conditions - i need to see if every if condition is hit during the execution of 47 test cases.
In other words i need to do code coverage. The complication is - i am working with cocotb stimulus for RTL verification. As a result, there is no direct way to drive the stimulus, so i dont see an easy way to use the standard coverage.py way to test coverage.
Is there a way to check the coverage so other way? I feel i am missing something.

If you truly can't use coverage.py, then I would write a helper function that used inspect.stack to find the caller, then linecache to read the line of source, and log that way. Then you only have to change if something: to if condition(something): throughout your file, which should be fairly easy.
Here's a proof of concept:
import inspect
import linecache
import re
debug = True
def condition(label, cond):
if debug:
caller = inspect.stack()[1]
line = linecache.getline(caller.filename, caller.lineno)
condcode = re.search(r"if condition\(.*?,(.*)\):", line).group(1)
print("CONDITION {}: {}".format(label, condcode))
return cond
x = 1
y = 1
if condition(1.1, x + y == 2):
print("it's two!")
This prints:
CONDITION 1.1: x + y == 2
it's two!

I have about 1200 lines of python code with about 180 if conditions - i need to see if every if condition is hit during the execution of 47 test cases. In other words i need to do code coverage. The complication is - i am working with cocotb stimulus for RTL verification.
Cocotb has support for coverage built in (docs)
export COVERAGE=1
# run cocotb however you currently invoke it

How to get results out of a Python exec()/eval() call?

I want to write a tool in Python to prepare a simulation study by creating for each simulation run a folder and a configuration file with some run-specific parameters.
study/
study.conf
run1
run.conf
run2
run.conf
The tool should read the overall study configuration from a file including (1) static parameters (key-value pairs), (2) lists for iteration parameters, and (3) some small code snippets to calculate further parameters from the previous ones. The latter are run specific depending on the permutation of the iteration parameters used.
Before writing the run.conf files from a template, I need to run some code like this to determine the specific key-value pairs from the code snippets for that run
code = compile(code_str, 'foo.py', 'exec')
rv=eval(code, context, { })
However, as this is confirmed by the Python documentation, this just leads to a None as return value.
The code string and context dictionary in the example are filled elsewhere. For this discussion, this snippet should do it:
code_str="""import math
math.sqrt(width**2 + height**2)
"""
context = {
'width' : 30,
'height' : 10
}
I have done this before in Perl and Java+JavaScript. There, you just give the code snippet to some evaluation function or script engine and get in return a value (object) from the last executed statement -- not a big issue.
Now, in Python I struggle with the fact that eval() is too narrow just allowing one statement and exec() doesn't return values in general. I need to import modules and sometimes do some slightly more complex calculations, e.g., 5 lines of code.
Isn't there a better solution that I don't see at the moment?
During my research, I found some very good discussions about Pyhton eval() and exec() and also some tricky solutions to circumvent the issue by going via the stdout and parsing the return value from there. The latter would do it, but is not very nice and already 5 years old.

The exec function will modify the global parameter (dict) passed to it. So you can use the code below
code_str="""import math
Result1 = math.sqrt(width**2 + height**2)
"""
context = {
'width' : 30,
'height' : 10
}
exec(code_str, context)
print (context['Result1']) # 31.6
Every variable code_str created will end up with a key:value pair in the context dictionary. So the dict is the "object" like you mentioned in JavaScript.
Edit1:
If you only need the result of the last line in code_str and try to prevent something like Result1=..., try the below code
code_str="""import math
math.sqrt(width**2 + height**2)
"""
context = { 'width' : 30, 'height' : 10 }
lines = [l for l in code_str.split('\n') if l.strip()]
lines[-1] = '__myresult__='+lines[-1]
exec('\n'.join(lines), context)
print (context['__myresult__'])
This approach is not as robust as the former one, but should work for most case. If you need to manipulate the code in a sophisticated way, please take a look at the Abstract Syntax Trees

Since this whole exec() / eval() thing in Python is a bit weird ... I have chose to re-design the whole thing based on a proposal in the comments to my question (thanks #jonrsharpe).
Now, the whole study specification is a .py module that the user can edit. From there, the configuration setup is directly written to a central object of the whole package. On tool runs, the configuration module is imported using the code below
import imp
# import the configuration as a module
(path, name) = os.path.split(filename)
(name, _) = os.path.splitext(name)
(file, filename, data) = imp.find_module(name, [path])
try:
module = imp.load_module(name, file, filename, data)
except ImportError as e:
print(e)
sys.exit(1)
finally:
file.close()

I came across similar needs, and finally figured out a approach by playing with ast:
import ast
code = """
def tf(n):
return n*n
r=tf(3)
{"vvv": tf(5)}
"""
ast_ = ast.parse(code, '<code>', 'exec')
final_expr = None
for field_ in ast.iter_fields(ast_):
if 'body' != field_[0]: continue
if len(field_[1]) > 0 and isinstance(field_[1][-1], ast.Expr):
final_expr = ast.Expression()
final_expr.body = field_[1].pop().value
ld = {}
rv = None
exec(compile(ast_, '<code>', 'exec'), None, ld)
if final_expr:
rv = eval(compile(final_expr, '<code>', 'eval'), None, ld)
print('got locals: {}'.format(ld))
print('got return: {}'.format(rv))
It'll eval instead of exec the last clause if it's an expression, or have all execed and return None.
Output:
got locals: {'tf': <function tf at 0x10103a268>, 'r': 9}
got return: {'vvv': 25}

PyCharm show full diff when unittest fails for multiline string?

I am writing some Python unit tests using the "unittest" framework and run them in PyCharm. Some of the tests compare a long generated string to a reference value read from a file. If this comparison fails, I would like to see the diff of the two compared strings using PyCharms diff viewer.
So the the code is like this:
actual = open("actual.csv").read()
expected = pkg_resources.resource_string('my_package', 'expected.csv').decode('utf8')
self.assertMultiLineEqual(actual, expected)
And PyCharm nicely identifies the test as a failure and provides a link in the results window to click which opens the diff viewer. However, due to how unittest shortens the results, I get results such as this in the diff viewer:
Left side:
'time[57 chars]ercent
0;1;1;1;1;1;1;1
0;2;1;3;4;2;3;1
0;3;[110 chars]32
'
Right side:
'time[57 chars]ercen
0;1;1;1;1;1;1;1
0;2;1;3;4;2;3;1
0;3;2[109 chars]32
'
Now, I would like to get rid of all the [X chars] parts and just see the whole file(s) and the actual diff fully visualized by PyCharm.
I tried to look into unittest code but could not find a configuration option to print full results. There are some variables such as maxDiff and _diffThreshold but they have no impact on this print.
Also, I tried to run this in py.test but there the support in PyCharm was even less (no links even to failed test).
Is there some trick using the difflib with unittest or maybe some other tricks with another Python test framework to do this?

The TestCase.maxDiff=None answers given in many places only make sure that the diff shown in the unittest output is of full length. In order to also get the full diff in the <Click to see difference> link you have to set MAX_LENGTH.
import unittest
# Show full diff in unittest
unittest.util._MAX_LENGTH=2000
Source: https://stackoverflow.com/a/23617918/1878199

Well, I managed to hack myself around this for my test purposes. Instead of using the assertEqual method from unittest, I wrote my own and use that inside the unittest test cases. On failure, it gives me the full texts and the PyCharm diff viewer also shows the full diff correctly.
My assert statement is in a module of its own (t_assert.py), and looks like this
def equal(expected, actual):
msg = "'"+actual+"' != '"+expected+"'"
assert expected == actual, msg
In my test I then call it like this
def test_example(self):
actual = open("actual.csv").read()
expected = pkg_resources.resource_string('my_package', 'expected.csv').decode('utf8')
t_assert.equal(expected, actual)
#self.assertEqual(expected, actual)
Seems to work so far..

A related problem here is that unittest.TestCase.assertMultiLineEqual is implemented with difflib.ndiff(). This generates really big diffs that contain all shared content along with the differences. If you monkey patch to use difflib.unified_diff() instead, you get much smaller diffs that are less often truncated. This often avoids the need to set maxDiff.
import unittest
from unittest.case import _common_shorten_repr
import difflib
def assertMultiLineEqual(self, first, second, msg=None):
"""Assert that two multi-line strings are equal."""
self.assertIsInstance(first, str, 'First argument is not a string')
self.assertIsInstance(second, str, 'Second argument is not a string')
if first != second:
firstlines = first.splitlines(keepends=True)
secondlines = second.splitlines(keepends=True)
if len(firstlines) == 1 and first.strip('\r\n') == first:
firstlines = [first + '\n']
secondlines = [second + '\n']
standardMsg = '%s != %s' % _common_shorten_repr(first, second)
diff = '\n' + ''.join(difflib.unified_diff(firstlines, secondlines))
standardMsg = self._truncateMessage(standardMsg, diff)
self.fail(self._formatMessage(msg, standardMsg))
unittest.TestCase.assertMultiLineEqual = assertMultiLineEqual

How to split python script in parts and to import the parts in a loop?

First, sorry for my stupid title :) And here is my problem.. Actually it's not a problem. Everything works, but I want to have better structure...
I have a python script with a loop "looped" each second.
In the loop there are many many IFs. Is it possible to put each IF in a separate file and then to include it in the loop? So this way every time the loop is "looped", all the IFs will be passed, too..
There are too many conditions in my script and all of them are different generally from the otheres so I want to have some kind of folder with modules - mod_wheather.py, mod_sport.py, mod_horoscope.py, etc..
Thanks in advance. I hope I wrote everything understandable..
EDIT:
Here is a structural example of what I have now:
while True:
if condition=='news':
#do something
if condition=='sport':
#so something else
time.sleep(1)
It will be good if I can have something like this:
while True:
import mod_news
import mod_sport
time.sleep(1)
And these IFs from the first example to be separated in files mod_news.py, mod_sport.py...

perhaps you wonder how to work with your own modules in general.
make one file named 'weather.py' and have it contain the appropriate if-statements like:
""" weather.py - conditions to check """
def check_all(*args, **kwargs):
""" check all conditions """
if check_temperature(kwargs['temperature']):
... your code ...
def check_temperature(temp):
-- perhaps some code including temp or whatever ...
return temp > 40
same for sport.py, horoscope.py etc
then your main script would look like:
import time, weather, sport, horoscope
kwargs = {'temperature':30}
condition = 'weather'
while True:
if condition == 'weather':
weather.check_all(**kwargs)
elif condition == 'sport':
sport.check_all()
elif condition == 'horoscope':
horoscope.check_all()
time.sleep(1)
edit: edited according to the edit in your question. Note that I suggest importing all modules only one time, at the beginning of the script, and using its functions. This is better than executing code by importing. But if you insist, you could use reload(weather), which actually performs a reload including code execution. But I cannot stress too much that using functions of external modules is a better way to go!

Put them in functions in separate files and then Import them:
"""thing1.py
A function to demonstrate
"""
def do_things(some_var):
print("Doing things with %s" % (some_var))
``
"""thing2.py
Demonstrates the same thing with a condition
"""
def do_things(some_var):
if len(some_var) < 10:
print("%s is < 10 characters long" % (some_var))
else:
print("too long")
``
"""main_program.py"""
import thing1, thing2
myvar = "cats"
thing1.do_things(myvar)
thing2.do_things(myvar)

I believe you are looking for some kind of PHP-like include() or C prepocessor #include. You would have a file such as the included.py below:
a = 2
print "ok"
and another file which has the following code:
for i in values:
import included
and you want the result to be equivalent to
for i in values:
a = 2
print "ok"
Is it what you are looking for? If so... no, it is not possible. Once Python imports a module, the code of the module is executed and following imports of the same mode only retrieve the already imported instance of the module. The code of a module is not executed everytime it is imported.
I can invent some crazy ways of doing it (let us say, file.read() + eval(), or calling reload() in an imported module.) but it would be a bad idea anyway. I bet we can think of a better solution to your real problem :)

Perhaps all you need is to call functions in your loop; and have those functions in other modules, which you import as needed.
while true:
if condition:
from module_a import f
f()
if condition2
from module_b import g
g()
Though the above is legal Python, and so answers your question, you should in practice write all the imports at the top of your file.

You could import the needed modules if they're needed, for example:
if condition:
import weather
... do something
However I'm not sure if that's what you really want.

I have a python script with a loop "looped" each second. In the loop
there are many many IFs.
Then you must optimize the repeatedly executed tests. Suppose there are 50 IFs blocks in your code and that in a turn of the for-loop, the N th condition is True: that means that the N-1 other conditions must be tested before the N th is tested and triggers the execution of the corresponding code.
It would be preferable to do so:
# to_include.py
def func_weather(*args,**kwargs):
# code
return "I'm the weather"
def func_horoscope(*args,**kwargs):
# code
return "Give me your birth'date"
def func_gastronomy(*args,**kwargs):
# code
return 'Miam crunch'
def func_sports(*args,**kwargs):
# code
return 'golf, swimming and canoeing in the resort station'
didi = {'weather':func_weather, 'horoscope':func_horoscope,
'gastronomy':func_gastronomy, 'sports':func_sports}
and the main module:
# use_to_include.py
import to_include
x = 'sports'
y = to_include.didi[x]()
# instead of
# if x =='weather' : y = func_weather()
# elif x=='horoscope' : y = func_horoscope()
# elif x=='gastronomy': y = func_gastronomy()
# elif x=='sports' : y = func_sports()
print y
result
golf, swimming and canoeing in the resort station