I'll do my best to describe the issue I am having. I am building a Python program that is built on multiple classes and uses the unittest framework. In a nutshell, the Main.py file has a "ValidateDriver" class that defines a "driver" variable as an ElementTree type. If I point this directly to the XML file I need to parse, (i.e. driver = ElementTree.parse(rC:\test.xml)) then I can access it from another class. However, in reality I don't have the actual XML file that is passed in from the command-line until you get to the Main function in the ValidateDriver class. So under the ValidateDriver class driver would really be driver = ElementTree and then in the main function I would reassign that variable to ValidateDriver.driver = ElementTree.parse(args.driver). However, this is the crux. When I go to the other class and try to call ValidateDriver.driver I don't have the "findall" method/attribute available. Again, the only way it will work is to do something like: ElementTree.parse(rC:\test.xml)). If I did this in C# it would work, but I am new to Python and this is kicking my butt. Any help/suggestions is appreciated. I've included the code for both classes.
Main Function:
import sys
import argparse
import xml.etree.ElementTree as ElementTree
import unittest
import Tests.TestManufacturer
class ValidateDriver:
driver = ElementTree
def main(argv):
parser = argparse.ArgumentParser(description='Validation.')
parser.add_argument('-d', '--driver', help='Path and file name xml file', required=True)
parser.add_argument('-v', '--verbosity',
help='Verbosity for test output. 1 for terse, 2 for verbose. Default is verbose',
default=2, type=int)
#args = parser.parse_args()
args = r'C:\test.c4i'
#print ("Validate Driver: %s" % args.driver)
#print ("Verbosity Level: %s" % args.verbosity)
ValidateDriver.driver = ElementTree.parse(r'C:\test.c4i')
loader = unittest.TestLoader()
suite = loader.loadTestsFromModule(Tests.TestManufacturer)
runner = unittest.TextTestRunner(verbosity=2) # TODO Remove this...
# TODO Uncomment this...
runner = unittest.TextTestRunner(verbosity=args.verbosity)
result = runner.run(suite)
if __name__ == "__main__":
main(sys.argv[1:])
Other Class, Test Manufacturer:
import unittest
import Main
manufacturer = ['']
class Tests(unittest.TestCase):
# Test to see if Manufacturer exists.
def test_manufacturer_exists(self):
for m in Main.ValidateDriver.driver.findall('./manufacturer'):
print m.text
Producing the following error:
C:\Python27\python.exe C:\Users\test\PycharmProjects\Validator\Main.py
Traceback (most recent call last):
File "C:\Users\test\PycharmProjects\Validator\Main.py", line 22, in <module>
class ValidateDriver:
File "C:\Users\test\PycharmProjects\Validator\Main.py", line 65, in ValidateDriver
main(sys.argv[1:])
File "C:\Users\test\PycharmProjects\Validator\Main.py", line 36, in main
ValidateDriver.driver = ElementTree.parse(r'C:\test.c4i')
NameError: global name 'ValidateDriver' is not defined
Process finished with exit code 1
The main problem seems to be that your main script is wrapped in a class. There's really no reason for this, and is quite confusing.
if __name__ == "__main__":
main_object = ValidateDriver()
main_object.main(sys.argv[1:])
This should go outside the class definition
This has nothing to do with "findall" being available. The issue is that the class itself hasn't yet been completely declared at the time you try to access it. In python, the file is read top to bottom. For example, this is not allowed:
if __name__ == "__main__":
f()
def f():
...
The call to f must happen at the bottom of the file after it is declared.
What you're doing with ValidateDriver is similar, because the class isn't defined until the statements directly in its body are executed (this is different from functions, whose bodies of course aren't executed until they are called). You call main(sys.argv[1:]) inside the class body, which in turn tries to access ValidateDriver.driver, which doesn't exist yet.
Preferably, the main function, as well as the code which calls it, should be outside the class. As far as I can tell, the class doesn't need to exist at all (this isn't C# or Java -- you can put code directly at the module level without a class container). If you insist on putting it in a class as a static method, it must be defined as a class method:
#classmethod
def main(cls, argv):
...
which can then be called (outside the class definition) like:
ValidateDriver.main(sys.argv[1:])
But I stress that this is non-standard and should not be necessary.
Related
I have three files that all contain classes with the same names but slightly different definitions. Some methods in these classes are identical across all three files, so I abstracted them out to another file, utils.py, where they are defined within a "template" version of the original class. The problem is that these methods invoke functions and modules that exist in the original files but not this new one.
My original approach was to use multiple class inheritance, which would initialize the template class within the scope of the parent class, allowing access to all the functions and modules it requires. However, I was instructed to avoid multiple class inheritance and to simply import the utils file.
Importing does not apply the same scoping logic as mentioned above with inheritance. So here arises my problem. I have created a small example to show what I mean. I am using a module called datajoint. You don't need to know much about it except that a schema is basically a table or collection of tables in a database.
schemas.py
import datajoint as dj
from datetime import datetime
import utils
dj.conn()
schema = dj.Schema('adib_example1')
schema.drop()
schema = dj.Schema('adib_example1')
def test_print():
print("test")
#schema
class Subject(dj.Lookup):
definition = """
subject_id: int
"""
contents = [dict(subject_id=1)]
#schema
class Session(dj.Computed):
definition = """
-> Subject
time: varchar(30)
"""
def make(self, key):
utils.SessionTemplate.make(self,key)
Session.populate() # invokes Session's make(), passing Subject's primary key
Approach 1
Import scoping not working like inheritance
utils.py
class SessionTemplate():
#staticmethod
def make(table, key):
test_print() # parent function usage example
table.time = f"{datetime.now()}" # parent module usage example
new_entry = dict(**key, time=table.time)
table.insert1(new_entry)
error
Traceback (most recent call last):
File "/home/.anaconda/imported_make/schemas.py", line 30, in <module>
Session.populate() # invokes Session's make(), passing Subject's primary key
File "/opt/conda/lib/python3.9/site-packages/datajoint/autopopulate.py", line 153, in populate
make(dict(key))
File "/home/.anaconda/imported_make/schemas.py", line 28, in make
utils.SessionTemplate.make(self,key)
File "/home/.anaconda/imported_make/utils.py", line 5, in make
test_print() # parent function usage example
NameError: name 'test_print' is not defined
Approach 2
Importing schemas.py into utils.py works, but requires including schemas. before every imported function and module, which is not practical in my case.
utils.py
import schemas
class SessionTemplate():
#staticmethod
def make(table, key):
schemas.test_print() # parent function usage example
table.time = f"{schemas.datetime.now()}" # parent module usage example
new_entry = dict(**key, time=table.time)
table.insert1(new_entry)
Approach 3
Import using * to avoid having to add schemas. before each parent function/module somehow does not provide access to the parents modules and functions.
from schemas import *
class SessionTemplate():
#staticmethod
def make(table, key):
test_print() # parent function usage example
table.time = f"{datetime.now()}" # parent module usage example
new_entry = dict(**key, time=table.time)
table.insert1(new_entry)
error
Traceback (most recent call last):
File "/home/.anaconda/imported_make/run.py", line 1, in <module>
import schemas
File "/home/.anaconda/imported_make/schemas.py", line 30, in <module>
Session.populate() # invokes Session's make(), passing Subject's primary key
File "/opt/conda/lib/python3.9/site-packages/datajoint/autopopulate.py", line 153, in populate
make(dict(key))
File "/home/.anaconda/imported_make/schemas.py", line 28, in make
utils.SessionTemplate().make(self,key)
File "/home/.anaconda/imported_make/utils.py", line 7, in make
test_print() # parent function usage example
NameError: name 'test_print' is not defined
I know import * is bad practice, but it would have been fine in this instance if it worked, and I'm not sure why it doesn't.
boss.py
class tasks():
def job1(input):
// do something
return output
def job2(input):
// do something
return output
worker.py
import boss.tasks
from boss.tasks import job1, job2
input_value = "xyz"
output1 = boss.tasks().job1(input_value)
output2 = boss.tasks().job2(input_value)
I have the following code:
theModule = locate(<the location of my python script in the file system>)
class_ = getattr(theModule, '<the class of my module>')
myAppli = class_()
But I always have an AttributeError exception when trying to perform the "getattr" method.
My objective was to create a Python class from a file somewhere in the file system, and execute methods on it by reflection without using the class declaration in the Python code.
Example
For example, I have the following python script, let's say that the name of the script file is: myScript.py
import sys
class PythonAppli:
def aMethod(self):
sys.echo('I am here\n')
And I perform this in another script in the same directory:
from pydoc import locate
theModule = locate('myScript.py')
class_ = getattr(theModule, 'pythonAppli')
myAppli = class_()
I have the following exception on the "getattr" line:
AttributeError: class PythonAppli has no attribute 'pythonAppli'
Note that if I add this line before my "getattr" method call:
inspect.getmembers(theModule)
I find 'pythonAppli' in the list of attributes.
I looked on the question about dynamic instanciation on StackOverflow but could not find my error here.
Per MisterMiyagi answer, my mistake was theModule was already my desired class. The following code did work:
from pydoc import locate
theModule = locate('myScript.py')
pythonAppli = theModule()
So it was much more simple that I imagined.
When I use HTMLTestRunner for Python 3.5，it shows an error.
I have changed the HTMLTestRunner for support python 3.5.
The code :
import pymysql
import pymysql
import unittest
import time
import unittest.suite
import HTMLTestRunner
import sys
def hell(a):
print(a)
return a
testunit = unittest.TestSuite()
testunit.addTest(hell('ad'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'print', description=u'简单')
runner.run(testunit)
When I run it, I got this error:
Traceback (most recent call last):
File "/Applications/Python 3.5/……/B.py", line 30, in <module>
testunit.addTest(hell('ad'))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/suite.py", line 47, in addTest
raise TypeError("{} is not callable".format(repr(test)))
TypeError: 'ad' is not callable
What should I do to make the script works?
You're adding the result of the call to hell('ad') to your tests, not the hell function itself. Since the hell function returns its argument, it returns the string ad, which is not a callable (function or the like).
Use
testunit.addTest(hell)
instead.
What about that argument then, how do you pass that?
Well, there are ways to do that, but generally, try not to let your unit test functions take an argument. Thus, hell() should siply not take an argument.
If you code your unit test correctly, you'll find that you rarely need to pass it an argument.
The line testunit.addTest(hell('ad')) doesn't do what you intend it to do. It doesn't tell the test suite to run hell('ad') later on. Rather, it calls that function immediately and passed the return value (which happens to be the string 'ad' you gave it as an argument) to addTest. That causes the exception later on, since a string is not a valid test case.
I'm not exactly sure how you should fix this. Your hell function doesn't appear to actually test anything, so there's not an obvious way to transform it into a proper test. There is a unittest.FunctionTestCase class that wraps a function up as a TestCase, but it doesn't appear to have any way of passing arguments to the function. Probably you should write a TestCase subclass class, and add various test_-prefixed methods that actually test things.
I had a right answer for my question, give the code:
# -*- coding: utf-8 -*-
import unittest
import HTMLTestRunner
def hell(a):
print(a)
return a
class HellTest(unittest.TestCase):
def setUp(self):
self.hell = hell
def tearDown(self):
pass
def testHell(self):
self.assertEqual(self.hell('aaa'), 'aaa')
if __name__ == '__main__':
testunit = unittest.TestSuite()
testunit.addTest(HellTest('testHell'))
filename = '/Users/vivi/Downloads/aa.html'
fp = open(filename, 'wb')
runner = HTMLTestRunner.HTMLTestRunner(stream=fp, title=u'不要生成error啦！', description=u'简单1112')
runner.run(testunit)
fp.close()
But, I do not know why add the class code 'class HellTest()'.The answer comes from a Chinese people whoes name is '幽谷奇峰'。Source code reference：https://segmentfault.com/q/1010000007427143?_ea=1345414
I am attempting to use multiprocessing to call derived class member function defined in a different module. There seem to be several questions dealing with calling class methods from the same module, but none from different modules. For example, if I have the following structure:
main.py
multi/
__init__.py (empty)
base.py
derived.py
main.py
from multi.derived import derived
from multi.base import base
if __name__ == '__main__':
base().multiFunction()
derived().multiFunction()
base.py
import multiprocessing;
# The following two functions wrap calling a class method
def wrapPoolMapArgs(classInstance, functionName, argumentLists):
className = classInstance.__class__.__name__
return zip([className] * len(argumentLists), [functionName] * len(argumentLists), [classInstance] * len(argumentLists), argumentLists)
def executeWrappedPoolMap(args, **kwargs):
classType = eval(args[0])
funcType = getattr(classType, args[1])
funcType(args[2], args[3:], **kwargs)
class base:
def multiFunction(self):
mppool = multiprocessing.Pool()
mppool.map(executeWrappedPoolMap, wrapPoolMapArgs(self, 'method', range(3)))
def method(self,args):
print "base.method: " + args.__str__()
derived.py
from base import base
class derived(base):
def method(self,args):
print "derived.method: " + args.__str__()
Output
base.method: (0,)
base.method: (1,)
base.method: (2,)
Traceback (most recent call last):
File "e:\temp\main.py", line 6, in <module>
derived().multiFunction()
File "e:\temp\multi\base.py", line 15, in multiFunction
mppool.map(executeWrappedPoolMap, wrapPoolMapArgs(self, 'method', range(3)))
File "C:\Program Files\Python27\lib\multiprocessing\pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "C:\Program Files\Python27\lib\multiprocessing\pool.py", line 567, in get
raise self._value
NameError: name 'derived' is not defined
I have tried fully qualifying the class name in the wrapPoolMethodArgs method, but that just gives the same error, saying multi is not defined.
Is there someway to achieve this, or must I restructure to have all classes in the same package if I want to use multiprocessing with inheritance?
This is almost certainly caused by the ridiculous eval based approach to dynamically invoking specific code.
In executeWrappedPoolMap (in base.py), you convert a str name of a class to the class itself with classType = eval(args[0]). But eval is executed in the scope of executeWrappedPoolMap, which is in base.py, and can't find derived (because the name doesn't exist in base.py).
Stop passing the name, and pass the class object itself, passing classInstance.__class__ instead of classInstance.__class__.__name__; multiprocessing will pickle it for you, and you can use it directly on the other end, instead of using eval (which is nearly always wrong; it's code smell of the strongest sort).
BTW, the reason the traceback isn't super helpful is that the exception is raised in the worker, caught, pickle-ed, and sent back to the main process and re-raise-ed. The traceback you see is from that re-raise, not where the NameError actually occurred (which was in the eval line).
If you create a new python module in eclipse, you have the option to use a argparse template for your new module. Below are code fragments from this template.
#!/usr/local/bin/python2.7
# encoding: utf-8
'''
eclipse_argparse -- shortdesc
eclipse_argparse is a description
It defines classes_and_methods
#author: user_name
#copyright: 2015 organization_name. All rights reserved.
#license: license
#contact: user_email
#deffield updated: Updated
'''
def main(argv=None): # IGNORE:C0111
'''Command line options.'''
if argv is None:
argv = sys.argv
else:
sys.argv.extend(argv)
...
program_shortdesc = __import__('__main__').__doc__.split("\n")[1]
...
if __name__ == "__main__":
sys.exit(main())
'...' means that I omitted the parts of the code that seem not important to my question.
I saved the file under eclipse_argparse.py.
As far as I understand, the main function is given the argument argv so that one can call the main function interactively from the python shell and give it a parameter:
>>> import eclipse_argparse
>>> eclipse_argparse.main('-bla')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "eclipse_argparse.py", line 57, in main
program_shortdesc = __import__('__main__').__doc__.split("\n")[1]
AttributeError: 'NoneType' object has no attribute 'split'
But this is not possible because __import__('__main__') evaluates to __main__, and not to eclipse_argparse as I would expect.
program_shortdesc should then evaluated to eclipse_argparse -- shortdesc
Why does this happen? Is my assumption, that you should be able to call the main method with an argument in an interactive session wrong?
Within a module, its own doc is available as __doc__. You don't need to import anything, or reference __main__ (which in the interactive shell points to shell environment, not any imported module).
This works when main is called from if __name__ and from a shell:
def main(argv=None): # IGNORE:C0111
# ...
print __doc__
In a shell I can do:
>>> import stack27864002
>>> stack27864002.__doc__
'\neclipse_argparse -- ...'
>>> stack27864002.main()
# same
I wonder why this is called a argparse template, since it has nothing to do with argparse.py module.
The default value for __doc__ is None, so it is a good idea to be prepared for that case - i.e. don't try to split it without first checking.
To make the code work in an interactive shell session, the code
program_shortdesc = __import__('__main__').__doc__.split("\n")[1]
should be changed to:
if __name__ == '__main__':
program_shortdesc = __import__('__main__').__doc__.split("\n")[1]
else:
program_shortdesc = __doc__.split("\n")[1]
What confused me was that they provide a main method with the possibility to supply an argument (which implies that you could import the module in a python shell and call the main function with some parameter, like import eclipse_argparse; eclipse_argparse.main('-some_parameter')) and then define a variable program_shortdesc that only works when you call the program through the '__main__' 'entry point', like python eclipse_argparse.py