I have a folder with all my unittests. They all include:
if __name__ == '__main__':
unittest.TextTestRunner(verbosity=2).run(suite())
So to test them I only have to import the test script. I have a test_all script that does all the unittests by importing them one by one. Currently this looks like this:
from pyMS.test import test_baseFunctions
from pyMS.test import test_compareFeatureXMLmzML
from pyMS.test import test_config
from pyMS.test import test_featureFunctions
from pyMS.test import test_fileHandling
from pyMS.test import test_mzmlFunctions
from pyMS.test import test_output
from pyMS.test import test_parseFeatureXML
from pyMS.test import test_rFunctions
from pyMS.test import test_rPlots
[...]
This means that every time I add or remove a new test I need to change the imports. So instead I want to use
from pyMS.test import *
However, this does not run any of the code. I'm curious for the reason why import * does not run the code.
Also, if someone knows a solution (that is note nose) to run all unittests without having to import them one by one would be great.
Thanks
Niek
If you use python 2.7 you can use the command line:
python -m unittest discover
This will automatically find and execute all tests in all subdirectories. For more options, see:
python -m unittest -h
This module has been backported to python 2.3+ and can be downloaded here. If you use the backport there's an included command line script called unit2 or unit2.py (your choice), invoked like this:
unit2.py discover
As for from XXX import *, this actually imports everything in the namespace of the file XXX/__init__.py. Put the following code in __init__.py to automatically load any direct submodules:
import os
all_files = os.listdir(os.path.dirname(__file__))
modules = [x for x in all_files if x.endswith(".py") and not x.startswith("_")]
__all__ = [x.rpartition(".")[0] for x in modules]
A detailed explanation of how this works can be found in the python docs for the __all__ global variable.
The easiest way (without using an external tool) to run all the tests is probably to use TestLoader.discover.
__name__ is only set to "__main__" for the initial python file being read by the interpreter. This allows the module to be imported by other modules without the code after the if __name__ == "__main__": being executed.
Any code that is not protected by the if __name__ == "__main__": will be executed. So you could remove it in each of the files then when you do the import unittest.TextTestRunner(verbosity=2).run(suite()) will be executed.
A better method is to use a method from unittest.TestLoader() to load your tests into a suite and then give that suite to unittest.TextTestRunner. The loader can then be automated without needing to change the imports in the test file. Add a test file to the directory structure and the tests will be executed automatically.
Related
I am trying to make my python script more modular -- it works properly when everything is in just one lengthy .py file, but I want to use the main python file to call other files to streamline the flow and make upgrades easier.
I'm struggling with package imports. One package I'm using is os, which I import in the main file:
import os
import pandas as pd
import numpy as np
from code_module_1 import *
if __name__ == '__main__':
code_module_1()
I also import it at the top of the python file that is called, code_module_1.py:
import os
import glob
import pandas as pd
import numpy as np
def function_called_from_main():
for root, dirs, files in os.walk('...file_path'):
etc.
When I do this, I receive an error saying the name 'os' is not defined, which I've taken to mean that the package isn't being imported in code_module_1.
I've attempted to fix this by placing the lines of code that import packages inside of the function that I'm calling from the main script, but I still run into the same error. Where should I be importing packages, and how do I make sure that other python files that are called have the packages that they need to run?
With the following line
from code_module_1 import *
you import everything defined in the module. So basically, all function definitions are available.
Next, you can't execute a module. You can only execute a function or a statement. Thus, the following line
if __name__ == '__main__':
code_module_1()
should be something like this:
if __name__ == '__main__':
function_called_from_main()
where you execute the function function_called_from_main that was previously imported.
My directory structure:
test.py
module/
importer.py
importee.py
__init__.py
So in my directory, I have test.py, then another directory which has been initialized as a module. Within that module, there is a file importer.py which imports a file importee.py. In order to test whether the import works, I made a simple function in importee.py and tried using it in importer.py (i.e. I ran importer.py directly); it worked just fine.
But when I go into test.py and have the import statement from module import * and try to run that (without any other code), it gives an error which traces back to the import statement in importer.py, saying No module named 'importee'
If it matters, the __init__.py in the module directory has the __all__ function specified properly.
I like to think this isn't a duplicate despite there being similarly titled posts, such as this or this or this or this; I've been searching for hours and still have no idea what could be causing this.
Any ideas? Any help is greatly appreciated.
Edit: content of the four files:
init.py
__ all __ = ["importee", "importer"]
importee.py
def example():
print("hey")
importer.py
from importee import *
example()
test.py
from module import *
When I run importer.py I get no errors, but when I run test.py I get a error which traces back to the first line of importer.py saying that No module named 'importee' found, even though I don't get that error when running importer.py directly...
The following runs and prints "hey" regardless of if you run python test.py from root or python importer.py from module.
You can learn more about relative imports from PEP 328. You can learn more about init here, but the important thing I want you to take away from this is that __init__ runs when you import from module.
Furthermore defining __all__ overrides identifiers that begin with _, and since you aren't using them I don't actually know that it would have any effect.
# test.py
from module import *
# module/__init__.py
from .importer import *
# module/importee.py
def example():
print("hey")
# module/importer.py
from .importee import *
example()
Say I am using Python 3 (and hence absolute imports) and my directory structure looks like this:
> package:
> sub_directory
__init__.py
sub_dir_file.py
sub_dir_file2.py
__init__.py
main_dir_file.py
In the file sub_dir_file.py I wish to import a function from sub_dir_file2.py. The catch is, I want to be able to run sub_dir_file.py with __name__ == '__main__', as well as import it in main_dir_file.py. Hence, if in sub_dir_file.py I use a relative import:
from .sub_dir_file2 import some_function
the module executes perfectly fine when run from main_dir_file.py, but throws an error when executed directly (as the relative import cannot be executed when __name__ == '__main__'. If I however use a normal absolute import, sub_dir_file.py will execute as a main, but cannot be imported from main_dir_file.py.
What would be the most elegant way of solving this problem? One obvious solution seems to be:
if __name__ == '__main__':
from sub_dir_file2 import some_function
else:
from .sub_dir_file2 import some_function
However, it doesn't seem very pythonic.
You should use the relative import syntax from .sub_dir_file2 import some_function or eventually the absolute syntax from package.sub_directory.sub_dir_file2 import some_function.
Then, in order to call one of the package sub module, it is simpler to use the -m option of the python interpreter to execute its content as the __main__ module.
Search sys.path for the named module and execute its contents as the
main module.
Since the argument is a module name, you must not give a file
extension (.py). The module name should be a valid absolute Python
module name, but the implementation may not always enforce this (e.g.
it may allow you to use a name that includes a hyphen).
For example:
> python -m package.main_dir_file
> python -m package.sub_directory.sub_dir_file
I would suggest using a main() function invoked if name is __main__. It's a good habit anyway, as far as I'm aware.
That way you can just call the imported module's main() yourself. It has other benefits, too, like allowing you to test or re-invoke the module without necessarily executing the file every time.
I am working on project euler and wanted to time all of my code. What I have is directory of files in the form 'problemxxx.py' where xxx is the problem number. Each of these files has a main() function that returns the answer. So I have created a file called run.py, located in the same directory as the problem files. I am able to get the name of the file through command prompt. But when I try to import the problem file, I continue to get ImportError: No module named problem. Below is the code for run.py so far, along with the command prompt used.
# run.py
import sys
problem = sys.argv[1]
import problem # I have also tired 'from problem import main' w/ same result
# will add timeit functions later, but trying to get this to run first
problem.main()
The command prompts that I have tried are the following: (both of which give the ImportError stated above)
python run.py problem001
python run.py problem001.py
How can I import the function main() from the file problem001.py? Does importing not work with the file name stored as a variable? Is there a better solution than trying to get the file name through command prompt? Let me know if I need to add more information, and thank you for any help!
You can do this by using the __import__() function.
# run.py
import sys
problem = __import__(sys.argv[1], fromlist=["main"]) # I have also tired 'from problem import main' w/ same result
problem.main()
Then if you have problem001.py like this:
def main():
print "In sub_main"
Calling python run.py problem001 prints:
In sub_main
A cleaner way to do this (instead of the __import__ way) is to use the importlib module. Your run.py needs to changes:
import importlib
problem = importlib.import_module(sys.argv[1])
Alternatives are mentioned in this question.
For sure! You can use __ import_ built-in function like __import__(problem). However this is not recommended to use, because it is not nice in terms of coding-style. I think if you are using this for testing purposes then you should use unittest module, either way try to avoid these constructions.
Regards
You can use exec() trick:
import sys
problem = sys.argv[1]
exec('import %s' % problem)
exec('%s.main()' % problem)
I have a Python project with following directory structure:
/(some files)
/model/(python files)
/tools/(more python files)
...
So, I have Python files in couple subdirectories and there are some
dependencies between directories as well: tools are used by model, etc. Now
my problem is that I want to make doctests for both models and tools,
and I want be able to run tests from command line like this: ./model/car.py .
I can make this work, but only with messy boilerplate code. I would like
to know what is the correct way, or is there any?
Question: How should I write my imports?
Thanx. Here is an example...
Content of tools/tool.py:
#!/usr/bin/env python
"""
>>> is_four(21)
False
>>> is_four(4)
True
"""
def is_four(val):
return val == 4
if __name__ == '__main__':
import doctest
doctest.testmod()
... and model/car.py:
#!/usr/bin/env python
"""
>>> car = Car()
>>> car.ok()
True
"""
from tools.tool import *
class Car(object):
def __init__(self):
self.tire_count = 4
def ok(self):
return is_four(self.tire_count)
if __name__ == '__main__':
import doctest
doctest.testmod()
By adding following lines in the begin of car.py it works, but doesn't look nice. :(
if __name__ == '__main__':
import sys
import os
sys.path.append(os.path.abspath(os.path.dirname('..')))
What you are trying to do is a relative import. It works fine in Python, but on the module level, not on the file system level. I know, this is confusing.
It means that if you run a script in a subdir, it doesn't see the upper dirs because for the running script, the root of the module is the current dir: there is no upper module.
So what are relative imports for?
Well, module in subdirs car import module in upper dirs as long as they are themself imported from a upperdir.
In your case it means you must run your scripts from "/" so it becomes the root of the module, and the submodules are allowed to use relative import.
A possible solution to your problem is to remove your if __name__ == "__main__" block and create /tests.py:
import doctest
from model import car
from tools import tool
doctest.testmod(car)
doctest.testmod(tool)
Then run in too launch all the tests.
Ultimately you will want to automatize the process, a simple solution is to use unittest so you can create test suites and just add the module names you want to test:
import unittest
import doctest
modules = ("model.car",
"tools.tool")
suite = unittest.TestSuite()
for mod in modules:
suite.addTest(doctest.DocTestSuite(mod))
runner = unittest.TextTestRunner()
runner.run(suite)
Another solution (recommended) is to use a tool such as nose that automates this for you.
easy_install nose
nosetests --with-doctest # done :-)
And by the way, avoid from x import *. This works for quick scripts, but when your program will grow, you really will need to explicitly name what you import. Either import x or from x import y
Use packages. Add an __init__.py file to your working directory and all subfolders then your imports will search the parent directories if it doesn't find the module in the current directory.
See http://www.network-theory.co.uk/docs/pytut/Packages.html
Also this question is a duplicate of:
Import a module from a relative path
Don't frob sys.path in this manner. Instead either use $PYTHONPATH to force the base directory in when invoking python, or use python -m model.car from that directory.