i'm using python 2.7 and trying to gather documentation for our testing project using pdoc.
pdoc is located here: D:\dev\Python27\Scripts
the regression project here: C:\views\md_LDB_RegressionTests_v03.1_laptop\mts\Tests\LDB\Regression\Tests
We are using proboscis for our tests and i'm trying to create html documentation for the separate group of tests, a separate python file in my case.
I run such command:
D:\dev\Python27\Scripts>python pdoc --html "C:\views\md_LDB_RegressionTests_v03.
1_laptop\mts\Tests\LDB\Regression\Tests\tests\check_system_management\check_capa
bilities_encoding_problems.py"
and get such answer:
Traceback (most recent call last):
File "pdoc", line 458, in <module>
module = imp.load_source('__pdoc_file_module__', fp, f)
File "C:\views\md_LDB_RegressionTests_v03.1_laptop\mts\Tests\LDB\Regression\Te
sts\tests\check_system_management\check_capabilities_encoding_problems.py", line
4, in <module>
from common.builders.system_request import default_create_system, create_cap
ability
ImportError: No module named common.builders.system_request
pdoc can't import the function from other modules in our regression...
The structure of our project looks like this:
-Tests (C:\views\md_LDB_RegressionTests_v03.1_laptop\mts\Tests\LDB\Regression\Tests)
-"common" package (with init file)
-"builders" packege
-system_request.py
-"test" package
-check_system_management package
-check_capabilities_encoding_problems.py - this is the file i want to get documentation to
Of course there are lots of other packages but im not sure if it makes sense to describe all the structure now
The import part of the check_capabilities_encoding_problems.py looks like this:
import urllib
from hamcrest import assert_that, all_of
from proboscis import test, before_class, after_class
from common.builders.system_request import default_create_system, create_capability
from common.entity.LDBChecks import LDBChecks
How can i point to pdoc where to look for the functions of other modules?
thank you!
You can set PYTHONPATH env variable. This is a path that say python where to find modules and packages by 3th party also you.
When using pdoc with my Spyder IDE, I use the following script to add a directory to pdoc path
import pdoc
libpath = r'C:\Path\To\Module'
pdoc.import_path.append(libpath)
mod = pdoc.import_module('ModuleName')
doc = pdoc.Module(mod)
string = doc.html()
The pdoc.import_path is a list of currently used paths to look for your module; pdoc.import_path equals sys.path in default. More info can be found in pdoc documentation.
pydoc and pdoc read your code!!!
if you will run it from the same directory pdoc3 --html . or pydoc -w . it should work if all the modules are in the same directory. but if they are not:
make sure your main module in each directory has it sys full path append to it (to the same directory).
sys.path.append("D:/Coding/project/....)
Relative path will not do the trick!
Related
Opening and loading data from a file that is situated in the same folder as the currently executing Python 3.x script can be done like this:
import os
mydata_path = os.path.join(os.path.dirname(__file__), "mydata.txt")
with open(mydata_path, 'r') as file:
data = file.read()
However once the script and mydata.txt files become part of a Python package this is not as straight forward anymore. I have managed to do this using a concoction of functions from the pkg_resources module such as resource_exists(), resource_listdir(), resource_isdir() and resource_string(). I won't put my code here because it is horrible and broken (but it sort of works).
Anyhow my question is; is there no way to manage the loading of a file in the same folder as the currently executing Python script that works regardles of wether the files are in a package or not?
You can use importlib.resources.read_text in order to read a file that's located relative to a package:
from importlib.resources import read_text
data = read_text('mypkg.foo', 'mydata.txt')
So imagine i have an external lib called printer with one file that looks like this.
# in file ExternalLibPrinter.py
def main():
print("hello external lib")
if __name__ == "__main__":
main()
Now i've installed this via pip.
What I would like to do is run that main function. If this was a python file that in my directory i would simply run
python3 ExternalLibPrinter.py
but I cant find such a file. I've also tried
python3 printer/ExternalLibPrinter.py
This can't find that file either. So is this not possible?
Use this code to call the main function :
import printer.ExternalLibPrinter as printer
printer.main()
If however the main() function was in the module's __init__.py, you would call it this way :
import printer
printer.main()
If you want to know where are the library's files stored, do this :
$ python3
>>> import printer
>>> printer
<module 'printer' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/printer/__init__.py'>
In this example it shows where printer is installed on my computer, but that may be different with yours.
If the library is a single file, it will be called printer.py. If is is in multiple files, there will be a directory named printer which will contain :
__init__.py — File that is used when you write import printer
and for each submodule there will be a file submodulename.py that is used when you write import printer.submodulename.
I am trying to import modules while running my main python script, using a smaller setup.py script. However the importlib command: importlib.util.spec_from_file_location(name, location) doesn't appear to be detecting my small python script. Presumably I'm not filling in the name or location fields correctly.
Example Script A (setup.py):
import os
import pandas as pd
print("success!") # So I can see it has run.
Example Script B (my_script.py):
import importlib
setup_path = ("/home/solebay/My Project Name/")
start_up_script = importlib.util.spec_from_file_location("setup.py", setup_path)
module = importlib.util.module_from_spec(start_up_script)
Running the above snippet returns:
AttributeError: 'NoneType' object has no attribute 'loader'
I subsequently investigated by running type(start_up_script) the result it gives is typeNone.
The paths are correct. I verified this by running the following:
"/home/solebay/My Project Name/"
sudo python3 "/home/solebay/My Project Name/setup.py"
These printed the messages is a directory and success! respectively.
Note: Maurice Meyer succeeded in answering my main question, and so I have marked it as correct. However, I have not achieved my main goal; namely importing modules via another script. So if that is your aim, this question might not be appropriate for you.
The location argument passed to spec_from_file_location has to be the full path to the python script:
import importlib.util
spec = importlib.util.spec_from_file_location(
name='something__else', # name is not related to the file, it's the module name!
location='/tmp/solebay/My Project Name/setup.py' # full path to the script
)
my_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(my_mod)
print(my_mod)
Out:
success!
<module 'something__else' from '/tmp/solebay/My Project Name/setup.py'>
I am writing a code organized in some (several) files. For the sake of organization of folders and the CMakeLists.txt, a pythonlibs folder is created during the build process, and some links to python files are created to libraries in the /build/src/XXXX/ folder.
In the python file, I add to the python path:
sys.path.insert(1,'/opt/hpc/softwares/erfe/erfe/build/pythonlibs')
import libmsym as msym
When I run the main python file, there is this one library lybmsym that fails with:
import libmsym as msym
File "/opt/hpc/softwares/erfe/erfe/build/pythonlibs/libmsym.py", line 15, in <module>
from . import _libmsym_install_location, export
ImportError: attempted relative import with no known parent package
I created a link using cmake, but I believe it does use the ln command (tried both hard and symbolic). Is there a way to prevent this behavior without changing the library itself, just another way to create this link?
Thanks.
I'm trying to build an app that uses some xml data using Python's built-in xml.etree.ElementTree class. It works properly when I run from the command line, but when I build it, I get an error "ImportError: No module etree.ElementTree." I'm guessing this is because I'm not importing that module correctly, but I haven't been able to figure out how. When I use the "includes" or "packages" directive, py2app complains with the same error, and when I specifically specify the package_dir (/System/Library/...), it compiles, but still gives me the error. I've included a short example to illustrate the issue.
macxml.py
from xml.etree.ElementTree import ElementTree
if __name__ == '__main__':
tree = ElementTree()
print tree.parse('lib.xml')
This should print out "< Element Library at xxxxxx>" where Library is the root name.
setup.py
from setuptools import setup
setup(name="Mac XML Test",
app=['macxml.py'],
)
What is the correct way to make the mac app utilize this library?
Python 2.6.4
Mac OS X 10.6.2
Edit: I also tried this on another mac (PPC 10.5.8) with Python 2.6.2 and achieved the same results.
After reinstalling and updating macholib, modulegraph, py2app, and setuptools to no avail, I did a little more digging and found the following error in the modulegraph module:
graphmodule.find_modules.find_modules(includes=['xml.etree'])
Traceback (most recent call last):
File "<stdin>", line 1 in <module>
File ".../modulegraph/find_modules.py", line 255 in find_modules
File ".../modulegraph/find_modules.py", line 182 in find_needed_modules
File ".../modulegraph/modulegraph.py", line 401 in import_hook
File ".../modulegraph/modulegraph.py", line 464 in load_tail
ImportError: No module named xml.etree
So I looked more into the load_tail and import_hook functions and found that for some reason it was importing the xml package correctly, but then went to an old install of _xmlplus to look for the etree subpackage (which of course it couldn't find). Removing the _xmlplus package eliminated the error and I was able to get the application to work with the following setup.py file:
from setuptools import setup
import xml.etree.ElementTree
setup(name="Mac XML Test",
app=['macxml.py'],
options={'py2app': {'includes': ['xml.etree.ElementTree']}},
data_files=[('', ['lib.xml'])]
)
The output shows up in the console.
Since the comment doesn't have good formating,
In find_modules.py I changed
REPLACEPACKAGES = {
'_xmlplus': 'xml',
}
To
REPLACEPACKAGES = {
#'_xmlplus': 'xml',
}
And the xml import worked.
If you're using macports (or fink etc.) make sure that py2app is using the correct interpreter. You may have to install a new version to work with 2.6.x (on OS X 10.5, py2app uses 2.4.x).
If that doesn't work my steps for working through path problems are:
Start up the python interpreter that your code (or py2app) uses (are you absolutely certain?? try which python)
import sys; print sys.path
If step 2. gives you a path in /System/Library..someotherpythonversion your code is running in the wrong interpreter.