My python package has a module called settings.py which looks for a settings.ini file, and if one isn't found raises an Exception.
My package works fine but when I try and use Sphinx Autodoc it fails because it can't find a settings file.
What options do I have for getting around this? The ones I can think of are
Put a settings.ini file in one of the Sphinx directories so it can be read.
Somehow mock the internal settings module if possible. The settings module is unimportant so I don't care if there is no documentation for it to the suer
Has anyone run into a similar issue when trying to generate Sphinx documentation on packages that need external files and if so what is your solution?
Pyramid has a command-line script pserve which loads an .ini settings file.
To document this script, we use autoprogram as follows:
.. autoprogram:: pyramid.scripts.pserve:PServeCommand.parser
:prog: pserve
and in your conf.py:
extensions = [
...
'sphinxcontrib.autoprogram',
...
]
It requires the use of argparse in your script.
Related
I am trying out Sphinx for generating documentation for a Python project. Just to make sure I can actually make it work, I have made a test project to try it out on: https://github.com/ThomasA/sphinxtest.
I have run sphinx-quickstart in the root of this repository. In the following questions, I specified 'doc' as the documentation root, named the project 'sphinxtest', entered 'Thomas Arildsen' as author, answered 'y' to the 'autodoc' option, and selected the default setting for everything else.
I expected the 'autodoc' option to cause the generation of a file 'amodule.rst' in the 'doc' folder. However, this does not get generated. I am puzzled by this. I thought this was what the 'autodoc' option was supposed to do and what I have seen examples of others apparently achieve with it. Sphinx completes without any error messages, so it seems to be doing what it thinks it should do. So, what could I be doing wrong?
I am using Sphinx v. 1.5.6 and Python 3.5.3, all installed with Anaconda.
autodoc does not generate the .rst source files.
Instead first use sphinx-apidoc to generate the source files. Then run Sphinx to make your documentation.
An alternative to Percy's answer is autoapi written by Carlos Jenkins. He provides a Sphinx extension that generates the ReST files per documentation generation. You can configure the output ReST by modifying a template file.
https://github.com/carlos-jenkins/autoapi
I have a custom module loader that basically does some redirection. I would like pylint to recognize this custom loader. This is my situation:
root/
__init__.py
new/
__init__.py
foo.py
bar.py
old/
__init__.py
I have a lot of clients importing old.foo. I wrote a custom loader in old/__init__.py to redirect these to import new.foo under the hood. How do I get pylint to recognize this? When it lints import old.foo, it complains that it can't find old.foo. This is only a problem with pylint. I can get client code to recognize the custom loader without any issue.
from the documentation on modules:
Packages support one more special attribute, __path__. This is initialized to be a list containing the name of the directory holding the package’s __init__.py before the code in that file is executed. This variable can be modified; doing so affects future searches for modules and subpackages contained in the package.
So if I understand correctly you want to redirect any references to old to redirect to new, so all you would need to do is replace the old folder with old.py that contains this:
__path__ = ["new"]
Then when anything tries to import old.foo it will end up importing new.foo.
You have to remember that pylint is a static analyser and as such doesn't actually load python file (except in some cases where it can't do otherwise, e.g. compiled code). As such it's not aware of custom importer or other tricks taking part of python's high dynamicity.
That being said:
you may still write a "brain" plugin for astroid (the library under pylint) that will help pylint understand your code's specificity
by relying on standard mecanism such as __path__ manipulation you'll get more chance to avoid such need, either because at some point pylint may understand this or because someone else will have contributed a plugin for that purpose.
We don't want to be maintaining documentation as well as the source code, which is evolving rapidly at the moment, yet Sphinx seems to require a frustrating amount of setup and configuration. (We just need some basic API docs.) Is there not a single command you can run inside a python project that will just iterate over all of the packages, modules, classes and functions generating documentation as HTML?
The sphinx-apidoc splats stuff into a directory, and after modifying the conf.py to have our packages in the sys.path we can run "make html", but it only lists packages and modules without documenting any classes or functions.
Thanks!
The sphinx-apidoc tool will autogenerate stubs for your modules, which might be what you want.
Instructions
Make sure the autodoc module was enabled during Sphinx configuration.
extensions = ['sphinx.ext.autodoc']
within Sphinx's conf.py should do the trick.
Make sure conf.py adjusts sys.path accordingly (see the comments at lines 16-19 in the file).
sys.path.insert(0, os.path.abspath('/my/source/lives/here'))
Run sphinx-apidoc to generate skeletons.
sphinx-apidoc -o /my/docs/live/here /my/source/lives/here
Rebuild the docs. If all goes well, you shouldn't get the following sort of warning:
mymodule.rst:4: WARNING: autodoc can't import/find module 'mymodule'
Your module RSTs should now be populated.
I have created a folder with all my modules for my GAE application and with external libraries like Jinja2 to keep everything sorted in one place. I have folders structure like this:
lib\
\utils\
\__init__.py
\firepython
\jinja2
\jsonpickle
__init__.py
sessions.py
When I try to load Jinja from utils__init__.py, I get error ImportError: No module named jinja2.environment. When I look at Jinja2 imports instructions, I see them look like from jinja2.loaders. I try to change them to be like from lib.jinja2.loaders but some other errors then appear about imports. More than that I don't think it's a good practice to change these imports in external libraries source if there is a more convenient and right way to import modules properly. I also have added some paths to PYTHONPATH but it doesn't solve all problems. How can I properly import an external package that is placed in another folder, may be with a deep structure?
Indeed you should not have to change imports in external libraries - though depending on your environment, you might even have too.
PYTHONPATH
Modifying your PYTHONPATH should suffice; PYTHONPATH should contain a 'lib' path that is either absolute or relative to your home, eg.
Then you could simply do
from jinja2 import WHATEVER
sys.path.append
Another way to go without PYTHONPATH is to use sys.path.append() and add your paths from your python code. I actually favor that, as it also allows to have per-application paths.
use virtualenv
Details would be a bit long to be put here, but please follow the official doc
These options applies to general python development rather than GAE specificities; if it does not work on your development machine you should post more details (exact imports, absolute paths, pythonpath...).
A proper project structure and use of appcfg.py should workout dependencies when uploading to google: please take a look at this good answer: How do I manage third-party Python libraries with Google App Engine? (virtualenv? pip?) and follow those guidelines.
A nice way to go with GAE is through yaml application directives. Please take a look at the doc for includes: http://code.google.com/appengine/docs/python/config/appconfig.html#Includes
Also remember that GAE officially supports python 2.5, and 2.7 support is experimental
Python 2.7 is now officially supported
To properly import a module, you need to make sure, that python knows where to find it.
To do so, for each external library append it's parent directory to the sys.path (in run-time), or setup PYTHONPATH environment (before running).
For example:
import sys
sys.path.append('/my/lib')
# now we can import from lib
import jsonpickle # will load /my/lib/jsonpickle/__init__.py
See http://docs.python.org/tutorial/modules.html#the-module-search-path . to understand what python does when you call import.
I have a python/django application that runs on the google app engine.
My views.py file has some imports...
from commands.userCommands import RegisterUserCommand
from commands.accountCommands import CreateNewAccountCommand, RenameAccountCommand
These imports work fine on my development environment (local machine). But when I upload to the google app engine, views.py fails with a "Could not import views. Error was: No module named userCommands" error.
Any idea why I can't import my commands.userCommands module?
My file structure looks as follows...
- app.yaml
- urls.py
- views.py
- etc...
- commands/__init__.py
- commands/userCommands.py
Note: I did try to append my application name to the module name/path. No luck.
Note: I did do an update with the --noisy argument, and it does appear to upload my commands folder successfully.
You could be running into a clash with Python's own commands module (which doesn't have submodules like yours) -- naming your own modules and packages in ways that are meant to hide ones in the standard library (just like naming your variables in ways that are meant to hide builtin names, like list or file) is always a perilous undertaking, even though it "should" work there's always potential for confusion.
Could you try renaming that commands package and its uses to something unambiguous and free from danger, such as mycommands, and see if that just makes the problem disappear? If that's the case you can then open a ticket on GAE's tracker (because it would show a minor but undeniable bug in GAE's runtime) but meanwhile your problem is solved!-) If the problem stays, ah well, at least we've eliminated one likely cause and can keep digging...
The __init__.py files are required to make Python treat the directories as containing packages, so you need a
commands/__init__.py
file in your directory structure. See http://docs.python.org/tutorial/modules.html.