I discoverd entry_points of setuptools:
http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins
quote: setuptools supports creating libraries that “plug in” to extensible applications and frameworks, by letting you register “entry points” in your project that can be imported by the application or framework.
But I have not seen a project using them.
Are there examples of projects which use them?
If not, why are they not used?
There are loads of examples. Any project that defines console scripts uses them, for example. A quick search on GitHub gives you plenty to browse through.
I'll focus on one specific example (one that is not on GitHub): Babel.
Babel uses both entry_points for both console scripts and to define extension points for translatable text extraction. See their setup.py source:
if have_setuptools:
extra_arguments = dict(
zip_safe = False,
test_suite = 'babel.tests.suite',
tests_require = ['pytz'],
entry_points = """
[console_scripts]
pybabel = babel.messages.frontend:main
[distutils.commands]
compile_catalog = babel.messages.frontend:compile_catalog
extract_messages = babel.messages.frontend:extract_messages
init_catalog = babel.messages.frontend:init_catalog
update_catalog = babel.messages.frontend:update_catalog
[distutils.setup_keywords]
message_extractors = babel.messages.frontend:check_message_extractors
[babel.checkers]
num_plurals = babel.messages.checkers:num_plurals
python_format = babel.messages.checkers:python_format
[babel.extractors]
ignore = babel.messages.extract:extract_nothing
python = babel.messages.extract:extract_python
javascript = babel.messages.extract:extract_javascript
""",
)
Tools like pip and zc.buildout use the console_scripts entry point to create commandline scripts (one called pybabel, running the main() callable in the babel.messages.frontend module).
The distutils.commands entry points defines additional commands you can use when running setup.py; these can be used in your own projects to invoke Babel command-line utilities right from your setup script.
Last, but not least, it registers its own checkers and extractors. The babel.extractors entry point is loaded by the babel.messages.extract.extract function, using the setuptools pkg_resources module, giving access to all installed Python projects that registered that entry point. The following code looks for a specific extractor in those entries:
try:
from pkg_resources import working_set
except ImportError:
pass
else:
for entry_point in working_set.iter_entry_points(GROUP_NAME,
method):
func = entry_point.load(require=True)
break
This lets any project register additional extractors; simply add an entry point in your setup.py and Babel can make use of it.
Sentry is a good example. Sentry's author even created a django package named Logan to convert standard django management commands to console scripts.
Related
I am developing a sphinx based collaborative writing tool. Users access the web application (developed in python/Flask) to write a book in sphinx and compile it to pdf.
I have learned that in order to compile a sphinx documentation from within python I should use
import sphinx
result = sphinx.build_main(['-c', 'path/to/conf',
'path/to/source/', 'path/to/out'])
So far so good.
Now my users want the app to show them their syntax mistakes. But the output (result in the example above) only gives me the exit code.
So, how do I get a list of warnings from the build process?
Perhaps I am being too ambitious, but since sphinx is a python tool, I was expecting to have a nice pythonic interface with the tool. For example, the output of sphinx.build_main could be a very rich object with warnings, line numbers...
On a related note, the argument to the method sphinx.build_main looks just like a wrapper to the command line interface.
sphinx.build_main() calls sphinx.cmdline.main(), which in turn creates a sphinx.application.Sphinx object. You could create such an object directly (instead of "making system calls within python"). Use something like this:
import os
from sphinx.application import Sphinx
# Main arguments
srcdir = "/path/to/source"
confdir = srcdir
builddir = os.path.join(srcdir, "_build")
doctreedir = os.path.join(builddir, "doctrees")
builder = "html"
# Write warning messages to a file (instead of stderr)
warning = open("/path/to/warnings.txt", "w")
# Create the Sphinx application object
app = Sphinx(srcdir, confdir, builddir, doctreedir, builder,
warning=warning)
# Run the build
app.build()
Assuming you used sphinx-quickstart to generate your initial Sphinx documentation set with a makefile, then you can use make to build docs, which in turn uses the Sphinx tool sphinx-build. You can pass the -w <file> option to sphinx-build to write warnings and errors to a file as well as stderr.
Note that options passed through the command line override any other options set in the makefile and conf.py.
I am writing my first package to be shared on the pypi database...
It took me a lot of fiddling to get everything to package correctly, but I finally did.
I have a structure like this.
---dist
---package.egg-info
---MANIFEST.in
---setup.py
---package/
---__init__.py
---file.py
---info.txt
---templates/
---template.html
now in my dist folder when I extract the tar.gz file I see everything. but when I run a pip install package then I only get the egg and the init.py and file.py files and not the other text files and template files.
Why is this?
Setup.py added...:
setup(name='django-g-recaptcha',
version='0.1.2',
description='Django view decorator to validate google recaptcha forms',
url='https://bitbucket.org/deltaskelta/django-g-recaptcha-validate/overview',
author='Jeff Willette',
author_email='jrwillette88#gmail.com',
keywords = ['django', 'recaptcha', 'catpcha'],
packages = ['g_recaptcha',],
)
Add include_package_data = True to setup() arguments:
setup(name='django-g-recaptcha',
version='0.1.2',
description='Django view decorator to validate google recaptcha forms',
url='https://bitbucket.org/deltaskelta/django-g-recaptcha-validate/overview',
author='Jeff Willette',
author_email='jrwillette88#gmail.com',
keywords = ['django', 'recaptcha', 'catpcha'],
packages = ['g_recaptcha',],
include_package_data = True
)
This should help, however I suggest to also use package_data along with your MANIFEST.in. And also you might want to add a setting specifying, that your package is intended to be used with Django.
See https://pythonhosted.org/setuptools/setuptools.html for more information.
Some background (not mandatory, but might be nice to know): I am writing a Python command-line module which is a wrapper around latexdiff. It basically replaces all \cite{ref1, ref2, ...} commands in LaTeX files with written-out and properly formatted references before passing the files to latexdiff, so that latexdiff will properly mark changes to references in the text (otherwise, it treats the whole \cite{...} command as a single "word"). All the code is currently in a single file which can be run with python -m latexdiff-cite, and I have not yet decided how to package or distribute it. To make the script useful for anybody else, the citation formatting needs to be configurable. I have implemented an optional command-line argument -c CONFIGFILE to allow the user to point to their own JSON config file (a default file resides in the module folder and is loaded if the argument is not used).
Current implementation: My single-file command-line Python module currently parses command-line arguments in if __name__ == '__main__', and loads the config file (specified by the user in -c CONFIGFILE) here before running the main function of the program. The config variable is thus available in the entire module and all is well. However, I'm considering publishing to PyPI by following this guide which seems to require me to put the command-line parsing in a main() function, which means the config variable will not be available to the other functions unless passed down as arguments to where it's needed. This "passing down by arguments" method seems a little cluttered to me.
Question: Is there a more pythonic way to set some configuration globals in a module or otherwise accomplish what I'm trying to? (I don't want to rely on 3rd party modules.) Am I perhaps completely off the tracks in some fundamental way?
One way to do it is to have the configurations defined in a class or a simple dict:
class Config(object):
setting1 = "default_value"
setting2 = "default_value"
#staticmethod
def load_config(json_file):
""" load settings from config file """
with open(json_file) as f:
config = json.load(f)
for k, v in config.iteritems():
setattr(Config, k, v)
Then your application can access the settings via this class: Config.setting1 ...
I am developing a product for Plone 4, inside the zeocluster/src/... directory of an installation, and I have an automated test. Unfortunately, when I run 'bin/client1 shell' and then (path to Plone's Python)/bin/python setup.py test, it fails. The error is
File "buildout-cache/eggs/Products.PloneTestCase-0.9.12-py2.6.egg/Products/PloneTestCase/PloneTestCase.py", line 109, in getPortal
return getattr(self.app, portal_name)
AttributeError: plone
What is the correct way to run automated tests in Plone 4?
In setup.py,
...
test_suite = "nose.collector"
...
The failing test:
import unittest
from Products.PloneTestCase import PloneTestCase as ptc
ptc.setupPloneSite()
class NullTest(ptc.PloneTestCase):
def testTest(self):
pass
def test_suite():
return unittest.TestSuite([
unittest.makeSuite(NullTest)
])
if __name__ == '__main__':
unittest.main(defaultTest='test_suite')
Best is to edit your buildout.cfg and add a part that creates a 'bin/test' script. Something like this:
[test]
recipe = zc.recipe.testrunner
# Note that only tests for packages that are explicitly named (instead
# of 'implicitly' added to the instance as dependency) can be found.
eggs =
# Use the name of the plone.recipe.zope2instance part here, might be zeoclient instead:
${instance:eggs}
defaults = ['--exit-with-status', '--auto-color', '--auto-progress']
Do not forget to add 'test' to the 'parts' in the main 'buildout' section of your buildout.cfg. Run bin/buildout and you should now have a bin/test script. See the PyPI page of this recipe for more options and explanation.
Now running 'bin/test' should run all tests for all eggs explicitly named in the instance part. This may run far too many tests. Use 'bin/test -s your.package' to run only the tests for your.package, provided your.package is part of the eggs in the instance.
Note that instead of the 'pass' that you now have in the test, it is better to add a test that you know for certain will fail, like 'self.assertEqual(True, False)'. Then it is easier to see that your test indeed has been run and that it fails as expected.
When I have a simple buildout for testing one specific package that I am developing, I usually extend one of the configs in the plonetest buildout, like this one for Plone 4; you can have a look at that for inspiration.
You need to use zope.testrunner and zope.testing to run your tests. Plone tests cannot be run via nose and we don't support the 'test_suite' argument to setup.py as invented by setuptools.
The other answers explain how to get a test runner script set up.
ptc.setupPloneSite() registers a deferred function that will be actually run when the zope.testrunner layer is set up. I'm guessing you're not using zope.testrunner and thus the layer isn't being setup so the Plone site is never created, hence the AttributeError when it tries subsequently to get the portal object.
I've written a setup.py script for py2exe, generated an executable for my python GUI application and I have a whole bunch of files in the dist directory, including the app, w9xopen.exe and MSVCR71.dll. When I try to run the application, I get an error message that just says "see the logfile for details". The only problem is, the log file is empty.
The closest error I've seen is "The following modules appear to be missing" but I'm not using any of those modules as far as I know (especially since they seem to be of databases I'm not using) but digging up on Google suggests that these are relatively benign warnings.
I've written and packaged a console application as well as a wxpython one with py2exe and both applications have compiled and run successfully. I am using a new python toolkit called dabo, which in turn makes uses of wxpython modules so I can't figure out what I'm doing wrong. Where do I start investigating the problem since obviously the log file hasn't been too useful?
Edit 1:
The python version is 2.5. py2exe is 0.6.8. There were no significant build errors. The only one was the bit about "The following modules appear to be missing..." which were non critical errors since the packages listed were ones I was definitely not using and shouldn't stop the execution of the app either. Running the executable produced a logfile which was completely empty. Previously it had an error about locales which I've since fixed but clearly something is wrong as the executable wasn't running. The setup.py file is based quite heavily on the original setup.py generated by running their "app wizard" and looking at the example that Ed Leafe and some others posted. Yes, I have a log file and it's not printing anything for me to use, which is why I'm asking if there's any other troubleshooting avenue I've missed which will help me find out what's going on.
I have even written a bare bones test application which simply produces a bare bones GUI - an empty frame with some default menu options. The code written itself is only 3 lines and the rest is in the 3rd party toolkit. Again, that compiled into an exe (as did my original app) but simply did not run. There were no error output in the run time log file either.
Edit 2:
It turns out that switching from "windows" to "console" for initial debugging purposes was insightful. I've now got a basic running test app and on to compiling the real app!
The test app:
import dabo
app = dabo.dApp()
app.start()
The setup.py for test app:
import os
import sys
import glob
from distutils.core import setup
import py2exe
import dabo.icons
daboDir = os.path.split(dabo.__file__)[0]
# Find the location of the dabo icons:
iconDir = os.path.split(dabo.icons.__file__)[0]
iconSubDirs = []
def getIconSubDir(arg, dirname, fnames):
if ".svn" not in dirname and dirname[-1] != "\\":
icons = glob.glob(os.path.join(dirname, "*.png"))
if icons:
subdir = (os.path.join("resources", dirname[len(arg)+1:]), icons)
iconSubDirs.append(subdir)
os.path.walk(iconDir, getIconSubDir, iconDir)
# locales:
localeDir = "%s%slocale" % (daboDir, os.sep)
locales = []
def getLocales(arg, dirname, fnames):
if ".svn" not in dirname and dirname[-1] != "\\":
mo_files = tuple(glob.glob(os.path.join(dirname, "*.mo")))
if mo_files:
subdir = os.path.join("dabo.locale", dirname[len(arg)+1:])
locales.append((subdir, mo_files))
os.path.walk(localeDir, getLocales, localeDir)
data_files=[("resources", glob.glob(os.path.join(iconDir, "*.ico"))),
("resources", glob.glob("resources/*"))]
data_files.extend(iconSubDirs)
data_files.extend(locales)
setup(name="basicApp",
version='0.01',
description="Test Dabo Application",
options={"py2exe": {
"compressed": 1, "optimize": 2, "bundle_files": 1,
"excludes": ["Tkconstants","Tkinter","tcl",
"_imagingtk", "PIL._imagingtk",
"ImageTk", "PIL.ImageTk", "FixTk", "kinterbasdb",
"MySQLdb", 'Numeric', 'OpenGL.GL', 'OpenGL.GLUT',
'dbGadfly', 'email.Generator',
'email.Iterators', 'email.Utils', 'kinterbasdb',
'numarray', 'pymssql', 'pysqlite2', 'wx.BitmapFromImage'],
"includes": ["encodings", "locale", "wx.gizmos","wx.lib.calendar"]}},
zipfile=None,
windows=[{'script':'basicApp.py'}],
data_files=data_files
)
You may need to fix log handling first, this URL may help.
Later you may look for answer here.
My answer is very general because you didn't give any more specific info (like py2exe/python version, py2exe log, other used 3rd party libraries).
See http://www.wxpython.org/docs/api/wx.App-class.html for wxPyton's App class initializer. If you want to run the app from a console and have stderr print to there, then supply False for the redirect argument. Otherwise, if you just want a window to pop up, set redirect to True and filename to None.