I have a tool written in Python which I would like to turn into a Python package, to make it easier to install dependencies and distribute it.
There are two usages:
you import the module mytool.py, call the wrapper function on it, a summary of some measurements is printed on screen and you obtain a dictionary with the full results. import mytool; results = mytool.mytool(param1=1, param2=2)
You call mytool.py from a shell, with the same parameters as above and a summary of the measurements is printed on screen. $ python mytool.py --param1 1 --param2 2
Would this be correct?
I understand that when you install a package from pypi, the main usage is the one in point 1. Once installed, It could be cumbersome to go find the exact path to mytool.py, and call it from the shell. Is there an alternative?
I think is fine, just make sure in your setup.py you specify a nice path to store the script playing a bit with the disutils lib
from distutils import setup
setup(
...,
scripts=['path/to/your/script',],
...
)
Some nice info can be found here https://docs.python.org/2/distutils/index.html#distutils-index
https://docs.python.org/2/distutils/packageindex.html
Related
pytest appears to be using old source code and failing tests because of it. I'm not sure how to update it.
Test code:
from nba_stats import league
class TestLeaders():
def test_default():
leaders = league.Leaders()
print(leaders)
Source code (league.py):
from nba_stats.nba_api import NbaAPI
from nba_stats import constants
class Leaders:
...
When I run pytest on my parent directory, I get an error that refers to an old import statement.
_____________________________ ERROR collecting test/test_league.py ______________________________
ImportError while importing test module '/home/mfb/src/nba_stats/test/test_league.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_league.py:1: in <module>
from nba_stats import league
../../../.virtualenvs/nba_stats_dev/lib/python3.6/site-packages/nba_stats/league.py:1: in <module>
from nba_stats import _api_scrape, _get_json
E ImportError: cannot import name '_api_scrape'
I tried resetting my virtualenvironment and also reinstalling my package via pip. What do I need to do to tell it to see the new import statement and why is this happening?
Edit: Deleting my virtual environment completely and then creating a new one seemed to fix it, but it seems to be a recurring issue with any further source code changes. Surely there must be a way to not have to reset my virtualenvironment each time?
Looks like you installed that package (possibly as a dependency through something else if not directly) and also have it cloned locally for development. You can look into local editable installs (https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs), but personally, I prefer to make the test refer directly to the package under which it is being run, since then it can be used "as-is" after cloning it. Do this by modifying sys.path in your test_league.py. Ie., assuming it has a structure with the python code under python/nba_stats, in the parent directory of `test
sys.path = [os.path.join(os.pardir, 'python')] + sys.path
at the top of test_league.py. This puts your local package up front and import will consider it first.
EDIT:
Since you tried and it still did not work (please do make sure that the snippet above does point to the local python package as in the actual structure; the above is just a common one but you may have a different structure), here is how you can see which directories are considered in order, and which are eventually selected:
python -vv -m pytest -svx
You will be able to see if there are spurious directories in sys.path, whether the one tried (as in the snippet above) matches as expected or not, any left-over .pyc files that get picked up, etc.
EDITv2: Since you stated that python -m pytest works, but pytest not, have a look where that pytest executable is coming from with which pytest. Likely it's a system one that refers to a different python then the one in your virtualenv. To see which python it picks up, do:
cat `which python`
and look at the top line.
If that is not the same as what which python gives you (with your desired virtualenv activated), you may have to install pytest for that current virtualenv (python -m pip install pytest).
I'm working through Learn Python the Hard way, and I'm currently on exercise 46 where you learn to create packages. I've created a basic package that does a few calculations, and uses a few different modules.
I've gotten the package to install in my python2.7 site packages, but I can't seem to run the module from my site packages after the fact. I'm wondering if the path that python is searching is different, because of the following:
After the install, I see this message Copying story-0.1-py2.7.egg to /usr/local/lib/python2.7/site-packages
However, when I try to run the module, I see this message /usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'story.py': [Errno 2] No such file or directory
Sorry if this makes absolutely no sense, I'm very new to the fantastical world of programing.
When you install a package, you access the internals differently. You can no longer just call python story.py.
To access the functions in story.py you need to import the module at the top of another python file, or in the interpreter.
If story.py contained
def my_test_function(blah):
print blah
You would use this function in another file by the following (after installing the module, as you've already done)
import story
story.my_test_function("Hello!")
By importing the module, you get access to all the functions and classes inside it by typing module_name.function_name. You could also just import the function you wanted to use directly, which would look something like
from story import my_test_function
my_test_function("Hello!")
I see there are some built in packages that I can import from any script like:
from datetime import date
today = date.today()
print today
How can I create a simple package and add it to the system library so that I can import it like datetime in the above example?
You're trying to make a module.
Start by installing the setuptools package; on either Windows or Linux you should be able to type pip install setuptools at a terminal to get that installed. You should now be able to write import setuptools at a python prompt without getting an error.
Once that's working, set up a directory structure containing a setup.py and a folder for your project's code to go in. The directory must contain a file called __init__.py, which allows you to import the directory as though it's a file.
some_folder/
| setup.py
| my_project/__init__.py
In setup.py, drop the following content:
# setup.py
from setuptools import setup
setup(name="My Awesome Project",
version="0.0",
packages=["my_project"])
In my_project/__init__.py, drop some stuff that you'd like to be able to import. Let's say...
# my_project/__init__.py
greeting = "Hello world!"
Now, in order to install the project at a system-wide level, run python setup.py install. Note that you'll need to run this as root if you're on Linux, since you're making changes to the system-wide python libraries.
After this, you should be able to run python from any directory you like and type:
>>> from my_project import greeting
>>> print greeting
Hello world!
>>>
Note that this is enough to tell you how to make a module, but there's one hell of a lot of stuff that setuptools can do for you. Take a look at https://pythonhosted.org/setuptools/setuptools.html for more info on building stuff, and https://docs.python.org/2/tutorial/modules.html for more info on how modules actually work. If you'd like to look at a package that (I hope) is reasonably simple, then I made my LazyLog module a couple of weeks ago on a train, and you're welcome to use it for reference.
The quick way, if you're just making something for your own use and not worrying about packaging, is to put the module (which could be as simple as a single file) in your system's site-packages directory. (On Debian-based systems, you probably want to use dist-packages instead).
To find out where your site-packages/dist-packages directory is, start Python and:
>>> from sys import path
>>> path
['', '/usr/lib/python3.4/site-packages/pip-7.1.2-py3.4.egg', '/usr/lib/python34.zip', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-cygwin', '/usr/lib/python3.4/lib-dynload', '/usr/lib/python3.4/site-packages']
Note the last item in that example: /usr/lib/python3.4/site-packages. That's the sort of thing you're looking for. So in this example, if I save the following to /usr/lib/python3.4/site-packages/foo.py:
def bar():
print('Hello world!')
Then from anywhere on my system:
>>> from foo import bar
>>> bar()
Hello world!
If you really need a package, you can do it with boost, which allows interacts with C++. You may implement algorithms using C++ and compile it as a Python lib. However it's poorly documented. And as doc described, C API should be a basic option. Boost is built on C API any way.
Sample: I made it several years ago on a class. You can do: import tfidf.
If you simply want the module for personal use, just drop it in a folder and add that folder to the PYTHONPATH environment variable.
For example, create a folder in your home directory called ~/python-packages, then add the following line to your .bashrc (assuming you are using bash):
export PYTHONPATH=$HOME/python-packages`
Then, simply drop any modules/packages you want to make available in ~/python-packages.
Add you python script or package's path to sys.path or just move them to one of the location in sys.path.
BUT i do not suggest to do this...
I'm trying to run a Python program, which uses the pygame modules, from MATLAB. I know I can use either
system('python program.py')
or just
! python program.py
However, I keep getting the error:
Traceback (most recent call last):
File "program.py", line 1, in <module>
import pygame
ImportError: No module named pygame
What is strange is that if I run the program from the command line, it works just fine. Does anyone know why if run from within MATLAB, Python can't find pygame?
The problem may be that MATLAB is not seeing your PYTHONPATH, which normally stores Python libraries and modules. For custom modules, PYTHONPATH should also include the path to your custom folders.
You can try setting the value of PYTHONPATH from within a MATLAB running session:
PATH_PYTHON = '<python_lib_folder>'
setenv('PYTHONPATH', PATH_PYTHON); % set env path (PYTHONPATH) for this session
system('python program.py');
See also the possibly relevant SO answer here: How can I call a Qtproject from matlab?
As I haven't used matlab too often and don't have the program available now I cannot say for sure, but matlab may be creating a custom environment with custom paths (this happens a lot, so the user has a very consistent experience in their software). When matlab installs it may not export paths to its own modules to your default environment. So when calling for pygame.py outside of matlab, python cannot find pygame.py under its usual lookup paths.
Solutions could be:
find the pygame.py, and map the path to it directly in your code, though this could cause you headaches later on during deployment
Try just copying the pygame.py file to your working directory, could have dependences that need to addressed.
Install pygame directly from its developer at http://www.pygame.org. Version differences could be a problem but pygame gets put under the usual lookup paths for python. (This would be my preferred solution personally.)
Or just export the location of path to pygame in matlab's library to your default enivronment. This could be a problem during deployment too.
For posterity, first try everything that Stewie noted here ("Undefined variable "py" or class" when trying to load Python from MATLAB R2014b?). If it doesnt work then it's possible that you have multiple pythons. You can try and check which python works (with all the related installed modules) on your bash/terminal. And then use
pyversion PYTHONPATH
to let matlab know the right path.
Also use py.importlib.import_module('yourmodule') to import the module after that.
That should get you started.
I'm trying to use the check-html flag with linkchecker. It gives an error that the tidy module is not installed. If I say tidy at a command line I can use it. I have MacPorts installed and have installed just about every option for libtidy or tidy that has to do with python. I'm thinking it has to do with a $PATH issue or something.
So if I am using python26 and I go into python and then say import tidy there are no errors and all is well. If I use python27 I get errors when I say import tidy. I'm assuming because I installed py26-utidylib py26-pytidylib. There were no py27s. I don't mind being in python26. So it appears if I am in python26 I should be able to run tidy.
But when I run linkchecker it says the tidy module is not installed. When I say which python it says my location is /opt/local/bin/python. Do I need to export the $PATH or $PYTHONPATH or something for it to work?
if you have 2 python compilers running, be sure you use the right one with the "python"-command.
you can see the used interpreter-version if you access the interactive-console with "python".
and you're right, the py26-...-library is for the python2.6 version. if there is no precompiled version you can always compile the library from the source for the required python-version. or at least, you can try.
but here you have to be sure to choose the right python-interpreter to execute the setup-process.
if you want the library for the 2.7-version you have to build it with 2.7-version.
you can also always check the used paths in which the current running python-interpreter is looking for modules, through the interactive python console:
python
>>> import sys
>>> sys.path
i assume you have the library installed for the wrong version, has import the wrong module ... or using it wrong :/.
maybe you could supply your code example!?