I see there are some built in packages that I can import from any script like:
from datetime import date
today = date.today()
print today
How can I create a simple package and add it to the system library so that I can import it like datetime in the above example?
You're trying to make a module.
Start by installing the setuptools package; on either Windows or Linux you should be able to type pip install setuptools at a terminal to get that installed. You should now be able to write import setuptools at a python prompt without getting an error.
Once that's working, set up a directory structure containing a setup.py and a folder for your project's code to go in. The directory must contain a file called __init__.py, which allows you to import the directory as though it's a file.
some_folder/
| setup.py
| my_project/__init__.py
In setup.py, drop the following content:
# setup.py
from setuptools import setup
setup(name="My Awesome Project",
version="0.0",
packages=["my_project"])
In my_project/__init__.py, drop some stuff that you'd like to be able to import. Let's say...
# my_project/__init__.py
greeting = "Hello world!"
Now, in order to install the project at a system-wide level, run python setup.py install. Note that you'll need to run this as root if you're on Linux, since you're making changes to the system-wide python libraries.
After this, you should be able to run python from any directory you like and type:
>>> from my_project import greeting
>>> print greeting
Hello world!
>>>
Note that this is enough to tell you how to make a module, but there's one hell of a lot of stuff that setuptools can do for you. Take a look at https://pythonhosted.org/setuptools/setuptools.html for more info on building stuff, and https://docs.python.org/2/tutorial/modules.html for more info on how modules actually work. If you'd like to look at a package that (I hope) is reasonably simple, then I made my LazyLog module a couple of weeks ago on a train, and you're welcome to use it for reference.
The quick way, if you're just making something for your own use and not worrying about packaging, is to put the module (which could be as simple as a single file) in your system's site-packages directory. (On Debian-based systems, you probably want to use dist-packages instead).
To find out where your site-packages/dist-packages directory is, start Python and:
>>> from sys import path
>>> path
['', '/usr/lib/python3.4/site-packages/pip-7.1.2-py3.4.egg', '/usr/lib/python34.zip', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-cygwin', '/usr/lib/python3.4/lib-dynload', '/usr/lib/python3.4/site-packages']
Note the last item in that example: /usr/lib/python3.4/site-packages. That's the sort of thing you're looking for. So in this example, if I save the following to /usr/lib/python3.4/site-packages/foo.py:
def bar():
print('Hello world!')
Then from anywhere on my system:
>>> from foo import bar
>>> bar()
Hello world!
If you really need a package, you can do it with boost, which allows interacts with C++. You may implement algorithms using C++ and compile it as a Python lib. However it's poorly documented. And as doc described, C API should be a basic option. Boost is built on C API any way.
Sample: I made it several years ago on a class. You can do: import tfidf.
If you simply want the module for personal use, just drop it in a folder and add that folder to the PYTHONPATH environment variable.
For example, create a folder in your home directory called ~/python-packages, then add the following line to your .bashrc (assuming you are using bash):
export PYTHONPATH=$HOME/python-packages`
Then, simply drop any modules/packages you want to make available in ~/python-packages.
Add you python script or package's path to sys.path or just move them to one of the location in sys.path.
BUT i do not suggest to do this...
Related
pytest appears to be using old source code and failing tests because of it. I'm not sure how to update it.
Test code:
from nba_stats import league
class TestLeaders():
def test_default():
leaders = league.Leaders()
print(leaders)
Source code (league.py):
from nba_stats.nba_api import NbaAPI
from nba_stats import constants
class Leaders:
...
When I run pytest on my parent directory, I get an error that refers to an old import statement.
_____________________________ ERROR collecting test/test_league.py ______________________________
ImportError while importing test module '/home/mfb/src/nba_stats/test/test_league.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_league.py:1: in <module>
from nba_stats import league
../../../.virtualenvs/nba_stats_dev/lib/python3.6/site-packages/nba_stats/league.py:1: in <module>
from nba_stats import _api_scrape, _get_json
E ImportError: cannot import name '_api_scrape'
I tried resetting my virtualenvironment and also reinstalling my package via pip. What do I need to do to tell it to see the new import statement and why is this happening?
Edit: Deleting my virtual environment completely and then creating a new one seemed to fix it, but it seems to be a recurring issue with any further source code changes. Surely there must be a way to not have to reset my virtualenvironment each time?
Looks like you installed that package (possibly as a dependency through something else if not directly) and also have it cloned locally for development. You can look into local editable installs (https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs), but personally, I prefer to make the test refer directly to the package under which it is being run, since then it can be used "as-is" after cloning it. Do this by modifying sys.path in your test_league.py. Ie., assuming it has a structure with the python code under python/nba_stats, in the parent directory of `test
sys.path = [os.path.join(os.pardir, 'python')] + sys.path
at the top of test_league.py. This puts your local package up front and import will consider it first.
EDIT:
Since you tried and it still did not work (please do make sure that the snippet above does point to the local python package as in the actual structure; the above is just a common one but you may have a different structure), here is how you can see which directories are considered in order, and which are eventually selected:
python -vv -m pytest -svx
You will be able to see if there are spurious directories in sys.path, whether the one tried (as in the snippet above) matches as expected or not, any left-over .pyc files that get picked up, etc.
EDITv2: Since you stated that python -m pytest works, but pytest not, have a look where that pytest executable is coming from with which pytest. Likely it's a system one that refers to a different python then the one in your virtualenv. To see which python it picks up, do:
cat `which python`
and look at the top line.
If that is not the same as what which python gives you (with your desired virtualenv activated), you may have to install pytest for that current virtualenv (python -m pip install pytest).
I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures
Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.
Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.
I'm working through Learn Python the Hard way, and I'm currently on exercise 46 where you learn to create packages. I've created a basic package that does a few calculations, and uses a few different modules.
I've gotten the package to install in my python2.7 site packages, but I can't seem to run the module from my site packages after the fact. I'm wondering if the path that python is searching is different, because of the following:
After the install, I see this message Copying story-0.1-py2.7.egg to /usr/local/lib/python2.7/site-packages
However, when I try to run the module, I see this message /usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file 'story.py': [Errno 2] No such file or directory
Sorry if this makes absolutely no sense, I'm very new to the fantastical world of programing.
When you install a package, you access the internals differently. You can no longer just call python story.py.
To access the functions in story.py you need to import the module at the top of another python file, or in the interpreter.
If story.py contained
def my_test_function(blah):
print blah
You would use this function in another file by the following (after installing the module, as you've already done)
import story
story.my_test_function("Hello!")
By importing the module, you get access to all the functions and classes inside it by typing module_name.function_name. You could also just import the function you wanted to use directly, which would look something like
from story import my_test_function
my_test_function("Hello!")
I have a tool written in Python which I would like to turn into a Python package, to make it easier to install dependencies and distribute it.
There are two usages:
you import the module mytool.py, call the wrapper function on it, a summary of some measurements is printed on screen and you obtain a dictionary with the full results. import mytool; results = mytool.mytool(param1=1, param2=2)
You call mytool.py from a shell, with the same parameters as above and a summary of the measurements is printed on screen. $ python mytool.py --param1 1 --param2 2
Would this be correct?
I understand that when you install a package from pypi, the main usage is the one in point 1. Once installed, It could be cumbersome to go find the exact path to mytool.py, and call it from the shell. Is there an alternative?
I think is fine, just make sure in your setup.py you specify a nice path to store the script playing a bit with the disutils lib
from distutils import setup
setup(
...,
scripts=['path/to/your/script',],
...
)
Some nice info can be found here https://docs.python.org/2/distutils/index.html#distutils-index
https://docs.python.org/2/distutils/packageindex.html
I am writing a little python app. I want to be able to easily deploy the app. I know that python 2.6 will allow one to execute an egg directly if there is a main module at the egg's root. I actually have that working.
The one kink is that when I try to use the argparse library, I cannot include the library in the egg without installing it into my source directory (or symlinking in the argparse.py into my source dir) since the argparse module is in the top-level package.
If I install it into a subdirectory called "argparse", I have to import it like "from argparse import argparse" instead of the normal "import argparse".
I would like to be able to specify a site-packages type directory in the egg where I could just install the third party modules/packages. Is there any way to do this with setuptools (or some other egg builder)?
Thanks!
I believe you can create a subdirectory called toplevel and in your entry point do
import sys
sys.path.insert(0, './toplevel')
Untested, though.