Develop Reference

How to update source files for pytest?

pytest appears to be using old source code and failing tests because of it. I'm not sure how to update it.
Test code:
from nba_stats import league
class TestLeaders():
def test_default():
leaders = league.Leaders()
print(leaders)
Source code (league.py):
from nba_stats.nba_api import NbaAPI
from nba_stats import constants
class Leaders:
...
When I run pytest on my parent directory, I get an error that refers to an old import statement.
_____________________________ ERROR collecting test/test_league.py ______________________________
ImportError while importing test module '/home/mfb/src/nba_stats/test/test_league.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test_league.py:1: in <module>
from nba_stats import league
../../../.virtualenvs/nba_stats_dev/lib/python3.6/site-packages/nba_stats/league.py:1: in <module>
from nba_stats import _api_scrape, _get_json
E ImportError: cannot import name '_api_scrape'
I tried resetting my virtualenvironment and also reinstalling my package via pip. What do I need to do to tell it to see the new import statement and why is this happening?
Edit: Deleting my virtual environment completely and then creating a new one seemed to fix it, but it seems to be a recurring issue with any further source code changes. Surely there must be a way to not have to reset my virtualenvironment each time?

Looks like you installed that package (possibly as a dependency through something else if not directly) and also have it cloned locally for development. You can look into local editable installs (https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs), but personally, I prefer to make the test refer directly to the package under which it is being run, since then it can be used "as-is" after cloning it. Do this by modifying sys.path in your test_league.py. Ie., assuming it has a structure with the python code under python/nba_stats, in the parent directory of `test
sys.path = [os.path.join(os.pardir, 'python')] + sys.path
at the top of test_league.py. This puts your local package up front and import will consider it first.
EDIT:
Since you tried and it still did not work (please do make sure that the snippet above does point to the local python package as in the actual structure; the above is just a common one but you may have a different structure), here is how you can see which directories are considered in order, and which are eventually selected:
python -vv -m pytest -svx
You will be able to see if there are spurious directories in sys.path, whether the one tried (as in the snippet above) matches as expected or not, any left-over .pyc files that get picked up, etc.
EDITv2: Since you stated that python -m pytest works, but pytest not, have a look where that pytest executable is coming from with which pytest. Likely it's a system one that refers to a different python then the one in your virtualenv. To see which python it picks up, do:
cat `which python`
and look at the top line.
If that is not the same as what which python gives you (with your desired virtualenv activated), you may have to install pytest for that current virtualenv (python -m pip install pytest).

Importing self-made package

I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures

Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.

Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.

Adding your own python project source code to site-packages folder using setuptools (setup.py install)

I'm aware there are similar questions to such as this one, but I have trouble understanding how to actually automatically create module in site-packages for my own project.
I tried using python setup.py install and it created .egg file in my site-packages but I can't import it in simple python script.
I tried googling for about an hour now and the only solution I found is to directly copy the sources to a directory which I don't want to do as I'd have to copy it every time I change my project.

I figured out my problem.
pip install .
Inside the project directory is actually giving me the wanted output, but problem is that I didn't nest the name of my package inside project.
Think, I have project foo, and I want to have imports such as:
from foo import bar
Well I had project named foo but I didn't have first package named foo rather I had straight away bar and all other packages, so that was the mistake.

Libraries act differently depending on if they are installed or not

I have a Python library I wrote which has been acting up on me. I have a set of variables which change the way the library works. In testing it all worked fine, but when I python lib.py install the variables have no effect on the library. I broke this down to the simplest example possible:
Library:
##lib.py
config="Original"
def run():
print config
Script:
import lib
lib.config="New"
lib.run()
print lib.config
If you place the library in the same directory as the script and run it the output is:
New
New
But if you install the library and then try the script using the library from the dist-packages the output is:
Original
New
Could someone explain what is going on? I'm a bit confused and terribly interested in the happenings and reason. Additionally am I doing programatical configuration totally wrong?
Edit
It turns out the problem is the init.py file. It's basically like importing a library that just imports another library. When you import an installed module it looks at the folder lib and the file init.py. init.py is just a one liner from lib import *. It simply pretends to be the actual library, but that causes an odd problem if you use a global variable. A simulated example of what is essentially going on:
##init.py
from lib import *
Script:
import init
init.config = 'New'
init.run()
print init.config
Output:
Original
New
The function run() looks for config in lib.py, but print init.config looks for it in init.py. Thanks for the help everyone. Fix is to change the way the module installs (no init.py). Eventually, I hope to remove all the global variables, but for the time being everything works perfect.

What you describe would be inconsistent with how Python works (read, if you like, "I don't believe you did precisely that and got precisely that result").
If you go importing lib from different places or in different ways, though, you could be ending up with two copies of it, either two copies of one of the modules or one of the current-directory lib and the other the installed lib. If you are getting this "Original"/"New" behaviour, that seems to me the most likely reason.

py2exe cannot import Module from other directory

I am bundling python source code with py2exe. The directory structure is as follows:
some_Mod.py
some_dir/another_dir/some_Mod.py
Inside the latter some_dir/another_dir/some_Mod.py I am trying to import the other Python Module with
from ..some_Mod import *
Using the import causes no problems with the python interpreter, but if I run the same constellation in the bundled package, I get an Exception:
ImportError: No module named some_Mod
Can somebody explain why?
Remark: Renaming the Modules is actually no problem, but I was just wondering, why py2exe cannot deal with this constellation.

If you have __init__.py files in each of those sub-directories then all import statements should work correctly.
Assuming that's not the problem, here's an excellent guide to importing best practices:
http://blog.habnab.it/blog/2013/07/21/python-packages-and-you/
In summary, never use relative imports - always absolute (see the link above for why).
Second (and I'm not entirely sure why), always keep your py2exe setup.py script in the exact folder where your main script is. I've tried modifying py2exe's 'script' option to allow my script to be somewhere else... but your exact problem happened to me. So, try making sure it is right where the main script is.
Finally, you can always give py2exe a little help. I usually have to add the root directory to the system path so the import statements are valid. Note, I'm not modifying sys.path in any of my application's code - only the py2exe script I use to build the exe.
At the top of my py2exe setup script:
import sys
sys.path.append(PATH_WHERE_PACKAGES_ARE)
# add any packages that need explicit importing here located in root directory:
import package1 # apparently it wasn't found...
import package2 # apparently same thing
Generally I don't import packages though, adding the project root where they exist usually is enough.

I'm not sure that py2exe now how to handle the from ..some_Mod import * syntax, check this to ensure that the some_Mod.py module is correctly packaged : python -m py2exe.mf -d some_dir/another_dir/some_Mod.py as explained in the py2exe FAQ