Develop Reference

How do you place a breakpoint inside a python package?

I'm working on contributing open source to the pandas package inside python. When I run import pandas as pd, it points to the installed version of python.
Wondering the best way to import the local version of the pandas library that I forked from github and set break-points inside to understand how the different functions work.

You should take a look at the pdb-module, using it for break-points goes as simple as:
import pdb # import the module
pdb.set_trace() # set break-point & open interactive shell
you can use the set_trace method whereever you want in any Python script you'd like to debug.
Here's a link to the official docs:
https://docs.python.org/3/library/pdb.html

So i installed numpy . But when i call it in a program an error occurs. Any method to solve it permanently in windows 10

Here is the error
import numpy
Exception has occurred: ModuleNotFoundError
No module named 'numpy'
File "C:\path\to\file\32.py", line 1, in <module>
import numpy

Let me know how did you install the NumPy package; using pip or something else?
If you have multiple python versions, i.e. 2.x and 3.x at the same time, please make sure your interpreter for the 32.py file is the version that you installed NumPy on.

To possibly fix your problem, you should first try installing it and see if there are any errors. You should also check the version of Python you are running on Windows 10, because when you update Python it sometimes switches names between py and python
As you can see, the version of Python has changed between py and python so you should try changing that first.
If this does not work, you should try finding the directory for NumPy and adding it to the system PATH in your script. The installer usually shows you the location by doing the following:
import sys
sys.path.append("<insert numpy location here>")
import NumPy
This should manually force it into finding the package. If none of this works, please tell us and we should be able to find a different solution.
Happy Coding!

If you're using a code editor like PyCharm, you could install it by clicking on
file then settings then the project interpreter setting and install new module! You can search for the module and install.

Make sure that the python version that you want to use is a Windows Environmental Variable. You can test this by running this line in your command line.
> python --version
If you get some other python version that is not the one that you wish to use you can set the python version you want by finding where exactly your Python folder is located and go into settings and set the path as a new variable (I can leave a tutorial for you). If that is too much of a hassle, the Python installers can set the python that you will install as an environmental variable for you. (You could uninstall and reinstall and make sure that you allow it to make it an environmental variable.
After that you should be able to import whatever external packages you want using pip
for example:
pip install numpy

How to import pandas using R studio

So, just to be clear, I'm very new to python coding... so I'm not exactly sure what's going wrong.
Yesterday, whilst following a tutorial on calling python from R, I successfully installed and used several python packages (e.g., NumPy, pandas, matplotlib etc).
But today, when trying to run the exact same code, I'm getting an error when trying to import pandas (NumPy is importing without any errors). The error states:
ModuleNotFoundError: No module named 'pandas'
I'm not sure what's going on!?
I'm using R-Studio (running on a Mac)... here's a code snippet of how I'm doing it:
library(reticulate)
os <- import("os") # Setting directory
os$getcwd()
repl_python() #used to make it interactive
import numpy as np. # Load numpy package
import pandas as pd # Load pandas package
At this point, it's throwing me an error. I've tried googling the answer and searching here, but to no avail.
Any suggestions as to how I'd fix this problem, or what is going on?
Thanks

Possibly your python path for reticulate changed upon reloading Rstudio. Here is how to set the path manually (filepath for Linux or Mac):
library(reticulate)
path_to_python <- "~/anaconda3/bin/python"
use_python(path_to_python)
https://stackoverflow.com/a/45891929/4549682
You can check your Python path with py_config(): https://rstudio.github.io/reticulate/articles/versions.html#configuration-info
I recommend using Anaconda for your Python distribution (you might have to use Anaconda anyway for reticulate, not sure). Download it from here: https://www.anaconda.com/distribution/#download-section
Then you can create the environment for reticulate to use:
conda_create('r-reticulate', packages = "python=3.5")
I use Python 3.5 for some specific packages, but you can change that version or leave it as just 'python' for the latest version.
https://www.rdocumentation.org/packages/reticulate/versions/1.10/topics/conda-tools
Then you want to install the packages you need (if they aren't already) with
conda_install('re-reticulate', packages = 'numpy')
The way I use something like numpy is
np <- import('numpy')
np$arange(10)

You need to set the second argument of the function use_python, so it should be:
For example, use_python("/users/my_user/Anaconda3/python.exe",required = TRUE)
DON'T forget required = TRUE

How to import a module from a directory?

On my system I have two versions of Python (to call them I type python and python2 in the command line). When I use the first version of Python, I cannot import sklearn module but I can do it in the second version of Python.
I would like to use the first version of python (because other modules are available there) and, at the same time, I would like to be able to import sklearn from this version of Python.
My solution was to use:
import sys
sys.path.append('location_of_the_sklearn_module')
To find the location of the sklearn module I started a python session (using the second version of python, in which sklearn works). The I type:
import sklearn
sklearn.__file__
As a result I got:
/home/name/my_name/numpy/local/lib/python2.7/site-packages/sklearn/__init__.pyc
In the session of the first version of Python I tried:
import sys
sys.path.append('/home/name/my_name/numpy/local/lib/python2.7/site-packages/sklearn')
import sklearn
Unfortunately it did not work. As a result I got: ImportError: No module named sklearn
Does anybody know what I am doing wrong and if it is possible to reach the goal in the way I try?

When importing packages, you need to add the parent directory of the package to PYTHONPATH, not the package directory itself, so just change...
sys.path.append('/home/name/my_name/numpy/local/lib/python2.7/site-packages/sklearn')
...to...
sys.path.append('/home/name/my_name/numpy/local/lib/python2.7/site-packages')
...although it may not necessarily import correctly in Python 3.x.

Checking a Python module version at runtime

Many third-party Python modules have an attribute which holds the version information for the module (usually something like module.VERSION or module.__version__), however some do not.
Particular examples of such modules are libxslt and libxml2.
I need to check that the correct version of these modules are being used at runtime. Is there a way to do this?
A potential solution wold be to read in the source at runtime, hash it, and then compare it to the hash of the known version, but that's nasty.
Is there a better solutions?

Use pkg_resources. Anything installed from PyPI at least should have a version number.
>>> import pkg_resources
>>> pkg_resources.get_distribution("blogofile").version
'0.7.1'

If you're on python >=3.8 you can use a module from the built-in library for that. To check a package's version (in this example lxml) run:
>>> from importlib.metadata import version
>>> version('lxml')
'4.3.1'
This functionality has been ported to older versions of python (<3.8) as well, but you need to install a separate library first:
pip install importlib_metadata
and then to check a package's version (in this example lxml) run:
>>> from importlib_metadata import version
>>> version('lxml')
'4.3.1'
Keep in mind that this works only for packages installed from PyPI. Also, you must pass a package name as an argument to the version method, rather than a module name that this package provides (although they're usually the same).

I'd stay away from hashing. The version of libxslt being used might contain some type of patch that doesn't effect your use of it.
As an alternative, I'd like to suggest that you don't check at run time (don't know if that's a hard requirement or not). For the python stuff I write that has external dependencies (3rd party libraries), I write a script that users can run to check their python install to see if the appropriate versions of modules are installed.
For the modules that don't have a defined 'version' attribute, you can inspect the interfaces it contains (classes and methods) and see if they match the interface they expect. Then in the actual code that you're working on, assume that the 3rd party modules have the interface you expect.

Some ideas:
Try checking for functions that exist or don't exist in your needed versions.
If there are no function differences, inspect function arguments and signatures.
If you can't figure it out from function signatures, set up some stub calls at import time and check their behavior.

I found it quite unreliable to use the various tools available (including the best one pkg_resources mentioned by this other answer), as most of them do not cover all cases. For example
built-in modules
modules not installed but just added to the python path (by your IDE for example)
two versions of the same module available (one in python path superseding the one installed)
Since we needed a reliable way to get the version of any package, module or submodule, I ended up writing getversion. It is quite simple to use:
from getversion import get_module_version
import foo
version, details = get_module_version(foo)
See the documentation for details.

You can use
pip freeze
to see the installed packages in requirements format.

For modules which do not provide __version__ the following is ugly but works:
#!/usr/bin/env python3.6
import sys
import os
import subprocess
import re
sp = subprocess.run(["pip3", "show", "numpy"], stdout=subprocess.PIPE)
ver = sp.stdout.decode('utf-8').strip().split('\n')[1]
res = re.search('^Version:\ (.*)$', ver)
print(res.group(1))
or
#!/usr/bin/env python3.7
import sys
import os
import subprocess
import re
sp = subprocess.run(["pip3", "show", "numpy"], capture_output=True)
ver = sp.stdout.decode('utf-8').strip().split('\n')[1]
res = re.search('^Version:\ (.*)$', ver)
print(res.group(1))