I built a python3 package called gender_univ using the Anaconda conda build command. I uploaded the package to the Anaconda cloud and then installed it into my conda environment. Though the package shows up in the list of installed packages when I type conda list, whenever I try to access the package using import gender_univ I get the error no module named gender_univ.
I want to understand why I can't seem to import a package that is apparently installed in my conda virtual environment? Any suggestions.
First, I would check that you are using the correct python (i.e. which python and confirm that it is the python in your conda environment). Next, you can check if your package is in the site-packages directory of that same python.
The most likely thing, I'd bet, is that the conda package doesn't include everything correctly. Are you sure that you have a build.sh (or bld.bat if you are on windows) and a setup.py? Did you try expanding your built conda package and looking for your python classes in there?
If you expand your built conda package, probably something like gender_univ-VERSION-py35_0.tar.bz2, you should see a lib/python3.5/site-packages/gender_univ directory (i.e. python package). Do you? If not, then the failure is with your building.
Related
I've been coding with R for quite a while but I want to start learning and using python more for its machine learning applications. However, I'm quite confused as to how to properly install packages and set up the whole working environment. Unlike R where I suppose most people just use RStudio and directly install packages with install.packages(), there seems to be a variety of ways this can be done in python, including pip install conda install and there is also the issue of doing it in the command prompt or one of the IDEs. I've downloaded python 3.8.5 and anaconda3 and some of my most burning questions right now are:
When to use which command for installing packages? (and also should I always do it in the command prompt aka cmd on windows instead of inside jupyter notebook)
How to navigate the cmd syntax/coding (for example the python documentation for installing packages has this piece of code: py -m pip install "SomeProject" but I am completely unfamiliar with this syntax and how to use it - so in the long run do I also have to learn what goes on in the command prompt or does most of the operations occur in the IDE and I mostly don't have to touch the cmd?)
How to set up a working directory of sorts (like setwd() in R) such that my .ipynb files can be saved to other directories or even better if I can just directly start my IDE from another file destination?
I've tried looking at some online resources but they mostly deal with coding basics and the python language instead of these technical aspects of the set up, so I would greatly appreciate some advice on how to navigate and set up the python working environment in general. Thanks a lot!
Python uses a different way of installing packages. Python has a thing named venv which stands for Virtual Environment. You install all of your packages in venv. Usually for each new project you make a new venv.
By using Anaconda on windows you install everything within the anaconda environment that you have specified.
python -m pip install "modulename" is a command that will install modulename to your default venv. You will be able to use this module when no other venv is specified. Here is the docs page. And here is a tutorial on how to use venv
By default python uses the same directory you have your code in. e.g. C:/Users/me/home/mypythonfile.py will run in C:/Users/me/home/ and will be able to access files in this directory. However you can use ../ to navigate directories or you can specify an absolute path to file you want to open e.g. with open("C:/system32/somesystemfile.sys") as file
Going over the technical differences of conda and pip:
So Conda is a packaging tool and installer that aims to do more than what pip does; handle library dependencies outside of the Python packages as well as the Python packages themselves. Both have many similar functionalities as well, you can install packages or create virtual environments with both.
It is generally advisable to generally have both conda and pip installed since there are some packages which might not be available with conda but with pip and vice versa.
The commands to install in both the ways is easy enough, but one thing to keep in mind is that
conda stores packages in the anaconda/pkgs directory
pip stores it in directory under /usr/local/bin/ for a Unix-based system, or \Program Files\ for Windows
You can use both pip or conda inside the jupyter notebook, it will work just fine, but it may be possible that you get multiple versions of the same package.
Most of the times, you will use cmd only to install a module used in your code, or to create environments, py -m pip install "SomeProject" here basically means that the module "SomeProject" will be downloaded in base env.
You could think of conda as python with a variety of additional functionalities, such as certain pre-installed packages and tools, such as spyder and jupyter. Hence, you must be precise when you say:
I've downloaded python 3.8.5 and anaconda3
Does it mean you installed python in your computer and then also anaconda?
In general, or at least in my opinion, using anaconda has advantages for development, but typically you'll just use a simple python installation in production (if that applies to you).
Anaconda has it's own package registry/repository . When you call conda install <package>, it will search for the package there and install it if available. You would better search it first, for instance matplotlib.
pip is a package manager for the Python Package Index. pip also ships with anaconda. Hence, in an anaconda environment you may install packages from either sources (either using pip install or conda install). For instance, pandas from PyPI and pandas from conda. There is no guarantee that packages exist in both sources. You must either search it first or simply try it.
In your first steps, I would suggest you to stick to only one dev env (either simple python or anaconda, recommend the second). Because that simplifies the question: "which python and which pip is executed in the cmd line?". That said, those commands should work as expected in any terminal, it be a simple cmd or an embedded one like in PyCharm or VS Code.
You could inspect that by running (on windows and linux at least):
which python, which pip.
Honestly, this is a question/answer that falls outside the scope of SO and for more info you would better check official websites, such as for anaconda or search for python vs anaconda blogs.
After updating macOs to Monterey, every new modules I install with pip won’t be imported in my Python programs. My sis.path are good and all the packages I need are in the site-packages. Can somebody help me ?
You need to check your python environment.
try which pip3 or which python3 to check locate your active python directory.
You can have different python versions installed in macOS which often leads to this issue.
Or maybe you are using virtual environments which creates isolated site-packages. If this is intended, then you should run your script within this environment to use the packages that you've installed.
I currently have scikit-learn 0.19 installed. I'd like to test my code using the latest development version as there seems to be a fix for Incremental PCA.
How do I go about installing this new version if I've previously installed scikit-learn using anaconda?
Also, how would I revert back to the stable release in the event that 0.20 does not solve my problem?
I am in need of some hand holding here, as I've read the docs on the website and not sure I completely understand the process (especially being able to revert back to the stable version if needed).
The whole point of the Anaconda Python distribution (apart from the convenience of having a bunch of useful packages included) is that you get the conda environment manager, which exists to meet exactly this sort of requirement.
What you want to do is to create a new conda environment by launching the Anaconda prompt and typing
conda create -n myenv scikit-learn other-package other-package2 etc
where myenv is the name you want to give the new environment and other-package other-package2 etc are the names of any other packages you will want to use (import) in your code. conda will figure out any dependencies of these packages and show you a list of what is going to be installed before it proceeds.
If you want to specify that a package should be a particular version, add that to the package name e.g. other-package=1.1.0, otherwise conda will install the latest versions of each package that are mutually compatible. You can also specify a particular version of Python by including it in the package list, e.g. python=3.4. You can check what versions of a package are available with conda search package-name (where package-name is the name of the package you want, obviously).
To run your code in the newly created environment, first activate the environment at the Anaconda prompt. If you use the Spyder IDE, launch it after activating the correct environment, or use the start menu shortcut specific to that environment if you have one. Other IDEs may have their own method of selecting a specific environment to work in.
To revert to the version(s) you were using before, activate the environment containing those versions - if you've never created a new environment before, that'll be root.
Just in case someone comes here looking for a solution without conda:
The website recommends that you download the latest code via
git clone git://github.com/scikit-learn/scikit-learn.git
and then include it in pip via (after changing to the directory)
pip install --editable .
You can also add the --user flag to have pip install to a local directory. Then, uninstalling should be as easy as pip uninstall sklearn.
I'm working on packaging up a suite of tools that can be installed in different environments, and I've run into many problems with dependencies, which are an issue since this package will be installed in air-gapped environments.
The package will be installed via Anaconda, and I have provided the installation script. In order to create the package, I ran the following command:
conda metapackage toolkit_bundle 0.0.1 --dependencies r-essentials tensorflow gensim spacy r-ggplot2 r-plotly r-dplyr r-rjson r-tm r-reshape2 r-shiny r-sparklyr r-slam r-nlp r-cluster r-ggvis r-plyr r-tidyr r-zoo r-magrittr r-xtable r-htmlwidgets r-formattable r-highcharter --summary "Toolkit Bundle"
This produced a tar.bzip2 file that I held on to and tried to install via the conda command
conda install toolkit_bundle.tar.bz2
The command seemed to run successfully, but I was unsuccessful in importing the modules in Python. I also tried creating a virtual conda environment and importing the package.
conda create -n myenv toolkit_bundle-0.0.1.tar.bz2
There was no error, but none of the modules were able to be imported either.
Am I missing a step in this process, or is my thought process flawed?
Update:
It looks like my thinking was pretty flawed. A quick skim of the conda metapackage command documentation revealed the following:
Tool for building conda metapackages. A metapackage is a package with no files, only metadata. They are typically used to collect several packages together into a single package via dependencies.
So my initial understanding was incorrect, and the package only contains metadata. Are there any other ideas for creating packages with dependencies resolved that can be installed in an air gapped environment?
I think you want to look at the command conda build for making packages, which just requires writing an appropriate meta.yaml file containing the dependencies, along with some other build parameters. There is good documentation for doing so on the conda website: https://conda.io/docs/user-guide/tasks/build-packages and there is a repo of examples.
If you have a working PIP package, you can also auto-generate a conda package recipe using conda skeleton.
Once you have built a set of packages locally, you can use the --use-local option to conda install to install from your local repo, with no need for an internet connection (as long as the packages for all the dependencies are in your local repo).
I was able to download the packages I needed via the pypi website, and after determining the dependencies, I manually downloaded them and wrote a script to install them in the required order.
I try to install Theano by Anaconda. It works, but when I enter the python -i, import theano shows No module named 'theano'. Do I need to switch another interpreter of Python, how? Also, for the packages installed by conda, if I don't double install them, can I find in Python? How is Python related to Python by Anaconda? Thanks!!!
I had have a similar issue, trying to install folium. If you are using the Anaconda:
When you install using conda install -c conda-forge folium, the package will be placed in:
./anaconda3/envs/[name env]/lib/python3.7/site-packages/folium
When you install using pip (with a anaconda env activated), pip install folium, the package will be placed in:
./anaconda3/lib/python3.7/site-packages/folium
Python use first the sites-packages as the target directory of manually built python packages. When you build and install python packages from source (using distutils, probably by executing python setup.py install ), you will find the installed modules in site-packages by default.
In this case you have two places: /anaconda3/lib/python3.7/site-packages/ and /anaconda3/envs/[name env]/lib/python3.7/site-packages/.
First the modules will be available as default in /anaconda3/lib/python3.7/site-packages/. Sometimes (and I really don't know why) the modules inside sites-packages conda env are not available to import automatically without export the PATH.
So, to solve this issue, you have 2 options:
Installing using pip install folium and import folium (don't need install by conda install), or
After conda install , run conda init, close the terminal and open a new one. So, try to import again.
Here are some tips about use a pip in a conda-environment.
You can refer to a specific version of python by using the following at the first line of your .py file
This is for python 2.7
#!/usr/bin/env python2.7
This is for python 3
#!/usr/bin/env python3
As other users already pointed out you need to check if your module is included in your sys path. Use code:
import sys
print(sys.path)
If not you can include this in your sys.path by using the command:
sys.path.append('/path/to/the/folder/of/your/module/file')
or place it in default PYTHONPATH itself.
Other great answers:
https://stackoverflow.com/a/19305076/5381704
The problem is that in the code editor you are using, you are running the default interpreter. Based on your code editor, change the python interpreter to the conda interpreter and it will work.
In my case that happened because conda screwed up the environment variables. Instead of using env-specific python and pip, it used the globally installed ones.
Solution:
conda deactivate your-env
conda activate your-env
In my workstation, I was able to solve No module named <module name> error using two different ways.
First method, I solved this temporarily by:
(1) Open a Terminal
(2) $ conda activate <Conda environment name>
(3) $ export PYTHONPATH=/home/<user name>/anaconda3/envs/<Conda environment name>/lib/<Python package version>/site-packages:$PYTHONPATH
It is a temporary solution. Whenever you run your virtual environment, you have to do this.
My runtime environment:
OS: Unbuntu 18.04
Conda version: 4.8.2
Conda-build version: 3.18,11
Python version 3.7.6.final.0
Second method, I removed the
alias python=/usr/bin/python3.6 line in bashrc file.
Somehow this line blocks using Python tools installed in Anaconda Virtual Environment if the Python version in the Virtual Environment is different.