Develop Reference

Problems installing python packages on Mac M1

I want to install python packages listed in the requirements file of my github repo. However, I have problems installing those python packages into my conda environment.
First of all, I installed conda with Miniforge3-MacOSX-arm64 which supports the M1 with arm64 architecture. However, some specific python packages like onnxruntime I wasn't able to install, because I encountered error messages like that:
ERROR: Could not find a version that satisfies the requirement onnxruntime
ERROR: No matching distribution found for onnxruntime
I assumed that for those specific python packages there is no support yet for the M1.
Therefore, I pursued another approach. I set the settings of Terminal to "Open with Rosetta". The plan is to install the applications of the intel x86_64 architecture and let Rossetta create the binaries to let run on arm64. Then I uninstalled miniforge for arm64 and installed miniforge for x86_64 named Miniforge3-MacOSX-x86_64. With that setup I was able to install all listed python packages of the requirement file and with pip freeze I can also confirm that they have been installed. However, I am somehow not able to use those python packages. For instance if I want to run pytest I get the following error:
zsh: illegal hardware instruction pytest
I assumed Rossetta takes care of that, that I can use applications for x86_64 also on arm64. But somehow it doesn't work. I tried a lot of different things and am out of ideas.
Does anyone know what the problem is? I would be also thankful for advice and suggestions how to properly set up a python environment on Mac M1.

I had the same problem back in 2days ago, I'm using m1 pro. I was trying to install the python packages only using pip but I got a numbers of errors, then I decided to install with conda.
In my case it worked, here is what I've done so far is:
First Enable the open with rosetta in your zsh.
And then,
# create environment in conda
conda create -n venv python=3.8 # with your python version
# activate
conda activate venv
and visit the conda website to look for the packages:
check packages
For suppose if you are looking for pytest packages then you can search it, and you'll get a result like this, with the available package and channel.
You need to enable that specific channel to get that package with this command:
# config channel
conda config --append channels conda-forge # available channel name
# then install
conda install --yes --file requirements.txt
Make sure, your have the same version of pytest in your requirements.txt file. (eg:pytest==6.2.5)
Hope this should work, if not try to install it with pip like:
pip install -r requirements.txt
after environment enable.

package installed in anaconda\lib\site-packages and found in spyder, but not in a jupyter notebook

I have installed the azureml package and can see it in . . .anaconda\lib\site-packages:
If I run import azureml.dataprep as dprep in a python script in Spyder (launched from Anaconda Navigator), it works. But, if I open one of my anaconda environments with jupyter notebook and try running the same line of code, I get an error about module not found for azureml:
I thought perhaps the problem was that the package needed to be installed for that specific environment, but azureml is not available as a package for install via the anaconda environments > install packages interface (there is an azure package but not an azureml package).
So, I followed instructions to use conda prompt to install a package into a specific environment. Instructions I followed:
(those are from this link)
And here is the result of following the instructions (it looked like it installed the package into the env):
But, I got the exact same error when trying to import the package in the environment started as a jupyter notebook. Then, I closed anaconda navigator completely just in case, but that also didn't change the result.
Any ideas about what I'm either doing wrong or how I can manually install this package into a specific anaconda environment?

You did the right thing to install the package into the environment. Btw, pip is automatically installed by conda into any environment that has Python, so installing it shouldn't have been necessary.
Are you sure that the environment that you installed into is the one in which your notebook kernel is running? Start the notebook and execute
!conda env list
That will give you a list of environments, and an asterisk * next to the one that is active.
You can also call pip directly from a notebook cell:
!pip install azureml
That will install into the conda environment in which the kernel is running.

Conda does not use package installed but using package from outside

I got a problem when I using conda environment.
I create a environment simply by:
conda create -n NAME python==3.5
So there is a python 3.5 virtual environment.
Then I enter into the virtual environment and install tensorflow-gpu with:
conda install tensorflow-gpu
Now I can see the tensorflow-gpu package along with the cudatoolkit and cudnn.
But when I run a simple program it just failed to use gpu.I don't think it is a nvidia driver problem or package from different resources because it work in the old conda environment but when I create a new it failed.
More interesting is it will using the package(tensorrt and uff) from outside of the enviroment.
When I create a new environment with python 3.6.It works as expected using tensorflow-gpu and cannot import package which isnt installed in the environment.
Does some one know how to fix the problem?
Ubuntu 16.04
Tensorflow 1.8.0
Python 3.5 and 3.6
TensorRT 5.0.0.10

Well I had fixed it.
This problem turn out to be the conda environment added a path from local and imported package from the local path instead of the environment.
If you run into the same problem.Try open python in your conda environment by:
python
and check the system path:
import sys
print(sys.path)
if there is a lib from outside just remove it using:
rm -r ~/PATH
This problem might origin from adding path unintentionally in some program.
Hope that helps.

This also happens if you use jupyter notebook. For example,
if you have jupyter notebook installed and THEN create your environment, the jupyter notebook is OUTSIDE the environment and IGNORES all the packages you installed in your environment. I needed to install jupyter notebook into every environment I created to keep these errors from happening.

Installing python packages in a different location than default by pip or conda

How do you use a Python package such as Tensorflow or Keras if you cannot install the package on the drive on which pip always saves the packages?
I'm a student at a university and we don't have permission to write to the C drive, which is where pip works out of (I get a you don't have write permission error when installing packages through pip or conda`).
I do have memory space available on my user drive, which is separate from the C drive (where the OS is installed).
So, is there any way I can use these Python libraries without it being installed?
Maybe I can install the package on my user drive and ask the compiler to access it from there? I'm just guessing here, I have no knowledge of how this works.

install conda
create new environment (conda create --name foobar python=3.x list of packages
use anaconda to activate foobar (activate foobar)
check pip location by typing in cmd 'where pip' to be sure you use pip from withing the python from withing the foobar environment and not the default python installed in your system outside of your conda environment
and next use the pip from above location to install requested library into your environment.
ps. you may want to consider to install Cygwin on your Windows machine to get use to work with Linux environment.

Conda PackageNotFoundError - unable to install packages

I'm trying to install a set of libraries used in a data science course, following the instructions from this url, under "Setting up the environment". I have installed Acaconda3 using Python 3.6 on a Windows 10 machine. I have a previously installed version of Python 2.7 which I suspect is the problem, but I´m not sure how to solve this (both versions are now listed in the environment settings).
The problem starts when trying to install the libraries, which gives me the following error when I open a command prompt and enter a "conda install ... " command. The .txt file I´m trying to read is in the project folder. How can I install the packages properly? (anaconda administrator allows a workaround by letting you select the packages and install them from there, but I´d like to know why the command prompt doesn´t work in this case).
(packt-py36) C:\Users\Eric\AnacondaProjects>conda install --file
conda-reqs.txt --yes
Fetching package metadata .............
PackageNotFoundError: Package not found: Conda could not find '

It isn't working because the list of packages was created for OS X. I was able to successfully import these packages on OS X but there are several packages and versions that are not compatible. I have gone through the file and eliminated any non-compatible packages for you. Quite a few packages are missing but you will be able to install from this.
This file may be used to create an environment using:
$ conda create --name <env> --file <this file>
platform: win-64
bleach=1.5.0
cycler=0.10.0=py36_0
decorator=4.0.11=py36_0
entrypoints=0.2.2=py36_1
freetype=2.5.5
html5lib=0.999=py36_0
icu=57.1
ipykernel=4.6.1=py36_0
ipython=6.0.0=py36_0
ipython_genutils=0.2.0=py36_0
ipywidgets=6.0.0=py36_0
jinja2=2.9.6=py36_0
jsonschema=2.6.0=py36_0
jupyter=1.0.0=py36_3
jupyter_client=5.0.1=py36_0
jupyter_console=5.1.0=py36_0
jupyter_core=4.3.0=py36_0
libpng=1.6.27
markupsafe=0.23=py36_2
matplotlib=2.0.0=np112py36_0
mistune=0.7.4=py36_0
mkl=2017.0.1=0
nbconvert=5.1.1=py36_0
nbformat=4.3.0=py36_0
notebook=5.0.0=py36_0
numpy=1.12.1=py36_0
openssl=1.0.2k
pandas=0.19.2=np112py36_1
pandocfilters=1.4.1=py36_0
path.py=10.3.1=py36_0
pickleshare=0.7.4=py36_0
pip=9.0.1=py36_1
prompt_toolkit=1.0.14=py36_0
pygments=2.2.0=py36_0
pyparsing=2.1.4=py36_0
pyqt=5.6.0=py36_2
python=3.6.1=0
python-dateutil=2.6.0=py36_0
pytz=2017.2=py36_0
pyzmq=16.0.2=py36_0
qt=5.6.2
qtconsole=4.3.0=py36_0
scikit-learn=0.18.1=np112py36_1
scipy=0.19.0=np112py36_0
simplegeneric=0.8.1=py36_1
sip=4.18=py36_0
six=1.10.0=py36_0
sqlite=3.13.0
testpath=0.3=py36_0
tk=8.5.18
tornado=4.5.1=py36_0
traitlets=4.3.2=py36_0
wcwidth=0.1.7=py36_0
wheel=0.29.0=py36_0
widgetsnbextension=2.0.0=py36_0
zlib=1.2.8