I use pip to install lightgbm on windows,Does it work?
Since I have an error while importing LGBMRegressor.
The error shows as below
"cannot import name 'LGBMRegressor'"
I tried installing it using an anaconda environment and was able to install it successfully.
Open Anaconda prompt and type:
conda install -c conda-forge lightgbm
It will take a few minutes for installation.
I am writing an answer as I don't have enough reputation to leave a comment yet. Valid solutions have been offered but I think that some subtleties should be addressed for those that come here and are new(ish) to Python.
There are differences between pip and conda and if I understand it well, one ought to keep using conda as much as possible within their conda environments to avoid running into time-wasting package conflicts. I advise you read this article to get a better grasp on which does what. Myth #2 and #10 are particularly relevant here.
For Lightgbm the obvious solution is to use conda-forge as mentioned above. That is if you are running a conda environment, be it anaconda or miniconda. But you should be aware that for the moment anaconda distribution (AD) and conda-forge (CF) are not 100% compatible, as you can read in this thread. From their own words:
Conda-forge and The Anaconda Distribution are incompatible with each other at present.
...Please pick one or the other for now. CF or AD, never both.
So far it seems not to be a problem for lightbgm though.
Happy coding!
I used this and it worked for me even with Python 3.6
import sys
!{sys.executable} -m pip install lightgbm
Related
To run transformers I installed it on CentOS 8 by
conda install -c conda-forge transformers=4.12.2
following the method on this page, but I still encountered the same error:
version `GLIBC_2.29’ not found
Then I tried to install that from the huggingface channel and got stuck by lots of conflicts.
conda install -c huggingface transformers=4.12.2
At last I tried to install the glibc231 myself, and downloaded the rpm from this link, but I saw the following error, leading me to think that I would be in the wrong direction:
Error: Problem: conflicting requests
nothing provides glibc-common = 2.31-3.gf.el7 needed by glibc231-2.31-3.gf.el7.x86_64
nothing provides glibc-langpack = 2.31-3.gf.el7 needed by glibc231-2.31-3.gf.el7.x86_64
Any suggestions? Thanks in advance.
I had the same issues, and I downgraded to the following version:
tokenizers=0.10.1
transformers=4.6.1
I also encountered this error. If you don't want to downgrade (which may or may not lose features), I solved it by building it from sources. You can find the instructions here:
https://huggingface.co/docs/tokenizers/python/latest/installation/main.html#installation-from-sources
Note at least two caveats in the guide above:
The process will ask you to install rust, which is a language very robust to different platforms. You might even have it on your system if you've worked with rust before, but in that case do remember to update your rust toolchain with rustup update.
If you follow the process exactly, you will install the latest commit on master, which might or might not be a good thing. I recommend you select your desired version and do a git checkout ... first before you run python setup.py install.
Edit: You might want to skip to the end of the question first, I've followed some advice in comments / answers and the current error is different from the original (appears to be related to numpy possibly).
This error ModuleNotFoundError: No module named 'torch' shows up in tons of threads, I've been trying solutions all day. I'll go through my troubleshooting steps one by one, using the solutions suggested in threads.
System info:
Windows 10
First thing I did was follow the instructions on Pytorch, installed Anaconda and did this using the correct settings for my machine (Note: I tried Python v3.7 before trying v3.8 in these screenshots, none of the solutions worked with that either):
As you can see, that should be good to go, according to the instructions.
So I go into the python terminal and try to import pytorch, like so:
ModuleNotFoundError: No module named 'torch' Great, so what now? Well I paste the error into Google and begin my 4 hour wild goose chase.
First result, stack overflow answer: No module named "Torch"
Let's try the selected answer, it requires some version-related syntax so lets check my python version:
Alright so as directed by the answer:
Try to install PyTorch using pip:
First create a conda environment using:
conda create -n env_pytorch python=3.6
Ok:
Activate the environment using:
source activate env_pytorch
That doesnt work, but if we activate using the instructions given by the prompt, we can do so:
Now install PyTorch using pip:
pip install torchvision --user ( this will install both torch and torchvision)
Hmmm.. well that went up in flames, so the following...
Now go to python shell and import using the command:
import torch
import torchvision
...doesn't do anything new, same error as before.
Well, to the next thread, on PyTorch GitHub: https://github.com/pytorch/pytorch/issues/4827
They're trying to use Jupyter, so I tried this, is was another long process like the above that went up in flames, and I really dont want to need to use Jupyter anyway, so we'll skip this one.
Another Pytorch GitHub thread: https://github.com/pytorch/pytorch/issues/12004
#edtky Could you please give me the output of the following commands
in CMD?
where conda.exe
where pip.exe
where python.exe
Sure I'll give it a shot:
#edtky Looks like you have two Python environments. Please try
importing torch in Anaconda Prompt.
Oh well, I already did that. No bueno.
Another thread: https://discuss.pytorch.org/t/modulenotfounderror-no-module-named-torch/7309 suggests:
In that case you’ve probably forgotten to activate the environment
where pytorch is installed. It can also be the library missing in your
PYTHONPATH variable.
Well I did activate the environment as shown above, but I dont know anything about a PYTHONPTH variable, seems like the PyTorch setup guide wouldve mentioned if I needed to manually do that, I have no clue how to do it and you aren't explaining, so lets look for other answers.
Someone made a whole article to give us this little gym of advice: https://medium.com/#valeryyakovlev/anaconda-no-module-named-torch-ead10946de66
Another beginner error I encountered when started to use pytorch in
anaconda environment
import torch ModuleNotFoundError: No module named ‘torch’ the proper way to install pytorch to anaconda is following
conda install -c pytorch pytorch It’s not enough to simply run “conda install pytorch” — the package won’t be found. So first
activate your conda profile with “source activate {your_profile}” and
then run the command conda install -c...
Ok thats new info, let's try that command again now that our env is activated:
Ok that's a lot of green, let's try now...
Well we can't win 'em all, so lets go onto the next thread: https://forums.fast.ai/t/modulenotfounderror-no-module-named-torch-windows-10/12438/2
I had also faced the similar problem , I just installed torch and torchvision using pip and it worked …
Ok! Let's try:
Oh well, another solution up in flames..
I ran into a similar issue with Windows 10. In the end I could only get torch installed with Miniconda.
Alrighty, lets try it!
Alright, cool, moment of truth:
Awesome! You just read through 25 minutes of me re-producing all my attempts to solve this problem, and it doesnt even include the hour I spend down a rabbit hole trying to use Jupyter, which failed equally as miserably. I think it's time to post the question to StackOverflow!
Edit 1:
An answer points out that one of my logs was an error python 3.8 isn't compatible with pytorch, good point I'll fix that. After unintalling 3.8 and installing 3.7:
And no luck! Remember I actually mentioned in my first paragraph that while I was trying 3.8 in these screenshots, the first time around I did all of this with 3.7
Edit 2:
I forgot to install after activating the environment in the previous edit. Once I fixed that, there's a new error:
Pytorch requires 3.5 <= python < 3.8. Setup an environment with:
conda create -n pytorch python=3.7
conda activate pytorch
conda install pytorch
You should also make sure that you launch the installed python interpreter from this environment (YourAnacondaInstallDirectory\envs\pytorch\python.exe) from the activated conda environment! The later is important because conda will export certain environment variables (have a look at this for a related issue caused by missing envionment variables).
Use This Let me Know if it will Work!
pip install https://download.pytorch.org/whl/cpu/torch-1.0.1-cp37-cp37m-win_amd64.whl
If you would like to install it with plain pip rather than conda:
As you know, Pytorch 1.5.1 requires 3.5 <= python < 3.8. Use Python 3.7 or slightly earlier.
Do NOT try to install with simple pip install torch. Instead, as you say, go to https://pytorch.org/, in the colorful grid, click on pip, copy the command, open a command prompt as an administrator (right-click and select "Run as Administrator") then paste the command, which should look something like:
pip install torch===1.5.1 torchvision===0.6.1 -f https://download.pytorch.org/whl/torch_stable.html
Then, edit the command to replace pip with the full path to your version of pip, e.g.:
"C:\Program Files\Python37\Scripts\pip.exe" pip install torch===1.5.1 torchvision===0.6.1 -f https://download.pytorch.org/whl/torch_stable.html
(You don't need to edit the command as long as Python 3.7 is in your path.)
Getting the error message when using matplotlib:
Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib
already initialized
OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can
degrade performance or cause incorrect results. The best thing to do
is to ensure that only a single OpenMP runtime is linked into the
process, e.g. by avoiding static linking of the OpenMP runtime in any
library. As an unsafe, unsupported, undocumented workaround you can
set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the
program to continue to execute, but that may cause crashes or silently
produce incorrect results. For more information, please see
http://www.intel.com/software/products/support/.
Do the following to solve the issue:
import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'
Answer found at: https://github.com/dmlc/xgboost/issues/1715
Be aware of potential side-effects:
but that may cause crashes or silently produce incorrect results.
This is a better solution, if applicable. Else, anyway gcamargo’s solution is likely to work. However, it comes with a warning "that it may cause crashes or silently produce incorrect results"
I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with
conda install nomkl
Answer found at: https://github.com/dmlc/xgboost/issues/1715
I had the same issue on macOS and found the following reasons:
Problem:
I had a conda environment where Numpy, SciPy and TensorFlow were installed.
Conda is using Intel(R) MKL Optimizations, see docs:
Anaconda has packaged MKL-powered binary versions of some of the most popular numerical/scientific Python libraries into MKL Optimizations for improved performance.
The Intel MKL functions (e.g. FFT, LAPACK, BLAS) are threaded with the OpenMP technology.
But on macOS you do not need MKL, because the Accelerate Framework comes with its own optimization algorithms and already uses OpenMP. That is the reason for the error message: OMP Error #15: ...
Workaround:
You should install all packages without MKL support:
conda install nomkl
and then use
conda install numpy scipy pandas tensorflow
followed by
conda remove mkl mkl-service
For more information see conda MKL Optimizations.
I had the same issue in a conda environment where TensorFlow was installed. After doing
pip uninstall tensorflow
pip install tensorflow
the problem was gone.
For me, this problem came up when I imported pytorch after numpy. Importing them in this order fixed my problem:
import torch
import numpy as np
Had same issue in OSX when updating tensoflow to 1.13 using conda.
Solution 1: /gcamargo worked but 3x slower per training epoch.
Solution 2: /sjcoding worked and removed serious warining but also 3x
slower in training.
Solution 3: that restored performance was: Install pip in new conda
env and use pip to install tensorflow. Using conda-forge also worked
but version of tf is old.
Apparently the new Intel-MKL optimizations in Anaconda are broken for OSX tensorflow.
Check if there's an update for the mkl package in your env (anaconda).
I was able to solve my case simply by updating mkl.
conda install -c intel mkl
(macOS Catalina 10.15.5)
I am using Macbook M1 and I faced the same issue. I solved this problem after removing the mkl package from my conda environment using the following command:
conda remove mkl
This issue happens because mkl is developed for Intel users and many of my packages were comming from mkl. After you remove this package, you will be required to reinstall many packages that you use through mkl. Also, in my case, when I tried to install pandas afterwards, there was a dependency issue. I solved this issue as well after updating all conda packages using the following command:
conda update --all
A useful link that helped me figure this out:
So, for those of you getting this same issue with lightgbm, I found in the documentation that you can
pip uninstall lightgbm
pip install lightgbm
Run the following in anaconda environmnet (if you're running Conda)
ln -sf `ls -d "$(brew --cellar libomp)"/*/lib`/* $CONDA_PREFIX/lib
These three things worked for me.
Try to change the backend of matplotlib.
For example, Tkagg backend causes this problem in my case. I changed it to Qt5Agg
matplotlib.use('Qt5Agg')
and it helps.
Confronted with the same error #15, none of the solutions to-date (5 Feb 2021) fully worked despite being helpful. However, I did manage to solve it while avoiding: dithering with dylib libraries, installing from source, or setting the environment variable KMP_DUPLICATE_LIB_OK=TRUE and its downsides of being an “unsafe, unsupported, undocumented workaround” and its potential “crashes or silently produce incorrect results”.
The trouble was that conda wasn’t picking up the non-mkl builds of tensorflow (v2.0.0) despite loading the nomkl package. What finally made this solution work was to:
ensure I was loading packages from the defaults channel (ie. from a channel with a non-mkl version of tensorflow. As of 5 Feb 2021, conda-forge does not have a tensorflow version of 2.0 or
greater).
specify the precise build of the tensorflow version I wanted: tensorflow>=2.*=eigen_py37h153756e_0. Without this, conda kept loading the mkl_... version of the package despite the nomkl package also being loaded.
I created a conda environment using the following environment.yml file (as per the conda documentation for managing environments) :
name: tf_nomkl
channels:
- conda-forge
- defaults
dependencies:
- nomkl
- python>=3.7
- numpy
- scipy
- pandas
- jupyter
- jupyterlab
- nb_conda
- nb_conda_kernels
- ipykernel
- pathlib
- matplotlib
- seaborn
- tensorflow>=2.*=eigen_py37h153756e_0
You could try to do the same without an environment.yml file, but it’s better to load all the packages you want in an environment in one go if you can.
This solution works on MacOS Big Sur v11.1.
conda install --revision 0 doesn't solve UnsatisfiableError: The following specifications... for me. So I manually install nomkl and remove mkl and mil-service in Anaconda-Navigator environment, and it works great for me!
I was getting the same error as mentioned in the original question when I ran a code with Tensorflow on my macOS Monterey. I tried installing nomkl and removing mkl as suggested in many of the previous answers. However this gave me trouble on running readcsv module of pandas and many other modules from different packages. A friend told me that newer versions of macOS have trouble with the usual Tensorflow and therefore pypi has launched a special version of TF called tf-nightly.
https://pypi.org/project/tf-nightly/#description
This installation solved the problem for me.
I had the same problem. Nothing you suggested solved the issue. I found that a possible cause is that you have multiple OpenMP libraries installed on your machine and they conflict with each other. Plus, I found that the problem was numpy and I did the upgrade (conda update numpy) and FINALLY IT WORKED!!!
I am trying to follow to the installation guide on tensorflow.org and have installed Python version 2 again for that reason using Homebrew.
When I run the installation as described
$ pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
I get this error message:
tensorflow-0.5.0-py2-none-any.whl is not a supported wheel on this platform.
I am obviously doing something wrong, but have no idea. Any clues?
I do not want to use virtualenv, since anaconda already comes with its own environment management conda. When installing the newest version 0.6.0 directly with pip install, I had a similar error. It seemed to not resolve the dependencies correctly.
Here is what you can try:
Install anaconda
Create a new conda workspace
Download the specific protobuf version that tensorflow needs: https://pypi.python.org/pypi/protobuf/3.0.0a3
Install it via sudo easy_install ~/Downloads/protobuf-3.0.0a3-py2.7.egg
Install a numpy version greater than 1.08.x via conda install numpy
Download the 0.6.0 version of tensorflow: https://storage.googleapis.com/tensorflow/mac/tensorflow-0.6.0-py2-none-any.whl
Install via pip install ~/Downloads/tensorflow-0.6.0-py2-none-any.whl
When you install tensorflow from the whl file directly, it should tell you when dependencies are not there. It seems not to be able to resolve these conflicts independently. My setup had issues with protobuf and numpy. After installing them manually everything worked fine.
I hope this helps!
It seems to be a common issue. Try to install it in the virtualenv. Its a much better solution, as you can always easily set up a new version of tensorflow without conflicts.
VirutalEnv Tutorial:
http://tensorflow.org/get_started/os_setup.md#virtualenv-based_installation
On the Mac, I didn't have any problem installing tensorflow with the anaconda version of python: https://www.continuum.io/downloads
The anaconda version also provides science, math, engineering, and data analysis packages. A lot of people on https://www.kaggle.com/ seem to use this...just a thought.
Sorry that I don't have enough reputation to post images.
The main problem is that it tells me that I need to install a C compiler and reinstall gensim or the train will be slow, and in fact it is really slow.
I have installed mingw32, Visual Studio 2008, and have added the mingw32 environment variable to my path.
Any ideas on how to solve it?
Just as you I installed mingw32 and added MinGW\bin to my PATH variable. I then uninstalled gensim with pip uninstall gensim and tried reinstalling with pip install gensim. That however gave the same warning so instead I downloaded gensim-0.12.3.tar.gz from here. I unzipped the archive and also the inner one, opened a cmd window in the folder and ran python setup.py install to install gensim. Now I could already from the output see that it could see the MinGW install. It installed correctly and I could run gensim without the warning and much faster.
Similar to user1151923, after adding MinGW\bin to my path variable and uninstalling\reinstalling gensim through pip, I still received the same warning message. I ran the following code to fix this problem (installed gensim from conda).
pip uninstall gensim
conda install gensim
I had the same problem and tried many solutions, but none of them worked except degrading to gensim version 3.7.1.
When I installed it from conda-forge then I obtained a version that is already compiled and fast:
conda install -c conda-forge gensim