I am trying to install keras-bert as explained here. Although it is successfully installed on the environment as, I cannot see keras-bert inside the R side.
(bert_env) C:\Users\xxxxx\Dropbox\Rcode\ProjectBERT>pip list
Package Version
-------------------------------- ---------
certifi 2022.6.15
keras 2.6.0
keras-bert 0.89.0
keras-embed-sim 0.10.0
keras-layer-normalization 0.16.0
keras-multi-head 0.29.0
keras-pos-embd 0.13.0
keras-position-wise-feed-forward 0.8.0
keras-self-attention 0.51.0
keras-transformer 0.40.0
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
numpy 1.23.1
pip 22.1.2
setuptools 61.2.0
six 1.16.0
wheel 0.37.1
wincertstore 0.2
Another proof which shows keras-bert is successfully install on the current environment (bert_env):
(bert_env) C:\Users\xxxxx\Dropbox\Rcode\ProjectBERT>pip install keras-bert
Requirement already satisfied: keras-bert in c:\anaconda3\envs\bert_env\lib\site-packages (0.89.0)
Requirement already satisfied: keras-transformer==0.40.0 in c:\anaconda3\envs\bert_env\lib\site-packages (from keras-bert) (0.40.0)
However, I cannot properly call the package on the R side which says:
> reticulate::conda_list()
name python
1 base C:\\Anaconda3/python.exe
2 bert_env C:\\Anaconda3\\envs\\bert_env/python.exe
3 py27 C:\\Anaconda3\\envs\\py27/python.exe
> reticulate::use_condaenv("bert_env", required=TRUE)
> reticulate::py_config()
C:\ANACON~2\envs\bert_env\lib\site-packages\numpy\__init__.py:138: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service
from . import _distributor_init
python: C:/Anaconda3/envs/bert_env/python.exe
libpython: C:/Anaconda3/envs/bert_env/python310.dll
pythonhome: C:/Anaconda3/envs/bert_env
version: 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Anaconda3/envs/bert_env/Lib/site-packages/numpy
numpy_version: 1.23.1
I have almost tried everything which offered as solution such as:
conda update conda
conda update --all
Then I have created a fully clean new environment in order to reinstall keras-bert on there. Also looked scipy and numpy package conflicts which is mentioned on their github side. No success at all!
Of course, once I check availability, it returns false!
reticulate::py_module_available('keras_bert')
[1] FALSE
I am aware of this post and applied exactly same patterns with YAML. No improvements as well. Any suggestions to properly install keras-bert or how should I approach the issue are greatly appreciated.
After a couple of days search, I found a way to set the reticulate path properly. This issue solves the main issue as well. .Renviron file is a user-controllable file that can be used to create environment variables. Hence, the .Renviron file can be used to store the information which can be accessed from all R projects in your computer.
First of all, you need to find the location of your .Renviron file. In order to find the path of your .Renviron file, you can type this on RStudio console:
usethis::edit_r_environ()
In my case, it returns:
* Modify 'C:/Users/xxxxx/Documents/.Renviron'
* Restart R for changes to take effect
Then, you might either navigate the location of .Renviron file manually. (Note: It was empty once I opened it in my scenario.) Then paste the full path of python.exe inside to the file as:
RETICULATE_PYTHON="YourEnvironmentPath/python.exe"
Tip: If you are not sure about your full python path, you can get it as:
reticulate::conda_python("keras_bert")
Please do not forget to type your environment name instead of my environment nam > keras_bert).
Then I got:
[1] "C:\\Anaconda3\\envs\\keras_bert/python.exe"
In my case, I pasted:
RETICULATE_PYTHON="C:\\Anaconda3\\envs\\keras_bert/python.exe"
Or (on Windows), you can click on start and open Windows Powershell. Then copy this code into powershell to handle same thing:
Add-Content C:\Users\xxxxx\Documents/.Renviron 'RETICULATE_PYTHON="C:\\Anaconda3\\envs\\keras_bert/python.exe"'
In order to see updates, you must close and reopen RStudio to have a clean RStudio session. Now, run the following line to check whether everything is fine or not:
Sys.getenv('RETICULATE_PYTHON')
or:
reticulate::py_config()
In my case, it returns:
[1] "C:/Anaconda3/envs/keras_bert/python.exe"
or:
python: C:/Anaconda3/envs/keras_bert/python.exe
libpython: C:/Anaconda3/envs/keras_bert/python38.dll
pythonhome: C:/Anaconda3/envs/keras_bert
version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 05:59:45) [MSC v.1929 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Anaconda3/envs/keras_bert/Lib/site-packages/numpy
numpy_version: 1.23.2
NOTE: Python version was forced by RETICULATE_PYTHON
Briefly, with this solution, you do not need to care about whether you declare the environment at the start of the R session or any other messy scenarios.
Related
I want to use PyProj. I follow the installation instructions given in the documentation
conda config --prepend channels conda-forge
conda config --set channel_priority strict
conda install pyproj
When I run
from pyproj import CRS
I am getting
C:\Users...\Miniconda3\envs\geopandas\lib\site-packages\pyproj_init_.py:89: UserWarning: pyproj unable to set database path.
_pyproj_global_context_initialize()
Some information on Python and my system
Python version
-> 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:15:42) [MSC v.1916 64 bit (AMD64)]
Operation System Information
-> Windows-10-10.0.19042-SP0
Quick an dirty
Make sure you are in you environment with conda activate YOUR_ENV:
conda remove --force pyproj
This removes the library from the conda environment. Then do a
pip install pyproj
Now I do not get the error upon calling from pyproj import CRS.
Proper way
However, this is not going through to the core of the problem. I had conflicting versions of the proj.db. Check the value of the PROJ_LIB environment variable. Mine pointed to an installation of postgis. This helps to solve it: https://pyproj4.github.io/pyproj/stable/gotchas.html#internal-proj-error-sqlite-error-on-select
How did I find the path to the installation of proj?
I looked into this file ...Miniconda3\envs\<ENV>\conda-meta\proj-<VERSION>-h1cfcee-<VERSION>.json and searched for \pkgs\\proj. Once found, I checked the versins of PROJ installed in this folder C:...\Miniconda3\pkgs\. There are folder named something like proj-<VERSION>-h1cfcee-<VERSION>. Make sure the folder name is the exact same name you found this file links to: ...Miniconda3\envs\<ENV>\conda-meta\proj-<VERSION>-h1cfcee-<VERSION>.json. Finally I set my environment variable to the version needed ...Miniconda3\pkgs\proj-<VERSION>-h1cfcee-<VERSION\Library\share\proj.
I'm trying to start using Python 3.9 and am unable to make this happen.
From the Anaconda Prompt I run and get this:
C:\Users\fname lname>python --version
Python 3.7.10
I thought I had installed 3.9 already, so I check to see if I have it and I run this command:
C:\Users\fname lname>conda search python
Loading channels: done
# Name Version Build Channel
python 2.7.13 h1b6d89f_16 pkgs/main
python 2.7.13 h9912b81_15 pkgs/main
...
python 3.9.1 h6244533_2 pkgs/main
python 3.9.2 h6244533_0 pkgs/main
Sure enough it looks like I have 3.9.2 so I run
conda install python=3.9.2
Here I'm getting a lot of conflict errors and I'm not sure really what actually is happening if anything.
It's a pretty long list of items that pop up and most of it scrolls long enough that I can't recover all of the lines:
...
Package hdf5 conflicts for:
hdf5
opencv -> hdf5[version='>=1.10.2,<1.10.3.0a0|>=1.10.4,<1.10.5.0a0|>=1.8.20,<1.9.0a0|>=1.8.18,<1.8.19.0a0']
Package pkginfo conflicts for:
conda-build -> pkginfo
pkginfo
Package pywin32-ctypes conflicts for:
keyring -> pywin32-ctypes[version='!=0.1.0,!=0.1.1']
pywin32-ctypes
spyder -> keyring[version='>=17.0.0'] -> pywin32-ctypes[version='!=0.1.0,!=0.1.1']
And running the same version command as before I get Python 3.7.10
I am trying to install requests module, and openssl keep causing the InvalidArchiveError.
I am using conda 4.7.12, python 3.8.2, on Windows 10 operating system. I had the same issue when installing other packages requiring openssl-1.1.1g. I have followed the advice from the error message to delete and re-download 'openssl-1.1.1g-he774522_0.tar.bz2', but I keep getting the same error.
Is this version of openssl broken or something? Any help will be greatly appreciated.
lykim#Louis MINGW64 ~/Desktop/master/Learning
$ conda install -c anaconda requests
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done
==> WARNING: A newer version of conda exists. <==
current version: 4.7.12
latest version: 4.8.3
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: C:\ProgramData\Anaconda3\envs\pytorch
added / updated specs:
- requests
The following packages will be downloaded:
package | build
---------------------------|-----------------
openssl-1.1.1g | he774522_0 5.8 MB anaconda
------------------------------------------------------------
Total: 5.8 MB
The following NEW packages will be INSTALLED:
brotlipy conda-forge/win-64::brotlipy-0.7.0-py38h1e8a9f7_1000
cffi anaconda/win-64::cffi-1.14.0-py38h7a1dbc1_0
chardet anaconda/win-64::chardet-3.0.4-py38_1003
cryptography anaconda/win-64::cryptography-2.9.2-py38h7a1dbc1_0
idna anaconda/noarch::idna-2.9-py_1
pycparser anaconda/noarch::pycparser-2.20-py_0
pyopenssl anaconda/win-64::pyopenssl-19.1.0-py38_0
pysocks anaconda/win-64::pysocks-1.7.1-py38_0
requests anaconda/win-64::requests-2.23.0-py38_0
urllib3 conda-forge/noarch::urllib3-1.25.9-py_0
win_inet_pton anaconda/win-64::win_inet_pton-1.1.0-py38_0
The following packages will be SUPERSEDED by a higher-priority channel:
certifi conda-forge::certifi-2020.4.5.1-py38h~ --> anaconda::certifi-2020.4.5.1-py38_0
openssl conda-forge --> anaconda
Proceed ([y]/n)? y
Downloading and Extracting Packages
openssl-1.1.1g | 5.8 MB | ########## | 100%
InvalidArchiveError('Error with archive C:\\ProgramData\\Anaconda3\\pkgs\\openssl-1.1.1g-he774522_0.tar.bz2. You probably need to delete and re-download or re-create this file. Message from libarchive was:\n\nCould not unlink')
(pytorch)
I encountered the same problem a couple of times. This time my problem was solved simply by closing down jupyter notebook, which was running and using openssl, so if your python is running somewhere, try to close it down. (In addition had already removed openssl-1.1.1g-he774522_0.tar.bz2 and a number of folders openssl-1.1.1g-he774522_0 containing also those .tar balls, so that might be necessary as well)
You may try to go to your packages directory
C:\ProgramData\Anaconda3\pkgs\
Then delete openssl-1.1.1g-he774522_0.tar.bz2 file, install libarchive and reinstall your package.
Another Solution
Simply you can install on your anaconda environment using pip instead of conda, anyway i recommend to update your conda too.
Encountered the same error, I think it was caused by an earlier failed installation which had left an incomplete openssl-1.1.1g-he774522_0 directory in users\username\Anaconda3\pkgs. Just delete it and then it completes just fine.
I had the same issue. There was some "openssl*" folders. Deleted them, and the "openssl*" .bz2 files as well, and tried to install again. Now works perfectly.
Eventually the folders can't be deleted because they're used by other programs. If it happens, go to Task Manager and close python.exe.
I was able to fix my error by elevating my command window. The delete method wasn't working.
I have tensorflow listed as a requirement in the install_requires section of the setup.py of my project.
When I attempt to install my project into a new Anaconda environment I get the following error:
$ python setup.py install
...
Searching for tensorflow
Reading https://pypi.org/simple/tensorflow/
No local packages or working download links found for tensorflow
error: Could not find suitable distribution for Requirement.parse('tensorflow')
I can get past this by installing tensorflow "manually" via conda:
$ conda install tensorflow
Once I do this the install via setup.py works without a hitch.
Am I mistaken in assuming that something is amiss with my environment? If not then what is going on and how can I avoid this issue? (My concern is that users of my package will not be able to install from source using setup.py)
Another oddity that I assume is related or may provide a clue is that the version of TensorFlow listed in my Anaconda environment is 2.0 but if I import it when running Python it appears to instead be using version 1.15. For example:
$ conda list tensorflow
# packages in environment at /home/james/miniconda3/envs/cvdata_test:
#
# Name Version Build Channel
tensorflow 2.0.0 mkl_py37h66b46cc_0
tensorflow-base 2.0.0 mkl_py37h9204916_0
tensorflow-estimator 2.0.0 pyh2649769_0
$ python
Python 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 22:33:48)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.15.0-rc2'
This is on a Dell laptop running Ubuntu 18.04 without a GPU, so perhaps the version shown in the interpreter is akin to tensorflow-cpu? If I run pip freeze I see tensorflow==2.0.0 and tensorflow-cpu==1.15.0rc2, which is a bit confusing...
This appears to have been caused by using Python version 3.8, which is currently an unsupported version of Python. Once I created a new Anaconda environment with Python version 3.7 this issue went away.
The only remaining issue is this error that I see when I run pip install -e . for my project which includes tensorflow:
ERROR: tensorflow-cpu 1.15.0rc2 has requirement tensorboard<1.16.0,>=1.15.0, but you'll have tensorboard 2.1.0 which is incompatible.
ERROR: tensorflow-cpu 1.15.0rc2 has requirement tensorflow-estimator==1.15.1, but you'll have tensorflow-estimator 2.1.0 which is incompatible.
So the issue of tensorflow-cpu version 1.15.0rc2 actually being installed when version 2.1.0 shows as being the installed version is still a mystery. To wit:
$ conda list tensorflow
# packages in environment at /home/james/miniconda3/envs/cvd:
#
# Name Version Build Channel
tensorflow 2.1.0 pypi_0 pypi
tensorflow-estimator 2.1.0 pypi_0 pypi
$ python -c "import tensorflow as tf; print(tf.__version__)"
1.15.0-rc2
I have been following the installation guide from http://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/
I got & am using:
conda 4.3.22
Python 3.5.3 :: Anaconda 4.4.0 (32-bit)
scipy: 0.19.0
numpy: 1.12.1
matplotlib: 2.0.2
pandas: 0.20.1
statsmodels: 0.8.0
sklearn: 0.18.2
I successfully installed theano & keras. HOWEVER, I FAIL at installing tensorflow. Please HELP.
I created a conda ‘tensorflow’ environment with python 3.5. With command
『pip install –ignore-installed –upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-1.2.1-cp35-cp35m-win_amd64.whl』
I got ERROR saying
『tensorflow-1.2.1-cp35-cp35m-win_amd64.whl is not a supported wheel on this platform』
So i changed to version 1.0.1 and same error.
Version 1.1.0 also same error.
So i deactivated the environment, and type command
『conda install -c conda-forge tensorflow』
I got ERROR
『PackageNotFoundError: Package missing in current win-32 channels』
Instead it says the close match found is “xtensor” which i know is a C++ library that I'm not looking for.
Is it because I’m using a 32-bit Windows 10?
So I also tried running the following :
『python -m pip install –upgrade tensorflow』
and got ERROR of
『Could not find a version that satisfies the requirement tensorflow (from versions: ) No matching distribution found for tensorflow』
What more requirements do i need for this?
I tried 『pip3 install tensorflow』 but somehow it could not recognized ‘pip3’. So i type 『where pip3』 and it could not find files for the given pattern. So i type『where python』. It ouput the directory of my python. Then checked if it’s already put under the path inside the environmental variable. And it has. But i still couldn't use pip3 command.
I have tried all the solutions provided from people having similar problem with me and none of them work.
This question has been answered here.
In short, yes, TensorFlow does not support 32-bit platforms. Although if you only plan on writing abstract high-level Keras code then Theano will do just fine.