While trying to following the instructions of installing Keras and TensorFlow on Rstudio link https://keras.rstudio.com/index.html I get the following error. It is a work computer that is running Windows 7. I am not familiar with python, but I believe I have python 3.6 installed correctly (I am able to run simple python code in the Spyder IDE). Thanks in advance for any suggestions on how to get this working.
> install_keras()
Creating r-tensorflow conda environment for TensorFlow installation...
Solving environment: ...working... failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url
<https://repo.continuum.io/pkgs/main/noarch/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your
way.
ConnectionError(MaxRetryError("HTTPSConnectionPool(host='repo.continuum.io', port=443): Max retries exceeded with url: /pkgs/main/noarch/repodata.json.bz2 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000000000474D860>: Failed to establish a new connection: [Errno 11004] getaddrinfo failed',))",),)
Error: Error 1 occurred creating conda environment r-tensorflow
In addition: Warning message:
running command '"C:\Users\...\...\Local\CONTIN~1\ANACON~1\Scripts\conda.exe" "create" "--yes" "--name" "r-tensorflow" "python=3.6"' had status 1
Installing Keras and TensorFlow using install_keras() isn't required to use the Keras R package. You can do a custom installation of Keras (and desired backend) as described on the Keras website and the Keras R package will find and use that version.
Source
So you can circumvent this issue with the firewall using a custom installation. The R package keras will then find that installation automatically. See the linked source for more information on how to do a custom installation.
edit: btw, there is a similar question that has been answered here. That poster goes into changing the proxy settings to cicrumvent the firewall. I cannot mark this question as a duplicate due to active bounty.
I suggest you first update your conda in terminal:
conda update --all
Then run the following commands in R:
install.packages("tensorflow")
library(keras)
to_categorical(0:3)
You could also test the followings if you get any error at the time of installing tensorflow and keras:
install.packages("tensorflow")
install.packages("keras")
install_keras()
install_tensorflow()
There is a good answer here:
https://github.com/rstudio/keras/issues/649
(scroll down to the answer by skeydan)
Here is the answer:
First install tensorflow directly from GitHub, as in
devtools::install_github("rstudio/tensorflow")
devtools::install_github("rstudio/keras")
Then do
tensorflow::install_tensorflow()
tensorflow::tf_config()
which should give you version 1.12.
We have been installing TF 1.10 until yesterday because of a bug in that will only be fixed in TF 1.13 (which should be out anytime but unfortunately isn't yet). Now with people having installation problems due to incompatibilities with numpy as installed by conda, we decided to switch to TF 1.12 now and as soon as TF 1.13 is actually available, we'll install that by default.
If you still run into problems installing, please open another issue and describe the problem, indicating the output of
reticulate::py_discover_config()
reticulate::use_condaenv("r-tensorflow")
reticulate::py_config()
Thanks!
Related
I am writing because I have a problem (silly and obvious introduction, I know).
I am trying to use the BERTopic package using the Python interpreter in RStudio and the reticulate extension:
Python 3.6.13 (C:/Users/Francesco/AppData/Local/r-miniconda/envs/r-reticulate/python.exe)
Reticulate 1.18.9008 REPL -- A Python interpreter in R.
I managed to install it with
pip3 install bertopic
At first, trying to install bertopic resulted in an error realating to its hdbscan dependence, specifically to the wheel used; I overcame it by installing hdbscan by conda (with pip the problem appeared unsolvable) and after doing it seemed that both were installed and fine (pip would confirm so).
Afterwards, I tried to follow the package tutorial in Medium/Towards Data Science (here the Colab version I’m following) to get accostumed with the package and to check that everything was working as supposed to.
I am basically copying and pasting the code of Colab on the Python chunks in the RMarkdown file I am using, but when I try to apply the same code of the tutorial to the same dataset used:
from sklearn.datasets import fetch_20newsgroups
docs = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes'))['data']
from bertopic import BERTopic
topic_model = BERTopic(language="english", calculate_probabilities=True, verbose=True)
topics, probs = topic_model.fit_transform(docs)
I get the following error:
Batches: 100%|##########| 589/589 [28:21<00:00, 2.89s/it]
2021-04-29 16:24:25,973 - BERTopic - Transformed documents to Embeddings
2021-04-29 16:24:35,752 - BERTopic - Reduced dimensionality with UMAP
OSError: [Errno 22] Invalid argument
In theory, following the output on colab, I should get:
....................... - BERTopic - Clustered UMAP embeddings with HDBSCAN
Since I had problem with hdbscan I do believe it is somehow related to it, and I read several GitHub and Stackoverflow pages pointing out problems with such a package, but I do not know how to solve this, but I really need to since I need to use package for my thesis.
Can someone help me, please?
PS: it's the first time I am asking stuff on stackoverflow: I hoped I wrote down everything necessary, but if some info is missing, please tell me.
I try to run Tensorflow with GPU support (GTX 1660 SUPER).
I created an enviroment using anaconda, than installed cudatoolkit (version 11.0.221) and tensorflow-gpu (version 2.4.1). Afterwards, I downloaded cuDNN (version 8.0.4), and copied all files from cuDNN's bin folder to my environment's bin folder at anaconda3\envs\<env name>\Library\bin.
In my script, I've set the memory limit to my GPU's memory using tf.config.experimental.set_memory_growth.
When I run the script (which uses convolutional algorithms), I get a warning that says Couldn't invoke ptxas.exe --version which comes after an Call to CreateProcess failed. Error code: 2 error.
After the launch failure, I get: Relying on driver to perform ptx compilation. Modify $PATH to customize ptxas location.
I've already tried switching to cuDNN version 8.1.1.
How I fix this?
I got a new fix for this.
First I tried using tensorflow=2.3, cudnn=7.6.5 and cudatoolkit=10.1 as mentioned in previous answers. However, every time I put a model to train, the process was going stale and the training seemed to be stuck in epoch 1.
I then managed to include ptxas in my conda environment by running conda install -c nvidia cuda-nvcc The packages I am using are:
tensorflow=2.9, cudnn=8.1.0, cudatoolkit=11.2.2, cuda-nvcc=11.7.99 and python=3.9
I am running everything on windows 10 flawlessly now.
For the benefit of community adding #Zuk Levinson comment
Solves the issue by using
tensorflow=2.3, cudnn=7.6.5 and cudatoolkit=10.1
I tried running this command but i get erros that i dont have tenserflow 2.2 or higher. But I checked and I have the correct version of tenserflow. I also did pip3 install keras command
I know for a fact that all of the code is correct because it worked for my teacher the other day and nothing has changed. I just need to run his commands but i keep running into problems
I am doing this course following everything he does in a recorded video so there must be no issue there but for some reason it just doesn't work
just install tensorflow as requested in the last line of the error message: pip install tensorflow. It is needed as backend for Keras.
Also, since keras is part of tensorflow now, I recommend to write imports as from tensorflow.keras.[submodule name] import instead of from keras.[submodule name] import
I have a problem with Apache MXNet machine learning library on OS X.
I have been able to run Python version of Lenet, convolutional neural network.
I installed these with pip under both Anaconda Python 2.7 and 3.6.
conda create -n mxnet27 python=2.7
conda info --envs
source activate mxnet27
conda list
pip install mxnet==0.12.1
But when I run C++ example files cpp-package/example/lenet.cpp I get the this segfault:
Segmentation fault: 11
This is the place in the code where the segfault is thrown:
Symbol conv1 =
Convolution("conv1", data, conv1_w, conv1_b, Shape(5, 5), 20);
I get similar segfault for the other C++ examples.
I have built MXNet on OS X 10.13.2
I disabled as many libraries as possible, e.g. OpenCV and CUDA.
On Simon Corston-Oliver suggestion I upgraded to MXNet 1.0.0, but that version did not compile with Clang on OS X. Error message:
operator_tune.h:150:36: note: add an explicit instantiation declaration to suppress this
warning if 'mxnet::op::OperatorTuneByType<float>::tuning_mode_' is explicitly instantiated in another translation unit
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/unordered_map:601:15: error: object of type 'std::__1::pair<int,
mxnet::test::perf::TimingInstrument::Info>' cannot be assigned because its copy assignment operator is implicitly deleted
I don't know of a specific issue with v0.12 that would lead to a segfault but before we dig in, I'd recommend upgrading to v1.0 which was released 2017-12-04.
If you still encounter the same problem with 1.0 we can work to debug.
I found a solution to compiling MXNet 1.0.0 posted here by helloniklas:
https://github.com/apache/incubator-mxnet/issues/9217
It involved only using make instead of CMake.
This solution worked me and compiled the code.
C++ examples runs without the seg fault, but documentation is scarce. I only got one of the to do training.
In development, I have been using the gpu-accelerated tensorflow
https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.2.1-cp35-cp35m-linux_x86_64.whl
I am attempting to deploy my trained model along with an application binary for my users. I compile using PyInstaller (3.3.dev0+f0df2d2bb) on python 3.5.2 to create my application into a binary for my users.
For deployment, I install the cpu version, https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.2.1-cp35-cp35m-linux_x86_64.whl
However, upon successful compilation, I run my program and receive the infamous tensorflow cuda error:
tensorflow.python.framework.errors_impl.NotFoundError:
tensorflow/contrib/util/tensorflow/contrib/cudnn_rnn/python/ops/_cudnn_rnn_ops.so:
cannot open shared object file: No such file or directory
why is it looking for cuda when I've only got the cpu version installed? (Let alone the fact that I'm still on my development machine with cuda, so it should find it anyway. I can use tensorflow-gpu/cuda fine in uncompiled scripts. But this is irrelevant because deployment machines won't have cuda)
My first thought was that somehow I'm importing the wrong tensorflow, but I've not only used pip uninstall tensorflow-gpu but then I also went to delete the tensorflow-gpu in /usr/local/lib/python3.5/dist-packages/
Any ideas what could be happening? Maybe I need to start using a virtual-env..