Cannot install pyarrow from pip or conda - python

I am relatively new to package management and have hit a roadblock getting pyarrow to install in my Windows x64 machine. Pyarrow docs say to use either
conda install -c conda-forge pyarrow
or
pip install pyarrow
https://arrow.apache.org/docs/python/install.html
conda command gives
PackagesNotFoundError: The following packages are not available from current channels:
pip command gives a very lengthy error dump under the heading
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
I tried both in a Python 3.7.9 and a Python 3.8.5 conda environment
since there is Github Jira discussion about this mattering at least
historically (https://issues.apache.org/jira/browse/ARROW-7076)
On the conda approach, I [successfully] added https://conda.anaconda.org/conda-forge/win-64 to current channels, but still 'Package not found' for pyarrow. I did this out of confusion that my list of channels appearing to be 32-bit specific, and my machine is 64-bit
There is ample evidence of pyarrow (2.0.0) presence in Conda-forge/win-64
Thanks for help in getting this package installed successfully.
This is the list of my current channels (with the 'win-64' one having been manually added) in the Conda 'Package not found' message
Current channels:
- https://conda.anaconda.org/conda-forge/win-32
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/win-32
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/win-32
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/msys2/win-32
- https://repo.anaconda.com/pkgs/msys2/noarch
- https://conda.anaconda.org/conda-forge/win-64

Will answer my own question: Arrow does not support 32-bit per per xhochy's response in this thread, so my 32-bit Anaconda Navigator running on my 64-bit hardware was the issue. I will work on getting knowledgeable enough to submit Pull Request to add to Apache documentation because it would have kept someone like me from being battered by this issue.
Solution for getting pyarrow installed in Windows
(Can also do from command line, but I'm working with consulting client who is based in Anaconda Navigator):
If running 32-bit Anaconda Navigator, uninstall it per their instructions Additional detail to their docs are that mentioned envs and pkgs subdirectories are typically at c:\users\username\anaconda3. First, throw these in the trash. Then uninstall Anaconda 32-bit using Add/Remove Programs in Windows as they instruct
Install 64-bit Anaconda from latest installer on their website
Open Anaconda Navigator and click on Environment
(to install for base(root) environment which will be default after fresh install of Navigator) choose Not Installed and click Update Index... button
pyarrow should show up in the updated list of available packages.
Click the Apply button and let it install
That's it to get pyarrow installed to be able to work with .feather format files in Windows Python
I did not test this directly in Conda (outside of Anaconda Navigator), but I presume that the Conda command line install instruction in Apache documentation works just fine if one is starting with 64-bit Python on their Windows machine. Also will mention that Mac OS X Python is apparently 64-bit by default, so this thread addresses a Windows-specific watchout.

Before install pyarrow, you need to install dependency in this page: https://www.microsoft.com/en-us/download/details.aspx?id=48145. Besides, if you have the installation of visual studio,you may have the packges like vc_redist-2015-2019, delete it to make sure you can install these two packages.

Related

vaex binary installation in windows

Installing Python packages can be as frustrating as it can be. Maybe I am the only poor pathetic who still stucks in Windows and the world is living their world happily in Mac and Linux
I am trying to install vaex in my venv environment in Windows. But I got the following unmovable enterprise obstacles:
pip install is not possible as I am unable to install the Visual C++ Build Tools which requires admin rights
Direct Git download or clone do require Visual C++ Build Tool
Conda only up-to version 4.3 which I am not allow to set the proxy when I try to use the conda-forge channel to install
Conda 4.3 is not accepting proxy server settings via command line. There is no way I am able to make conda to use .condarc file. The solution I was proposed is to upgrade to 4.4 which I am not able to do that
Other dependencies pop-up when I try to upgrade conda to 4.4. conda upgrade conda forces me into a whole list of packages that I need to upgrade/install along where I was given no choice to select only conda
Or alternatively, what can I do to compile all the required elsewhere and repackage the whole package for installation?
You could build the wheel distribution of this project (and all its dependencies) on another Windows machine of the same bitness (32 or 64 bits) and with same (minor) version of the Python interpreter. This wheel distribution can then be reused on the target machine.

How to use OpenCV in Python with VS as IDE

I have problems getting a project started. I want to work with the library OpenCV in Python, using Visual Studio 2017 as IDE (on windows 10). I added Python extensions (e.g. Anaconda3 64 Bit) to VS via the VS Installer. After that, the struggle begins. Someone told me, Anaconda would be nice to handle multiple packages, so I opened the Python IDE window and set Anaconda as default. Then in the same window I clicked on packages. After executing
conda install -c conda-forge opencv
conda install -c conda-forge/label/gcc7 opencv
conda install -c conda-forge/label/broken opencv
with the text field above the listed packages I was able to see the opencv package in that list. But when I tried to import opencv in a new python file, it didn't work. (The package was not found)
I also tried to download the latest OpenCV version on the official site, but there I got a folder. I navigated throught this folder and found some .pyd (python extension) files. Can I include them in VS?
I hope anyone can help, thanks in advance!
You can use/try/check the Python Environment Tab (View, Other Windows, Python Environment) to install stuff to any "environment" you have installed:
No idea if it is more comfy or not - I generally use that to install missing dependencies (using pypi - not conda) and the search input to check if it went ok:
I mostly use the python 3.6 for dev - so not much installed over anaconda defaults.
HTH

OpenCV - Can't import cv2 [duplicate]

How to install opencv with python 3.6 and anaconda 3.6?
I tried conda install -c https://conda.binstar.org/menpo opencv3
but i get the following error:
UnsatisfiableError: The following specifications were found to be in conflict:
- opencv3 -> python 2.7*
- python 3.6*
Use "conda info <package>" to see the dependencies for each package.
I am using Windows 10 64-bit, with python 3.6, and anaconda 3.6 installed.
Is it even available for python3.6 at the moment or should i rollback my python version to 3.5.*?
search anaconda prompt
open and run the command.
> pip install opencv-python
this single command help's you to install opencv easily.
you can take help from the video link below.
video link
From menpo file page, it shows that the OpenCV 3.2 binary there are only for Python 2.7/3.4/3.5 and on linux-64 platform
You may go to the this site to get the exact version you need.
opencv_python‑3.2.0‑cp36‑cp36m‑win_amd64.whl is the basic one.
opencv_python‑3.2.0+contrib‑cp36‑cp36m‑win_amd64.whl is the one
with opencv-contrib modules such as the text module for binding to tesseract OCR engine and many others.
Both binary are for OpenCV 3.2 with Python 3.6 binding for Windows 64-bit. To install it, 1) download the binary to local drive, 2) open your Anaconda command prompt and 3) type the command below in the directory the binary locates.
pip install opencv_python‑3.2.0+contrib‑cp36‑cp36m‑win_amd64.whl
Hope this help.
Update on 2018-02-22:
OpenCV 3.4.0 wheel files are now available in the unofficial site and replaced OpenCV 3.3.0
Update on 2019-01-30:
OpenCV 4.0.1 wheel files are now available in the unofficial site with CPython 3.5/3.6/3.7 support.
I managed to get it working by doing the following:
Download and install python3.6 from official python site
https://www.python.org/downloads/release/python-360/
Download and install Anaconda 4.4.0 from the official anaconda site
https://www.continuum.io/downloads
Open command line and run:
pip install opencv-python
Open command line and run:
pip install opencv-contrib-python
I am using Windows 10 and it worked for me.
It's pretty simple..
Install Anaconda 3.6. Check anaconda is added to System Variable Path.
Open CMD and type conda install -c conda-forge opencv.
This will install latest OpenCV version available (3.6).
Open IDE editor and try import cv2.
It will probably don't work...don't worry.
You have to add cv2 command to editor.
For Eclipse (with PyDev):
Create firs a project and then do the following:
For PyCharm:
cv2 module probably won't work. Go to the Anaconda folder/Lib/site-packages/cv2 and copy the file cv2.cp36-win_amd64.pyd to the site-packages folder. Rename it cv2.pyd
Now try to write a command... cv2.imread(). If auto-completition don't work, try cv2.cv2.imread().
This will work for sure.
I am using Python 3.6.2 and Anaconda 4.3.23 (It should also work with your case).
I did the following:
Download the Numpy version corresponding to your Python installation from here. In my case, I’ve used numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl
Download the OpenCV version corresponding to your Python installation from here. In my case, I’ve used opencv_python-3.3.0-cp36-cp36m-win_amd64.whl
Now go to the folder where you downloaded these files and run the following:
pip install numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl
pip install opencv_python-3.3.0-cp36-cp36m-win_amd64.whl
Note the Successfully installed … message after each command.
At this point, you should be able to play with OpenCV and Python. Let’s try a small test first. Start the Python interpreter or Jupyter Notebook and write:
import cv2
print(cv2.__version__)
If everything was correctly installed, you should see the version number of your OpenCV install, in my case this was 3.3.0.
I see you found a solution but this may be helpful for others. The package is not available for Python 3.6. You can check this by going to that package channel on anaconda.org and selecting the files tab. You will see the package tarballs with the Python version listed as py27, py34, py35,etc. This is a good way to check for Python versions of a specific package.
You can also run the following to see the package versions and Python versions available for your OS from the Anaconda channel:
conda search <package_name>
Or to search a particular channel and package you can do this:
conda search -c <channel_name> <package_name>
As of March 2018, OpenCV 3.4 can be installed directly from conda-forge or anaconda in Windows/OSX/Linux for Python 3.6
conda install -c conda-forge opencv
or
conda install -c anaconda opencv
Using:
conda install -c conda-forge opencv
worked for me
If you have installed anaconda then you should uninstall it, then try
pip install opencv_python‑3.2.0+contrib‑cp36‑cp36m‑win_amd64.whl
It worked for me.
Thank You.
I am using python 3.6 and the following worked for me:
Download and install opencv (Win pack) on your computer from the official website:
https://opencv.org/releases.html (I took version 3.4.2)
Go to the website of Christoph Gohlke and download the wheel file corresponding to your system. (I took opencv_python-3.4.2-cp36-cp36m-win_amd64.whl)
As mentioned on the website of Christoph Gohlke, make sure you installed 'numpy1.14' & 'mkl' package. Also make sure you use pip with version 9 or newer.
Start the 'Anaconda Prompt'
Change the directory in the 'Anaconda Prompt' to the folder where you downloaded the wheel file from Gohlke's website (via the MS-DOS command 'cd').
In the 'Anaconda Prompt' type 'pip install opencv_python-3.4.2-cp36-cp36m-win_amd64.whl') (change the name of the wheel file accordingly).
When starting spyder, test your installation as follows:
import cv2
print(cv2.__version__)
If the version is printed in the console (in my case 3.4.2), your installation was successful.
IMPORTANT REMARK:
If you created a dedicated environment within Anaconda (in my case 'py36'), make sure you installed spyder for this dedicated environment ('conda install spyder'). If not, your installation of opencv will not be recognised within the environment you are working in. Maybe this is obvious and straightforward but in my case I struggled to find this solution.
First Download Anaconda Python 3.6 from official site. After installing anaconda, simply open command prompt and type following statement and press enter of course -
conda install -c conda-forge opencv
It may take some time. After the completion, check your conda packages by typing conda list - opencv should be there.
However, Before proceed to install opencv, you can check whether opencv for python 3.6 is available or not. We can check it by typing conda info opencv in command prompt and press enter of course, you'll see following -
opencv 3.3.1 py36h20b85fd_1
---------------------------
file name : opencv-3.3.1-py36h20b85fd_1.tar.bz2
name : opencv
version : 3.3.1
build string: py36h20b85fd_1
build number: 1
channel : https://repo.anaconda.com/pkgs/main/win-64
size : 96.7 MB
arch : None
constrains : ()
license : BSD 3-clause
license_family: BSD
md5 : e65c68524073445511ace8ade7ae3641
platform : None
subdir : win-64
timestamp : 1512689066576
url : https://repo.anaconda.com/pkgs/main/win-64/opencv-3.3.1-py36h20b85fd_1.tar.bz2
dependencies:
jpeg >=9b,<10a
libpng >=1.6.32,<1.7.0a0
libtiff >=4.0.9,<5.0a0
numpy >=1.11.3,<2.0a0
python >=3.6,<3.7.0a0
vc 14.*
zlib >=1.2.11,<1.3.0a0
By this we can also get ensure that opencv 3.3.1 py36h20b85fd_1 is available. And this is available for python 3.6
I think this way is straight forward. Just install anaconda from official page and follow the image.
Using Anaconda3's package manager directly will be more reliable and cross-platform:
conda install opencv

How to upgrade to the latest Anaconda 5.0.1

I have Anaconda 4.4.0 (Windows, Python 3.6., 64 bit).
I would like to upgrade to latest Anaconda 5.0.1
Few options:
Download the full installer and run it
From existing installation (of 4.4.0) run "conda update --all"
From existing installation run "conda update anaconda"
What is the tradeoff among these options? What is the recommended one?
Download the full installer: Provided that you uninstall your existing Anaconda, this method will be least likely to cause upgrade problems. It will also probably be slower. Note that I think you should uninstall the old Anaconda so that you don't end up with two conda[.exe] files, two Anaconda Prompt shortcuts, and so forth. You may end up trying to install a package with the wrong conda and be very confused about what's happening.
conda update --all: This will update all of your packages in the environment to their latest version, regardless of their version in the Anaconda installer. This is not recommended because you will end up with package versions that are different from the ones in the Anaconda installer and you may end up with an error message about packages that are incompatible.
conda update anaconda: This will update the "metapackage" called anaconda to the latest version. This package has dependencies on specific versions of all of the packages and Anaconda (the company) give some assurance that these will all work together. So, updating the anaconda package will update all your packages to the version used in the latest version of the Anaconda installer.
My suggestion (based on some experience, I am not an employee of Anaconda) would be to try #3 and if it fails, try #1.

Python - pip install pandas, not working

I am trying to install pandas via pip install pandas but when I do, I get the error:
Command python setup.py egg_info failed with error code 1 in
c:\users[username]\appdata\local\temp\pip_build_[username]\pandas
I followed the answer given here and installed ez_setup.py without incident, but still get the error when doing pip install pandas.
Thanks for any help, if I can provide more information please let me know.
The easiest way to install pandas and its dependencies on Windows is to download the relevant packages from Christoph Gohlke's Python Extension Packages for Windows repository. You'll find the files for pandas here as well as a list of other required dependencies.
On Linux (Debian / Ubuntu varieties), when NOT installing inside a virtual environment, but in the main system, I find it best to just use the Synaptic Package Manager (because even the --user switch seems to fail when trying to install pandas without sudo). Search for pandas inside Synaptic PM. There's varieties for python 2 and 3.
However, on linux, I have generally found the cleanest, easiest, and overall safest approach to be creating virtual environments and then use pip install <package name> inside the virtual environment. I believe this would be best on Windows too.
I installed Pandas package following procedure listed after the following disclaimer section:
Disclaimer:
- I don't consider myself a computer expert so follow the instructions at your own risk.
- My procedure worked on my windows computer
- My windows computer has python 2.7 installed from python.org
- My python GUI is IDLE
- I don't recall installing pip, it is possible that it installs at the time of installing python 2.7 from python.org (not sure)
- The directory in which my pip.exe is located is under C:\Python27\Scripts
Procedure:
Open a command window for the directory under which you have pip.exe, (in my case is Scripts). The prompt looks like this in my case: C:\Python27\Scripts>
At the prompt type pip install pandas. The prompt looks like this in my case: C:\Python27\Scripts>pip install pandas
Press ENTER key. You should see message: "Collecting pandas" being displayed in the command window.
Once the system completes collecting pandas, you should see message "Successfully installed pandas-0.22.0" , or similar depending on version collected.
Picture shows steps 1 -4 as shown in my computer command window.collecting_pandas

Categories

Resources