I've downloaded python libraries (tar.gz) from external links. How to use these libraries, I mean in which directory under the python should I unzip such libraries.
How to use downloaded libraries with python when we are using:
windows 7
Ubuntu
Also how to install these libraries without using pip in Linux? Do we have to set path in windows7 to use the libraries?
Guys better to post comments then to give negative points. With your suggestion I can modify my post. I dont think you to be smart guy by rating negative points in any comments.
For Linux based systems (e.g., Ubuntu):
$ tar xzf package.tar.gz
$ cd package
$ python setup.py install
Also, look input pip https://pypi.python.org/pypi/pip
On Windows and Linux, all you need to do is unzip the archive somewhere, cd into and run the following command as administrator (usually by running cmd.exe as Administrator on Windows or by using sudo on Linux):
python setup.py install
If the package does not have a setup.py, it can get tricky. Most of the time the package just needs to be copied into your Python's site-packages folder.
You will find some packages that need to be built. On Linux, you can usually do this pretty easily. On Windows, you will need the appropriate compiler, usually a free version of Visual Studio will do, although you have to make sure you get the correct version of Visual Studio. I've heard reports that you can also use MingW.
Related
A little background: I am working on some python modules that other developers on our team will use. A common theme of each module is that one or more messages will be published to Kafka. We intend at this time to use the Confluent Kafka client. We are pretty new to python development in our organization -- we have traditionally been a .NET shop.
The complication: while the code that we create will run on Linux (rhel 7), most of the developers will do their work on Windows.
So we need the librdkafka C library compiled on each developer machine (which has dependencies of its own, one of which is OpenSSL). Then a pip install of confluent-kafka should just work, which means a pip install of our package will work. Theoretically.
To start I did the install on my Linux laptop (Arch). I knew I already had OpenSSL and the other zip lib dependencies available, so this process was painless:
git clone librdkafka repo
configure, make and install per the README
pip install confluent-kafka
done
The install of librdkafka went into /usr/local:
/usr/local/lib/librdkafka.a
/usr/local/lib/librdkafka++.a
/usr/local/lib/librdkafka.so -> librdkafka.so.l
/usr/local/lib/librdkafka++.so -> librdkafka++.so.l
/usr/local/lib/librdkafka.so.l
/usr/local/lib/librdkafka++.so.l
/usr/local/lib/pkgconfig/rdkafka.pc
/usr/local/lib/pkgconfig/rdkafka++.pc
/usr/local/include/librdkafka/rdkafkacpp.h
/usr/local/include/librdkafka/rdkafka.h
Now the painful part, making it work on Windows:
install precompiled OpenSSL
git clone librdkafka repo
open in VS2015
install libz via NuGet
build solution
install to where???
This is where I'm stuck. What would a standard install on a Windows 7/8/10 machine look like?
I have the following from the build output, but no idea what should go where in order to make the pip install confluent-kafka "just work":
/librdkafka/win32/Release/librdkafka.dll
/librdkafka/win32/Release/librdkafka.exp
/librdkafka/win32/Release/librdkafka.lib
/librdkafka/win32/Release/librdkafkacpp.dll
/librdkafka/win32/Release/librdkafkacpp.exp
/librdkafka/win32/Release/librdkafkacpp.lib
/librdkafka/win32/Release/zlib.dll
<and the .h files back in the src>
Any recommendations on an install location?
I'm not sure where the ideal place to install on Windows would be, but I ran the following test with some success.
I copied my output and headers to C:\test\lib and C:\test\include, then ran a pip install with the following options:
pip install --global-option=build_ext --global-option="-LC:\test\lib" --global-option="-IC:\test\include" confluent-kafka
Unfortunately, this doesn't quite work because the confluent-kafka setup does not support Windows at this time: https://github.com/confluentinc/confluent-kafka-python/issues/52#issuecomment-252098462
It's an old question but seems still no easy answer yet. Also Confluent seems too busy to work on Windows supporting...
I had the same headache couple weeks ago and after some research, I managed to make it work for me on Windows. I logged my finding and uploaded a pre-compiled library to my Git, please check and see if it helps. :D
https://github.com/MichaelZhangCA/confluent-kafka-python
My environment is Python 3.6 64 bit version but ideally, it should also work for 32 bit if you follow the same approach.
I assume you have successfully followed instructions from MichaelZhangCA (https://github.com/MichaelZhangCA/confluent-kafka-python/) from previous post.
If you did so, these probably were the last two commands executed:
::Install confluent-kafka
cd C:\confluent-kafka-python\confluent-kafka-python-0.11.4
python setup.py install
If that is correct, those DLLs were created under C:\confluent-kafka-python\librdkafka-reference\release\.
All you have to do is to copy them to a diretory already in Widnows' PATH.
For example, I use Anaconda 5.2 For Windows, with python 3.6. My Anaconda Prompt has an empty directory in PATH, so I copied there those DLL's:
::Anaconda Prompt - copy DLLs to a directory already in PATH
mkdir %CONDA_PREFIX%\Library\usr\bin
copy C:\confluent-kafka-python\librdkafka-reference\release %CONDA_PREFIX%\Library\usr\bin
If you don't use Anaconda, just copy those DLL's to any other directory in Windows' PATH. You may also leave them in C:\confluent-kafka-python\librdkafka-reference\release, and add this directory to PATH.
I installed distribute and pip using the links I have just given. I also installed the Microsoft Visual C++ 2008 redistributable package. However when I try to use pip.exe I get
error: Unable to find vcvarsall.bat
How can I fix this?
Installing the Microsoft Visual C++ 2008 Redistributable Package is not sufficient to compile packages. You need to install a compiler, not just the support files.
There are three ways to do this:
Install Visual C++.
Use mingw's port of gcc instead of Visual C++.
Use cygwin's port of gcc instead of either, and a cygwin build on Python instead of the native one.
If you want to go with option 1, you need to install Visual C++ itself. The free version should work just as well as the paid version, as long as you're not going to build binary packages to redistribute to others. Unfortunately, I'm not sure where to find the 2008 version anymore. As of May 2013, the download page only has 2010 and 2012.
When you install this, it will create a batch file called vcvarsall.bat (not vcvarshall.bat!), and give you the option of putting that batch file in your PATH. Running that batch file sets up a DOS prompt for building with that version of Visual C++. (This is handy if you have multiple versions of Visual C++, or other compilers, around.) If you skip that option, you will have to do it manually.
This question shows how to use a newer Visual Studio with older Python, and also shows how to point distutils at a vcvarsall.bat that's not on your PATH, and has links to a whole lot of other relevant questions and blog posts.
Many people find option 2 simpler. Install mingw, modify your PATH in the environment to include C:\MinGW\bin (or wherever you choose to install it), and pass -c mingw32 whenever you run a setup.py script.
The problem is that it's not as clearly documented how to tell easy_install and pip to use mingw instead of VC++. To do that, you need to find or create a distutils.cfg file, find or create a [build] section within it, and add compiler=mingw32. Not too hard. This blog post looks like it explains things pretty well, or see this answer.
Option 3 is by far the simplest. Install cygwin, tell it to install the Python and gcc packages, and you're done.
The problem is that you don't have native Windows Python, you have a Unix Python running in a fake Unix environment on top of Windows. If you like Cygwin, you'll love this; otherwise, you won't.
You'll receive such error only for packages (or one of package's dependencies) that has CPython extensions. Pip internally:
downloads the source
runs distutils python setup install
install prepares setup files and tries to build CPython extensions in windows environment
windows environment calls MS Visual Studio vcvarsall.bat script which setups DOS environment variables to enable MS Visual Studio's C compiler in the shell
if vcvarsall.bat is not found - you'll get this message
Usual solution
For python libraries which have CPython extensions that are portable on windows, it is usual to have windows binary package, which are downloadable from pypi or library web site.
In such cases it is more suitable (and painless) to install library by downloading and running windows binary package.
There is a feature request for pip to Add support for installation of binary distutils packages on Windows.
New way to do it - wheels
Thanks to comment from #warren-p: That feature request has been superseeded by Wheels support in PIP.
Official description: A wheel is a ZIP-format archive with a specially formatted filename and the .whl extension.
As I have understood, if there is windows binary package with extension .whl then start by installing wheel first:
# Make sure you have the latest pip that supports wheel
pip install --upgrade pip
pip install wheel
and then install .whl like this:
pip install full-path-or-url-to-your-library.whl
References:
pythonwheels.com
https://pypi.python.org/pypi/wheel
http://wheel.readthedocs.org/en/latest/
You can download Visual Studio 2008 Express SP1 from
http://visual-studio-2008.en.malavida.com/
You can deselect the two add to browser options it offers.
I found these links on microsoft.com that still work to Install Visual C++.
http://download.microsoft.com/download/8/B/5/8B5804AD-4990-40D0-A6AA-CE894CBBB3DC/VS2008ExpressENUX1397868.iso
2008 SP1 here
http://download.microsoft.com/download/E/8/E/E8EEB394-7F42-4963-A2D8-29559B738298/VS2008ExpressWithSP1ENUX1504728.iso
I cannot figure out how to install pyopencl with Cygwin. Never used Cygwin before so I am very lost as to how I initiate python and use it to run my .py setup files.
You said, you want to install pyopencl without cygwin on windows, right?
There we go:
In case of you need 32-Bit version, download win32 Packages.
Download Python 2.7.5 Windows X86-64 Installer from http://www.python.org/download/ and install python (you rather install it in C:\Python27)
Go to http://www.lfd.uci.edu/~gohlke/pythonlibs/
Download numpy, scipy, pyopencl and install them
Go to pypi.python.org/pypi/pytools and download pytools.
Extract the file (using i.e. WinRAR) or tar (tar cxvf my.tar.gz)
Open commandline aka Dos (cmd.exe) and go in cmd to the folder you have extracted pytools by using cd command
Enter in cmd: python setup.py install
Open Python IDLE or simply type python in cmd and type import pyopencl
There should be no errors. If so, you did well.
Did you install Python into Cygwin?
If not, launch setup.exe, get to the packages screen, and do a search for python.
You can install 2.6.8 and 3 side by side if you want.
After that it's like using python anywhere else. You can do a $ python my.py to run my.py. You can install easy_install or pip, etc. if they'll help. Otherwise, follow the directions for PyOpenCL and you should be good!
I'm moving from a computer with windows xp to one with windows 7 (64 bit) and I want to install Python on the new machine. I'm currently using python 2.7 and 3.2 and a bunch of packages
1) Should I install the 64 bit versions?
2) How can I tell which packages I currently have, so that I can get those for the new machine?
If these are duplicates (my search skills seem to be failing), please point me to the threads where they're discussed.
You can use pip now for this purpose.
using pip list will give you a list of the packages that you have installed and the versions.
You could also use pip freeze and then copy the output of this to a file and then use this as a requirements files so you can install those exact modules and versions again on the new computer.
More information on pip is here is here
but to install from a requirements files you would use
pip install -r requirements.txt
Type help('modules') on the Python shell to get a list of all installed modules. If you can find stable 64-bit versions, then by all means select 64-bit builds for installations.
From your Python home folder go to ../Lib/site-packages/ and save the folder listing to a file, e.g. ls > ~/my-python-modules.txt on Linux/OSX, or dir > my-python-modules.txt on Windows. The file will contain all the additional modules that have been installed on your system. Also, from the same folder search for *.pth files: they might contain the names of modules that were installed as Python Eggs.
On Linux/OSX the location of the site-packages folder might vary: use locate -b site-packages to quickly find out where they are.
To reinstall the modules it is best to use your system's built-in package manager if it has one (this will be the case if you use Linux or MacPorts), otherwise you will need to use easy_install (the package is called setuptools) or pip (url). On Windows with modules that need C/C++ compilation it is easiest to use binary executable installers, unless you have a sane build environment like MS Visual Studio or MinGW. If you install binary packages then it is probably best to get the 64-bit version to match your architecture. Not sure if 32 bit versions will work or will event install.
I want to install a library called Glasslab-Cluster Library (Glasslab Cluster) for python. But I found that it just provided Linux and Ubuntu install commands.
Are there any ways that I can install this library on windows?
Thank you and please help.
As far as I can see in the link you provided, there is a "setup.py" file in the source package.
So, download the sources, unzip them in a separate folder, go to that folder and type
python setup.py install
You will probably need the MinGW GCC compiler (free) to make this run.
Hope this helps ! (I have no windows systems anymore and can not test this)