I have Anaconda pack for Python installed on my windows 7 laptop.
I followed all the installation steps as explained at:
http://www.pyopt.org/install.html#installation
After some trial and error, I could install pyOpt using the following command:
python setup.py install --compiler=mingw32
But still, I can't run even the simplest tutorial example at ..
http://www.pyopt.org/tutorial.html
.. maybe because the optimizer SLSQP is missing in the installation. I get following error message when I run the tutorial example:
slsqp = pyOpt.SLSQP()
AttributeError: 'module' object has no attribute 'SLSQP'
How can I install pyOpt completely with optimizers etc. on windows?
I'm going to preface this with: I know very little about what I am doing...
I've been stuck on this for a long time too. I have a feeling it has something to do with how the modules are built during installation. I think the pyOpt modules all point to fortran and c libraries which are "translated" to python when you run
python setup.py install --compiler=mingw32
I have been playing around using different compilers and command environments (administrator privledges, powershell, cygwin, etc.) to no avail. The msvc compiler fails at a certain point. Cygwin, MinGW32, IntelEM, all compile but produce the same error when going to run the actual examples.
If you look in the C:\PathToPython\Lib\site-packages\pyOpt\ folder you'll see the folders with all the modules. When you call import, you're importing these modules... so by entering the following (for example... I think):
from pyOpt import pySNOPT
you are more or less instructing the imports to navigate through the folders to find pySNOPT and import it ... I think. You'll notice that in \pyOpt\pySNOPT\ there is a file pySNOPT.py. When importing, using from pyOpt import SNOPT it looks in pyOpt for SNOPT.py... but you'll find (as python has) that there is NO such file to import. Instead, there is only a pySNOPT.py. Knowing this we would expect the following to fix the problem:
from pyOpt.pySNOPT import pySNOPT
But this, too, fails. This time producing an import error stating "SNOPT shared library failed to import" (this is a pretty big clue, I suspect).
If you open the pySNOPT.py (this could be replaced with the acronyms for any of the other solvers) file you'll see the familiar format of a python module staring back at you. The first lines of all of these files goes something like:
try:
import snopt
except ImportError:
raise ImportError('SNOPT shared library failed to import')
This is where the breakdown occurs. The "import snopt" line is supposed to load the snopt.dll (snopt.so, snopt.a for Linux, Unix). When running setup.py, if I use a cygwin environment it creates DLLs in each of the C:\PathToCygwinDrive\lib\python2.7\site-packages\pyOpt\pyACRONYM folders. These DLLs do NOT appear to be placed in the location corresponding to the version of Python I am telling PyCharm to use (c:\PathToPython\Lib\site-packages\pyOpt\pyACRONYM). I used all of the compilers (including cygwin) in other environments (powershell, Visual C++ 2008 64-bit command prompt, any other command prompt for that matter). Simply copying and pasting the DLLs from the Cygwin pyOpt folder to the regular Python pyOpt folder did not work. I don't know if there's a special way that DLLs have to be imported or something. Reading up on building C extensions and linking them to python projects got over my head and tedious way too fast.
I thought this was local to my computer (since a group member managed to get his installation working) but when going through setup on another group member's computer I ran into the EXACT same error.
One important thing to note... I am doing everything in x64 architecture (as was the group member who I tried to help but also got stuck here). This might be the real secret to fixing this. Perhaps there are too many hoops to jump through to get pyOpt working on 64-bit Python. Which brings me to my last point... there some workarounds I can think of:
install 32-bit Python, restart the process.
install Spyder IDE (through anaconda or something), then install pyOpt using:
conda install --channel https://conda.anaconda.org/melund pyopt
or
conda install --channel https://conda.anaconda.org/mutirri pyopt
(both for x64 windows)
DO NOT use the default conda pyOpt (conda install pyopt) as this is version 0.84 and will not work.
Switch to Linux (since programming with linux is like using lube with anal... you CAN make it work without it... but everything will be a lot more painful)
These are the workarounds I've found so far. I hope someone posts an actual solution SOON!
I succeeded to install pyOpt on Win64 on Anaconda. Here is how:
One can use the command from the comment above to install pyOpt 1.2 from mutirri channel:
conda install --channel https://conda.anaconda.org/mutirri pyopt
Installation goes almost smoothly. With the only exception that it asks to install Microsoft MPI package in order for pyOpt to work properly (that is easy: just go to https://www.microsoft.com/en-us/download/details.aspx?id=49926 and download the latest installer).
However, when we try to run some test script for pyOpt, an attempt to use the SLSQP solver fails. Surprise! So, some research is needed.
If we load Lib\site-packages\pyOpt\pySLSQP\slsqp.pyd file into Dependency Walker we can see that LIBGFORTRAN-3.DLL file is missing.
Quik google search shows that LIBGFORTRAN-3.DLL file comes bundled with MinGW64. So, we need to download and install MinGW64 using installer from here:
https://sourceforge.net/projects/mingw-w64/files/mingw-w64/
Then locate LIBGFORTRAN-3.DLL in the mingw64\bin folder and copy all DLLs from this folder somewhere into the PATH (I just copied all mingw64 DLLs to C:\Anaconda2\Library\bin).
Now SLSQP solver works!
I had a similar problem when using pyOpt on Windows, I couldn't use any pyOpt library that were written in fortran. The above solutions didn't work for me so I add my own answer here, I hope it helps:
Install these libraries:
conda install libpython
conda install -c msys2 m2w64-toolchain
Check if fortran compiler are available:
f2py -c --help-fcompiler
You can install one:
conda install -c conda-forge fortran-compiler
Delete or uninstall any prior version of pyOpt
Download pyOpt-master from https://github.com/madebr/pyOpt
Go in the appropriate folder and try to run the following line in anaconda prompt:
python setup.py install --compiler=mingw32
If the option --compiler is not recognized, try to run the following lines:
python setup.py build --compiler=mingw32
python setup.py install
If it doesn't work try the following:
Go in the following directory (or equivalent):C:\Users...\Anaconda3\Lib\distutils
Open or create file distutils.cfg
(You can do it with the following line if you have notepad, it will create and open the file):
notepad distutils.cfg
In distutils.cfg add these two lines:
[build]
compiler=mingw32.exe
Then go back in the pyOpt master directory and run this line in anaconda prompt:
python setup.py build --compiler=mingw32
If it worked:
In distutils.cfg, remove '.exe', so you have these two lines:
[build]
compiler=mingw32
And run this line:
python setup.py install
If everything went well pyOpt should have appeared where you wanted (in my case, C:\Users...\Anaconda3\Lib\site-packages).
You can try to run python examples\parallel_gradient.py or any other exemple to see if it worked.
I think it should do but other things I have done before might be necessary:
If this doesn't work, try that:
If you have installed mingGW with conda, uninstall it, I think you can do it with:
conda uninstall -c anaconda MinGW
Download the latest version of mingw-w64: http://mingw-w64.org/doku.php/download/mingw-builds
In Anaconda3\DLLs you may put:
libatomic-1.dll
libgcc_s_seh-1.dll
libgfortran-3.dll
libcomp-1.dll
libquadmath-0.dll
libssp-0.dll
libstdc++-6.dll
libwinpthread-1.dll
One was already there, libfortran-3.dll was downloaded from internet, the rest are copies from minGW-w64
(I found them in C:\Program Files\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin)
The path depends on where you installed minGW-w64 and may depend on your version. Copy ALL the DLL somewhere the sys.path have access to(Anaconda3\DLLs for me).
I got this to work in Windows using 64-bit Python 3:
unzip swigwin and add to environment variables
unzip wingw-w64 and add to environment variables, place at very beginning
Run the following command from the pyOpt folder:
python setup.py install --compiler=mingw32
If you have tried to compile it, you may need to delete the build folder first
Related
A little background: I am working on some python modules that other developers on our team will use. A common theme of each module is that one or more messages will be published to Kafka. We intend at this time to use the Confluent Kafka client. We are pretty new to python development in our organization -- we have traditionally been a .NET shop.
The complication: while the code that we create will run on Linux (rhel 7), most of the developers will do their work on Windows.
So we need the librdkafka C library compiled on each developer machine (which has dependencies of its own, one of which is OpenSSL). Then a pip install of confluent-kafka should just work, which means a pip install of our package will work. Theoretically.
To start I did the install on my Linux laptop (Arch). I knew I already had OpenSSL and the other zip lib dependencies available, so this process was painless:
git clone librdkafka repo
configure, make and install per the README
pip install confluent-kafka
done
The install of librdkafka went into /usr/local:
/usr/local/lib/librdkafka.a
/usr/local/lib/librdkafka++.a
/usr/local/lib/librdkafka.so -> librdkafka.so.l
/usr/local/lib/librdkafka++.so -> librdkafka++.so.l
/usr/local/lib/librdkafka.so.l
/usr/local/lib/librdkafka++.so.l
/usr/local/lib/pkgconfig/rdkafka.pc
/usr/local/lib/pkgconfig/rdkafka++.pc
/usr/local/include/librdkafka/rdkafkacpp.h
/usr/local/include/librdkafka/rdkafka.h
Now the painful part, making it work on Windows:
install precompiled OpenSSL
git clone librdkafka repo
open in VS2015
install libz via NuGet
build solution
install to where???
This is where I'm stuck. What would a standard install on a Windows 7/8/10 machine look like?
I have the following from the build output, but no idea what should go where in order to make the pip install confluent-kafka "just work":
/librdkafka/win32/Release/librdkafka.dll
/librdkafka/win32/Release/librdkafka.exp
/librdkafka/win32/Release/librdkafka.lib
/librdkafka/win32/Release/librdkafkacpp.dll
/librdkafka/win32/Release/librdkafkacpp.exp
/librdkafka/win32/Release/librdkafkacpp.lib
/librdkafka/win32/Release/zlib.dll
<and the .h files back in the src>
Any recommendations on an install location?
I'm not sure where the ideal place to install on Windows would be, but I ran the following test with some success.
I copied my output and headers to C:\test\lib and C:\test\include, then ran a pip install with the following options:
pip install --global-option=build_ext --global-option="-LC:\test\lib" --global-option="-IC:\test\include" confluent-kafka
Unfortunately, this doesn't quite work because the confluent-kafka setup does not support Windows at this time: https://github.com/confluentinc/confluent-kafka-python/issues/52#issuecomment-252098462
It's an old question but seems still no easy answer yet. Also Confluent seems too busy to work on Windows supporting...
I had the same headache couple weeks ago and after some research, I managed to make it work for me on Windows. I logged my finding and uploaded a pre-compiled library to my Git, please check and see if it helps. :D
https://github.com/MichaelZhangCA/confluent-kafka-python
My environment is Python 3.6 64 bit version but ideally, it should also work for 32 bit if you follow the same approach.
I assume you have successfully followed instructions from MichaelZhangCA (https://github.com/MichaelZhangCA/confluent-kafka-python/) from previous post.
If you did so, these probably were the last two commands executed:
::Install confluent-kafka
cd C:\confluent-kafka-python\confluent-kafka-python-0.11.4
python setup.py install
If that is correct, those DLLs were created under C:\confluent-kafka-python\librdkafka-reference\release\.
All you have to do is to copy them to a diretory already in Widnows' PATH.
For example, I use Anaconda 5.2 For Windows, with python 3.6. My Anaconda Prompt has an empty directory in PATH, so I copied there those DLL's:
::Anaconda Prompt - copy DLLs to a directory already in PATH
mkdir %CONDA_PREFIX%\Library\usr\bin
copy C:\confluent-kafka-python\librdkafka-reference\release %CONDA_PREFIX%\Library\usr\bin
If you don't use Anaconda, just copy those DLL's to any other directory in Windows' PATH. You may also leave them in C:\confluent-kafka-python\librdkafka-reference\release, and add this directory to PATH.
I'm trying to build slycot, a python wrapper around the fortran library SLICOT.
I'm using the anaconda python distribution Anaconda-2.0.1-Windows-x86_64 on a 64-bit windows 7.
The first naive python setup.py build command resulted in
NotImplementedError("Only MS compiler supported with gfortran on win64")
So far so good. After googling for a while, someone recommended simply commenting out the line in numpy. This seems to work. gfortran is called and all the libraries are compiled, but then it fails at linking with the following message
The command line is too long
It seems that listing all the *.o files generates a huge command string which can not be processed.
Any suggestions?
If you are still having this problem, try upgrading to Anaconda-2.1.0. If that still fails and you are not tied to Python2, then install a Python3 environment using the Anaconda3-2.1.0-Windows-x86_64 distribution. Once I did that and a manual package install (steps below) I didn't get the "command line too long" error. I have Anaconda installed in C:\Anaconda - can't tell if that helped or not.
To manually install a package:
Download the package you need (for me it was numpy).
Extract to a directory (I used c:\temp).
Fix gnu.py in the distutils\fcompiler subdirectory per instructions here.
Exit any apps using python and kill any remaining python.exe processes with task manager; not doing this caused my first compile to fail.
Open an Anaconda Command Prompt (a standard admin cmd.exe window will not work).
If you have multiple environments, activate the correct one.
Change to the directory you extracted your package to.
Type python setup.py install
Has anyone managed to install QuTIP 2.2.0 with an existing Python 2.7.5 distribution (on Win7)? The instruction manual suggests that I need to install Python(x,y) first, but the instructions are pretty vague. I'm still a Python newbie.
Understanding installation instructions
The installation instructions are pretty clear, but I remember the times, I got lost in those short lines assuming I know something obvious.
I will try translating it
install Python(X,Y) - do it. Follow the link, download the exe file and run it.
Do not forget to set the options, following defaults will fail, Cython option must be included.
edit the distutils.cfg file as instructed
download tar.gz archive for QuTIP from PyPi, unpack it in some directory, cd into it to be in the directory, where you see setup.py, and run $ python setup.py install
The distutils.cfg refers to mingw32, this is needed for compilation. If it is not installed with Python(X,Y), you would have to install it separately. Be careful and install proper version, even on 64 bit systems use 32 bit one (this I assume from proposed name of compiler in config).
Good luck. I am not on Windows for about 2 years, so I cannot confirm, it works, but I hope, it will move you on.
I also had a lot of problems installing it correctly.
Here is my working solution.
As the installation instructions suggest:
Install PythonXY (I am using 2.7.6.1) (including Cython package) (Edit: The newer versions of PythonXY do not include a compiler. Try installing from here instead: https://code.google.com/p/pythonxy/wiki/AdditionalPlugins)
Edit C:\Python27\Lib\distutils\distutils.cfg to include:
[build]
compiler = mingw32
[build_ext]
compiler = mingw32
Add C:/MinGW32-xy/bin to your PATH. It has to be before other paths with e.g. gcc in them. You can do this:
set PATH=C:/MinGW32-xy/bin;%PATH%
for a temporary add (in that console) or use the answer here.
For a permanent change go properties of your Computer. Go to advanced system settings --> Environment Variables. Change the System Variable to have the MinGW path as the first entry. It doesn't work if it is last or in the user path!
Run a Python interpreter:
import qutip; qutip.testing
qutip.testing.run()
If it doesn't crash on the 7th test you probably have a working copy of qutip.
I get 320 test in 2194.236s and SKIP=7, errors=5.
Details of which tests failed for my can be seen here.
I would use Anaconda 2.7 with the added mingw and libpython libraries and then edit the distutils.cfg as stated. The skipped tests mentioned are fortran tests that you cannot run on windows, while the errors are time dependent tests that generate cython code at run time. If you follow the above suggestions then those tests will pass.
I've tried to install qutip for several hours, unsuccessfully. Fortunately, kind people from University of California have a solution:
wheels for lots of packages for python computing
This resource is not official, 'as is', but works better!
Type 'pip install package.whl' to download and build.
So I was really eager to try generating a django model diagram with django-extensions by doing the following:
manage.py graph_models -a -g -o my_project_visualized.png
Then I realized I need pygraphviz to make it work. I'm running Windows 7 64bit, so I installed Activestate and Graphviz as suggested by the documentation as a requirement. Graphviz 2.28 was installed quite successfully and then Activestate was installed with some tweaks msiexec /package activestate_install.exe /qr because it always stuck at "finish checking disk spaces required".
Later, it turns out activestate wasn't really helping in installing pygraphviz. So after a lot of trial and errors, I managed to install pygraphviz by following this guide http://blog.ropardo.ro/2011/11/28/installing-pygraphviz-on-windows/. Just as I tried to run the graph_models command again, python once again tells me that pygraphviz module is not installed... so I pip freeze to see if it's really not installed. Now comes the good part, pip freeze returns a traceback of ImportError: DLL load failed: %1 is not a valid Win32 application. But that only happens in virtualenvs, when get out of virturalenvs and pip freeze, it works as expected.
By googling around, I realized I might have installed some 64bit version package while having 32bit python 2.7.3 on my machine, and that's indeed what I did. Just that I don't remember which ones I installed were 64bit. It's late at night, and I'm tired and frustrated, so I started uninstalling things that I thought might be causing the problem. I uninstalled mingw32 and Activestate, and the next thing I know, I can't even get into python from command line. pip is not recognized either; every python package is not recognized. Checking back at the Python27 directory, half of the folders are gone in C:\Python27\Lib. There used to be a lot of files and folders which it's impossible for me to recall what they are, and now there's only 4 directories:
bsddb, importlib, site-packages, and test
and the site-packages folder is a lot "cleaner", too! I'm sure there used to be a lot of package files and scripts (some .pth files and .py files). Now there's only two files easy-install.pth and PIL.pth and some folders for some packages I installed. However, the most critical issue is I can't even use python in cmd now.
I thought maybe uninstalling activestate messed up some of the environmental variables I set, then I went into the system settings and added in things like C:\Python27\Scripts;C:\Python27\Lib\site-packages to PATH. I went back into cmd, and typed "python" and "pip", the response I get for python is The system cannot find the path specified., and for pip it's still 'pip' is not recognized as an internal or external command, operable program or batch file.
This is kind of the only time that I regret for not setting up a system restore point..
Could anyone please tell me what's going on here, what am I doing wrong, and how am I supposed to get everything working again? Thanks a lot!
Boy, you indeed messed up your installation. "ActiveState" is a python distribution for Windows, so you installed a new python distribution over your old one. When you uninstalled, it obligingly removed your python installation, including the default library.
You'll have to reinstall your Python installation, I'm afraid. Start from scratch or backup.
You have my sympathies, if that's any consolation.
I'm working with PyInstaller under Python 2.6, which is only partially supported due to the mess MS have created with their manifest nonense which now affects Python since it is now MSVC8 compiled.
The problem is that the manifest embedding support relies on the pywin32 extensions in order to build which is a pain because without including the host's site-packages folder when I create the virtualenv (kinda defeats the point in a build environment) I cannot find a way to install the required extensions so they are accessible to PyInstaller.
Has anyone found a solution to this issue?
I found http://old.nabble.com/Windows:-virtualenv-and-pywin32--td27658201.html (now a dead link) which offered the following solution:
Browse http://sourceforge.net/projects/pywin32/files/ for the URL of the exe you want
Activate your virtualenv
Run easy_install http://PATH.TO/EXE/DOWNLOAD
This works with modern versions of setuptools (circa February 2014, reported by tovmeod in the comments).
If you are using an old version of setuptools (or distribute it merged back into setuptools), you may get this error message:
error: c:\users\blah\appdata\local\temp\easy_install-ibkzv7\pywin32-214.win32-py2.6.exe is not a valid distutils Windows .exe
In which case:
Download the exe yourself
Activate your virtualenv
Run easy_install DOWNLOADED_FILE.exe
I rather hopefully tried "pip install" rather than "easy_install", but this didn't work, and likely never will (citation needed).
Finally, I found but haven't tested a solution at http://www.mail-archive.com/python-list#python.org/msg272040.html which is:
Solved this by copying the pywin32.pth file into my virtualenv site-packages
and editing the file to point to the path.
If the other options don't work for you, maybe this will?
For Python 2.7 or 3.x use pypiwin32.
pip install pypiwin32
OK, well since I had to find a way forward I improvised. I've internally created a git repository with a hacked-together version of pywin32 that will install within a virtualenv using the standard setup.py script. It took a lot of fiddling to make it work right but I managed to get it to load and the dependent code now works as I need it to. If people feel this would be of benefit to the community please post a comment: if I get enough I'll try and put something up on my github account.
This may have been improved since previous answer, since I've successfully installed pywin32 on sandbox on several machines without any specific "hacks" :
$ virtualenv sandbox
$ sandbox\scripts\activate
(sandbox) $ git clone https://github.com/Travis-Sun/pywin32.git
(sandbox) $ cd pywin32
(sandbox) $ python setup.py install
Tested with following environment :
windows 7
git
python 2.7.10 with virtualenv
VS2008. It may also work (but I've not tested yet) with
http://www.microsoft.com/en-us/download/details.aspx?id=44266
Edit: Scratch this for now, appears to be some problems with the installation still...
I got rather tired of the whole situation, and just created a set of converted wheels ("wheel convert <.exe>"). I'll try and keep them maintained for the most recent build, but do shout if there are any issues.
https://tr00st.co.uk/python/wheel/pywin32/
Installation can be done easily using pip and pointing to the package matching your version and architecture. For example, for Python 3.5/amd64:
pip install https://tr00st.co.uk/python/wheel/pywin32/pywin32-219-cp35-none-win_amd64.whl
Caveat: The --upgrade process currently fails, as the uninstall procedure is unable to clean up after itself (Access Denied when cleaning up win32api.pyd) - this is only when removing the temporary directory, which can be manually deleted. Easiest way around this is to uninstall and reinstall instead of upgrading, then manually delete the temporary folder.