I'm new to Python. I'm familiar with pip... but how do I install something that pip doesn't know about?
Specifically, this is a Jupyter SQL Magics extension (GitHub link)
How do I install an extension "from scratch"?
Not everything is uploaded to pip and can be installed by it.
This seems to be a python script not extension.
Just download the code and use it as import to your script.
Here is how imports work
Python extensions, modules, library or programs (call them how you like) have setup.py script that makes them install-able. Or if its a program that requires library in order to use it it might have requirements.txt file.
This file is used by pip to install every dependency like so:
pip install -r requirements.txt
you can read more about setup.py here
Can you give more details what are you trying to accomplish?
We want to exclude the Python source code from the package we create. But after configuring setup.py, I failed excluding the py files. I have been using python setup.py bdist_wheel as command. Is there any way to exclude to source code from Python package? Basically we do not want to expose the source codes.
The wheel plugin for setuptools that handles bdist_wheel does not have the --exclude_source_files which is only supported by bdist_egg. The egg packages are however deprecated and not supported by pip for example. What you can do however:
pip3 install wheel
python3 setup.py bdist_egg --exclude-source-files
wheel convert dist/mypackage-1.0-py3.6.egg
The wheel utility takes a source-stripped egg and converts it into whl package.
This came up for us putting Python libs on an embedded system with very minimal available memory.
It can be achieved with:
./setup.py bdist_egg --exclude_source_files
Works for both distutils and setuptools.
Well, we also encountered this issue (around 2 years ago) and didn't find a sensible automation process for it. So I wrote my own.
You're welcomed to it: setup.py template
I have Anaconda pack for Python installed on my windows 7 laptop.
I followed all the installation steps as explained at:
http://www.pyopt.org/install.html#installation
After some trial and error, I could install pyOpt using the following command:
python setup.py install --compiler=mingw32
But still, I can't run even the simplest tutorial example at ..
http://www.pyopt.org/tutorial.html
.. maybe because the optimizer SLSQP is missing in the installation. I get following error message when I run the tutorial example:
slsqp = pyOpt.SLSQP()
AttributeError: 'module' object has no attribute 'SLSQP'
How can I install pyOpt completely with optimizers etc. on windows?
I'm going to preface this with: I know very little about what I am doing...
I've been stuck on this for a long time too. I have a feeling it has something to do with how the modules are built during installation. I think the pyOpt modules all point to fortran and c libraries which are "translated" to python when you run
python setup.py install --compiler=mingw32
I have been playing around using different compilers and command environments (administrator privledges, powershell, cygwin, etc.) to no avail. The msvc compiler fails at a certain point. Cygwin, MinGW32, IntelEM, all compile but produce the same error when going to run the actual examples.
If you look in the C:\PathToPython\Lib\site-packages\pyOpt\ folder you'll see the folders with all the modules. When you call import, you're importing these modules... so by entering the following (for example... I think):
from pyOpt import pySNOPT
you are more or less instructing the imports to navigate through the folders to find pySNOPT and import it ... I think. You'll notice that in \pyOpt\pySNOPT\ there is a file pySNOPT.py. When importing, using from pyOpt import SNOPT it looks in pyOpt for SNOPT.py... but you'll find (as python has) that there is NO such file to import. Instead, there is only a pySNOPT.py. Knowing this we would expect the following to fix the problem:
from pyOpt.pySNOPT import pySNOPT
But this, too, fails. This time producing an import error stating "SNOPT shared library failed to import" (this is a pretty big clue, I suspect).
If you open the pySNOPT.py (this could be replaced with the acronyms for any of the other solvers) file you'll see the familiar format of a python module staring back at you. The first lines of all of these files goes something like:
try:
import snopt
except ImportError:
raise ImportError('SNOPT shared library failed to import')
This is where the breakdown occurs. The "import snopt" line is supposed to load the snopt.dll (snopt.so, snopt.a for Linux, Unix). When running setup.py, if I use a cygwin environment it creates DLLs in each of the C:\PathToCygwinDrive\lib\python2.7\site-packages\pyOpt\pyACRONYM folders. These DLLs do NOT appear to be placed in the location corresponding to the version of Python I am telling PyCharm to use (c:\PathToPython\Lib\site-packages\pyOpt\pyACRONYM). I used all of the compilers (including cygwin) in other environments (powershell, Visual C++ 2008 64-bit command prompt, any other command prompt for that matter). Simply copying and pasting the DLLs from the Cygwin pyOpt folder to the regular Python pyOpt folder did not work. I don't know if there's a special way that DLLs have to be imported or something. Reading up on building C extensions and linking them to python projects got over my head and tedious way too fast.
I thought this was local to my computer (since a group member managed to get his installation working) but when going through setup on another group member's computer I ran into the EXACT same error.
One important thing to note... I am doing everything in x64 architecture (as was the group member who I tried to help but also got stuck here). This might be the real secret to fixing this. Perhaps there are too many hoops to jump through to get pyOpt working on 64-bit Python. Which brings me to my last point... there some workarounds I can think of:
install 32-bit Python, restart the process.
install Spyder IDE (through anaconda or something), then install pyOpt using:
conda install --channel https://conda.anaconda.org/melund pyopt
or
conda install --channel https://conda.anaconda.org/mutirri pyopt
(both for x64 windows)
DO NOT use the default conda pyOpt (conda install pyopt) as this is version 0.84 and will not work.
Switch to Linux (since programming with linux is like using lube with anal... you CAN make it work without it... but everything will be a lot more painful)
These are the workarounds I've found so far. I hope someone posts an actual solution SOON!
I succeeded to install pyOpt on Win64 on Anaconda. Here is how:
One can use the command from the comment above to install pyOpt 1.2 from mutirri channel:
conda install --channel https://conda.anaconda.org/mutirri pyopt
Installation goes almost smoothly. With the only exception that it asks to install Microsoft MPI package in order for pyOpt to work properly (that is easy: just go to https://www.microsoft.com/en-us/download/details.aspx?id=49926 and download the latest installer).
However, when we try to run some test script for pyOpt, an attempt to use the SLSQP solver fails. Surprise! So, some research is needed.
If we load Lib\site-packages\pyOpt\pySLSQP\slsqp.pyd file into Dependency Walker we can see that LIBGFORTRAN-3.DLL file is missing.
Quik google search shows that LIBGFORTRAN-3.DLL file comes bundled with MinGW64. So, we need to download and install MinGW64 using installer from here:
https://sourceforge.net/projects/mingw-w64/files/mingw-w64/
Then locate LIBGFORTRAN-3.DLL in the mingw64\bin folder and copy all DLLs from this folder somewhere into the PATH (I just copied all mingw64 DLLs to C:\Anaconda2\Library\bin).
Now SLSQP solver works!
I had a similar problem when using pyOpt on Windows, I couldn't use any pyOpt library that were written in fortran. The above solutions didn't work for me so I add my own answer here, I hope it helps:
Install these libraries:
conda install libpython
conda install -c msys2 m2w64-toolchain
Check if fortran compiler are available:
f2py -c --help-fcompiler
You can install one:
conda install -c conda-forge fortran-compiler
Delete or uninstall any prior version of pyOpt
Download pyOpt-master from https://github.com/madebr/pyOpt
Go in the appropriate folder and try to run the following line in anaconda prompt:
python setup.py install --compiler=mingw32
If the option --compiler is not recognized, try to run the following lines:
python setup.py build --compiler=mingw32
python setup.py install
If it doesn't work try the following:
Go in the following directory (or equivalent):C:\Users...\Anaconda3\Lib\distutils
Open or create file distutils.cfg
(You can do it with the following line if you have notepad, it will create and open the file):
notepad distutils.cfg
In distutils.cfg add these two lines:
[build]
compiler=mingw32.exe
Then go back in the pyOpt master directory and run this line in anaconda prompt:
python setup.py build --compiler=mingw32
If it worked:
In distutils.cfg, remove '.exe', so you have these two lines:
[build]
compiler=mingw32
And run this line:
python setup.py install
If everything went well pyOpt should have appeared where you wanted (in my case, C:\Users...\Anaconda3\Lib\site-packages).
You can try to run python examples\parallel_gradient.py or any other exemple to see if it worked.
I think it should do but other things I have done before might be necessary:
If this doesn't work, try that:
If you have installed mingGW with conda, uninstall it, I think you can do it with:
conda uninstall -c anaconda MinGW
Download the latest version of mingw-w64: http://mingw-w64.org/doku.php/download/mingw-builds
In Anaconda3\DLLs you may put:
libatomic-1.dll
libgcc_s_seh-1.dll
libgfortran-3.dll
libcomp-1.dll
libquadmath-0.dll
libssp-0.dll
libstdc++-6.dll
libwinpthread-1.dll
One was already there, libfortran-3.dll was downloaded from internet, the rest are copies from minGW-w64
(I found them in C:\Program Files\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin)
The path depends on where you installed minGW-w64 and may depend on your version. Copy ALL the DLL somewhere the sys.path have access to(Anaconda3\DLLs for me).
I got this to work in Windows using 64-bit Python 3:
unzip swigwin and add to environment variables
unzip wingw-w64 and add to environment variables, place at very beginning
Run the following command from the pyOpt folder:
python setup.py install --compiler=mingw32
If you have tried to compile it, you may need to delete the build folder first
My setup.py script is simple:
from distutils.core import setup
setup(name='my-awesome-app',
version='1.0',
scripts=['my-awesome-app.py'],
)
And the file structure is:
my-awesome-app/
my-awesome-app.py
setup.py
In theory I am only including my-awesome-app.py in the distribution. In practice setup.py ends up in the RPM too.
I don't see a point of including setup.py there, is there a way to force distutils to leave this file out?
I am using python 2.7, I build my RPM by running python setup.py bdist_rpm.
Thanks for help :)
setup.py is required because when the package is installed in your environment, the following command is run:
$ python setup.py install
Running python setup.py bdist_rpm only creates a distribution package that you can give to others. setup.py is still required to do the installation.
You can always create the spec file manually and leave out the setup.py.
For example and more details see:
https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file
While I was installing Python libraries on Linux, it threw the below error. How can I fix it?
I ran the below command to install:
python setup.py install
Output:
running install
running build
running build_ext
error: pyconfig.h: No such file or directory
The first thing to try is to install the appropriate module via your package manager. Each distro has many, many, many Python modules already packaged and ready to go.
If you insist on installing a module from source then you'll need the development package for your distro's Python. This is usually called python-devel, python-dev, or something similar. Use the package manager to install it.