it's my first post here so apologies if my style is wrong!
I've been trying to install mpi4py (as SU2 requires it for parallel processing) on Ubuntu 16.04 using python 3.6. I downloaded mpi4py and tried to build from source as I would like to use the OpenMPI libraries I had configured, and tried the following:
sudo python setup.py build --mpicc=/opt/openmpi/openmpi-2.1.0/bin/mpicc
However, it ultimately fails with "error: Cannot link MPI programs. Check your configuration!!!"
Looking at the SU2 and mpi4py documentation, they recommend using pip instead. I also tried this:
sudo pip install mpi4py
and env MPICC=/opt/openmpi/openmpi-2.1.0/bin/mpicc pip install mpi4py but they also gave the same error.
I have saved both outputs - please see the link for the attachments. It seems to pick up on a lot of the files from my NVIDIA CUDA installation, though I'm not sure why.
I'm a beginner at at compiling programs so I'd appreciate it if anyone could shed any light on this? Thanks.
Though its an old post but I will post my answer might help others.
I fixed this problem by following commands:
sudo apt install libopenmpi-dev
pip install mpi4py`
For anyone else who is struggling with this problem, it turned out to be a number of factors. Mainly, mpi4py was not building correctly as my implementation of MPI (openMPI) had not been properly built. There were many factors that differentiated building programs on an existing supercomputer and a fresh Linux install.
Some troubleshooting tips I can recommend are:
Make sure you clear away any environmental variables which are unnecessary for the install using unset
Unload any modules which may interfere with the install using module unload. This is especially true when building your own MPI implementation on a supercomputer which has several others loaded already.
Check your $PATH variable for any locations which may contaminate your install (not forgetting that the order is important)! You may want to temporarily set a cleaner PATH variable to make sure the build picks up the correct libraries.
Hope this helps.
Related
I'm installing a bunch of python modules on my system that are specific to this code I am going to be working with. Specifically, I used pip install pyda to get the pyda module. To make sure I had gotten all the modules, I went through and ran some of the code snippets, and came across the following error:
ModuleNotFoundError: No module named 'pyda.utilities'
I tried using pip install pyda.utilities, but that honestly doesn't make sense, it should have just come with the pyda module. According to this website https://pypi.org/project/pyda/ it seems like it should come with the package. I tried determining if I just installed it in the wrong python version, but I'm having a difficult time forcing it to use the specific python version that I installed the module in (specifically, I tried to create an alias for /usr/bin/python3.7 or something like this as I have seen on other websites, but it just fusses at me that this is simply a directory, incredibly unhelpful because I can't find the corresponding executable, so I'm a bit confused here).
This is a very long question likely with a very simple answer, any thoughts or help on what the issue might be would be appreciated.
Edit: I have determined that it's a package problem, not a python problem. The command 'pip install pyda' is not actually installing everything, oddly enough, which is why it cannot find the pyda.utilities module. Unfortunately, I think this means I will have to install the package manually. I will keep this question posted because of the useful answer on virtual environments, so thanks everyone.
The answer is indeed straightforward. As #Chris indicated in the comments, start using virtual environments.
It's not as complicated as it sounds and there's plenty of tutorials on getting started with virtualenv for Python, like https://uoa-eresearch.github.io/eresearch-cookbook/recipe/2014/11/26/python-virtual-env/
The basic steps:
check you're using the version of Python you want in your virtual environment
if you don't, change directories to where it lives
ensure you have pip and it works
check if you have virtualenv and if you don't pip install virtualenv
create a virtual environment virtualenv /your/env/folder/here
activate the virtual environment with /your/env/folder/here/Scripts/activate
After that, just install the packages you need with pip and they will end up in your virtual environment, with no interference from other Python versions or packages.
Check your python version, if it does not work restart your computer and try run setup.py install on the python command line
Over the last few days of headaches, I've found 3 possible methods to do this, all of which have issues.
PyGObject's pip install fails due to a lack of Cairo and probably other dependencies. While this would be my preferred method, it would probably be the hardest to fulfill.
Using MSYS2 allows me to use GObject through the mingw64 python, but using pip to get my other modulues such as pylint fails. I'd like to have an MSYS2 or similar install on my system anyways to make some windows binaries, so I'm very open to this as well.
EDIT: Got pip to work in MSYS2. Make sure to sync toolchain with pacman.
PyGObject for Windows seems messy at best and isn't up-to-date. Would require having a second python installation anyways, giving no benefit over MSYS2.
Note I'm a complete newbie to Unix and have little experience with CLI's in general, so any help regarding MSYS2 will need to be explained as if to a child. My only other experience with this stuff involved an endless cycle of hell that was installing Arch to a separate partition, breaking said install, then reinstalling again.
I also tried Cygwin, but I couldn't get that to run Python with GObject at all with my "install all the needed packages then pray" method. Creating a Gtk.Window() caused the terminal to use none of the memory it had and explode.
MSYS2 is currently the only "officially" supported way: https://pygobject.readthedocs.io/en/latest/getting_started.html#windows
pip in MSYS2 should work in general, if something doesn't please file a bug at https://github.com/Alexpux/MINGW-packages/issues .
As you said in the third alternative to install a new Python version, you can create and run multiple Python versions using pyenv. You can go through the installation section of their Github repo for installation and an overview: https://github.com/pyenv/pyenv/
For a comprehensive guide on how to use it, you read this amazing post by Real Python:
https://realpython.com/intro-to-pyenv/
I am trying to use Pythonanywhere to run some of my scripts. Eventually I would like to turn the entire code into a webapp, but I am not there yet.
Anyway, I would like to install a couple of bioinformatics packages on Pythonanywhere. Biopython has wrappers for these tools, but you still need the actual tools instaled. The tools in question do not have pip or easyinstall compatibility. So I think I will have to download/compile from source? However, I keep on running into "no permission" issues. Would a virtualenv be a solution?
As always, any help is greatly appreciated
Virtualenv works for Python packages.
However if you're trying to use non Python packages/libraries it won't be of much use.
Since the tools don't have easyinstall or pip compatibility I don't think using virtualenv will work out.
Download and compile from source should work though.
Note that if Biopython is installable via easyinstall or pip then use virtualenv to get that.
I am trying to install sklearn-pandas.
On my attempt:
easy_install sklearn-pandas
I get the result:
The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.
This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand. Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.
I'm on windows 7 (I admit it!), using Python 2.7.3
This is the first time I've gotten any error like this. Possible ideas I've explored are the more basic solutions:
The authors didn't write this package to be installed with easy_install
I have some sort of file permission problem (?)
There is some sort of dependencies issues
If anyone has ever gotten this error or has any insight into this please let me know! Much thanks.
Don't use easy_install, use pip!
(To install pip, follow this other StackOverflow question).
Additionally, you're probably going to need a compiler, development tools, and header files for Python (it's a whole 'nother bag of worms in Windows).
Assuming all is well, you can now
pip install sklearn-pandas
I am not a total newbie but I am trying to install modules for quite a long time and at this point i would like to have a fresh start and install python and all the modules I need so i really understand them. My problem is that some of them import, but most of them install either to the wrong site-packages or dont import maybe because i messed up my system/python. Also I tried the PYTHONPATH and PATH to set this up right, but it never worked.
So my questions are:
Is there a way to ensure I can clean everything up and start from zero ?
Ideally this would be without having to set up Mac OSX new.
Is there a way to install all the modules in the correct place (whatever the directory is I dont care, it should just work)?
Is there a good step-by-step description on how installing modules works. And I dont mean just the info to use easy_install, pip install etc, but a way to fully understand what I need to consider, where I need to put them, why these modules are recognized in certain directories, how the system finds them and most important what are the most common pitfalls and how to avoid them.
I also tried Macports and various other similiar ways to install but even if some of them worked and while I am sure that these are really great tools, most I had to hack to work.
So if someone can recommend a good and stable way to install a lot of modules at once, this would be incredibly useful.
Thanks a lot !!!!
And sorry for the long questions.
Buildout and virtualenv should be what you are looking for.
Buildout helps you configure a python installation and virtualenv allows you to isolate multiple different configurations from each other.
Here's a nice blog post explaining how to use them together.
Also, see this other question: Buildout and Virtualenv
You can safely install an up-to-date Python 2 and/or Python 3 on OS X using the python.org installers here. They will coexist with any other Pythons you have installed or that were shipped by Apple with OS X. To install packages, for each Python instance first install Distribute which will install a version-specific easy_install command, i.e. easy_install-2.7 or easy_install-3.2. Many people prefer to use pip to manage packages; you can use easy_install to install a version-specific copy of it. If you want something fancier, you could also install virtualenv but, with the isolation provided by Python framework builds on OS X, that isn't as necessary as on most other platforms.
Is there a way to install all the modules in the correct place?
Download and untar/gunzip/etc the module source (Most of the modules ares available in gzip form at http://pypi.python.org/pypi), then run configure with --prefix set to the same thing for every install:
[ 11:06 jon#hozbox.com ~ ]$ ./configure --prefix=/usr/local
/usr/local is usually the default, but it doesn't hurt to specify it and will ensure that every module you install will be placed in /usr/local/lib/python/...
Is there a good step-by-step description on how installing modules works?
The Python website has a great page called: Installing Python Modules
http://pypi.python.org/pypi
http://docs.python.org/install/index.html