I have some pyhton-scripts which are working perfectly on my local computer, but as you would guess, not so good on the server!
I wanted to use "pythonbrew" since at the server I have only Python 2.7.6 installed, but I need hdf5-package running ... OK
I installed pythonbrew (according to https://pypi.python.org/pypi/pythonbrew/) and switched to version 2.7.10 (as I use on my local computer).
Now I got some "numpy-issues". So I installed numpy with pip successfully (according to How do you install numpy when you're not a superuser? and https://gist.github.com/audy/4012573), however, when I am running the programme:
pythonbrew py startup.py
I got the error message saying numpy was not found ...
I really have no clue what I am missing here? I appreciate any help!
I found out myself how to run a python script with pythonbrew properly!
According installation guide for pythonbrew (see: https://pypi.python.org/pypi/pythonbrew/), the following line should be used to run a script:
pythonbrew py myprogramm.py
However, I can run my script by using python instead of pythonbrew py:
python myprogramm.py
I cannot give an explanation for that, sorry. But maybe other people can. It is just working very well like this and I can use python and all packages normally!
Further instructions of how to use pythonbrew, installing or switching between different version etc., see the link to the installation guide I posted above!
More details about pythonbrew and usage (from my perspective of view):
The installation guide I mentioned explains step by step how to install pythonbrew in your home directory (e.g. on a server) and helps to set up python on a system where you are not superuser (e.g. a server where you are allowed to run scripts but not to execute any further installations!).
I had the problem that I needed a more up to date python-version and the server where the data is located is equipped with an older version. The other half of the story is that I also needed some packages (e.g. h5py or cosmolopy) which are also not installed at the server.
For me the easiest and perfectly working solution to that problems was to use pythonbrew and install it at my home directory on the server!
Related
This is probably a really dumb question but I am stuck and wasting too much time on this so I would SO appreciate any help.
I am using a RHEL 7 box and installed Apache Zeppelin on it. Everything works except for the life of me I can't import Python packages such as Pandas.
I realized I didn't have PIP so I installed it with these steps: https://pip.pypa.io/en/stable/installing/ (notice I had to use the "--user" argument for the command "python get-pip.py").
Finally, I did "pip install pandas --user" which worked perfectly. I then go into my Zeppelin notebook and I cannot import pandas, even after restarting the Python interpreter.
I did some research and I think the problem is that "which python" and "which pip" are installed in different directories as the former results in "/usr/bin/python" while the latter in "~/.local/bin/pip".
So I suspect the packages installed with pip are basically getting loaded into a different version of python? If it helps, when I do "whereis python" I get 5 different results such as "/usr/bin/python" and "/usr/bin/python2.7" etc.
First thing to understand is: Python packages aren't installed globally, every installed Python has its own set of packages. BTW, pip being a Python package with a script is also not global. If you have a few different pythons you need different pips for them. I don't know Apache Zeppelin so I cannot guess if it uses the system Python (/usr/bin/python) or has its own Python; in the latter case you need to install pip specifically for Zeppelin so its pip install packages available for Zeppelin.
To investigate to what Python pip installs packages you need to find out under what python it runs. Start with shebang:
head -1 `which pip`
The command will prints something like ~/.local/bin/python. If it's not the version of Python you need to install packages for you need to install a different pip using that Python.
The most complex case would be if the shebang is PATH-dependent, something like #!/usr/bin/env python. In that case pip runs Python that you can find with which python.
PS. AFAIK the simplest way to install pip at RedHat is dnf install python-pip.
phd's answer was very helpful but I found that it was just a matter of using the root account to install the python packages. Then my Zeppelin was able to see any packages.
I think i somehow messed up installing packages for python and the whole environment. Whenever i want to install something i run into problems like:
directory is not owned by the current user and the cache has been disabled
Also PyCharm thinks i still need to install referenced packages (no modules. Which is wrong since i can use them. When i click on 'install referenced packages' i get the mentioned warning from above plus:
OSError: [Errno 1] Operation not permitted
Hopefully one of you can help! :(
Best regards
Matthias
Edit: At some point i also installed a venv for python. Maybe this relates to my problem. I'm rly sorry i am new to it and have no idea what i'm doing.
You have an issue with your permissions, if you run
which python
you will see where your current version of python is installed. If you're on osx, I seriously recommend downloading brew here: https://brew.sh/. Just paste that command into your terminal, then you should install python by running:
brew install python
This will give you some later instructions, but the brew bundled version of python is alot more robust than the one that gets bundled with your system. Another note is that if you're using virtual environments, consider using virtualenvwrapper here. This should make things run alot more smoothly for you :)
Hope that helps.
I've started to use python and recently i've bought my first mac.
I saw that Mountain Lion comes with their own python versions, despite i've decided to install a new python 2.7.5, and also Django and MySQL-Python. Everything works fine.
I've decided to try scipy and following the installed steps suggested by the official site with macports It seemed that macports installed another version 2.7.5 of python. Now if i try to run python in my terminal and import numpy it fails.
I saw that when i run python in my terminal, it execute
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
But if I go in
cd /System/Library/Frameworks/Python.framework/Versions/2.7/bin
and run ./python i can import django and numpy modules but not MySQLdb modules.
I'm really confused how I can fix that mess.
I would like to have just one Python version installed in one place with all the modules I need.
Or please suggest something to clean that mess, know and have the control to what modules I have, where they are and how to import them! And also control which version of Python I want to run.
Edit 10/27/2013: i've solved following the link suggested in the comment, deleting all versions in /Library/ and all simlinks in /usr/local/bin, after that i changed to the path i wanted the py interpreter in my bash_profile.
Actually the problems seems to be created by macports that has installed numpy using the py interpreter installed under /System/, so i changed my default py interpreter to that installed under /System/ and installed the rest of packages i need. Now everything works as i want.
Personally, I wouldn't mess around with the built-in version of Python. I had some real issues with my Mac when I did that a few months ago.
You could try Canopy instead. It's a Python development environment which is great for writing and executing code. Any packages you need can be easily added (or removed) using the package manager. https://www.enthought.com/products/canopy/
I was trying to setup Django dev environment on Mac and arrived into a hell. It all started when trying to install PIL, which failed after trying 15 or so different recipes I found on blogs. So I wanted to install the Python, this time 2.7, and reinstall setuptools, easy_install, pip from scratch.
After just installing Python 2.7, and easy_install with setuptools for 2.7, this all in turn created such a mess that is unbelievable. Different version of Python are installed everywhere, easy_install is installed everywhere and points randomly to different python hashbangs (sometimes to #!/usr/bin, #!/usr/local/, #!/Library/...)
Now I can't even do easy_install pip, which I always could. So I'm already in a hell and I haven't even attempted to install MySQL yet.
My question finally is did anyone bump into such problems, it would help enough to know that I'm not alone.
Second, would it be easier to set up the entire environment on Ubuntu than it is on a Mac?
Thirdly, is there any guide that can really clearly explain how to set up but also tear down the stack for Python development on a Mac?
It wouldn't hurt to run a VM with vagrant. This post should tell you more:
http://stevelosh.com/blog/2011/06/django-advice/
Of course using virtualenv should also help alleviate some of these issues.
I've gone through the same hell 2 weeks ago :)
I needed to make working python 2.7 and virtualenv on OSX 10.6.8.
You haven't mentioned virtualenv in your question but I strongly recommend it. That way you minimize amount of globally installed packages. Everything is... cleaner.
My idea is to only have following things globally:
python (from brew)
pip (via easy_install)
virtualenv (via pip)
virtualenvwrapper (via pip)
other through either virtualenv or buildout
I've just checked and pip PIL installs fine within my virtualenv.
Here are notes from this battle (gist.github.com):
#NOTE: .pydistutils.cfg seems to be not compatible with brew install python
#areas I needed to clean before installation
#clean up ~/Library/Python
#clean up .local
brew install python
easy_install pip
pip install virtualenv
pip install virtualenvwrapper
mkdir $HOME/.virtualenvs
Example .bash_profile:
#homebrew
export PATH=/usr/local/bin:/usr/local/sbin:${PATH}
# homebrew python 2.7
export PATH="/usr/local/share/python:${PATH}"
#virtualenv wrapper
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/share/python/virtualenvwrapper.sh
Good luck!
Second, would it be easier to set up
the entire environment on Ubuntu than
it is on a Mac?
To answer this question (though I never used Mac though): I never had problems setting up a python environment for Django development on Ubuntu. Though in any case you should go with the built-in Python version if possible. Attempting to install any other Python versions usually ends up messy. Luckily with Ubuntu 11.04 the standard version is already 2.7.
I'm using django development environment on a MAC OS X 10.8 with python 2.7. I don't use virtualenv ore some other things.
With all the respect can say that there is NO ANY PROBLEMS to develop on a mac. Mac is a UNIX like system and you've probably seen that all tools for developers have MAC ports.
As for the setup mess. It's a good idea to use virtualenv. As for PIL installation. I needed to compile it with TrueType. As I'm in common with UNIX like environments it was not heavy task for me to compile PIL from sources using GCC (it's already installed on a MAC)... There are some mess with Django to setup virtualenv... There are certainly lots of articles to setup it on Google.
I use Eclipse and write all my PYTHONPATH variables there. You can forget installing everything like in Linux and try not to make anymore mess with installed tools. Try to read THIS article if you feel like you're ok to use Eclipse for your development on a MAC. It also has a recipe to avoid mess with installation of many copies of Python and other dev utils.
Yes I have had problems with MacOS. I think rather than trying to figure it out I just switched to Ubuntu. I use a mac with Ubuntu installed in VMware Fusion. I have developed on both and prefer the Ubuntu because I'm just more comfortable with installing packages and the file structure.
I love using the VM because I'm never scared of having to start over. I can get a whole new OS installed and get the packages with what I use in just a few hours. Not to mention with 6month rollouts I can do complete installs of new versions instead of updates.
Depending on your production environment, it may be beneficial to use an OS that is similar, if you can install a package on ubuntu desktop, you already know how to do it on ubuntu server.
I'm working with PyInstaller under Python 2.6, which is only partially supported due to the mess MS have created with their manifest nonense which now affects Python since it is now MSVC8 compiled.
The problem is that the manifest embedding support relies on the pywin32 extensions in order to build which is a pain because without including the host's site-packages folder when I create the virtualenv (kinda defeats the point in a build environment) I cannot find a way to install the required extensions so they are accessible to PyInstaller.
Has anyone found a solution to this issue?
I found http://old.nabble.com/Windows:-virtualenv-and-pywin32--td27658201.html (now a dead link) which offered the following solution:
Browse http://sourceforge.net/projects/pywin32/files/ for the URL of the exe you want
Activate your virtualenv
Run easy_install http://PATH.TO/EXE/DOWNLOAD
This works with modern versions of setuptools (circa February 2014, reported by tovmeod in the comments).
If you are using an old version of setuptools (or distribute it merged back into setuptools), you may get this error message:
error: c:\users\blah\appdata\local\temp\easy_install-ibkzv7\pywin32-214.win32-py2.6.exe is not a valid distutils Windows .exe
In which case:
Download the exe yourself
Activate your virtualenv
Run easy_install DOWNLOADED_FILE.exe
I rather hopefully tried "pip install" rather than "easy_install", but this didn't work, and likely never will (citation needed).
Finally, I found but haven't tested a solution at http://www.mail-archive.com/python-list#python.org/msg272040.html which is:
Solved this by copying the pywin32.pth file into my virtualenv site-packages
and editing the file to point to the path.
If the other options don't work for you, maybe this will?
For Python 2.7 or 3.x use pypiwin32.
pip install pypiwin32
OK, well since I had to find a way forward I improvised. I've internally created a git repository with a hacked-together version of pywin32 that will install within a virtualenv using the standard setup.py script. It took a lot of fiddling to make it work right but I managed to get it to load and the dependent code now works as I need it to. If people feel this would be of benefit to the community please post a comment: if I get enough I'll try and put something up on my github account.
This may have been improved since previous answer, since I've successfully installed pywin32 on sandbox on several machines without any specific "hacks" :
$ virtualenv sandbox
$ sandbox\scripts\activate
(sandbox) $ git clone https://github.com/Travis-Sun/pywin32.git
(sandbox) $ cd pywin32
(sandbox) $ python setup.py install
Tested with following environment :
windows 7
git
python 2.7.10 with virtualenv
VS2008. It may also work (but I've not tested yet) with
http://www.microsoft.com/en-us/download/details.aspx?id=44266
Edit: Scratch this for now, appears to be some problems with the installation still...
I got rather tired of the whole situation, and just created a set of converted wheels ("wheel convert <.exe>"). I'll try and keep them maintained for the most recent build, but do shout if there are any issues.
https://tr00st.co.uk/python/wheel/pywin32/
Installation can be done easily using pip and pointing to the package matching your version and architecture. For example, for Python 3.5/amd64:
pip install https://tr00st.co.uk/python/wheel/pywin32/pywin32-219-cp35-none-win_amd64.whl
Caveat: The --upgrade process currently fails, as the uninstall procedure is unable to clean up after itself (Access Denied when cleaning up win32api.pyd) - this is only when removing the temporary directory, which can be manually deleted. Easiest way around this is to uninstall and reinstall instead of upgrading, then manually delete the temporary folder.