We need to use an external library which only runs on Python 3.6 or higher. Unfortunately, HDInsight only has Python 3.5 installed by default.
Is there a way to upgrade the minor version for Python on HDInsight?
The official Azure documentation does not mention about it.
https://learn.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-python-package-installation
As stated in Microsoft Q&A:
It's not recommended as using non-cluster built-in Python versions are unsupported scenario.
WARNING!: HDInsight cluster depends on the built-in Python environment - Python 3.5. Directly installing custom packages in those default built-in environments may cause unexpected library version changes. And break the cluster further.
If you want to install, you can use Python 3.6, change “python=3.5” in this command to python=3.6, and follow the rest steps in the document works.
sudo /usr/bin/anaconda/bin/conda create --prefix /usr/bin/anaconda/envs/py36new python=3.6 anaconda –yes
Related
I have just installed repo on my machine.
When I run repo to check whether repo is installed properly, there's a warning:
$ repo
warning: Python 3 support is currently experimental. YMMV.
Please use Python 2.6 - 2.7 instead.
error: repo is not installed. Use "repo init" to install it here.
I added the installation path of repo and python to the user environment.
Please tell me how to use python 2.7 when I need to run repo. Meanwhile, there are some tools that need to run with python 3.6 also.
Modern versions of repo (that is, the 2.x series) support Python 3 just fine, and in fact, do not support Python 2 at all, according to the documentation. It looks like you're using an older version that wants Python 2, so you should upgrade to a newer version.
Since Python 2.7 is now end of life, it's not a good idea to use it for anything anymore since it receives no security updates.
Okay, so I am trying to generate and deploy the documentation of a Python 2.7 based package. The documentation is generated using sphinx, python 2.7.
However, to actually deploy the documentation, I need to use a tool called doctr. This automatically deploys the generated html to github pages. However it requires me to work on Python 3.5 or newer.
So the question is : When the entire travis build is running on python 2.7, how do I temporarily shift to python 3.5 and deploy the documentation?
You can use pyenv to install 3.5 if not already present, and change to it, do your thing, and change back.
Basically this;
pyenv install 3.5.0
pyenv global 3.5.0
<do the deploy>
pyenv global <previous version>
I'm developing a website using a server with Debian 8.10 (Jessie) as its OS and Python 3.4.2 (the supported Python version for Debian Jessie) while my notebook is using Ubuntu 16.04 and Python 3.5.2 (I think it's also the default version for Ubuntu 16.04). I was planning to build my website using Django 1.11 which both Python versions (3.4 and 3.5) support.
Is there any compatibility issues when I develop it using my Python 3.5.2 and deploy it to a Python 3.4.2 server? If any, how much the trouble it will be?
I know I can install any version of Python by adding someone's repository, but it seems unofficial so I avoid doing it. And there is a workaround that come to my mind: intall a specific version of Python by download its tarball file from the official website
Which will you recommend most?
Upgrade my server's Python version to 3.5.2 by adding someone's repo
Download Python 3.4.2's tarball and install it to my local machine
Upgrade my server's OS to Debian 9 Stretch which its default Python version is 3.5.3
Or any other better idea? Or perhaps you have a way to install specific Python version?
*I have some concern on security and safety
When you are working with different Python versions, it is recommended that you use some kind of virtual environment so each project has its own python version with its own modules that you need.
In this way, you can always keep each project with the Python version that you know it will work and with total compatibility with all the modules that you are using, making sure that any update to your working modules does not break anything in your past projects.
You should install a virtual environment in your local machine that matches the server machine and match the Python version and the modules that you have available, then start developing from there.
This space is a little bit too short to explain how to work with them, but you can find information about them here: https://docs.python.org/3/tutorial/venv.html
Optionally, you could use anaconda and its own version of enviroments, that may be simpler if you are familiar with conda
I'm trying to follow the instructions here but no matter what I do it seems to be stuck on using Python 2.7.1 which is causing me errors currently.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install-osx.html
Attached is an image showing my logs of upgrading to the newest awsebccli but its still stuck saying 2.7.1 when it should say 3.6 (or newer)
What could I be doing wrong?
I want to note that I also installed the newest version of python3 and python via brew.
Here is the exact error I'm getting if it helps also.
You do not want to upgrade the default python 2.7x installation or the python 3.x installation available on linux or OSX. There are usually lots of other libraries and applications that depend on this default installation. When you want to use a specific version of python the approach is to rely on a virtual enviorenment.
virtualenv is a tool to create isolated Python environments.
You are then leaving the system installation untouched. Getting the hang of virtualenv is quite easy. Once you create it (a one off task). All you need is to activate it and then you can use it as you would normally use the default python interpreter. How to copy packages from one virtualenv (or the system installation) to another is discussed here:
Installing python3 in a python2 virtual environment
I encounter a problem with pip installation on linux. I've python 2.7 and 3.4, also Django in 1.7 installed. Currently I'm working on a project which uses different versions and I'm unable to install packages trough pip on python 2.7. Everything goes to directory of 3.4.
Is there any way to "force" pip to install packages in concrete version of python?
The usual, and recommended by most users, way of working with Django is to use a separate, virtual environment per project.
Use virtualenv to set up your Python 2.x environment and venv for Python 3.x. Both will install their own, local version of pip. Google lists lots of tutorials if you need help beyond the documentation.