I am working with Apache Airflow in AWS (MWAA) and I have deployed a python package that was developed on python 3.8. It was running fine locally, however, on MWAA it starts giving syntax errors probably because the python version in MWAA is 3.7. Is there any possibility to upgrade the python version in MWAA or any workaround for this?
As you can see here https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-faqs.html#python-version MWAA uses python 3.7
It means plugins and DAGs will have to be in python 3.7 (no walrus for you!)
But tasks can still be run on python 3.8+ if:
you run them in docker (via EC2, Sagemaker or other)
you upgrade python on a worker using the bashOperator (I do not recommand it)
I hope they upgrade soon.
Related
You have already done all the configuration to implement python in the IIS, but it gives me an error.
I already installed python 3.9 on my machine and the IIS has everything installed
Configuration python in IIS:
This is my first attempt at python. What is going wrong?
I am trying to run a django project on an EC2 server, however, when I run python3 manage.py runserver, it returns this error, django.core.exceptions.ImproperlyConfigured: SQLite 3.9.0 or later is required (found 3.7.17).. I then check to see what version of SQLite3 is running on my python installation on my EC2 server by running sqlite3.sqlite_version, and it returns 3.7.17. So I then try to update SQLite3 using the default AWS EC2 Amazon Linux package manager, yum, by running yum install sqlite. It then returns this, Package sqlite-3.7.17-8.amzn2.1.1.x86_64 already installed and latest version, even though it is not the latest version. How can I install the latest version of SQLite3 to fix this?
I had the same problem. Since my app is very small with little dependency, I was able to quickly switch to EC2 sever running Ubuntu. It is necessary to learn how to use Ubuntu (apt).
You can find right now in the installation:
Package: sqlite3
Version: 3.31.1-4ubuntu0.2
We need to use an external library which only runs on Python 3.6 or higher. Unfortunately, HDInsight only has Python 3.5 installed by default.
Is there a way to upgrade the minor version for Python on HDInsight?
The official Azure documentation does not mention about it.
https://learn.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-python-package-installation
As stated in Microsoft Q&A:
It's not recommended as using non-cluster built-in Python versions are unsupported scenario.
WARNING!: HDInsight cluster depends on the built-in Python environment - Python 3.5. Directly installing custom packages in those default built-in environments may cause unexpected library version changes. And break the cluster further.
If you want to install, you can use Python 3.6, change “python=3.5” in this command to python=3.6, and follow the rest steps in the document works.
sudo /usr/bin/anaconda/bin/conda create --prefix /usr/bin/anaconda/envs/py36new python=3.6 anaconda –yes
I'm developing a website using a server with Debian 8.10 (Jessie) as its OS and Python 3.4.2 (the supported Python version for Debian Jessie) while my notebook is using Ubuntu 16.04 and Python 3.5.2 (I think it's also the default version for Ubuntu 16.04). I was planning to build my website using Django 1.11 which both Python versions (3.4 and 3.5) support.
Is there any compatibility issues when I develop it using my Python 3.5.2 and deploy it to a Python 3.4.2 server? If any, how much the trouble it will be?
I know I can install any version of Python by adding someone's repository, but it seems unofficial so I avoid doing it. And there is a workaround that come to my mind: intall a specific version of Python by download its tarball file from the official website
Which will you recommend most?
Upgrade my server's Python version to 3.5.2 by adding someone's repo
Download Python 3.4.2's tarball and install it to my local machine
Upgrade my server's OS to Debian 9 Stretch which its default Python version is 3.5.3
Or any other better idea? Or perhaps you have a way to install specific Python version?
*I have some concern on security and safety
When you are working with different Python versions, it is recommended that you use some kind of virtual environment so each project has its own python version with its own modules that you need.
In this way, you can always keep each project with the Python version that you know it will work and with total compatibility with all the modules that you are using, making sure that any update to your working modules does not break anything in your past projects.
You should install a virtual environment in your local machine that matches the server machine and match the Python version and the modules that you have available, then start developing from there.
This space is a little bit too short to explain how to work with them, but you can find information about them here: https://docs.python.org/3/tutorial/venv.html
Optionally, you could use anaconda and its own version of enviroments, that may be simpler if you are familiar with conda
I'm quite newbiew with server configuration so I've created a droplet from DigitalOcean that has Django on Ubuntu 14.04.
My app was running pretty fine but after a few changes I started facing a few issues related to environment. I'm developing with Python 3.4 and Django 1.7 and my server is running with Python 2.7 and Django 1.7.
Question: how can I make the server run with the proper python version? This may help:
# python --version
Python 2.7.6
# python3 --version
Python 3.4.0
This is the tutorial I've followed: https://www.digitalocean.com/community/tutorials/how-to-use-the-django-one-click-install-image
Thanks!