Develop Reference

I'm hitting a roadblock when trying to install Cython

I'm trying to install Cython, which I need for using another package. I have Xcode and a C compiler. I'm using a Mac (Big Sur) with python 3.9, but I have no experience with C.
I first tried using pip:
pip3 install Cython
This resulted in the message
Requirement already satisfied: Cython in /usr/local/lib/python3.9/site-packages (0.29.23)
and I was unable to import the package into my editor, Idle. Usually, every package I install goes to
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages
so I tried downloading the zip file from https://github.com/cython/cython and placing it in the above site-packages directory. At the terminal, I changed to the directory cython-master and entered
python3 setup.py install
which seemed to work since a number of messages popped up on my screen, none indicating errors, and the whole thing ending with
Installed /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-
packages/Cython-3.0a7-py3.9-macosx-10.9-x86_64.egg
Processing dependencies for Cython==3.0a7
Finished processing dependencies for Cython==3.0a7
However, I'm still receiving a module not found error a my shell in Idle.

Typically happens with a wrong pip executable.
Use $ python -mpip install
Use virtualenverapper (or the standard library venv module)

PyCharm, can not import modules

I need to import
1.yaml
2.pypiper
modules to the latest PyCharm IDE. I can not find pypiper in the list of available modules. When I am trying to add a new repository for the pypiper:
https://github.com/epigen/pypiper.git
still the same: typing pypiper in the search field does not yield anything.
yaml at the same time can be found, but it is giving me an error when I am trying to add it:
Could not find a version that satisfies the requirement yaml (from versions: )
No matching distribution found for yaml

PyCharm looks for packages in the repositories that it has been told to look in. By default it looks at https://pypi.python.org/simple.
Effectively this means that if you cannot install something through pip, you cannot install through the PyCharm interface.
What you need to do is to figure out:
How to install yaml
sudo pip install pyyaml
How to install pypiper
pip install --user https://github.com/epigen/pypiper/zipball/master
When I run into these issues it's easiest for me to just install on the command line opposed to through PyCharm unless it's a virtual env. Just make sure your pip is referencing the same python interpreter you are using in PyCharm or you still wont see it in PyCharm.

How to handle different versions of python protobuf

My python package contains a lot of files compiled by python-protobuf (python2-protobuf-2.5.0 on Arch Linux), I installed the package on Ubuntu server 12.04.3 (which have python-protobuf-2.4.1), tried to run the code, and hit the following error:
from google.protobuf.internal import enum_type_wrapper
ImportError: cannot import name enum_type_wrapper
I think it's because the protobuf modules in my package are compiled by protobuf-2.5.0 and they do not work with protobuf-2.4.1.
I have no idea of the environments in which my code may run, the version of protobuf may vary. How to make my package work with both protobuf 2.4 and 2.5?
(A possible way: include two different sets of protobuf libraries (one compiled by 2.4.1, the other compiled by 2.5.0) in my package, get google.protobuf version at runtime and select the protobuf libraries to import. Is it possible?

You need to specify the version of protobuf that will work with in your setup.py in the list install_requires=['protobuf>=2.5.0']. With a Python package, you can put just the name or the exact versions that will run with the package using ==. I believe you can also specify != for specific versions.
If you are not packaging it with a setup.py, you should set up a virtualenv and put a file install_requires.txt with all the specific python packages and versions in the root of the project.
That might look like:
$ cd ../project
$ virtualenv project_venv
$ source project_venv/bin/activate
$ cd project
$ pip install protobuf>=2.5.0
$ pip freeze > ./requirements.txt
Then someone you distribute to can activate their virtualenv and do:
$ pip install -r requirements.txt
Make sure your package will work from a fresh virtualenv by installing with that method. This is also good to check before installing via a setup.py. You want to make sure your requirements will get anyone working who just does a fresh sudo python setup.py install, or python setup.py install in a virtualenv context.
You can exit a virtualenv context with:
$ deactivate

Your best bet may be to include a copy of the protobuf runtime library with your package, maybe under a different package name. Then you can make sure that it matches the version of your generated code.
Another option is to invoke protoc as part of the installation process, so you get whatever version is available on the host.
I don't think packaging multiple versions of your generated code sounds like a good idea -- you'll just have problems again when the next protobuf release comes out.

How to prevent setuptools install package as an .egg

For example, installing IPython on Linux (where setuptools is not installed) I've got IPython installed in site-packages\IPython.
Installing IPython on Windows (where IPython requires setuptools), after executing the same command
python setup.py install
I get IPython installed in site-packages\ipython-0.13.2-py2.7.egg\IPython
Is there a way to install the module "old way" i.e. into site-packages\IPython?

I've discovered that
python setup.py install --old-and-unmanageable
does the job, but I am not sure it is a good way as --old-and-unmanageable is marked "Try not to use this!".

I don't know if it's applicable in your case, but the --root option also does this. For example, the Fedora packaging guidelines make use of this, since the versioning is managed externally by RPM. https://fedoraproject.org/wiki/Packaging:Python_Eggs

setup.py: restrict the allowable version of the python interpreter

I have a Python library. Unfortunately I have not updated it to work with Python 3 yet.
In its setup.py, I added
install_requires=['python<3'],
My intent was to not allow this package to be installed/used under Python 3, because I know it doesn't (yet) work. I don't think this is the right way to do it, because pip then tries to download and install python 2.7.3 (which is already the installed version!).
How should I specify my library dependency on a particular range of Python interpreter versions? Should I add a Programming Language :: Python :: 2 :: Only tag? Will this actually prevent installation under Python 3? What if I also want to restrict the minimum version to Python 2.6?
I'd prefer a solution that works everywhere, but would settle for one that only works in pip (and hopefully doesn't cause easy_install to choke).

As of version 9.0.1 pip will honor a new python_requires string, specifying the Python version required for installation, e.g, for example if one wishes to enforce minimum Python version of 3.3:
setup(
...,
python_requires=">=3.3"
)
See here for more details. See also this answer on SO.

A possible solution is to test for the Python version, since pip can't satisfy the Python version except for the version it's currently running in (it installs in the current Python environment):
import sys
if not sys.version_info[0] == 2:
sys.exit("Sorry, Python 3 is not supported (yet)")
setup(...

After commenting in the answer above and receiving feedback, I thought to turn my comment into an answer. Note that the answers above are all fine, yet from my experience, I found one thing that is "missing" in these answers, that needs to be pointed out, so here I will illustrate this issue.
For simplicity and completeness of illustration, I have composed a very minimal and simple Python 3 project. The only 3rd party package it uses, is the famous SSH client package paramiko (it's official PyPi page can be found here).
The Python interpreter in the virtual environment of my project is of version 3.6.9
Now, in order to check the python_requires attribute "in action", I have added it to the project's setup.py script, which looks as follows:
from setuptools import setup, find_packages
setup(name='mySampleProject',
version='1.0',
description='Sample project in Python 3',
author='Guy Avraham',
license='MIT',
packages=find_packages(),
include_package_data=True,
python_requires='>=3.8',
install_requires=['paramiko'])
Note that I "required" that the Python version will be 3.8+. This of course should NOT work with the current Python version in the project's virtual environment which is 3.6.9.
Now, when I build the project using the "normal" use in the setup.py, meaning by running: python3 setup.py install, the project was built successfully. See the following output of the pip3 list command after running the python3 setup.py install command:
(mySampleProject_env) guya#ubuntu:~/mySampleProject$ pip3 list
DEPRECATION: The default format will switch to columns in the future. You can use --
format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
bcrypt (3.2.0)
cffi (1.14.3)
cryptography (3.1.1)
mySampleProject (1.0)
paramiko (2.7.2)
pip (9.0.1)
pkg-resources (0.0.0)
pycparser (2.20)
PyNaCl (1.4.0)
setuptools (39.0.1)
six (1.15.0)
As you can see, the project, along with all its "sub dependencies" was installed EVEN though I was NOT expecting it to.
On the other hand, when I installed the project using the command: pip3 install -e . (note the . to indicate the "current working directory"), I got the following output:
(mySampleProject_env) guya#ubuntu:~/mySampleProject$ pip3 install -e .
Obtaining file:///home/guya/mySampleProject
mySampleProject requires Python '>=3.8' but the running Python is 3.6.9
Which now, indeed, "considers" the python_requires attribute, thus "failing" the build of the project.
It is detailed in the very first paragraph in the tutorial in this page
and also during minutes ~09:00 - 11:00 in this video
NOTE: I did NOT check all the above for Python 2 (or pip for Python 2).