Develop Reference

Python3 with Bazel 0.5.4 - hard-coded to python executable?

I'm maintaining a project that is using Bazel 0.5.4, and it needs to run on an environment where python does not exist. I'd like to point the application to using python3.
This issue is similar to Bazel 0.26.1 use Python3 on py_test, but the version of rules_python mentioned in that ticket is so new that it's not compatible with bazel 0.5.4
GOAL: Issue python commands with /usr/bin/python3, not python
As-is, the application fails when it's unable to find python
bazel build completes without an issue, but bazel test continues to look for python, and fail.
bazel test ... --test_output=all --python_top=//sandbox_app:python-3.8.10
...
/usr/bin/env: 'python': No such file or directory
================================================================================
INFO: Elapsed time: 0.658s, Critical Path: 0.10s
INFO: Build completed, 1 test FAILED, 2 total actions
//sandbox_app:sandbox_test FAILED in 0.1s
This didn't work
The following configuration worked for a similar user, described here:
https://groups.google.com/g/bazel-discuss/c/nVQ48R94S_8
But even after a bazel clean, bazel still continues to invoke python when running the tests.
WORKSPACE
git_repository(
name = "io_bazel_rules_go",
remote = "https://github.com/bazelbuild/rules_go.git",
tag = "0.5.4",
)
BUILD.bazel
package(default_visibility = ["//visibility:public"])
py_test(
name = "sandbox_test",
srcs = ["sandbox_test.py"],
default_python_version = "PY3",
srcs_version = "PY3",
)
py_runtime(
name = "python-3.8.10",
files = [],
interpreter_path = "/usr/bin/python3",
)
Version details:
$ bazel info release
release 0.5.4
It would minimize disruption to the application if this can be done without upgrading bazel.
sandbox_test.py
#!/usr/bin/python3
import unittest
class SandboxTest(unittest.TestCase):
def testRunSandbox(self):
# Valid for Python 3, syntax error for python2
print(print("Hello, Python!"))
if __name__ == '__main__':
unittest.main()
Possibly related issues
https://github.com/bazelbuild/bazel/issues/4815
https://github.com/bazelbuild/bazel/issues/200
Rules spec:
https://bazel.build/reference/be/python
Similar issues:
How do I select the runtime in bazel for Python and pip?
Bazel, python3 and python interpreter options
Getting Bazel to run using Python3 (syntax error)

Poetry install doesn't seem to install the packages in the right place

So I'm having a problem for quite some time which I cannot get solved. Basically I took over a project which uses Poetry for package managing (It is a Django project). Adding packages with 'poetry add' and then installing them via 'poetry install' all works fine locally (I use a Docker container). But when pushing the changes to my server and then running 'poetry install' it says the packages are already installed. But when running the Django application, I get an internal server error saying the package doesn't exist. An example is given with the 'openpyxl' package.
pyproject.toml
...
[tool.poetry.dependencies]
openpyxl = "^3.0.10"
...
poetry.lock
...
[[package]]
name = "openpyxl"
version = "3.0.10"
description = "A Python library to read/write Excel 2010 xlsx/xlsm files"
category = "main"
optional = false
python-versions = ">=3.6"
[package.dependencies]
openpyxl = {version = ">=2.6.0", optional = true, markers = "extra == \"xlsx\""}
[package.extras]
all = ["markuppy", "odfpy", "openpyxl (>=2.6.0)", "pandas", "pyyaml", "tabulate", "xlrd", "xlwt"]
cli = ["tabulate"]
html = ["markuppy"]
ods = ["odfpy"]
pandas = ["pandas"]
xls = ["xlrd", "xlwt"]
xlsx = ["openpyxl (>=2.6.0)"]
yaml = ["pyyaml"]
openpyxl = [
{file = "openpyxl-3.0.10-py2.py3-none-any.whl", hash = "sha256:0ab6d25d01799f97a9464630abacbb34aafecdcaa0ef3cba6d6b3499867d0355"},
{file = "openpyxl-3.0.10.tar.gz", hash = "sha256:e47805627aebcf860edb4edf7987b1309c1b3632f3750538ed962bbcc3bd7449"},
]
...
error:
Anyone experienced with Poetry who can help me with this?

I'm making an educated guess on few ideas you can try (without knowing more about the problem/context). For starters:
when running / starting your app, are you using poetry run... / poetry run python... ? I'm listing this one first, since it seems like the py environment isn't accessing the installed libs. would like to know the cmd "when running the Django application"
on the server, you can try to remove the lock file and re-run poetry lock, poetry install to have a 'fresh start'. The lock file you provided is the one from the server?
how do you use Docker, is this only on local? do you use Docker on the server as a running container? You can try dockerizing your app, and as part of the Dockerfile you copy pyproject.toml/poetry.lock files have docker RUN poetry cmds to install your deps.
again, these are shots in the dark. Some areas that can use some more understanding/detail are (a) how you start up Django app and (b) how you use Docker. If you're looking at point #3, I can elaborate on a solution for you. I've worked with quite a bit of docker+poetry apps/services.

Can Poetry add package from AWS CodeArtifact

I find multiple answers for how to publish package using Python Poetry to CodeArtifact and this is quite simple. But now I try to add the published package poetry add sample-package and it does not work. Poetry error:
Could not find a matching version of package sample-package
With pip install it works. But not with Poetry.
My pyproject.toml specifies to you my CodeArtifact repo as default. No problem with this:
[[tool.poetry.source]]
name = "artifact"
url = "https://test-domain-1234.d.codeartifact.region.amazonaws.com/pypi/test-repo"
default = true
Did anyone figure out how to do it?

I found my mistake. In the package that I publish I need to specify repository without /simple at the end. But for project where I use the package from CodeArtifact the repository needs to end with /simple.
Example: Publish package config looks like:
[[tool.poetry.source]]
name = "artifact"
url = "https://test-domain-1234.d.codeartifact.region.amazonaws.com/pypi/test-repository/"
secondary=true
And the publish command is: poetry publish --build -r artifact
For project where I use my package sample-lib the config should be:
[[tool.poetry.source]]
name = "artifact-repo"
url = "https://test-domain-1234.d.codeartifact.region.amazonaws.com/pypi/test-repository/simple"
secondary=true
And then Poetry command is: poetry add sample-lib --source artifact-repo

poetry tries to install project folder as a dependency: EnvCommandError

I am trying to run poetry install command, but I get the following error:
[EnvCommandError]
Command ['pip', 'install', '-e', '<PROJECT_FOLDER_PATH>'] errored with
the following return code 1, and output:
Obtaining file:///<PROJECT_FOLDER_PATH>
ERROR: Package '<subfolder>' requires a different Python: 3.6.8 not in '>=3.7,<4.0'
Where my project directory containing .toml file is marked as <PROJECT_FOLDER> (PROJECT_FOLDER_PATH is, correspondingly, it's full path), and it contains <subfolder>.
Part of my toml file:
[tool.poetry]
name = "<PROJECT_FOLDER>"
version = "0.1.0"
description = ""
authors = ["Your Name <you#example.com>"]
[tool.poetry.dependencies]
python = "^3.7"
It seems that poetry tries to install the project itself as a dependency, but for some reason that I don't understand it seems the conflicting Python version. I temporarily solved it by setting python = "^3.6", but now the issue is back, as I need some package which only accepts python = "^3.7".

Pypi upload problems: Must be a valid Python identifier

I'm having trouble uploading my package to pypi. I used to be able to just use python setup.py sdist upload -r pypi but this now causes an error:
Upload failed (400): requires: Must be a valid Python identifier.
error: Upload failed (400): requires: Must be a valid Python identifier.
I've tried a few things to get this working but everything has failed with the same error.
I removed the current dist, build and egg folders in my root directory. Then I increased my package version number by 1 micro version. I ensured my ~/.pypirc file is as it should be according to instructions:
[distutils]
index-servers =
pypi
[pypi]
username: c.welsh2
password: ...
and updated pip, twine and setuptools. I create a build using
python setuptools.py bdist_wheel
which created the build in /package_root/dist/* and I try uploading to pypi using
twine upload dist/*
And again I get:
HTTPError: 400 Client Error: requires: Must be a valid Python identifier. for url: https://upload.pypi.org/legacy/
Does anybody know what is causing this problem?
For completeness, here is my setup file:
from distutils.core import setup
import setuptools
#version
MAJOR = 4
MINOR = 0
MICRO = 5
#=======
__version__ = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
setup(
name = 'PyCoTools',
packages = ['PyCoTools'], # this must be the same as the name above
version = __version__,
description = 'A python toolbox for COPASI',
author = 'Ciaran Welsh',
requires=['lxml','argparse','pandas','numpy','scipy','matplotlib.pyplot','scipy','seaborn','sklearn'],
package_data={'PyCoTools':['*.py','Documentation/*.pdf',
'logging_config.conf',
'Documentation/*.html','Licence.txt',
'ReadMe.md',
'Examples/KholodenkoExample/*',
'Examples/BioModelsWorkflowVersion1/*',
'Scripts/*.py',
'Tests/*.py',
'Tests/*.cps',
'PyCoToolsTutorial/*.pickle',
'PyCoToolsTutorial/*.py',
'PyCoToolsTutorial/*.ipynb',
'PyCoToolsTutorial/*.html',
'PyCoToolsTutorial/*.cps']},
author_email = '--<hidden>',
##
url = 'https://pypi.python.org/pypi/PyCoTools',
keywords = ['systems biology','modelling','biological',
'networks','copasi','identifiability analysis','profile likelihood'],
license='GPL4',
install_requires=['pandas','numpy','scipy','matplotlib',
'lxml'],
long_description='''Tools for using Copasi via Python and calculating profile likelihoods. See Github page and documentation for more details''')

Turns out there was quite an unforgiving typo. I couldn't give matplotlib.pyplot to the requires argument since its called matplotlib!