I have setup a pypiserver behind an nginx proxy which uses htpasswd for authentication. I am currently able to upload sdists, but I can't figure out how to download them. I want to be able to download them when running setup.py test and somehow by using pip. Is this possible?
[distutils]
index-servers =
private
[private]
repository = https://example.com/pypi
username = remco
password = mypass
To make it extra hard the server is currently using a non verified ssl connection.
I tried the following setup based on http://pythonhosted.org/setuptools/setuptools.html#setuptools-package-index, but the only documentation on this is 'XXX'
#!/usr/bin/env python2.7
from setuptools import setup
setup(
name='asd',
version='0.0.1',
package_index='https://example.com/pypi/simple',
test_suite='test',
tests_require=['foo==0.0.1'])
for using your index with pip create ~/.pip/pip.conf with this content:
[global]
index-url = https://remco:mypass#build.d-centralize.nl/pypi/simple
cert = /etc/ssl/certs/your_cert_CA.pem
A little bit documentation on pip.conf is here and on pypiserver here
Perhaps you can also try using package_index='https://user:pass#example.com/pypi/simple
in setup.py.
The server certificate had to be setup properly.
For uploading using pip one must create a valid ~/.pypirc file:
[distutils]
index-servers = example
[example]
repository = https://example.com/pypi
username = myname
password = mypass
For installing packages one needs to add the following section to .pip/pip.conf
[global]
extra-index-url = https://myname:mypass#example.com/pypi/simple
As knitti noted in a previous answer it is also possible to user index-url instead of extra-index-url. This does mean that the cheese shop is not used as a second server.
For using a private server with setuptools unittesting you need to add the following to your setup.py:
from setuptools import setup
setup(
...
dependency_links=[
'https://myname:mypass#example.com/pypi/packages/'
])
Related
So I'm having a problem for quite some time which I cannot get solved. Basically I took over a project which uses Poetry for package managing (It is a Django project). Adding packages with 'poetry add' and then installing them via 'poetry install' all works fine locally (I use a Docker container). But when pushing the changes to my server and then running 'poetry install' it says the packages are already installed. But when running the Django application, I get an internal server error saying the package doesn't exist. An example is given with the 'openpyxl' package.
pyproject.toml
...
[tool.poetry.dependencies]
openpyxl = "^3.0.10"
...
poetry.lock
...
[[package]]
name = "openpyxl"
version = "3.0.10"
description = "A Python library to read/write Excel 2010 xlsx/xlsm files"
category = "main"
optional = false
python-versions = ">=3.6"
[package.dependencies]
openpyxl = {version = ">=2.6.0", optional = true, markers = "extra == \"xlsx\""}
[package.extras]
all = ["markuppy", "odfpy", "openpyxl (>=2.6.0)", "pandas", "pyyaml", "tabulate", "xlrd", "xlwt"]
cli = ["tabulate"]
html = ["markuppy"]
ods = ["odfpy"]
pandas = ["pandas"]
xls = ["xlrd", "xlwt"]
xlsx = ["openpyxl (>=2.6.0)"]
yaml = ["pyyaml"]
openpyxl = [
{file = "openpyxl-3.0.10-py2.py3-none-any.whl", hash = "sha256:0ab6d25d01799f97a9464630abacbb34aafecdcaa0ef3cba6d6b3499867d0355"},
{file = "openpyxl-3.0.10.tar.gz", hash = "sha256:e47805627aebcf860edb4edf7987b1309c1b3632f3750538ed962bbcc3bd7449"},
]
...
error:
Anyone experienced with Poetry who can help me with this?
I'm making an educated guess on few ideas you can try (without knowing more about the problem/context). For starters:
when running / starting your app, are you using poetry run... / poetry run python... ? I'm listing this one first, since it seems like the py environment isn't accessing the installed libs. would like to know the cmd "when running the Django application"
on the server, you can try to remove the lock file and re-run poetry lock, poetry install to have a 'fresh start'. The lock file you provided is the one from the server?
how do you use Docker, is this only on local? do you use Docker on the server as a running container? You can try dockerizing your app, and as part of the Dockerfile you copy pyproject.toml/poetry.lock files have docker RUN poetry cmds to install your deps.
again, these are shots in the dark. Some areas that can use some more understanding/detail are (a) how you start up Django app and (b) how you use Docker. If you're looking at point #3, I can elaborate on a solution for you. I've worked with quite a bit of docker+poetry apps/services.
I find multiple answers for how to publish package using Python Poetry to CodeArtifact and this is quite simple. But now I try to add the published package poetry add sample-package and it does not work. Poetry error:
Could not find a matching version of package sample-package
With pip install it works. But not with Poetry.
My pyproject.toml specifies to you my CodeArtifact repo as default. No problem with this:
[[tool.poetry.source]]
name = "artifact"
url = "https://test-domain-1234.d.codeartifact.region.amazonaws.com/pypi/test-repo"
default = true
Did anyone figure out how to do it?
I found my mistake. In the package that I publish I need to specify repository without /simple at the end. But for project where I use the package from CodeArtifact the repository needs to end with /simple.
Example: Publish package config looks like:
[[tool.poetry.source]]
name = "artifact"
url = "https://test-domain-1234.d.codeartifact.region.amazonaws.com/pypi/test-repository/"
secondary=true
And the publish command is: poetry publish --build -r artifact
For project where I use my package sample-lib the config should be:
[[tool.poetry.source]]
name = "artifact-repo"
url = "https://test-domain-1234.d.codeartifact.region.amazonaws.com/pypi/test-repository/simple"
secondary=true
And then Poetry command is: poetry add sample-lib --source artifact-repo
I created a virtual environment on my Desktop called project_env. I then installed the Google API Python Client from github (https://github.com/googleapis/google-api-python-client). Then created a Python file called youtube.py with the following code. When I run the file I get "ImportError no module googleapiclient"
from googleapiclient.discovery import build
api_key = "My Key"
youtube = build('youtube', 'v3', developerKeys=api_key)
request = youtube.channels().list(
part='statistics',
forUsername='livelifetothefull'
)
response = request.execute()
print(response)
I'm very new to this, so any help would be much appreciated.
It would have been great if you had provided details on how you are running it and what version of Python you are using. Without that, I have provided solutions that applies in a general context.
You may need to add your venv to your path so that it can recognize the site-packages. See here on how you can do that.
It may also mean that the interpreter may not be resolving to the correct venv. For example, if you running through cmd line try running it with python3 if you doing with python or vice-versa.
Change of library issue, see here on that.
You can try these commands :
cd project_env
source bin/activate
bin/pip install wheel
bin/pip install google-api-python-client
bin/python -c "from googleapiclient.discovery import build"
My problem is that I can't upload my module to PyPI. When I run
twine upload dist/easy-email-0.0.1.tar.gz
I get
HTTPError: 400 Client Error: 'Easy-email-0.0.1.tar.gz' is an invalid value for Download-URL. Error: Invalid URI see https://packaging.python.org/specifications/core-metadata for url: https://test.pypi.org/legacy/
What am I doing wrong?
Here is the setup.py:
from distutils.core import setup
setup(
name = 'easy-email',
packages = ['easy-email'],
version = '0.0.1', # Ideally should be same as your GitHub release tag varsion
description = 'Send emails in python!',
author = 'myname',
author_email = 'myemail',
url = 'https://github.com/marmadukeandbob05/Easy-Email/',
download_url = 'Easy-Email-0.0.1.tar.gz',
keywords = ['email', 'gmail'],
classifiers = [],
)
Your download_url is invalid, it is not a valid URL. Note that you don't need to set that value at all when uploading your installation archive to PyPI, because the download URL is on PyPI.
Only set download_url when you are going to host your packages elsewhere, not on PyPI. You would have to use a full URL, so one that starts with http:// or https://, and pip or easy_install would then follow that URL from PyPI to find the installation archive. You'd only use the twine register to register the metadata and just not use twine upload at all.
The error message linked you to the documentation for the field:
A string containing the URL from which this version of the distribution can be downloaded.
Bold emphasis mine; Easy-Email-0.0.1.tar.gz is not a URL. It is merely a filename.
You'd use this when you want people to download the archive from a different host, for example, from GitHub. For example, if the requests project wanted people to download the release from GitHub instead of from the PyPI servers, they could use download_url = 'https://github.com/requests/requests/archive/v2.18.4.tar.gz', and then only use twine register to put the metadata on PyPI.
I'm having trouble uploading my package to pypi. I used to be able to just use python setup.py sdist upload -r pypi but this now causes an error:
Upload failed (400): requires: Must be a valid Python identifier.
error: Upload failed (400): requires: Must be a valid Python identifier.
I've tried a few things to get this working but everything has failed with the same error.
I removed the current dist, build and egg folders in my root directory. Then I increased my package version number by 1 micro version. I ensured my ~/.pypirc file is as it should be according to instructions:
[distutils]
index-servers =
pypi
[pypi]
username: c.welsh2
password: ...
and updated pip, twine and setuptools. I create a build using
python setuptools.py bdist_wheel
which created the build in /package_root/dist/* and I try uploading to pypi using
twine upload dist/*
And again I get:
HTTPError: 400 Client Error: requires: Must be a valid Python identifier. for url: https://upload.pypi.org/legacy/
Does anybody know what is causing this problem?
For completeness, here is my setup file:
from distutils.core import setup
import setuptools
#version
MAJOR = 4
MINOR = 0
MICRO = 5
#=======
__version__ = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
setup(
name = 'PyCoTools',
packages = ['PyCoTools'], # this must be the same as the name above
version = __version__,
description = 'A python toolbox for COPASI',
author = 'Ciaran Welsh',
requires=['lxml','argparse','pandas','numpy','scipy','matplotlib.pyplot','scipy','seaborn','sklearn'],
package_data={'PyCoTools':['*.py','Documentation/*.pdf',
'logging_config.conf',
'Documentation/*.html','Licence.txt',
'ReadMe.md',
'Examples/KholodenkoExample/*',
'Examples/BioModelsWorkflowVersion1/*',
'Scripts/*.py',
'Tests/*.py',
'Tests/*.cps',
'PyCoToolsTutorial/*.pickle',
'PyCoToolsTutorial/*.py',
'PyCoToolsTutorial/*.ipynb',
'PyCoToolsTutorial/*.html',
'PyCoToolsTutorial/*.cps']},
author_email = '--<hidden>',
##
url = 'https://pypi.python.org/pypi/PyCoTools',
keywords = ['systems biology','modelling','biological',
'networks','copasi','identifiability analysis','profile likelihood'],
license='GPL4',
install_requires=['pandas','numpy','scipy','matplotlib',
'lxml'],
long_description='''Tools for using Copasi via Python and calculating profile likelihoods. See Github page and documentation for more details''')
Turns out there was quite an unforgiving typo. I couldn't give matplotlib.pyplot to the requires argument since its called matplotlib!