I'm trying to use Poetry to manage my python projects, but some PyPI dependencies don't have a version number such as this one.
I thus get such errors
$ poetry update
Updating dependencies
Resolving dependencies... (0.5s)
SolverProblemError
Because wworkflow depends on waapi-client-python (^0) which doesn't match any versions, version solving failed.
at /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/poetry/puzzle/solver.py:241 in _solve
237│ packages = result.packages
238│ except OverrideNeeded as e:
239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240│ except SolveFailure as e:
→ 241│ raise SolverProblemError(e)
242│
243│ results = dict(
244│ depth_first_search(
245│ PackageNode(self._package, packages), aggregate_package_nodes
I got the similar error when using any as the version value of the dependency in .toml.
Is it that Poetry does not support such a usecase?
Thanks to #Lain Shelvington's comments. I got the package name wrong. I took the name from its GitHub repo, but the PyPI package is named differently.
poetry update works after fixing the name. However, Poetry's inability to distinguish the package name error from its version tag issue is a bit confusing.
Related
I’m publishing a Python package.
It has dependencies a, b, c. It also has pytest as a dependency, defined in the group.dev as per poetry’s documentation.
However, I couldn’t find a conclusive answer for the following question:
When some installs my library with:
pip install my_library
Is pytest (defined in any group other than the main group, in this case dev) also installed? That would be very undesirable.
You can mention the dev dependency like this. It will not install the pytest. Reference
# rp_poetry/pyproject.toml (Excerpt)
[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.26.0"
[tool.poetry.dev-dependencies]
pytest = "^5.2"
black = "^21.9b0"
poetry is may go to change the label of dev-dependencies in the future.
I recently updated python (3.9.13)/ pip (22.1.2)/ pipenv (2022.9.20) while setting up a new environment for a project. I had an existing Pipfile, using AWS CodeArtifact as the source containing PyPI and our private projects, which had worked in the past:
[[source]]
url = "https://aws:${AUTH}#${CODEARTIFACT_URI}"
verify_ssl = true
[packages]
boto3 = "*"
...
[dev-packages]
pytest = "*"
...
[requires]
python_version = "3.9"
On running pipenv sync --dev, I received a stack trace ending with:
pipenv.vendor.plette.models.base.ValidationError: {'url': 'https://aws:${AUTH}#${CODEARTIFACT_URI}', 'verify_ssl': True}
name: required field
Adding a name to the source section fixes it, but I'm trying to understand why. I was reading the Advanced use of Pipenv documentation. It says Should you wish to use an alternative default index other than PyPI: simply do not specify PyPI as one of the sources in your Pipfile which is what we do. I'm curious if this is a new validation in a newer version of Pipenv, or if we are doing something incorrectly, since I still see Pipfiles out there with only a url and verify_ssl specified.
This was answered by #matteius over on https://github.com/pypa/pipenv/discussions/5370:
We converted to plette which is more strict, primarily because we have access to make modifications to that library, which we need to do for named package categories (slated for October 🤞). Plette already requires url, verify_url and name be supplied per each source entered, hence it is more strict than Pipfile was. It is good practice anyway to name your sources, because while the first source is the default install source for all unspecified packages, to specify any other index would require a name. From the sound of it we need to update the documentation (which is quiet true in a number of ways).
I'm trying to run a script that requires the datasets python package. I've tried installing this unsuccessfully using pip by calling:
pip install datasets
I know this hasn't worked because when I run the script I get the message:
Traceback (most recent call last):
File "lda.py", line 2, in <module>
import lda
File "/Users/deepthought/lda.py", line 3, in <module>
import datasets
ImportError: No module named datasets
I've installed python via homebrew.
When I run pip install datasets I get the error:
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/ch/84cpkwc52zx0rsh4k5v4_7h40000gn/T/pip-build-gZWyT3/datasets/
I'm fairly new to scripting python or going under the hood of an OS X, so there's a risk I've missed something elementary.
I've been researching & trying to overcome this for about a week now including looking at similar questions on stackoverflow.com and haven't gotten past this stage for the duration. One of the tutorials I was working through told me to edit ~/.profile
This has been left like so:
# The orginal version is saved in .profile.pysave
#PATH="/Library/Frameworks/Python.framework/Versions/3.5/bin:${PATH}"
#export PATH
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
/etc/paths contains:
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
I'm running OS X El Capitan - 10.11.5 (15F34)
Python 2.7.11
Brew doctor flagged multiple items, but I've no idea whether it is worth fixing none/all of them:
Warning: Your XQuartz (2.7.7) is outdated
Please install XQuartz 2.7.9:
https://xquartz.macosforge.org
Warning: Python is installed at /Library/Frameworks/Python.framework
Homebrew only supports building against the System-provided Python or a
brewed Python. In particular, Pythons installed to /Library can interfere
with other software installs.
Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected dylibs:
/usr/local/lib/libtcl8.6.dylib
/usr/local/lib/libtk8.6.dylib
Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected header files:
/usr/local/include/fakemysql.h
/usr/local/include/fakepq.h
/usr/local/include/fakesql.h
/usr/local/include/itcl.h
/usr/local/include/itcl2TclOO.h
/usr/local/include/itclDecls.h
/usr/local/include/itclInt.h
/usr/local/include/itclIntDecls.h
/usr/local/include/itclMigrate2TclCore.h
/usr/local/include/itclTclIntStubsFcn.h
/usr/local/include/mysqlStubs.h
/usr/local/include/node/ares.h
/usr/local/include/node/ares_version.h
/usr/local/include/node/nameser.h
/usr/local/include/node/node.h
/usr/local/include/node/node_buffer.h
/usr/local/include/node/node_internals.h
/usr/local/include/node/node_object_wrap.h
/usr/local/include/node/node_version.h
/usr/local/include/node/openssl/opensslconf.h
/usr/local/include/node/uv-private/ngx-queue.h
/usr/local/include/node/uv-private/stdint-msvc2008.h
/usr/local/include/node/uv-private/tree.h
/usr/local/include/node/uv-private/uv-bsd.h
/usr/local/include/node/uv-private/uv-darwin.h
/usr/local/include/node/uv-private/uv-linux.h
/usr/local/include/node/uv-private/uv-sunos.h
/usr/local/include/node/uv-private/uv-unix.h
/usr/local/include/node/uv-private/uv-win.h
/usr/local/include/node/uv.h
/usr/local/include/node/v8-debug.h
/usr/local/include/node/v8-preparser.h
/usr/local/include/node/v8-profiler.h
/usr/local/include/node/v8-testing.h
/usr/local/include/node/v8.h
/usr/local/include/node/v8stdint.h
/usr/local/include/node/zconf.h
/usr/local/include/node/zlib.h
/usr/local/include/odbcStubs.h
/usr/local/include/pqStubs.h
/usr/local/include/tcl.h
/usr/local/include/tclDecls.h
/usr/local/include/tclOO.h
/usr/local/include/tclOODecls.h
/usr/local/include/tclPlatDecls.h
/usr/local/include/tclThread.h
/usr/local/include/tclTomMath.h
/usr/local/include/tclTomMathDecls.h
/usr/local/include/tdbc.h
/usr/local/include/tdbcDecls.h
/usr/local/include/tdbcInt.h
/usr/local/include/tk.h
/usr/local/include/tkDecls.h
/usr/local/include/tkPlatDecls.h
Warning: Unbrewed .pc files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected .pc files:
/usr/local/lib/pkgconfig/tcl.pc
/usr/local/lib/pkgconfig/tk.pc
Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected static libraries:
/usr/local/lib/libtclstub8.6.a
/usr/local/lib/libtkstub8.6.a
Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
git
python3
Warning: Broken symlinks were found. Remove them with `brew prune`:
/usr/local/bin/github
/usr/local/lib/perl5/site_perl/Git/I18N.pm
/usr/local/lib/perl5/site_perl/Git/IndexInfo.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Editor.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Fetcher.pm
/usr/local/lib/perl5/site_perl/Git/SVN/GlobSpec.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Log.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Memoize/YAML.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Migration.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Prompt.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Ra.pm
/usr/local/lib/perl5/site_perl/Git/SVN/Utils.pm
/usr/local/lib/perl5/site_perl/Git/SVN.pm
/usr/local/lib/perl5/site_perl/Git.pm
/usr/local/share/git-core/templates/description
/usr/local/share/git-core/templates/hooks/applypatch-msg.sample
/usr/local/share/git-core/templates/hooks/commit-msg.sample
/usr/local/share/git-core/templates/hooks/post-update.sample
/usr/local/share/git-core/templates/hooks/pre-applypatch.sample
/usr/local/share/git-core/templates/hooks/pre-commit.sample
/usr/local/share/git-core/templates/hooks/pre-push.sample
/usr/local/share/git-core/templates/hooks/pre-rebase.sample
/usr/local/share/git-core/templates/hooks/prepare-commit-msg.sample
/usr/local/share/git-core/templates/hooks/update.sample
/usr/local/share/git-core/templates/info/exclude
/usr/local/share/man/man1/git-add.1
/usr/local/share/man/man1/git-am.1
/usr/local/share/man/man1/git-annotate.1
/usr/local/share/man/man1/git-apply.1
/usr/local/share/man/man1/git-archimport.1
/usr/local/share/man/man1/git-archive.1
/usr/local/share/man/man1/git-bisect.1
/usr/local/share/man/man1/git-blame.1
/usr/local/share/man/man1/git-branch.1
/usr/local/share/man/man1/git-bundle.1
/usr/local/share/man/man1/git-cat-file.1
/usr/local/share/man/man1/git-check-attr.1
/usr/local/share/man/man1/git-check-ignore.1
/usr/local/share/man/man1/git-check-mailmap.1
/usr/local/share/man/man1/git-check-ref-format.1
/usr/local/share/man/man1/git-checkout-index.1
/usr/local/share/man/man1/git-checkout.1
/usr/local/share/man/man1/git-cherry-pick.1
/usr/local/share/man/man1/git-cherry.1
/usr/local/share/man/man1/git-citool.1
/usr/local/share/man/man1/git-clean.1
/usr/local/share/man/man1/git-clone.1
/usr/local/share/man/man1/git-column.1
/usr/local/share/man/man1/git-commit-tree.1
/usr/local/share/man/man1/git-commit.1
/usr/local/share/man/man1/git-config.1
/usr/local/share/man/man1/git-count-objects.1
/usr/local/share/man/man1/git-credential-cache--daemon.1
/usr/local/share/man/man1/git-credential-cache.1
/usr/local/share/man/man1/git-credential-store.1
/usr/local/share/man/man1/git-credential.1
/usr/local/share/man/man1/git-cvsexportcommit.1
/usr/local/share/man/man1/git-cvsimport.1
/usr/local/share/man/man1/git-cvsserver.1
/usr/local/share/man/man1/git-daemon.1
/usr/local/share/man/man1/git-describe.1
/usr/local/share/man/man1/git-diff-files.1
/usr/local/share/man/man1/git-diff-index.1
/usr/local/share/man/man1/git-diff-tree.1
/usr/local/share/man/man1/git-diff.1
/usr/local/share/man/man1/git-difftool.1
/usr/local/share/man/man1/git-fast-export.1
/usr/local/share/man/man1/git-fast-import.1
/usr/local/share/man/man1/git-fetch-pack.1
/usr/local/share/man/man1/git-fetch.1
/usr/local/share/man/man1/git-filter-branch.1
/usr/local/share/man/man1/git-fmt-merge-msg.1
/usr/local/share/man/man1/git-for-each-ref.1
/usr/local/share/man/man1/git-format-patch.1
/usr/local/share/man/man1/git-fsck-objects.1
/usr/local/share/man/man1/git-fsck.1
/usr/local/share/man/man1/git-gc.1
/usr/local/share/man/man1/git-get-tar-commit-id.1
/usr/local/share/man/man1/git-grep.1
/usr/local/share/man/man1/git-gui.1
/usr/local/share/man/man1/git-hash-object.1
/usr/local/share/man/man1/git-help.1
/usr/local/share/man/man1/git-http-backend.1
/usr/local/share/man/man1/git-http-fetch.1
/usr/local/share/man/man1/git-http-push.1
/usr/local/share/man/man1/git-imap-send.1
/usr/local/share/man/man1/git-index-pack.1
/usr/local/share/man/man1/git-init-db.1
/usr/local/share/man/man1/git-init.1
/usr/local/share/man/man1/git-instaweb.1
/usr/local/share/man/man1/git-log.1
/usr/local/share/man/man1/git-lost-found.1
/usr/local/share/man/man1/git-ls-files.1
/usr/local/share/man/man1/git-ls-remote.1
/usr/local/share/man/man1/git-ls-tree.1
/usr/local/share/man/man1/git-mailinfo.1
/usr/local/share/man/man1/git-mailsplit.1
/usr/local/share/man/man1/git-merge-base.1
/usr/local/share/man/man1/git-merge-file.1
/usr/local/share/man/man1/git-merge-index.1
/usr/local/share/man/man1/git-merge-one-file.1
/usr/local/share/man/man1/git-merge-tree.1
/usr/local/share/man/man1/git-merge.1
/usr/local/share/man/man1/git-mergetool--lib.1
/usr/local/share/man/man1/git-mergetool.1
/usr/local/share/man/man1/git-mktag.1
/usr/local/share/man/man1/git-mktree.1
/usr/local/share/man/man1/git-mv.1
/usr/local/share/man/man1/git-name-rev.1
/usr/local/share/man/man1/git-notes.1
/usr/local/share/man/man1/git-p4.1
/usr/local/share/man/man1/git-pack-objects.1
/usr/local/share/man/man1/git-pack-redundant.1
/usr/local/share/man/man1/git-pack-refs.1
/usr/local/share/man/man1/git-parse-remote.1
/usr/local/share/man/man1/git-patch-id.1
/usr/local/share/man/man1/git-peek-remote.1
/usr/local/share/man/man1/git-prune-packed.1
/usr/local/share/man/man1/git-prune.1
/usr/local/share/man/man1/git-pull.1
/usr/local/share/man/man1/git-push.1
/usr/local/share/man/man1/git-quiltimport.1
/usr/local/share/man/man1/git-read-tree.1
/usr/local/share/man/man1/git-rebase.1
/usr/local/share/man/man1/git-receive-pack.1
/usr/local/share/man/man1/git-reflog.1
/usr/local/share/man/man1/git-relink.1
/usr/local/share/man/man1/git-remote-ext.1
/usr/local/share/man/man1/git-remote-fd.1
/usr/local/share/man/man1/git-remote-testgit.1
/usr/local/share/man/man1/git-remote.1
/usr/local/share/man/man1/git-repack.1
/usr/local/share/man/man1/git-replace.1
/usr/local/share/man/man1/git-repo-config.1
/usr/local/share/man/man1/git-request-pull.1
/usr/local/share/man/man1/git-rerere.1
/usr/local/share/man/man1/git-reset.1
/usr/local/share/man/man1/git-rev-list.1
/usr/local/share/man/man1/git-rev-parse.1
/usr/local/share/man/man1/git-revert.1
/usr/local/share/man/man1/git-rm.1
/usr/local/share/man/man1/git-send-email.1
/usr/local/share/man/man1/git-send-pack.1
/usr/local/share/man/man1/git-sh-i18n--envsubst.1
/usr/local/share/man/man1/git-sh-i18n.1
/usr/local/share/man/man1/git-sh-setup.1
/usr/local/share/man/man1/git-shell.1
/usr/local/share/man/man1/git-shortlog.1
/usr/local/share/man/man1/git-show-branch.1
/usr/local/share/man/man1/git-show-index.1
/usr/local/share/man/man1/git-show-ref.1
/usr/local/share/man/man1/git-show.1
/usr/local/share/man/man1/git-stage.1
/usr/local/share/man/man1/git-stash.1
/usr/local/share/man/man1/git-status.1
/usr/local/share/man/man1/git-stripspace.1
/usr/local/share/man/man1/git-submodule.1
/usr/local/share/man/man1/git-svn.1
/usr/local/share/man/man1/git-symbolic-ref.1
/usr/local/share/man/man1/git-tag.1
/usr/local/share/man/man1/git-tar-tree.1
/usr/local/share/man/man1/git-unpack-file.1
/usr/local/share/man/man1/git-unpack-objects.1
/usr/local/share/man/man1/git-update-index.1
/usr/local/share/man/man1/git-update-ref.1
/usr/local/share/man/man1/git-update-server-info.1
/usr/local/share/man/man1/git-upload-archive.1
/usr/local/share/man/man1/git-upload-pack.1
/usr/local/share/man/man1/git-var.1
/usr/local/share/man/man1/git-verify-pack.1
/usr/local/share/man/man1/git-verify-tag.1
/usr/local/share/man/man1/git-web--browse.1
/usr/local/share/man/man1/git-whatchanged.1
/usr/local/share/man/man1/git-write-tree.1
/usr/local/share/man/man1/git.1
/usr/local/share/man/man1/gitk.1
/usr/local/share/man/man1/gitremote-helpers.1
/usr/local/share/man/man1/gitweb.1
/usr/local/share/man/man3/Git.3pm
/usr/local/share/man/man3/Git::I18N.3pm
/usr/local/share/man/man3/Git::SVN::Editor.3pm
/usr/local/share/man/man3/Git::SVN::Fetcher.3pm
/usr/local/share/man/man3/Git::SVN::Memoize::YAML.3pm
/usr/local/share/man/man3/Git::SVN::Prompt.3pm
/usr/local/share/man/man3/Git::SVN::Ra.3pm
/usr/local/share/man/man3/Git::SVN::Utils.3pm
/usr/local/share/man/man5/gitattributes.5
/usr/local/share/man/man5/githooks.5
/usr/local/share/man/man5/gitignore.5
/usr/local/share/man/man5/gitmodules.5
/usr/local/share/man/man5/gitrepository-layout.5
/usr/local/share/man/man5/gitweb.conf.5
/usr/local/share/man/man7/gitcli.7
/usr/local/share/man/man7/gitcore-tutorial.7
/usr/local/share/man/man7/gitcredentials.7
/usr/local/share/man/man7/gitcvs-migration.7
/usr/local/share/man/man7/gitdiffcore.7
/usr/local/share/man/man7/gitglossary.7
/usr/local/share/man/man7/gitnamespaces.7
/usr/local/share/man/man7/gitrevisions.7
/usr/local/share/man/man7/gittutorial-2.7
/usr/local/share/man/man7/gittutorial.7
/usr/local/share/man/man7/gitworkflows.7
Warning: Your Homebrew is outdated.
You haven't updated for at least 24 hours. This is a long time in brewland!
To update Homebrew, run `brew update`.
How do I make progress in diagnosing the issue with the installation of the datasets package?
Update
Here is the script I'm trying to run:
import sys
egg_path = '/usr/local/lib/python2.7/site-packages/datasets-0.0.9-py2.7.egg'
sys.path.append(egg_path)
import numpy as np
import lda
import datasets
X = lda.datasets.load_reuters()
vocab = lda.datasets.load_reuters_vocab()
titles = lda.datasets.load_reuters_titles()
X.shape
(395, 4258)
X.sum()
84010
model = lda.LDA(n_topics=20, n_iter=1500, random_state=1)
model.fit(X) # model.fit_transform(X) is also available
topic_word = model.topic_word_ # model.components_ also works
n_top_words = 8
for i, topic_dist in enumerate(topic_word):
topic_words = np.array(vocab)[np.argsort(topic_dist)][:-(n_top_words+1):-1]
print('Topic {}: {}'.format(i, ' '.join(topic_words)))
Using pip install datasets I was also not able to properly install this package. It seems like there is a bug in this particular package.
The DESCRIBE.rst file is simply missing. To fix this just download the plain package from PyPi. https://pypi.python.org/pypi/datasets/0.0.9
Then adjust the setup.py file (remove the description).
Afterwards you need to install using python setup.py install. Don't forget to add the installed package to your Python path!
To do so, I would recommend that you add the following to your script.
import sys
egg_path = '__MODULE_PATH__/datasets-0.0.9-py3.5.egg'
sys.path.append(egg_path)
import datasets
Otherwise, you can also add your module using:
export PATH=__MODULE_PATH__:$PATH
Alternatively, you could also simply pull the source code from the Github repository and just include it in your project. https://github.com/realtimeweb/datasets
Hope this was kind of helpful to your problem. If you got any further questions just let me know.
I just hit the same issue on a rapsberry pi, just found out this had been fixed but the error comes from the lack of ram to extract properly the package.
You can fix this by disabling the creation of a cache dir in ram adding the parameter
--no-cache-dir
for example
pip2 install --user --no-cache-dir datasets
I am creating a setup.py file for a project which depends on private GitHub repositories. The relevant parts of the file look like this:
from setuptools import setup
setup(name='my_project',
...,
install_requires=[
'public_package',
'other_public_package',
'private_repo_1',
'private_repo_2',
],
dependency_links=[
'https://github.com/my_account/private_repo_1/master/tarball/',
'https://github.com/my_account/private_repo_2/master/tarball/',
],
...,
)
I am using setuptools instead of distutils because the latter does not support the install_requires and dependency_links arguments per this answer.
The above setup file fails to access the private repos with a 404 error - which is to be expected since GitHub returns a 404 to unauthorized requests for a private repository. However, I can't figure out how to make setuptools authenticate.
Here are some things I've tried:
Use git+ssh:// instead of https:// in dependency_links as I would if installing the repo with pip. This fails because setuptools doesn't recognize this protocol ("unknown url type: git+ssh"), though the distribute documentation says it should. Ditto git+https and git+http.
https://<username>:<password>#github.com/... - still get a 404. (This method doesn't work with curl or wget from the command line either - though curl -u <username> <repo_url> -O <output_file_name> does work.)
Upgrading setuptools (0.9.7) and virtualenv (1.10) to the latest versions. Also tried installing distribute though this overview says it was merged back into setuptools. Either way, no dice.
Currently I just have setup.py print out a warning that the private repos must be downloaded separately. This is obviously less than ideal. I feel like there's something obvious that I'm missing, but can't think what it might be. :)
Duplicate-ish question with no answers here.
I was trying to get this to work for installing with pip, but the above was not working for me. From [1] I understood the PEP508 standard should be used, from [2] I retrieved an example which actually does work (at least for my case).
Please note; this is with pip 20.0.2 on Python 3.7.4
setup(
name='<package>',
...
install_requires=[
'<normal_dependency>',
# Private repository
'<dependency_name> # git+ssh://git#github.com/<user>/<repo_name>#<branch>',
# Public repository
'<dependency_name> # git+https://github.com/<user>/<repo_name>#<branch>',
],
)
After specifying my package this way installation works fine (also with -e settings and without the need to specify --process-dependency-links).
References
[1] https://github.com/pypa/pip/issues/4187
[2] https://github.com/pypa/pip/issues/5566
Here's what worked for me:
install_requires=[
'private_package_name==1.1',
],
dependency_links=[
'git+ssh://git#github.com/username/private_repo.git#egg=private_package_name-1.1',
]
Note that you have to have the version number in the egg name, otherwise it will say it can't find the package.
I couldn't find any good documentation on this, but came across the solution mainly through trial & error. Further, installing from pip & setuptools have some subtle differences; but this way should work for both.
GitHub don't (currently, as of August 2016) offer an easy way to get the zip / tarball of private repos. So you need to point setuptools to tell setuptools that you're pointing to a git repo:
from setuptools import setup
import os
# get deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']
setup(
# ...
install_requires='package',
dependency_links = [
'git+https://{github_token}#github.com/user/{package}.git/#{version}#egg={package}-0'
.format(github_token=github_token, package=package, version=master)
]
A couple of notes here:
For private repos, you need to authenticate with GitHub; the simplest way I found is to create an oauth token, drop that into your environment, and then include it with the URL
You need to include some version number (here is 0) at the end of the link, even if there's no package on PyPI. This has to be a actual number, not a word.
You need to preface with git+ to tell setuptools it's to clone the repo, rather than pointing at a zip / tarball
version can be a branch, a tag, or a commit hash
You need to supply --process-dependency-links if installing from pip
I found a (hacky) workaround:
#!/usr/bin/env python
from setuptools import setup
import os
os.system('pip install git+https://github-private.corp.com/user/repo.git#master')
setup( name='original-name'
, ...
, install_requires=['repo'] )
I understand that there are ethical issues with having a system call in a setup script, but I can't think of another way to do this.
Via Tom Hemmes' answer I found this is the only thing that worked for me:
install_requires=[
'<package> # https://github.com/<username>/<package>/archive/<branch_name>.zip']
Using archive URL from github works for me, for public repositories. E.g.
dependency_links = [
'https://github.com/username/reponame/archive/master.zip#egg=eggname-version',
]
With pip 20.1.1, this works for me
install_requires=[ "packson3#https://tracinsy.ewi.tudelft.nl/pubtrac/Utilities/export/138/packson3/dist/packson3-1.0.0.tar.gz"],
in setup.py
Edit: This appears to only work with public github repositories, see comments.
dependency_links=[
'https://github.com/my_account/private_repo_1/tarball/master#egg=private_repo_1',
'https://github.com/my_account/private_repo_2/tarball/master#egg=private_repo_2',
],
Above syntax seems to work for me with setuptools 1.0. At the moment at least the syntax of adding "#egg=project_name-version" to VCS dependencies is documented in the link you gave to distribute documentation.
This work for our scenario:
package is on github in a private repo
we want to install it into site-packages (not into ./src with -e)
being able to use pip install -r requirements.txt
being able to use pip install -e reposdir (or from github), where the dependencies are only specified in requirements.txt
https://github.com/pypa/pip/issues/3610#issuecomment-356687173
I recently started out using pypi for some packaging of a few tools that are useful in my everyday life, but I'm having trouble actually making sure that I can download the most recent version of my package.
The package in question is pyfuzz and I just upgraded to version 0.1.1, but for some reason when I pip install it, even with the --upgrade flag I can only pull down 0.1.0.
The file is clearly recognized on the pypi site (See: https://pypi.python.org/pypi/PyFuzz/0.1.1) and if I try to upload again I get an error saying that I've already uploaded 0.1.1.
This is my setup file:
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
setup(
name="PyFuzz",
version="0.1.1",
author="Slater Victoroff",
author_email="Slater.R.Victoroff#gmail.com",
packages=["pyfuzz"],
url="http://pypi.python.org/pypi/PyFuzz/",
license="LICENSE.txt",
description="Simple fuzz testing for unit tests, i18n, and security",
long_description=open("README.txt").read(),
install_requires=[
"lxml >= 2.3.2",
"requests >= 1.2.3",
"numpy >= 1.6.1",
"cssselect >= 0.8"
],
)
And I uploaded using python setup.py sdist upload am I doing something silly here? Any help is appreciated.
It looks like the PyPi Index hasn't been updated yet:
https://pypi.python.org/simple/PyFuzz/
(Output at time of writing):
PyFuzz-0.1.0.tar.gz
This is a known issue with PyPi - package indexes and mirrors for the actual packages are often out of sync, if not down alltogether. Usually its the other way round though - the index listing a version that some package mirrors don't have yet. http://www.pypi-mirrors.org/ may be useful to check mirror freshness.
So it's not a mistake on your side, but a failure of PyPi. Just wait, and eventually it should update and resolve itself.
What you can do in the meantime is to install the most recent version of your package by explicitely giving the URL to the source tarball:
pip install https://pypi.python.org/packages/source/P/PyFuzz/PyFuzz-0.1.1.tar.gz