Develop Reference

Importing module from Github to use in Python

I'm a beginner in Python and I have no experience with GitHub at all. I want to import the module semsimlib from the following URL: https://github.com/timvdc/semsimlib
I have looked on the internet for help on how to do this but most of it is very unclear and doesn't seem to work for me. Can anyone provide a detailed explanation on how to do this in a easy way?

It looks the repo does not provide appropriate scripts to simply install the package. There is no setup.py file and there is no distribution on pypi.
What you can do is go to site-packages folder inside your python installation or inside your virtual environment. Then run git clone https://github.com/timvdc/semsimlib. You should now be able to import semsimlib. Keep in mind that you will also have to install all the other dependencies your self one by one since there is also no requirements file.
You can also clone the repo into any folder on your computer and at the top of your script put:
import sys
sys.path.append("path/to/semsimlib/folder")
semsimlib will now be importable. However, I would try to get it to work with the first method.

Installing a github package without setup.py?

I am trying to install https://github.com/ExaVault/evapi-python.git using pip, but it doesn't have a setup.py so how am I supposed to use it? I want to use the packages it contains, but with no setup.py how am I supposed to use it?

Just an unfinished, broken package, it isn't supposed to be installed by Python tools. Install it manually — clone the repo and copy files to site-packages.
And send them a bug report. Or better yet a pull-request.
Upd. Oh, I see, you've sent a bug report alredy, nice!

Installing it looks like a manual job, but you can still use it. Download it as zip here https://github.com/ExaVault/evapi-python/archive/master.zip, extract, go to evapi-python-master folder, go to src, and simply place your code there.
import V1Api
import ApiClient
...

How to include examples or test programs in a package?

The Python Cookbook suggests the following tree structure for a "typical library package":
projectname/
README.txt
Doc/
documentation.txt
projectname/
__init__.py
foo.py
bar.py
utils/
__init__.py
spam.py
grok.py
examples/
helloworld.py
You 'll notice that the examples/ are not part of the actual package, which resides under projectname/projectname/ (that's where you 'll find the top-level __init__.py of the package).
Well, examples/helloworld.py obviously needs to import the projectname package.
I am aware that there are at least 2-3 relevant questions in StackOverflow. I do not believe that this is a duplicate because the other questions either involve intra-package imports or the general case of importing one python module from another when they do not reside in the same directory. I am specifically asking for the suggested way to do this when packaging a library.
Is there a way to achieve this without modifying the path? If modifying the path is the only way, is there a way for this to be done in an elegant manner?
Let me elaborate on that last point. In Repository Structure and Python by Kenneth Reitz, a similar structure appears, with tests/ instead of examples/. This is exactly the same problem. He suggests using "a simple (but explicit) path modification to resolve the package properly." OK, but this is the actual code:
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
I really don't like the .. part. I would hope for a more general solution, hopefully one that would work from whichever directory I would choose to run the example (or the test).

While the folder tree looks alright, I believe that implicitly assuming that modules are available for import is wrong from both a developer's perspective and certainly from a user's perspective.
It's true that you can technically use sys.path.insert(0, os.path.abspath('..')) to add any path for python to allow imports from, but that means you, the developer, have to make sure that the added path is always in the right location.
It is common for users to install packages to use them. There's a clear workflow for developers:
Have pip and virtualenv installed (and even better, virtualenvwrapper)
Install the package in editable mode using pip's -e flag which means that any changes you make to the code will directly effect execution. You won't have to reinstall everytime you make changes to your code.
Since your code is always installed (specifically, under site-packages as a user but in editable mode when you develop), you can always import your package using its explicit name from any example or test.
A common workflow:
$ pip install virtualenv
...
$ virtualenv distro
New python executable in /home/nir0s/work/distro/bin/python3
Also creating executable in /home/nir0s/work/distro/bin/python
Installing setuptools, pip, wheel...done.
$ source distro/bin/activate
# install in editable mode
$ pip install -e ~/repos/nir0s/distro/
Obtaining file:///home/nir0s/repos/nir0s/distro
Installing collected packages: distro
Running setup.py develop for distro
Successfully installed distro
(distro) $ pip freeze
appdirs==1.4.3
-e git+git#github.com:nir0s/distro#e8a182f9d1dbe6391f25...#egg=distro
packaging==16.8
pyparsing==2.2.0
six==1.10.0
The package, being installed in editable mode, means that there's an egg-link file pointing to the directory of the package:
$ cat distro/lib/python3.6/site-packages/distro.egg-link
/home/nir0s/repos/nir0s/distro
Users will do the same thing without dealing with editable mode. Simply creating virtual environments and install packages in them. Then, when they're done working, they'll delete those virtual environments. Easy peasy.

Python: Custom package installation not importing module

I'm having a problem with this package that I installed in Python 3.5. After installing it, I try to run requestProxy.py but it won't import any of its own packages. Here's what I did, and what's happening.
I cloned it and created a private repo using these instructions.
I installed in an activated virtualenv, created without using sudo, using:
pip3 install -e HTTP_Proxy_Randomizer
Terminal said it installed ok.
I can find the egg link in my virtualenv's site-packages folder, but when I try to run the main file, it says:
from project.http.requests.parsers.freeproxyParser import freeproxyParser
ImportError: No module named project.http.requests.parsers.freeproxyParser
I had to write a setup.py for the package, which didn't seem to come with its own. I came up with:
setup(name='HTTP_Request_Randomizer',
version='1.0',
description='HTTP Proxy Request Randomizer',
package_dir={'project': 'project','http':'project/http',\
'requests':'project/http/requests','errors':'project/http/requests/errors',\
'parsers':'project/http/requests/parsers','proxy':'project/http/requests/proxy'},
packages=['project','http','requests','errors','parsers','proxy']
Here's the package structure:
pip3 freeze
gives me:
Complete output from command git config --get-regexp remote\..*\.url:
fatal: bad config file line 4 in /home/danny/.gitconfig
----------------------------------------
Error when trying to get requirement for VCS system Command "git config --get-regexp remote\..*\.url" failed with error code 128 in /home/danny/Documents/HTTP_Request_Randomizer, falling back to uneditable format
Could not determine repository location of /home/danny/Documents/HTTP_Request_Randomizer
Django==1.9.7
## !! Could not determine repository location
HTTP-Request-Randomizer==1.0
mysqlclient==1.3.7
So I want to have requestProxy.py install the other necessary packages and not fail at line 1. I'm sure this is a problem with my implementation and not the original author's coding. I was experimenting with this package a couple of weeks ago before I was aware of virtualenvs or pip install -e, and just copied it manually to site-packages. It worked then. Now I understand the concepts to do it more cleanly, but I can't get those to work.
It feels as though I have done something wrong with my git config or with my package_dir structure in setup.py, perhaps?
I've been pythoning for maybe a month and have a lot to learn. I normally find what I need on Stack Overflow without having to bother anyone, but after trying everything with this, I really need some help. Any advice much appreciated.

I figured it out. I was using Ninja IDE, and even though I entered the virtualenv for the project and restarted, it still wasn't recognizing it. I was able to run it from the terminal, and also in Pycharm and Liclipse.

use a relative path in requirements.txt to install a tar.gz file with pip

We're using a requirements.txt file to store all the external modules needed. Every module but one is gathered from internet. The other one is stored on a folder under the one holding the requirements.txt file.
BTW, this module can be easily installed with pip install
I've tried using this:
file:folder/module
or this:
file:./folder/module
or even this:
folder/module
but always throws me an error.
Does anyone know which is the right way to do this?
Thanks

In the current version of pip (1.2.1) the way relative paths in a requirements file are interpreted is ambiguous and semi-broken. There is an open issue on the pip repository which explains the various problems and ambiguities in greater detail:
https://github.com/pypa/pip/issues/328
Long story short the current implementation does not match the description in the pip documentation, so as of this writing there is no consistent and reliable way to use relative paths in requirements.txt.
THAT SAID, placing the following in my requirements.txt:
./foo/bar/mymodule
works when there is a setup.py at the top level of the mymodule directory. Note the lack of the file:: protocol designation and the inclusion of the leading ./. This path is not relative to the requirements.txt file, but rather to the current working directory. Therefore it is necessary to navigate into the same directory as the requirements.txt and then run the command:
pip install -r requirements.txt

Its based off the current working directory (find with os.getcwd() if needed) and the relative path you provide in the requirements file.
Your requirements file should look like this:
fabric==1.13.1
./some_fir/some_package.whl
packaging==16.8
Note this will only work for .whl files not .exe
Remember to keep an eye on the pip install output for errors.

For me only the file: directive worked. This even works with AWS SAM, i.e. sam build. Here is my requirements.txt and the englishapps is my own custom Python package that I need in AWS Lambda
requests
file:englishapps-0.0.1-py3-none-any.whl

As was mentioned before, the files are relative to the current working directory, not the requirement.txt.
Since v10.0 requirements files support environment variables in the format: ${VAR_NAME}. This could be used as a mechanism to specify a file location relative to the requirements.txt. For example:
# Set REQUIREMENTS_DIRECTORY outside of pip
${REQUIREMENTS_DIRECTORY}/folder/module

Another option is to use the environment manager called Pipenv to manage this use case.
The steps after you do the pipenv install for a new project:
pipenv install -e app/deps/fastai (-e is editable, and is optional)
then you will see the following line in your Pipfile:
fastai = {editable = true,path = "./app/deps/fastai"}
here's similar issues:
https://github.com/pypa/pipenv/issues/209#issuecomment-337409290
https://stackoverflow.com/a/53507226/7032846

A solution that worked for me both for local and remote files (via Windows share). Here an example of requirements.txt
file:////REMOTE_SERVER/FOLDER/myfile.whl