Develop Reference

Importing module from Github to use in Python

I'm a beginner in Python and I have no experience with GitHub at all. I want to import the module semsimlib from the following URL: https://github.com/timvdc/semsimlib
I have looked on the internet for help on how to do this but most of it is very unclear and doesn't seem to work for me. Can anyone provide a detailed explanation on how to do this in a easy way?

It looks the repo does not provide appropriate scripts to simply install the package. There is no setup.py file and there is no distribution on pypi.
What you can do is go to site-packages folder inside your python installation or inside your virtual environment. Then run git clone https://github.com/timvdc/semsimlib. You should now be able to import semsimlib. Keep in mind that you will also have to install all the other dependencies your self one by one since there is also no requirements file.
You can also clone the repo into any folder on your computer and at the top of your script put:
import sys
sys.path.append("path/to/semsimlib/folder")
semsimlib will now be importable. However, I would try to get it to work with the first method.

How to include examples or test programs in a package?

The Python Cookbook suggests the following tree structure for a "typical library package":
projectname/
README.txt
Doc/
documentation.txt
projectname/
__init__.py
foo.py
bar.py
utils/
__init__.py
spam.py
grok.py
examples/
helloworld.py
You 'll notice that the examples/ are not part of the actual package, which resides under projectname/projectname/ (that's where you 'll find the top-level __init__.py of the package).
Well, examples/helloworld.py obviously needs to import the projectname package.
I am aware that there are at least 2-3 relevant questions in StackOverflow. I do not believe that this is a duplicate because the other questions either involve intra-package imports or the general case of importing one python module from another when they do not reside in the same directory. I am specifically asking for the suggested way to do this when packaging a library.
Is there a way to achieve this without modifying the path? If modifying the path is the only way, is there a way for this to be done in an elegant manner?
Let me elaborate on that last point. In Repository Structure and Python by Kenneth Reitz, a similar structure appears, with tests/ instead of examples/. This is exactly the same problem. He suggests using "a simple (but explicit) path modification to resolve the package properly." OK, but this is the actual code:
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
I really don't like the .. part. I would hope for a more general solution, hopefully one that would work from whichever directory I would choose to run the example (or the test).

While the folder tree looks alright, I believe that implicitly assuming that modules are available for import is wrong from both a developer's perspective and certainly from a user's perspective.
It's true that you can technically use sys.path.insert(0, os.path.abspath('..')) to add any path for python to allow imports from, but that means you, the developer, have to make sure that the added path is always in the right location.
It is common for users to install packages to use them. There's a clear workflow for developers:
Have pip and virtualenv installed (and even better, virtualenvwrapper)
Install the package in editable mode using pip's -e flag which means that any changes you make to the code will directly effect execution. You won't have to reinstall everytime you make changes to your code.
Since your code is always installed (specifically, under site-packages as a user but in editable mode when you develop), you can always import your package using its explicit name from any example or test.
A common workflow:
$ pip install virtualenv
...
$ virtualenv distro
New python executable in /home/nir0s/work/distro/bin/python3
Also creating executable in /home/nir0s/work/distro/bin/python
Installing setuptools, pip, wheel...done.
$ source distro/bin/activate
# install in editable mode
$ pip install -e ~/repos/nir0s/distro/
Obtaining file:///home/nir0s/repos/nir0s/distro
Installing collected packages: distro
Running setup.py develop for distro
Successfully installed distro
(distro) $ pip freeze
appdirs==1.4.3
-e git+git#github.com:nir0s/distro#e8a182f9d1dbe6391f25...#egg=distro
packaging==16.8
pyparsing==2.2.0
six==1.10.0
The package, being installed in editable mode, means that there's an egg-link file pointing to the directory of the package:
$ cat distro/lib/python3.6/site-packages/distro.egg-link
/home/nir0s/repos/nir0s/distro
Users will do the same thing without dealing with editable mode. Simply creating virtual environments and install packages in them. Then, when they're done working, they'll delete those virtual environments. Easy peasy.

Python: Custom package installation not importing module

I'm having a problem with this package that I installed in Python 3.5. After installing it, I try to run requestProxy.py but it won't import any of its own packages. Here's what I did, and what's happening.
I cloned it and created a private repo using these instructions.
I installed in an activated virtualenv, created without using sudo, using:
pip3 install -e HTTP_Proxy_Randomizer
Terminal said it installed ok.
I can find the egg link in my virtualenv's site-packages folder, but when I try to run the main file, it says:
from project.http.requests.parsers.freeproxyParser import freeproxyParser
ImportError: No module named project.http.requests.parsers.freeproxyParser
I had to write a setup.py for the package, which didn't seem to come with its own. I came up with:
setup(name='HTTP_Request_Randomizer',
version='1.0',
description='HTTP Proxy Request Randomizer',
package_dir={'project': 'project','http':'project/http',\
'requests':'project/http/requests','errors':'project/http/requests/errors',\
'parsers':'project/http/requests/parsers','proxy':'project/http/requests/proxy'},
packages=['project','http','requests','errors','parsers','proxy']
Here's the package structure:
pip3 freeze
gives me:
Complete output from command git config --get-regexp remote\..*\.url:
fatal: bad config file line 4 in /home/danny/.gitconfig
----------------------------------------
Error when trying to get requirement for VCS system Command "git config --get-regexp remote\..*\.url" failed with error code 128 in /home/danny/Documents/HTTP_Request_Randomizer, falling back to uneditable format
Could not determine repository location of /home/danny/Documents/HTTP_Request_Randomizer
Django==1.9.7
## !! Could not determine repository location
HTTP-Request-Randomizer==1.0
mysqlclient==1.3.7
So I want to have requestProxy.py install the other necessary packages and not fail at line 1. I'm sure this is a problem with my implementation and not the original author's coding. I was experimenting with this package a couple of weeks ago before I was aware of virtualenvs or pip install -e, and just copied it manually to site-packages. It worked then. Now I understand the concepts to do it more cleanly, but I can't get those to work.
It feels as though I have done something wrong with my git config or with my package_dir structure in setup.py, perhaps?
I've been pythoning for maybe a month and have a lot to learn. I normally find what I need on Stack Overflow without having to bother anyone, but after trying everything with this, I really need some help. Any advice much appreciated.

I figured it out. I was using Ninja IDE, and even though I entered the virtualenv for the project and restarted, it still wasn't recognizing it. I was able to run it from the terminal, and also in Pycharm and Liclipse.

packaging python application for linux

I have made a GUI application using python and PyQt5. I want to package this app but there doesn't seems to be a straight forward way to do this. Moreover what I have found answers to is to package a python module and not an application. I have read various articles and the official docs but still don't seem to have a proper answer to this, though there are several workarounds through which I could achieve the same, I just want to know what is the standard way.
This is my directory structure :
Moodly/
Moodly/
__init__.py
controller.py
logic.py
models.py
view.py
resoure.py
style.py
sounds/
notify.wav
message.wav
setup.py
MANIFEST.in
setup.cfg
run.py
moodly.png
Moodly.desktop
What do I want to achieve: The user is given with a tar file of Moodly. The user extracts it, runs the command
python setup.py install
in the terminal, the setup places all the files in the proper place and creates a Moodly.desktop file probably in usr/local/share/applications clicking on which user can run the app.
My way of achieving this:
setup.py
from setuptools import setup
setup(
name="Moodly",
version="1.0",
author="Akshay Agarwal",
author_email="agarwal.akshay.akshay8#gmail.com",
packages=["Moodly"],
include_package_data=True ,
url="http://github.com/AkshayAgarwal007/Moodly",
entry_points = {
'gui_scripts': [
'moodly = Moodly.controller:main',
],
},
# license="LICENSE.txt",
description="Student Intimation system",
# long_description=open("README.txt").read(),
# Dependent packages (distributions)
)
MANIFEST.in
include Moodly/sounds/notify.wav
include Moodly/sounds/message.wav
Now with no setup.cfg I run the command:
python setup.py install
This succesfully installs Moodly to /usr/lib/python-3.4/site-packages
alongwith the sounds directory.And now from the terminal when I type in moodly(as specified in entry points in setup.py) my GUI application launches successfully.
Now I just need the setup to create the Moodly.desktop alongwith moodly.png in usr/local/share/applications which I am trying to achieve through this:
setup.cfg
[install]
install_data=/usr/local/share/applications
Adding this to setup.py
data_files = [("Moodly", ["moodly.png","Moodly.desktop",])],
But this somehow seems to copy the files inside python-3.4/site-packages/Moodly rather than the specified destination but it used to work well with distutils
This guy also seems to have faced the same issue
Some other links I have used:
python-packaging
starting with distutils
So the way I am trying to do it , how much of it is correct and what is the standard way to do it. How can I possibly place that Moodly.desktop in the right place or what could be a better alternative way to do the entire process.
Moreover would using Pyinstaller be a better idea. Pyinstaller would package the app with PyQt5, requests and beautifulsoup4 (external modules that I have used) which I don't want. I want to use the install_requires option provided by setuptools and not unnecessary make the user download the modules which they already might have.

The .desktop file isn't supposed to be installed using Distutils. Distutils is only concerned with installing Python packages.
To install .desktop files, icons and other files incidental to distribution level packaging, you should look at build automation systems, such as CMake.
The first step in this process is to get CMake to build a Python project. You should take a look here for how to do that: https://bloerg.net/2012/11/10/cmake-and-distutils.html
Beyond that, installing .desktop files is easy. Assuming you've written a .desktop file and put it somewhere, installing it is a matter of doing:
install(PROGRAMS com.akshay.moodly.desktop DESTINATION ${XDG_APPS_INSTALL_DIR})
in your CMakeLists.txt file.
Note that you install the .desktop file to ${XDG_APPS_INSTALL_DIR} (that's a CMake variable), not a hardcoded path like /usr/local/share/applications or something. The user (and pretty much every automated distro package builder) will always install your package to a temporary path and then copy files over into their packages. Never assume that your app will live in /usr/bin or /usr/local/bin or whatever. The user could install things into /opt/Moodly or even $HOME/Moodly.

Unable to install python-setuptools: ./configure: No such file or directory

The question is related to the answer to "Unable to install Python without sudo access".
I need to install python-setuptools to install python modules.
I have extracted the installation package.
I get the following error when configuring
[~/wepapps/pythonModules/setuptools-0.6c9]# ./configure --prefix=/home/masi/.local
-bash: ./configure: No such file or directory
I did not find the solution at the program's homepage.
How can I resolve this error?

As Noah states, setuptools isn't an automake package so doesn't use ‘./configure’. Instead it's a pure-Python-style ‘setup.py’ (distutils) script.
You shouldn't normally need to play with .pydistutils.cfg, as long as you run it with the right version of Python. So if you haven't added the .local/bin folder to PATH, you'd have to say explicitly:
/home/masi/.local/bin/python setup.py install
AIUI this should Just Work.
I did not find the solution at the program's homepage.
Yeah, they want you to install it from a shell script egg which uses the default version of Python. Which you don't want.
(Another approach if you can't get setuptools to work is to skip it and install each module and dependency manually. Personally I have a bit of an aversion to setuptools/egg, as it contains far too much “clever” magic for my tastes and makes a mess of my filesystem. But I'm an old curmudgeon like that. Most Python modules can be obtained as simple Python files or plain old distutils scripts, but unfortunately there are some that demand eggs.)

You might want to check http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-locations.
EasyInstall is a python module with some shell scripts (or some shell scripts with a python module?) and does not use the unix make tool that gets configured with the "./configure" command. It looks like your best bet is to try editing ~/.pydistutils.cfg to include:
[install]
install_lib = /home/masi/.local/lib/python/site-packages/
install_scripts = /home/masi/.local/bin
You'll also presumably have made the ~/.local/bin/ folder part of your PATH so you can run the easy_install script. (I'm not sure exactly where the site-packages directory will be under .local, but it shouldn't be hard to find.)
Hope this helps.