I have several related projects that I think will be a good fit for Python's namespace-packages. I'm currently running python 3.8, and have created the following directory structure for testing.
├── namespace-package-test.package1
│ ├── LICENSE.txt
│ ├── README.md
│ ├── setup.cfg
│ ├── setup.py
│ ├── src
│ │ └── pkg1
│ │ ├── cli
│ │ │ ├── __init__.py
│ │ │ └── pkg1_cli.py
│ │ └── __init__.py
│ └── tests
├── namespace-package-test.package2
│ ├── AUTHORS.rst
│ ├── CHANGELOG.rst
│ ├── LICENSE.txt
│ ├── README.md
│ ├── setup.cfg
│ ├── setup.py
│ ├── src
│ │ └── pkg2
│ │ ├── cli
│ │ │ ├── __init__.py
│ │ │ └── pkg2_cli.py
│ │ └── __init__.py
│ └── tests
The entire project is on a private bitbucket (cloud) server at;
git#bitbucket.org:<my-company>/namespace-package-test.git
I would like to install, locally, only package 1. I've tried every iteration I can imagine of the following, but nothing seems to get me there. I either get a repository not found error or a setup.py not found error.
pip install git+ssh://git#bitbucket.org:<my-company>/namespace-package-test.package1.git
Is this possible?
Is my project structure correct for what I am doing?
What should the pip install command look like?
Bonus, what if I only want to install a specific spec using pipx?
pipx install "namespace-package-test.package1[cli] # git+ssh://git#bitbucket.org:<my-company>/namespace-package-test.package1.git"
I think I figured it out ... for posterity sake
Pip install (into virtual environment)
pip install git+ssh://git#bitbucket.org/<company name>/namespace-package-test.git/#subdirectory=namespace-package-test.package1
pipx install - with spec
pipx install "namespace-package-test.package1[cli] # git+ssh://git#bitbucket.org/<company name>/namespace-package-test.git/#subdirectory=namespace-package-test.package1"
So i'm trying to use this github repository, i put it on my site-packages folder and tested this example, but i got the error cannot import name 'market_candles'. What could be causing this problem? I already made sure that TA-Lib, Pandas and Matplotlib are installed, so where could be the problem? I'm looking at the __init__py and it seems fine.
Cloning the repository directly into your sites-packages will result in nested pyttrex folders. Copy only the pyttrex/pyttrex directory into sites-packages
pyttrex
├── LICENSE
├── README.md
├── pyttrex
│ ├── ADX.py
│ ├── __init__.py
│ ├── average_n
│ ├── average_true_range.py
│ ├── backtest.py
│ └── test.py
└── tgnotifier.py
I have a structure similar to this:
/home/user/nginx_http/
├── nginx_http
│ ├── common
│ │ ├── autodiscovery.py
│ │ ├── __init__.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── nginx_http.py
├── README.rst
└── setup.py
When I'm in the /home/user/nginx_http folder and I run this:
python -m nginx_http
it works fine. But soon as I try to run it with full path it gives me an error;
python -m /home/user/nginx_http/nginx_http
/bin/python: Import by filename is not supported.; '/home/user/nginx_http/nginx_http' is a package and cannot be directly executed
I want to run my python program as a cronjob which would require the full path. The two options I wanted clarification on:
Should I consider moving my python program into python's module folder?
/usr/lib/python2.7/site-packages
Or should I set PYTHONPATH env to the location of /home/user/nginx_http. Will that break anything as far as other modules are concerned?
Thanks.
I have a multiplatform project in which I required to ship few third party executable/ data files which are not part of python. In the source file I kept it under data directory and from main script calling the executable using this line
trd_prt_exe = os.path.join("tools", "syslinux", "bin", "executable_name")
It works perfectly fine while testing/ developing from source. Problem comes when I distribute the same using setup.py. After installing application using setup.py, I get this error
for path, subdirs, files in os.walk(os.path.join("tools"))
File "/usr/lib/python2.7/os.py", line 276, in walk
names = listdir(top)
TypeError: coercing to Unicode: need string or buffer, NoneType found
Clearly, python could not find my executables under data directory.
How can we access these executables/ data files during development and after distributing it.
Update I
I could have included but simply forgot. Here is my complete project strecture:-
[sundar#arch multibootusb-7.0.0]$ tree
.
├── data
│ ├── multibootusb.desktop
│ └── multibootusb.png
├── LICENSE.txt
├── multibootusb
├── PKG-INFO
├── README.txt
├── scripts
│ ├── admin.py
│ ├── detect_iso.py
│ ├── __init__.py
│ ├── install_distro.py
│ ├── install_syslinux.py
│ ├── isodump.py
│ ├── multibootusb_ui.py
│ ├── qemu.py
│ ├── uninstall_distro.py
│ ├── update_cfg.py
│ └── var.py
├── setup.py
└── tools
├── checking.gif
├── mbr.bin
├── multibootusb
│ ├── chain.c32
│ ├── extlinux.cfg
│ ├── grub.exe
│ ├── memdisk
│ ├── menu.c32
│ ├── menu.lst
│ ├── syslinux.cfg
│ └── vesamenu.c32
├── multibootusb.png
├── syslinux
│ └── bin
│ ├── syslinux3
│ ├── syslinux4
│ ├── syslinux5
│ └── syslinux6
└── version.txt
Here is what I have in setup.py:-
from distutils.core import setup
import os
mbusb_version = open(os.path.join("tools", "version.txt"), 'r').read().strip()
setup(
name='multibootusb',
version=mbusb_version,
packages=['scripts'],
scripts = ['multibootusb'],
platforms = ['Linux'],
url='http://multibootusb.org/',
license='General Public License (GPL)',
author='Sundar',
author_email='feedback.multibootusb#gmail.com',
description='Create multi boot Live linux on a USB disk...',
long_description = 'The multibootusb is an advanced cross-platform application for installing/uninstalling Linux operating systems on to USB flash drives.',
data_files = [("/usr/share/applications",["data/multibootusb.desktop"]),
('/usr/share/pixmaps',["data/multibootusb.png"]),
('multibootusb/tools',["tools/checking.gif"]),
('multibootusb/tools',["tools/mbr.bin"]),
('multibootusb/tools',["tools/version.txt"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/chain.c32"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/extlinux.cfg"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/grub.exe"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/memdisk"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/menu.c32"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/menu.lst"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/syslinux.cfg"]),
('multibootusb/tools/multibootusb',["tools/multibootusb/vesamenu.c32"]),
('multibootusb/tools/syslinux/bin',["tools/syslinux/bin/syslinux3"]),
('multibootusb/tools/syslinux/bin',["tools/syslinux/bin/syslinux4"]),
('multibootusb/tools/syslinux/bin',["tools/syslinux/bin/syslinux5"]),
('multibootusb/tools/syslinux/bin',["tools/syslinux/bin/syslinux6"])]
#('multibootusb/tools',["tools/multibootusb.png"])]
)
The problem what I found is that the main executable script "multibootusb" is available in usr/bin/multibootusb but other data/ third party executables are under /usr/multibootusb/ and other modules/s scripts required by main program multibootusb is under /usr/lib/python2.7/site-packages/scripts. Therefore, the main program is unable to locate my third party data/ executables.
How to overcome this issue? where am I doing wrong?
i have the following file structure:
ihe/
├── dcmt
│ ├── actions
│ ├── calendar_observer
│ ├── cms
│ ├── consumption
│ ├── data_mining
│ ├── dcmt
│ ├── dcmt_db
│ ├── dcmt_db.bak.bak
│ ├── dcmt_db.sqlite
│ ├── devices
│ ├── d.py
│ ├── gadgets
│ ├── history
│ ├── houses
│ ├── hwc_settings
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── manage.py
│ ├── notifications
│ ├── profitable
│ ├── rules
│ └── schedule
├── hwc
│ ├── configuration
│ ├── daemons
│ ├── database
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── utils
│ └── wrapper
├── __init__.py
├── __init__.pyc
dcmt is a django project. hwc is pure python. however for instance in hwc/daemons there is a runme.py script. in that runme.py script i want to be able to import the models from the django project. Now as i understand it i have to have the correct python path and then somehow set the django settings. My question is how do i best do this so that for the whole hwc modules I only have to do that once?
Your project structure seems a bit confused.
It's probably not a good idea to have a Django project inside another package hierarchy. A lot of the import paths assume your project is in a top-level package and the only reason you're probably not running into issues already is that Python 2.x still supports relative imports (which have been removed in 3.x). This makes references to packages very ambiguous and can cause weird bugs.
From what I can see your settings package is actually called (fully-qualified) ihe.dcmt.hwc_settings. If ihe is in your Python path (check the value of sys.path in the script you're trying to run), that (i.e. the fully-qualified path) is probably what DJANGO_SETTINGS_MODULE should point at.
If you want to hook into Django's functionality in your scripts, you might want to look into the documentation for writing manage.py commands. This would let you write Django-related scripts more consistently and save you the worry about referencing and initialising Django's settings correctly yourself.