I'm developing a python package foo. My project structure looks like this:
.
├── foo
│ ├── foo
│ │ ├── bar.py
│ │ ├── foo.py
│ │ ├── __init__.py
│ ├── README.md
│ └── setup.py
├── footest
│ ├── test.py
test.py only has 1 line: import foo
In order for test.py to be able to import the package foo I install it with the command pip3 install -e foo.
Now a new folder called foo.egg-info is created under foo/
.
├── foo
│ ├── foo
│ │ ├── bar.py
│ │ ├── foo.py
│ │ ├── __init__.py
│ ├── foo.egg-info
│ │ ├── dependency_links.txt
│ │ ├── PKG-INFO.txt
│ │ ├── requires.txt
│ │ ├── SOURCES.txt
│ │ ├── top_level.txt
│ ├── README.md
│ └── setup.py
├── footest
│ ├── test.py
What's the purpose of this folder? I tried deleting it and test.py still ran properly. Is is just leftover garbage, similar to the .o files when compiling C projects? If so, is there a way to automatically remove it?
The package.egg-info saves meta data about your installed package like version.
It is used when you for example uninstall it, or "pip list" to see what is installed.
you can open it and take a look.
Related
Is there a way to create a requirements.txt file that only contains the modules that my script actually needs?
I usually just do a pip freeze and then remove unused modules.
You can use pipreqs to analyze your project files, and automatically generate a requirements.txt for you.
First step is to install pipreqs. You can do so, by executing the following command in your console:
pip install pipreqs
Then, the only thing you need to do is to execute the command pipreqs, specifying the path where your files are located at. For example, to generate a requirements.txt, based on the modules on the current working directory, you could execute:
pipreqs .
For some other directory:
pipreqs "PATH/TO/YOUR/PROJECT/PYTHON/FILES"
Real World Example
Here's the tree-view of the folder we're going to use as example:
.
├── data
│ └── EXCEL_FILES
│ ├── DIVISION_MAP.xlsx
│ ├── INPUT_CONTAINER.xlsx
│ ├── KITS.xlsx
│ ├── LOADING.xlsx
│ ├── MOQ_CBM_SHIPPING_TYPE.xlsx
│ ├── OUTBOUND_OTM.XLSX
│ ├── PLANT_SOURCE_MAP.xlsx
│ ├── PORT_STATE.xlsx
│ └── SALABLE_STORAGE_LOCATION.xlsx
├── loading
│ ├── __init__.py
│ ├── configs
│ │ ├── Initialization.py
│ │ ├── __init__.py
│ │ ├── config.ini
│ │ └── logconfig.py
│ ├── constants.py
│ ├── read_files.py
│ ├── reports.py
│ └── utils
│ ├── __init__.py
│ ├── date_utils.py
│ ├── file_utils.py
│ └── utils.py
├── logs
│ └── po_opt.log
├── main.py
└── outputs
├── 2022-10-03
├── 2022-10-04
├── 2022-10-05
├── 2022-10-06
├── 2022-11-23
├── 2022-11-24
└── 2022-11-28
Executing the command pipreqs . from the root path I get the following requirements.txt:
❯ pipreqs .
INFO: Successfully saved requirements file in ./requirements.txt
Output:
holidays==0.17.2
numpy==1.20.1
pandas==1.2.4
python_dateutil==2.8.2
six==1.16.0
I have several related projects that I think will be a good fit for Python's namespace-packages. I'm currently running python 3.8, and have created the following directory structure for testing.
├── namespace-package-test.package1
│ ├── LICENSE.txt
│ ├── README.md
│ ├── setup.cfg
│ ├── setup.py
│ ├── src
│ │ └── pkg1
│ │ ├── cli
│ │ │ ├── __init__.py
│ │ │ └── pkg1_cli.py
│ │ └── __init__.py
│ └── tests
├── namespace-package-test.package2
│ ├── AUTHORS.rst
│ ├── CHANGELOG.rst
│ ├── LICENSE.txt
│ ├── README.md
│ ├── setup.cfg
│ ├── setup.py
│ ├── src
│ │ └── pkg2
│ │ ├── cli
│ │ │ ├── __init__.py
│ │ │ └── pkg2_cli.py
│ │ └── __init__.py
│ └── tests
The entire project is on a private bitbucket (cloud) server at;
git#bitbucket.org:<my-company>/namespace-package-test.git
I would like to install, locally, only package 1. I've tried every iteration I can imagine of the following, but nothing seems to get me there. I either get a repository not found error or a setup.py not found error.
pip install git+ssh://git#bitbucket.org:<my-company>/namespace-package-test.package1.git
Is this possible?
Is my project structure correct for what I am doing?
What should the pip install command look like?
Bonus, what if I only want to install a specific spec using pipx?
pipx install "namespace-package-test.package1[cli] # git+ssh://git#bitbucket.org:<my-company>/namespace-package-test.package1.git"
I think I figured it out ... for posterity sake
Pip install (into virtual environment)
pip install git+ssh://git#bitbucket.org/<company name>/namespace-package-test.git/#subdirectory=namespace-package-test.package1
pipx install - with spec
pipx install "namespace-package-test.package1[cli] # git+ssh://git#bitbucket.org/<company name>/namespace-package-test.git/#subdirectory=namespace-package-test.package1"
My tests were working a few hours before. But suddenly tests located in root folder/tests are giving me ModuleNotFoundError: No module named 'tests.test_* I imported the tests.test_a in a top level python file and it worked. Any guesses why cant pytest load them?
I am asking this because this is weird that the test folder located inside root folder/some folder/tests also contains __init__.py and they work fine. I read this question and deleted the __init__.py and it worked. But it is confusing as to why the top level tests would not work.
The folder structure is along the lines of:
.
├── authenticate
│ ├── exceptions.py
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── exceptions.cpython-38.pyc
│ │ ├── __init__.cpython-38.pyc
│ │ ├── repository.cpython-38.pyc
│ │ └── use_case.cpython-38.pyc
│ ├── Readme.md
│ ├── repository.py
│ ├── tests
│ │ ├── __init__.py
│ │ ├── __pycache__
│ │ └── test_authenticate.py
│ └── use_case.py
├── tests
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── test_authenticate.cpython-38-pytest-5.4.2.pyc
│ ├── test_authenticate.py
As weird as it sounds, I had forgotten to add __init__.py in one of my packages. I don't know why it was causing tests in my top level directory to fail
ImportError while loading conftest '/home/rohit/flask/src/tests/conftest.py'.
conftest.py:6: in <module>
from mycode import create_app
E ModuleNotFoundError: No module named 'mycode'
when running pytest form the src folder above mycode folder, it gives ModuleNotFoundError
The directory structure is as follow
├── mycode
│ ├── auth
│ ├── auth.py
│ ├── db_insert.py
│ ├── db.py
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── auth.cpython-37.pyc
│ │ ├── db.cpython-37.pyc
│ │ ├── db_insert.cpython-37.pyc
│ │ └── __init__.cpython-37.pyc
│ ├── README.md
│ ├── schema.sql
│ └── templates
│ ├── auth
│ │ ├── login.html
│ │ └── register.html
│ └── base.html
└── tests
├── conftest.py
├── data.sql
├── __pycache__
│ └── conftest.cpython-37-pytest-5.4.1.pyc
└── test_factory.py
Your __init__.py file needs to go in the folder named mycode. If the above is already true then try the following.
Rename __init__.py to mycode.py
You can see here for more solutions that may solve your problem.
I am writing a custom Django module but I seem to have something wrong. I cannot import a class that lives in a certain file. I get the error
ValueError: Unable to configure handler 'admins': Cannot resolve 'myPackage.handlers.MyHandlerClass': No module named handlers
This is the directory structure. I believe I can import views and models with no problem.
myPackage
├── CHANGELOG.rst
├── myPackage
│ ├── handlers .py
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── migrations
│ │ ├── 0001_initial.py
│ │ ├── 0001_initial.pyc
│ │ ├── __init__.py
│ │ └── __init__.pyc
│ ├── models.py
│ ├── models.pyc
│ ├── tests.py
│ ├── views.py
│ └── views.pyc
├── myPackage.egg-info
│ ├── dependency_links.txt
│ ├── PKG-INFO
│ ├── requires.txt
│ ├── SOURCES.txt
│ └── top_level.txt
├── MANIFEST.in
├── README.rst
├── requirements.txt
└── setup.py
There is a space in the filename of handlers .py, so python can't find a module names handlers. Obviously the easiest fix is to correct the filename, but for anyone actually wanting a space in the filename, import name with spaces is a syntax error, so the only way to import such a name is using __import__. But this is really a very bad idea.