I'm having some import issues using a Python module I cloned from GitHub.
My project structure looks like such
-Jupyter_notebook.ipynb
-gsdmm/
The gsdmm/ folder was cloned into the same directory as the notebook from GitHub here using
git clone https://github.com/rwalk/gsdmm.git
Now my import error happens anytime I try to import the 'MovieGroupProcess' class from the module as shown in the readme. I'm running it from a Jupyter notebook but same issue exists in Python script
from gsdmm import MovieGroupProcess
where I get the following error:
ImportError: cannot import name 'MovieGroupProcess' from 'gsdmm' ([current_directory]\gsdmm\__init__.py)
I'm not sure if something was messed up with my paths or needs to be changed.
If this is your project structure
.
├── gsdmm
│ ├── gsdmm
│ │ ├── __init__.py
│ │ └── mgp.py
│ └── test
└── Jupyter_notebook.ipynb
Then trying to import gssdmm from your notebook Jupyter_notebook won't work as there is not __init__.py file inside the first gsdmm directory.
But if you change the project structure to
.
└── gsdmm
├── gsdmm
│ ├── __init__.py
│ └── mgp.py
├── test
└── Jupyter_notebook.ipynb
Now when you import gsdmm python can find the __init__.py file and can import the module successfully.
Related
I am creating a Python package and want to make the example files in the example folder runnable with something like python example_file.py. How do I do it? My example_file.py and __init__.py look like below:
# example_file.py
from demandforecast import DemandForecast # module coded in demandforecast.py
if __name__ == '__main__':
# Example code here
# __init__.py
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
When I navigate to the examples directory and run python example_file.py I get the following error:
Exception has occurred: ModuleNotFoundError
No module named 'demandforecast'
This can be solved by adding using the old sys.path.insert() trick mentioned in Importing files from different folder, but I would rather not include the absolute path because this will vary from user to user.
Here is my directory structure:
├───demandforecast
│ demandforecast.py
│ __init__.py
│
└───examples
example_file.py
I have tried a few posts here, such as Relative imports in Python 3, but without luck. Adding sys.path.append('../') to the example file before the demandforecast import also did not work.
I think your import is simply incorrect, and should be:
from demandforecast.demandforecast import DemandForecast
Since demandforecast is both the package name (the directory is definitely a package, with the __init__.py file) and a module name within that package. (I assume DemandForecast is a class within the demandforecast module.)
from src.pro.demandforecast.demandforecast import foo
if __name__ == '__main__':
foo()
output:
foo
create a folder src and inside of it create a package pro and now inside pro create two packages demandforecast and examples.
Directory structure:
/src$ tree
.
└── pro
├── demandforecast
│ ├── demandforecast.py
│ ├── __init__.py
│ └── __pycache__
│ ├── demandforecast.cpython-38.pyc
│ └── __init__.cpython-38.pyc
├── examples
│ ├── example_file.py
│ └── __init__.py
├── __init__.py
└── __pycache__
└── __init__.cpython-38.pyc
IDE:
I am trying to better understand importing modules. I read about how to do this from here https://stackoverflow.com/a/14132912/14179793 and I can get it to work using solution 1. There is an additional variable that I need to figure out though.
This is a dummy project I am testing with:
.
├── a_package
│ ├── __init__.py
│ └── lib_a.py
├── b_package
│ ├── __init__.py
│ └── test_script.py
├── main.py
└── src
└── src_lib
└── src_lib.py
With this setup I can do:
python -m b_package.test_script
this is lib a function
This is src_lib_function.
test_script.py:
from a_package.lib_a import lib_a_function
from src.src_lib.src_lib import src_lib_function
if __name__ == '__main__':
lib_a_function()
src_lib_function()
pass
The goal is to make b_package/test_script.py executable without using python test_script ie ./test_script
However, adding the shebang at the top #!/usr/bin/env python causes an import error:
$ ./b_package/test_script.py
Traceback (most recent call last):
File "./b_package/test_script.py", line 2, in <module>
from a_package.lib_a import lib_a_function
ModuleNotFoundError: No module named 'a_package'
I assume it is because python is not loading it as a module based off the above mentioned question but I am not sure how to resolve this.
I ended up using setuptools as suggested by kosciej16 to achieve the desired results.
New project structure:
.
├── a_package
│ ├── __init__.py
│ └── lib_a.py
├── b_package
│ ├── __init__.py
│ └── test_script.py
├── main.py
├── pyproject.toml
├── setup.cfg
└── src
├── __init__.py
└── src_lib
├── __init__.py
└── src_lib.py
setup.cfg:
[metadata]
name = cli_test
version = 0.0.1
[options]
packages = find:
[options.entry_points]
console_scripts =
test_script = b_package.test_script:main
This allows the user to clone the repo and run pip install . from the top level then they can execute the script by just typing test_script
I have a python project and want to write a command line launcher. My project works fine when I run tests or launch from my IDE (pycharm). However when I try to launch via the command line I get a ModuleNotFoundError: No module named 'python' error.
Here is my directory structure:
.
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── create_jenkins_jobs.sh
└── python
├── __init__.py
├── create_credentials.py
├── create_jenkins_jobs_cli.py
├── credential_builder.py
├── jenkins_multibranch_job_builder.py
├── templates
│ ├── job.xml
│ ├── multibranch_job_template.xml
│ ├── ssh_cred_template.xml
│ └── user_pw_cred_template.xml
└── tests
├── __init__.py
├── integration
│ ├── __init__.py
│ ├── build_credential_test.py
│ ├── build_job_test.py
│ ├── conftest.py
│ ├── test_id_rsa
│ └── test_id_rsa.pub
└── unit
The shell script create_jenkins_jobs.sh contains:
#!/bin/bash
python3 python/create_jenkins_jobs_cli.py "${#}"
the file create_jenkins_jobs_cli.py contains the following code:
from typing import Set
from shared.common import preflight_env_check
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
def configure_jenkins_jobs(git_repos: Set[str]):
jenkins_job_builder = JenkinsMultibranchJobBuilder(jenkins_server_url=JENKINS_BASE_SERVER_URL,
jenkins_folder=JENKINS_ROOT_JOB_FOLDER,
user=JENKINS_USER, token=JENKINS_TOKEN,
git_cred_id=GIT_SSH_CRED_ID,
git_repos=git_repos)
jenkins_job_builder.configure_jobs()
configure_jenkins_jobs(GIT_REPOS)
When I launch create_jenkins_jobs.sh or create_jenkins_jobs_cli.py I always get
ModuleNotFoundError: No module named 'python'
$>./create_jenkins_jobs.sh
/Users/pswenson/dev/cc-cicd-automation/.venv/bin/python3: Error while finding module specification for 'python/create_jenkins_jobs_cli.py' (ModuleNotFoundError: No module named 'python/create_jenkins_jobs_cli')
I've tried all sorts of different techniques to get this to work with PYTHONPATH, working directories, etc...
It seems there is something basic about how python modules work that I don't understand.
Does changing the line:
from python.jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
to
from jenkins_multibranch_job_builder import JenkinsMultibranchJobBuilder
Sort you?
When I call python __main__.py
I receive error:
from facebook.fb_api import FacebookApi
ModuleNotFoundError: No module named 'facebook'
# __main__.py
from facebook.fb_api import FacebookApi
if __name__ == "__main__":
api = FacebookApi()
api.start()
Project structure
facebook/
├── cache.py
├── configs.py
├── fb_api.py
├── __init__.py
├── __main__.py
├── parser
│ ├── cfg.py
│ ├── example.json
│ ├── __init__.py
│ ├── models.py
│ ├── run_parser.py
│ └── utils.py
├── __pycache__
│ └── fb_api.cpython-36.pyc
├── request_handler.py
└── services
├── case_service.py
└── __init__.py
Your fb_api.py is in the same directory as __main__.py so no facebook folder exists for your application. Remove it from your import and it should work
Because __main__.py is in the same directory. So just import fb_api. To use Facebook.fb_api then move main.py to Facebook directory.
You need to run your code from a higher level in your directory structure. Rather than running from within the facebook folder, run from its parent folder with python -m facebook. The -m flag tells Python you're running module by its Python name, which in this case is the facebook package. A package being run that way has its __main__.py file run as the main script, which is exactly what you want.
Try local import
# __main__.py
from .fb_api import FacebookApi
if __name__ == "__main__":
api = FacebookApi()
api.start()
The structure of my_dir is
├── README.md
├── main
│ ├── functions
│ │ ├── __pycache__
│ │ ├── my_function.py
│ ├── pipeline.py
│ ├── options
│ │ └── pipeline_options.py
│ └── transforms
│ ├── __pycache__
│ ├── my_transform.py
├── poetry.lock
├── pyproject.toml
└── tests
On pipeline.py:
from main.functions.my_function import MyFunction
On my_function.py:
import apache_beam as beam
class MyFunction(beam.DoFn):
...
I have read similar questions here, including this one which is the solution I have currently. Also read this on Python imports.
When I run pipeline.py, I get
Traceback (most recent call last):
File "pipeline.py", line 7, in <module>
from main.functions.my_function import MyFunction
ModuleNotFoundError: No module named 'main'
Additionally, I'm using VSCode and if I run the file via "Python: Run Python File in Terminal" I get back the error. However, if I run via the debugger the paths are all found and the error is not thrown, which I found odd. Also, VSCode isn't throwing any path warnings.
I know there are similar questions but I haven't been able to figure out what is wrong here and have spent quite some time on this already. Any help/pointer is much appreciated.
Additional info
I'm using python 3.8.1
Using poetry run python main/pipeline.py to run the code
Running from command from my_dir