Import Custom Module in Python [Although it works in PyCharm] - python

I see the question has already been asked thousands times, but I'm utterly confused.
I'm trying to run hello.py which import utils.common into hello.py
from utils.common import function
And I get the following error:
ModuleNotFoundError: No module named 'utils'
So my structure is the following:
── gig
├── __init__.py
├── src
│ ├── __init__.py
│ ├── hello
│ │ ├── __init__.py
│ │ └── hello.py
│ └── utils
│ ├── __init__.py
│ ├── common.py
Provided __init__.py in gig and in src as well, nothing changes though.
P.S. Imports work fine in PyCharm, the issue arises in docker container or when I try to run it locally from terminal.
Any pointers are very much appreciated.
Cheers,
Giga

usually your full import path: gig.src.utils.common should work.

Related

pytest no module named common

I'm trying to get started with python and pytest, I have the following project structure :
.
├── my_module
│ ├── __init__.py
│ ├── common
│ │ ├── __init__.py
│ │ └── utils.py
│ └── my_script.py
└── test
├── __init__.py
└── test_my_script.py
When I run tests (using pytest), I get an error:
no module named 'common'.
I have also the following all configs files:
tox.ini
setup.py
setup.cfg
pyproject.toml
someone know what I missed?
EDIT
here is how I import utils from test_my_script.py :
from common.util import func1,func2
common.util module is not accessible from your test execution because it is located in my_module package.
You should use import starting from working directory which is your project root.
Consider using following import instead:
from my_module.common.util import func1, func2

Accessing Folder Structure for Spark Submit zipped PyFile

I have a folder structure that looks like this:
├── repo
│ ├── src
│ └── main.py
│ ├── __init__.py
│ ├── utils
│ ├── __init__.py
│ └── helpers.py
│ └── predict
│ ├── __init__.py
│ └── predict.py
│
I am submitting a spark job with the --file src/predict/predict.py and a py-files src.zip (the spark submit command is built from main.py) of the entire source directory.
In predict, I want to reference a method in helpers.py. I do this by attempting an import: from src.utils.helpers import helper_func
However this does not work: ModuleNotFoundError: No module named 'src'. The imports work locally, but I am trying to understand how this works in the EMR. Ideally I dont want to have to change the importing from local to EMR. I just want them to be the same.

Why my test files from a test folder can't import the source files from a source folder?

I have a problem where this is my project structure:
.
├── Resources/
├── src/
│ ├── __init__.py
│ ├── main.py
│ ├── utils/
│ │ ├── __init__.py
│ │ ├── util.py
│ │ └── otherUtil.py
│ └── calculations/
│ ├── __init__.py
│ └── financials.py
└── tests/
├── __init__.py
└── test.py
My problem is that I can't reach the classes from the src/ folder from the tests, although the code in src/ can reach the Resources folder, through the first shown method.
I have tried:
To append the home library path this way:
Here I used the from src import util after these lines, I even tried from .src import util.
Then this way:
Here I used the from src import util after these lines, I even tried from .src import util.
Than without the sys.path.append() with no use.
I have tried every combination I know, but for no use, and I don't want to install them as individual packages. Does someone have an idea, witch will solve my problem?
Clarification edit:
I don't want to put the tests in the source folder, i want to keep them separate.
You can use this code found here:
# test.py
import sys
# insert at 1, 0 is the script path (or '' in REPL)
sys.path.insert(1, '/path/to/utils/')
import utils

Python unable to import nested module

I am currently writing a library in Python. I am trying to use a class that I have defined in a module but I am unable to use it in my main.py. Inside Selector and SeasonSelector I have 2 classes defined with the same name of the file. I get the following error:
No name 'Selector' in module 'Selectors'
main.py:
import pyf1
testSeasonSelector = pyf1.Selectors.SeasonSelector()
testSelector.loadData()
pyf1/__init__.py:
from Selectors.Selector import Selector
from Selectors.SeasonSelector import SeasonSelector
Directory
├── main.py
└── pyf1
├── Selector.pyc
├── Selectors
│ ├── SeasonSelector.py
│ ├── SeasonSelector.pyc
│ ├── Selector.py
│ ├── Selector.pyc
│ └── __init__.pyc
├── __init__.py
├── __init__.pyc
├── __pycache__
│ └── __init__.cpython-38.pyc
└── data
Your directory should look like this
├── main.py
└── pyf1
├── selector.pyc
├── selectors
| ├── __init__.py
│ ├── season_selector.py
│ ├── season_Selector.pyc
│ ├── selector.py
│ ├── selector.pyc
│ └── __init__.pyc
├── __init__.py
├── __init__.pyc
├── __pycache__
│ └── __init__.cpython-38.pyc
└── data
Your directory is not a certified package without init.py as such you cannot make import statements. I also took the liberty of correcting how you named your modules to fit good python naming convention\practices. With this you should be able to import from your built package without issues
I managed to find a solution.
The issue was I wasn't using the correct syntax. Once I added the missing __init__() files I used the import statement Import .parent_file from ParentClass Which then allowed me to pass functionality up the chain how I wanted.
Inside the Selectors directory I could then use from .selector import __Selector to use in SeasonSelector
Furthermore - I hadn't included the PyF1 in my sys.path, therefore python wasn't able to scan the directories. This included with the syntax above allowed me to do what I wanted.

Import self made package issues

Hey guys I am trying to import a self made package. I currently have the structure below
.
├── LICENSE
├── README.md
├── unifipy
│ ├── connect.py
│ ├── exceptions.py
│ ├── __init__.py
│ ├── __pycache__
│ └── tests.py
└── venv
all I am trying to do is import the package into my tests file
import unittest
import unifipy
However I keep having a module exception saying it can't be found
I have tried moving tests.py out of the directory and putting it one level up but it doesn't help
Any help would be appreciated

Categories

Resources