Develop Reference

How to fix Pylint's false positive Unable to import error?

For last couple of hours I was trying to figure out what's the Pythonic way of importing modules from parent directory and from sub-directories. I made a project just for testing purposes and hosted it on github, so you could take a look at it to better understand my question.
In the project we have the following files structure:
├── __init__.py
├── main.py
└── project
├── a.py
├── __init__.py
└── subdirectory
├── b.py
└── __init__.py
I'm trying to figure out how to import modules from subdirectories and from the parent directories.
If I try to import the modules ./project/subdirectories/b.pyand ./project/a.py into the main.py module without specifying the root directory's name in the import statement then pylint starts to complain that it's unable to locate the modules, but the program runs fine:
If I do specify the root directory in the import statement then the pylint stops complaining, but the program doesn't run anymore:
Can someone, please, explain to me why do I have those false positive from pylint when the program does work and if I make the pylint happy, by specifying the root directory in the import statement, then the program stops working?

Your IDE (and thus pylint) expect the code to be launched from the root of your git repository not from the directory above it. There is no test directory in your git project structure. So I think the import from test.project import a is working only because your git repository itself is named test and because you launch the program from above the project directory.
Ie your real structure is this:
test
├── .git # root of git repository inside test
├── __init__.py
├── main.py
└── project
├── a.py
├── __init__.py
└── subdirectory
├── b.py
└── __init__.py
But the test directory is not inside git and for the IDE the git root as the source root. If you start launching __main__ from the same directory you're launching pylint from, then pylint indication will be accurate. So the fix is to create the test directory inside your git project:
.git # root of git repository outside test/
test
├── __init__.py
├── main.py
└── project
├── a.py
├── __init__.py
└── subdirectory
├── b.py
└── __init__.py
You could also launch pylint from the same directory you launch the code by changing the root directory of your IDE.

pyImporterror running subfolder python scripts from the parent folder

I am trying to run from the directory folder:
$ python subdirectoryTwo/file.py command (Python 2.7).
Folders structure:
-directory
-subdirectoryOne
__init.py__
config.py
-subdirectoryTwo
__init.py__
file.py
My file.py has:
from subdirectoryOne.config import config
However I am getting an error:
file.pyImportError: No module named subdirectoryOne.config`
(I guess it still looks in the directory folder)

There few thins you need to change.
(public)landpacks-MacBook-Pro:qx frank$ tree
.
├── __init__.py
├── __init__.pyc
├── a
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── config.py
│   └── config.pyc
└── b
├── __init__.py
└── test.py
Create a __init__.py with with your subdirectoryOne and subdirectoryTwo like here I used a and b. and then add few codes at the begin of your file.py. And I name it as test.py here. the code is:
import sys
sys.path.append("..")
from project.a.config import myconf
print(myconf)
You can see I import it by project.a.config instead of a.config. because you run your code under the project.
UPDATE
My a/config.py just simple with:
(public)landpacks-MacBook-Pro:qx frank$ cat a/config.py
myconf='127.0.0.1'

One of the solutions (not the optimum one) is to set PYTHONPATH to your directory:
$ export PYTHONPATH='/absolute/path/to/directory'

unit tests: invoke test scripts from any directory

I am trying to get my unit test scripts to work no matter where they are invoked from and have looked into a dozen questions on relative import issues now. But relative paths in combination with the unittest module seem to add some extra complications.
My directory structure is this:
importtest
├── importtest
│   ├── importtest.py
│   └── __init__.py
└── test
├── data.txt
├── importtest_test.py
└── __init__.py
the __init__.py files are empty, importtest.py contains
def func(filename):
with open(filename) as f:
return True
and importtest_test.py contains
import unittest
from importtest import importtest
class Test(unittest.TestCase):
def test(self):
self.assertEqual(importtest.func('data.txt'), True)
This works when I execute py -m unittest test/importtest_test.py from the root folder of the project (the top most importtest folder). But it does not work from any other directory.
Why is that and how can I fix this?

Import python project modules from tests subdirectory

├── ledger
│   ├── __init__.py
│   ├── ledger_data.py
│   └── ledger_model.py
├── main.py
├── sscommon
│   ├── __init__.py
│   └── logging.py
└── tests
└── test_ledger_data.py
I need to import classes from ledger_data module when running test_ledger_data.py. I currently do sys.path.append("../") in test_ledger_data.py or I have to add symbolik links to all modules being used to tests directory. Both options seem incorrect. How to do it correctly?
If I just run the file either from project root or tests directories I get error:
from ledger.ledger_data import LedgerData
ImportError: No module named 'ledger'

You can create an __init__.py file in your folder, and import the parent dir using:
parent_dir = os.path.abspath(os.path.join(os.path.abspath(__file__), os.pardir))
sys.append(parent_dir)
This uses os.path to find out the directory based on your file location.
Update: create the above __init__.py and reside it inside tests/ folder.
Then, in your test_ledge_data.py put at the head of the file from __init__ import *; this will import everything in your init file to your module namespace.

How to import python file from git submodule

I've a project which uses git submodules. In my python file I want to use functions from another python file in the submodule project.
In order to work I had to add the init.py file to all subfolders in the path. My folder tree is the following:
myproj
├── gitmodules
│   ├── __init__.py
│   ├── __init__.pyc
│   └── mygitsubmodule
│   ├── __init__.py
│   ├── __init__.pyc
│   └── file.py
└── myfile.py
Is there any way to make it work without touching mygitsubmodule ?
Thanks

you can add to sys.path in the file you want to be able to access the module, something like:
import sys
sys.path.append("/home/me/myproj/gitmodules")
import mygitsubmodule
This example is adding a path as a raw string to make it clear what's happening. You should really use the more sophisticated, system independent methods described below to determine and assemble the path.
Also, I have found it better, when I used this method, to use sys.path.insert(1, .. as some functionality seems to rely of sys.path[0] being the starting directory of the program.

I am used to avoiding modifying sys.path.
The problem is, when using git submodule, submodule is a project directory, not a Python package. There is a "gap" between your module and that package, so you can't import.
Suppose you have created a submodule named foo_project, and there is a foo package inside.
.
├── foo_project
│ ├── README.rst
│ └── foo
│ └── __init__.py
└── main.py
My solution will be creating a soft link to expose that package to your module:
ln -s foo_project/foo foo
.
├── foo_project
│ ├── README.rst
│ └── foo
│ └── __init__.py
├── foo -> foo_project/foo
└── main.py
Now you can import foo in the main.py.

For reference,
from submodulefolder.file import func_name
or
import submodulefolder.file as lib_name
where file excludes the extension of file.py, seems to work in relative terms without modifying the subfolder / git submodule with a init.py since python 3.3+,
as shown here.
Tested on py3.8.5 linux native and py3.7.8 anaconda Windows, both in Spyder's Ipython-console, as well as natively on linux via terminal.