Develop Reference

Python Path Issues with Sphinx?

I'm running into a lot of trouble trying to use Sphinx autodoc for a python package, and I think the issue is a path one, but I'm not sure how to solve? Wondering if anyone might know?
I'm trying it out with just a super basic module setup with a single main.py inside of 'my_project' that looks like:
In both cases below, I have just a basic conf.py and index.rst file:
Scenario 1 (works fine)
When I use the sphinx-quickstart command to build out the folder tree and specify 'N' when asks if I want to separate source and build directories, it seems to all work fine and I am able to successfully build the html and it finds and reads my_project.main without issue. In this case, I get a folder tree that looks like:
.
├── docs
│ ├── build
│ ├──_static
│ ├──_templates
│ ├──conf.py
│ └──index.rst
└── my_project
└── main
Scenario 2 (not working)
However: When I use the sphinx-quickstart command to build out the folder tree and specify 'Y' when it asks about separating source and build, now I get a nicer more ordered folder tree within my ./docs. So now I have:
.
├── docs
│ ├── build
│ └── source
│ ├──_static
│ ├──_templates
│ ├──conf.py
│ └──index.rst
└── my_project
└── main
But the nesting of the conf.py (I think?) seems to be causing path trouble? Now if I try and build the html, I get an import error:
WARNING: autodoc: failed to import module 'main' from module 'my_project';
the following exception was raised:
No module named 'my_project'
So it seems that the conf.py is not probably finding and adding the top-level path in this case?
So, I wonder:
How can I add a folder 2 levels 'up' from current to the sys.path? I can get one level 'up' using .. but ... does not work to go up two? It does not work when I try something like:
sys.path.insert(0, os.path.abspath('...')) # three dots does doesn't work?
Am I missing something else here? how should I be using the 'nested' sphinx-quickstart setup to properly find my modules using autodoc?

As you may have surmised, autodoc needs to be able to import your code in order to make your docs. There are two ways to do this:
The easy way is to edit the sys.path.insert line in conf.py. You almost had this right: instead of ..., it should be ../...
The better way (in my opinion) is to (i) install your package in a virtual environment and (ii) install/run sphinx in that same environment. This way, you don't need to mess with the import path at all. Installing your code also makes it a lot easier to share/reuse (even if only between other scripts on your machine).
If you don't know how to do this, I'd recommend looking into a tool called flit. The whole process should look something like this (although note that I haven't actually tested any of these commands):
# Create a `pyproject.toml` file:
flit init
# Create a new virtual environment
python -m venv sphinx-env
. sphinx-env/bin/activate
# Install your package and sphinx
pip install . sphinx
# Run sphinx
cd docs
make html

How to import from subpackage (without exposing) when building using setuptools?

I am packaging a personal Python project and uploading it to pypi. It has the following structure:
root
├── package
│ ├── __init__.py
│ ├── foo.py
│ └── subpackage
│ ├── bar.py
│ └── __init__.py
├── setup.py
└── test.py
Now, foo.py has a function called public_function that depends on bar.py:bar_function , so I am structuring my code as:
# package/foo.py
from package.subpackage import bar_function
def public_function():
return bar_function()
# package/subpackage/__init__.py
from .bar import bar_function
My goal is to let people call public_function while hiding all its dependencies like this:
import package
package.bar_function()
So package/__init__.py looks like:
from .bar import bar_function
This is working correctly when importing from test.py, but when building using setuptools, things start getting weird.
After checking these similar questions (one, two and three) these are the two approaches I have tried, but don't work as I want:
Approach 1
As I don't want to expose subpackage, I tried:
# package/setup.py
from setuptools import setup
setup(
name='package',
packages=['package']
)
But when I do import package (after uploading to pypi and running pip install package), I get the following error:
File /venv/lib/python3.8/site-packages/package/foo.py:1
----> 1 from package.subpackage import bar_function
ImportError: cannot import name 'bar_function' from 'package.subpackage' (unknown location)
(I also tried with a relative import from .subpackage ... but didn't work)
Approach 2
As it looked like the subpackages are not making it to the build, I tried:
# package/setup.py
from setuptools import setup, find_packages
setup(
name='package',
packages=find_packages()
)
Which doesn't yield the error anymore, but now I end up exposing everything else (i.e I can do import foo and import subpackage) which I don't want.
What should be the right way to set up my package? Is it even possible to achieve what I want in Python?

Sphinx can't find my functions (probably due to imports or folder structure)

I have the following folder structure for a package project (simplified):
projectname
├── docs
├── packagename
│ ├── somemodule
│ | ├── __init__.py (second)
| │ └── somescript.py
│ └── __init__.py (first)
├── setup.cfg
└── pyproject.toml
In first __init__.py I do from . import somemodule. In the second __init__.py I do from .somescript import *. And somescript.py contains some function my_sum. The problem is that sphinx doesn't see the function. It only sees module packagename.somemodule - there is no function doscstring for my_sum in generated documentation. The function works well if I install the library using pip install . from the projectname folder.
I'm sure the problem is in the folder structure or imports because there was no problem when somescript.py was placed directly in packagename folder.
Additional information (maybe useful, maybe not):
I use Read The Docs.
Part of .readthedocs.yaml:
python:
install:
- method: pip
path: .
UPD:
Read The Docs generates the following warnings:
WARNING: autodoc: failed to import module 'somescript' from module 'packagename'; the following exception was raised:
No module named 'packagename.somescripts'
WARNING: html_static_path entry '_static' does not exist
UPD2:
I fixed the warning from autodoc by fixing docs/source/packagename.rst but I still have the problem

I once had a similar problem and it could be solved by manually adding all directories of the module to the path. See also this answer.

Import python from sibling folder without -m or syspath hacks

So I've spent the last three days trying to figure out a workable solution to this problem with imports.
I have a subfolder in my project where I have scripts for database control, which has sibling folders that would like to call it. I have tried many online solutions but couldn't find anything that properly works. It seems some changes in Python 3.3/4 nullify a lot of solutions, or something.
So I made a very simple test case.
IMPORTS/
├─ folder1/
│ ├─ script1.py
│ ├─ __init__.py
├─ folder2/
│ ├─ script2.py
│ ├─ __init__.py
├─ __init__.py
How do I, from script1.py, call a function inside script2.py?

I generally prefer to install my module as a dependency so I can import from the project root. This seems to be the correct approach, though I've rarely seen it talked about online.
E.g. from IMPORTS you would run pip install -e . (install the package in this folder in editable mode). This will require that you have a setup.py:
from setuptools import setup, find_packages
setup(
name='IMPORTS',
version='x.x.x',
description='What the package does.',
author='Your Name',
author_email='x#x.com',
install_requires=[],
packages=find_packages()
)
Here is an example from one of my personal packages.
Then you can import from the root folder (where setup.py is). Following your example:
from folder1 import script1
Or vice versa.
In summary:
Write a setup.py.
Install your package in editable mode with pip install -e .
Write import statements from the package root.

why can I not use my package normally? PyPi

I'm building a very simple package, and I managed to upload it to pypi.
I followed this post: https://www.freecodecamp.org/news/build-your-first-python-package/
But when I tried to import it and use it, a weird thing happened.
import testsimpleprinter as tsp
tsp.testsimpleprinter("Hello") # <- does not work
tsp.testsimpleprinter.testsimpleprinter("Hello there!") # <- this works just fine
TestSimplePrinter
├── testsimpleprinter
│ ├── testsimpleprinter.py <- inside this, I have a func called testsimpleprinter again
│ └── __init__.py <- "from testsimpleprinter import testsimpleprinter"
├── setup.py
At first I thought i had created setup.py somewhere wrong, so I moved it inside testsimpleprinter folder. But when i tried with it I got something like this:
ERROR: File "setup.py" not found for legacy project testsimpleprinter==0.1.0 from https://files.pythonhosted.org/packages/08/36/6446d90277fa8899aaa68b811b/a89ba43807c58be2bab666197ff0e9f41c/testsimpleprinter-0.1.0.tar.gz#sha256=713fc48338620adfd4ef71102478d5e740ad77164fafbc19363f4cf1816922cc.
As in the post, I want to use my package like this
import testsimpleprinter as tsp
tsp.testsimpleprinter("Hello") # I want it to work
Thank you in advance.

The issue is with your testsimpleprinter/__init__.py file. With the import statement from testsimpleprinter import testsimpleprinter you are reimporting the package, not the local testsimpleprinter.py file. To specify the local file, you need to add a . as a prefix, which should be like:
# testsimpleprinter/__init__.py file
from .testsimpleprinter import testsimpleprinter