Develop Reference

Using Pylint with PyModule generated using PyO3 and maturin

Pylint will not recognize any of the functions from a PyModule that I created using PyO3 and maturin. All of the functions import and run fine in the python code base, but for some reason Pylint is throwing E1011: no-member warnings.
Below is a (likely) incomplete dummy example, but is provided in order to show the way I am decorating using pymodule and pyfunction:
#[pyfunction]
fn add_nums(
_py: Python<'_>,
a: f32,
b: f32,
) -> PyResult<f32> {
let res:f32 = a+b;
Ok(res)
}
#[pymodule]
fn my_module(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(add_nums, m)?)?;
Ok(())
}
Then if I build that using maturin build --release and install the module, from the resulting wheelfile, into my python environment and import into a script:
import my_module
my_module.add_nums(5, 6) # ignore that these are not f32 - irrelevant this is a dummy example
If I then run pylint on that file (from terminal - VS Code pylint extension actually does not complain about this...), I end up with something like: E1101: Module 'my_module' has no 'add-nums' member (no-member) even though the code (not this code - but the real code which I cannot include here) runs just fine.
Has anyone successfully built wheelfiles using maturin, used them in another project, then had Pylint play nicely with that project and recognize that the methods do actually exist?

Pylint has a extension-pkg-allow-list setting which you can use to inspect non-python modules. It will need to load the extension into pylint's interpreter though, which is why it's not enabled by default.
There's also requests to support (and lint) pyi, but AFAIK that's not supported yet, cf #2873 and #4987.
Before Pylint 2.8, the setting is extension-pkg-whitelist.

Similar to the answer by #Masklinn except it looks like the term 'extension-pkg-whitelist' exists in older versions and later the 'extension-pkg-allow-list' does not (though it was introduced for obvious societal reasons).
add the following into the [MASTER] section of your .pylintrc:
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code.
extension-pkg-allow-list=
my_module
for versions where this is not supported (someone please update which version it changed here) use the extension-pkg-whitelist instead.

Python Extensions cannot see installed packages in DevContainer IDE

I am trying to create a basic Development Container to use with VS Code.
I've been through a few iterations no of versions but keep coming up against the same issue, my VS Code extensions cannot seem to see what packages are installed in my venv.
Files in my workspace:
.devcontainer/devcontainer.json
{
"name": "Existing Dockerfile",
"context": "..",
"dockerFile": "../Dockerfile"
}
venv/ containing pip installed pandas
Dockerfile:
FROM python:3.9
WORKDIR .
COPY my_file.py .
my_file.py
import sys
import pandas
print(sys.path)
Output of sys.path incase relevant is ['/workspaces/yt', '/usr/local/lib/python39.zip', '/usr/local/lib/python3.9', '/usr/local/lib/python3.9/lib-dynload', '/workspaces/yt/venv/lib/python3.9/site-packages']
The code executes fine when ran but in VS Code the linting tools raise an error that pandas is not accessed.
Any help would be greatly appreciated.

The most likely reason is that the environment in which the pandas library was installed is not the same as the environment you are currently using.
Solution
Use Ctrl+Shift+P to open the command palette and search for Python:Select Interpreter to select a correct interpreter.
If the error is still not resolved, go ahead and try the following methods:
Add configuration in settings.json to point to the pandas library.
// Just an example, please modify it to your own path
"python.analysis.extraPaths": [
// Path to pandas library
"C:\\WorkSpace\\PyTest0802\\.venv\\Lib\\site-packages"
],
The most simple and rude method -- cancel this type of error (use with caution, this method will cause the prompt message that there is such an error to not be displayed)
Add the following configuration to the settings.json file
"python.analysis.diagnosticSeverityOverrides": {
"reportMissingModuleSource": "none",
},

Python module import statement runs in PyCharm but not in VSCode

Here's my file structure
test/
-dir1
-thing.py
-dir2
-__init__.py
-thing2.py
I am using python 3.7 and windows 10.
In thing.py, I'm trying to import a function called foo from thing2.py and have it execute when I run thing.py. My code works perfectly in PyCharm when I press run. However, when I run thing.py from the terminal directly or through code runner in VSCode, I get the following error:
from dir2.thing2 import foo
ERROR: ModuleNotFoundError: No module named 'dir2
Is the issue something to do with my PYTHONPATH or something else?

Based on the information you provided, I reproduced the problem you described. And you could use the following methods to solve it:
Please add the following code at the beginning of the "thing.py" file, which adds the path of the currently opened file to the system path so that VSCode can find "foo" according to "from dir2.thing2 import foo":
import os, sys
sys.path.append('./')
If you don't want to add code, you could add the following setting in "launch.json", which adds the path of the project when debugging the code:
"env": {
"PYTHONPATH": "${workspaceFolder}"
}

Import statement works on PyCharm but not from terminal

PyCharm 2016.2.3, Mac OS X 10.11.1, Python 3.5 (Homebrew);
I have this folder structure
project
/somepackage
/subpackage
__init__.py
bar.py
__init__.py
foo.py
foo.py:
import somepackage.subpackage.bar
print("foo")
bar.py:
print("bar")
So my expected output is
bar
foo
This works fine when run from PyCharm. However, when I run it from my terminal I get an ImportError:
$ pwd
$ /home/project (not the actual path; just omitting some personal stuff)
$ python3.5 somepackage/foo.py
File "foo.py", line 1, in <module>
import somepackage.subpackage.bar
ImportError: No module named 'somepackage'
I have found this question, which is about the same problem. However, none of the suggested solutions work for me, as I am indeed using the same Python interpreter as PyCharm does and I am currently in the folder that contains the /somepackage folder.
Does anyone have any other suggestions about how to solve this issue?

You are running foo.py like a script, but you are really using it like a module. So the proper solution is to run it as a module:
python3 -m somepackage.foo
For the record, another alternative is to edit your path like:
export PYTHONPATH=.
(Or you could put the absolute directory in there, and of course you should append any other directories that are already in your PYTHONPATH.) This is closer to what PyCharm does, but is less philosophically correct.

Setting PYTHONPATH is what makes it work, as noted above. I use the following VSCODE .env content so that it works for any project:
PYTHONPATH=${PROJ_DIR}:${PYTHONPATH}
This is essentially what PyCharm does when you check "Add Content Roots to PYTHONPATH" in your run/debug configuration. It's a helpful setting, but it spoils you because your code fails outside PyCharm.
Or, if you run in terminal, first export:
export PYTHONPATH=...
Took me days to work all this out.

i solved my problem by two steps on Linux:
first step
go to the root directory of your project and set:
export PYTHONPATH=$PATHONPATH:`pwd`
second step
run python3 -m somepackage.foo
remember Without '.py' suffix

I just had the same problem using scapy.layers.http Module,
this problem occurred on my Kali (linux-Debian) but run fine on Win-10 (after few modifications.)
packet was installed (scapy-http) correctly and the program was running in PyCharm but not as script (from terminal)
I tried solving it with reinstalling in main root, and messing with the sys.path but None have worked.
Troubleshoot & Solution
I found that it looks for the http module in:
/usr/local/lib/python3.7/dist-packages/scapy/layers/init.py
and got the ImportError:
from scapy.layers import http --->
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'http' from 'scapy.layers' (/usr/local/lib/python3.7/dist-packages/scapy/layers/__init__.py)
So then I checked where scapy-http is really installed, which is module called http.py, so I just copied it to:
/usr/local/lib/python3.7/dist-packages/scapy/layers/
found the http.py file in :/usr/local/lib/python3.7/dist-packages/scapy_http/
And that did it :)!
I know its a bit glitchie but that worked!

Python for .NET: ImportError: No module named warnings

I am trying to embed python 2.6 in .NET 4.0.
Following the very minimal documentation in "Python for .NET", I wrote a fairly straightforward code as follows:
const string pythonModulePath = #"C:\Projects\PythonImport\PythonImport\test.py";
Environment.SetEnvironmentVariable("PYTHONHOME", Path.GetDirectoryName(python
ModulePath));
PythonEngine.Initialize();
var oldWorkingDirectory = Directory.GetCurrentDirectory();
var currWorkingDirectory = Path.GetDirectoryName(pythonModulePath);
Directory.SetCurrentDirectory(currWorkingDirectory);
var pyPlugin = PythonEngine.ImportModule(Path.GetFileNameWithoutExtension(python
ModulePath));
if (pyPlugin == null)
{
throw new PythonException();
}
Directory.SetCurrentDirectory(oldWorkingDirectory);
Even if test.py is empty I get an import error saying "No module named warnings".
'import site' failed; use -v for traceback
Unhandled Exception: Python.Runtime.PythonException: ImportError : No module nam
ed warnings
The reason becomes apparent when you run test.py using "python -v" (verbose mode). Notice that python calls #cleanup[2] warnings.
Why is Python .NET not able to resolve warnings? The module is present in Lib directory.
Any ideas?

I think by default the Python Engine does not set the paths you need automatically.
http://www.voidspace.org.uk/ironpython/custom_executable.shtml has an example for embedding ironpython. Looks like you are missing something like
PythonEngine engine = new PythonEngine();
engine.AddToPath(Path.GetDirectoryName(Application.ExecutablePath));
engine.AddToPath(MyPathToStdLib);
Unless I know where and if Ironpython is installed, I prefer to find all of the standard modules I need, compile them to a IPyStdLibDLL and then do th following from my code
import clr
clr.addReference("IPyStdLib")
import site

#SilentGhost I don't know if you figured this out, but you need to set your PYTHONPATH environment variable rather than your PYTHONHOME variable. I, too, had the problem, set the environment variable with the ol' Environment.SetVariable("PYTHONPATH", ...) and everything worked out well in the end.
Good luck with the Python.NET.

I'm working with the 2.4 version of Python.NET with Python 3.6 and I had similar issues. I didn't have much luck setting environment variables at runtime but I found the following approach worked.
I found Python.NET was picking up all packages in the folders defined in the static PythonEngine.PythonPath variable. If you have other directories set in the PYTHONPATH system variable, it will also include these too.
To include module directories at runtime you can do something like
PythonEngine.PythonPath = PythonEngine.PythonPath + ";" + moduleDirectory;
using (Py.GIL())
{
dynamic module = Py.Import("moduleName");
...etc
}
Make sure you set the PythonEngine.PythonPath variable before making any calls to the python engine.
Also needed to restart visual studio to see any system variable changes take effect when debugging.
As a side note, I also found that my \Python36\Lib\site-packages folder path needed to be added to PYTHONPATH to pick up anything installed through pip.