I'm having trouble understanding Intellij's import policy for python for import os. As far as I know, the import order is supposed to be standard library first, then third party packages, then company packages, and finally intra-package or relative imports. For the most part Intellij orders everything correctly, but keeps pushing import os into third party packages. Am I missing smth? Isn't import os a standard library package?
It might happen if the corresponding module comes from a virtual environment that itself is located inside a project directory, and it confuses the detection of the right import group. There was a similar request in the tracker, but it was fixed quite a while ago. Which version of Python plugin do you use? Would you mind creating a dedicated issue in YouTrack so that we could investigate the problem further there?
The answer I got from a co-worker a couple of years age is that os was originally a third-party package; IntelliJ left it where it is for some backward compatibility issue.
Related
I have an environment with some extreme constraints that require me to reduce the size of a planned Python 3.8.1 installation. The OS is not connected to the internet, and a user will never open an interactive shell or attach a debugger.
There are of course lots of ways to do this, and one of the ways I am exploring is to remove some core modules, for example python3-email. I am concerned that there are 3rd-party packages that future developers may include in their apps that have unused but required dependencies on core python features. For example, if python3-email is missing, what 3rd-party packages might not work that one would expect too? If a developer decides to use a logging package that contains an unreferenced EmailLogger class in a referenced module, it will break, simply because import email appears at the top.
Do package design requirements or guidelines exist that address this?
It's an interesting question, but it is too broad to be cleanly answered here. In short, the Python standard library is expected to always be there, even though sometimes it broken up in multiple parts (Debian for example). But you say it yourself, you don't know what your requirements are since you don't know yet what future packages will run on this interpreter... This is impossible to answer. One thing you could do is to use something like modulefinder on the future code before letting it run on that constrained Python interpreter.
I was able to get to a solution. The issue was best described to me as cascading imports. It is possible to stop a module from being loaded, by adding an entry to sys.modules. For example, when importing the asyncio module ssl and _ssl modules will be loaded, even though they are not needed outside of ssl. This can be stopped with the following code. This can be verified both by seeing the python process is 3MB smaller, but also by using module load hooks to watch each module as it loads:
import importhook
import sys
sys.modules['ssl'] = None
#importhook.on_import(importhook.ANY_MODULE)
def on_any_import(module):
print(module.__spec__.name)
assert module.__spec__.name not in ['ssl', '_ssl']
import asyncio
For my original question about 3rd-party design guidelines, some recommend placing the import statements within the class rathe that at the module level, however this is not routinely done.
[Edit]: My problem is unrelated to what I’m asking here. My machine has python 2.7 and 3.6 installed. Requests module was installed for 2.7 site-packages but not 3.6
I’m trying to import requests into a sub directory. I’m not aware of the best way to do this
My folder structure is
Project
- project1
- sub1
- sub1.py <—- trying to use requests here
- tests
But in the future if I want to use requests across the whole project, what’s the most effective was to import it?
If you're importing a third-party module, it's fine to just have e.g. import requests at the top of any .py file where you want to use it. As long as it's installed for your project (e.g. you've done pip install requests) you can use it wherever you want.
If you're importing your own code from one place in your project to use in another place, that can require a bit more thought, but I don't think that's your question here.
I am trying to debug a file for a project I am working on, and the first thing I made sure to do is install/build all of the modules that the file is importing. Thisis the first line of the file:
from scitbx.array_family import flex
which in turn reads from flex.py,
from __future__ import division
import boost.optional # import dependency
import boost.std_pair # import dependency
import boost.python
I entered the commands in ipython individually and get stuck on importing boost.optional. Since they are all from the same module I tried searching for the module named boost.
I found the site: http://www.boost.org/doc/libs/1_57_0/more/getting_started/unix-variants.html
and installed the related .bz2 file in the same directory as my other modules to make sure it is within sys.path. However I still can't get ipython to import anything. Am I completely off base in my approach or is there some other boost module that I can't find? I should mention that I am a complete novice with computers, and am learning as I go along. Any advice is much appreciated!
The library you have installed is called Boost. This is a collection of C++ libraries, one of which is Boost.Python. However this library doesn't provide Python modules that you can import directly - it doesn't provide boost.optional. Instead it enables interoperability between Python and C++ - you can write a C++ library using Boost.Python that can then be used in a normal Python interpreter.
In you case boost.optional is provided by the CCTBX collection of software, which does depend on Boost and Boost.Python. So you are not too far off. This thread in the mailing list covers your error message and some potential solutions.
Essentially you need to use the custom cctbx.python command (or scitbx.python, they are equivalent) to run python as this sets the PYTHONPATH correctly for their requirements. It's also documented on this page.
I am creating a Python package with multiple modules. I want to make sure that when I import modules within the package that they are importing only from the package and not something outside the package that has the same name.
Is the correct way of doing this is to use relative imports? Will this interfere when I move my package to a different location on my machine (or gets installed wherever on a customer's machine)?
Modern relative imports (here's a reference) are package-relative and package-specific, so as long as the internal structure of your package does not change you can move the package as a whole around wherever you want.
While Joran Beasley's answer should work as well (though does not seem necessary in those older versions of Python where absolute imports aren't the default, as the old style of importing checked within the package's directory first), I personally don't really like modifying the import path like that when you don't have to, especially if you need to load some of those other packages or modules that your modules or packages now shadow.
A warning, however: these do require that the module in question is loaded as part of a package, or at least have their __name__ set to indicate a location in a package. Relative imports won't work for a module when __name__ == '__main__', so if you're writing a simple/casual script that utilizes another module in the same directory as it (and want to make sure the script will refer to the proper directory, things won't work right if the current working directory is not set to the script's), you could do something like import os, sys; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) (with thanks to https://stackoverflow.com/a/1432949/138772 for the idea). As noted in S.Lott's answer to the same question, this probably isn't something you'd want to do professionally or as part of a team project, but for something personal where you're just doing some menial task automation or the like it should be fine.
the sys.path tells python where to look for imports
add
import sys
sys.path.insert(0,".")
to the top of your main python script this will ensure local packages are imported BEFORE builtin packages (although tbh I think this happens automagically)
if you really want to import only packages in your folder do
import sys
sys.path = ["."]
however I do not recommend this at all as it will probably break lots of your stuff ...
most IDE's (eclipse/pycharm/etc) provide mechanisms to set up the environment a project uses including its paths
really the best option is not to name packages the same as builtin packages or 3rd party modules that are installed on your system
also the best option is to distribute it via a correctly bundled package, this should more than suffice
I use intellij with python plugin.
when I want to import python libs like
import random
I got editor error.
No module named random less... (Ctrl+F1)
This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level and class-level items are supported better than instance items.
when I run the code every thing is ok
what can I do to make the intelij recognize this libs?
You may have fixed this by now, but I was having the same problem and finally solved it, so I figured I'd post the solution for anyone who came here by Googling/Binging the error message:
I went to File > Project Structure > Modules, highlighted the main module, then pressed the plus sign and then "Python" under "Framework".
Hope that helps you or someone else.
may be my intellij version is different with you guys.
on Windows platform I fix this problem by:
1.File > Project Structure > Modules
2.on the module's dependencies panel,change the module SDK from JDK to python
3.done