Does importing a specific function from a module is a faster process than importing the whole module?
That is, is from module import x debugs faster than import module?
I would say there is little or no peformance difference, as importing a module for the first time will execute the entire module - all classes, variables and functions are all built, regardless of the actually symbol you need.
The 2nd time you import the module in the same program that will be much quicker, as the module is not reloaded, and all existing definitions are used.
No, it shouldn't be faster, and that shouldn't matter anyway: importing things is not usually considered a performance-critical operation, so you can expect it to be fairly slow compared to other things you can do in Python. If you require importing to be very fast, probably something is wrong with your design.
The whole module has to compile before you can import the specific function.
Instead, it's just a difference in namespace. (ie. you call module_x.function_y vs just calling function_y)
Related
Hi I'm trying to learn python, when looking at examples sometimes I see a whole package simply imported, while other times a specific module within the package is imported. What is the advantage of this?
For example:
import face_detection
vs
from face_detection import model
The former seems better to me as then it would require you to be more explicit by needing to use face_detection.model every time you'd like to use the model module rather than being vague and simply calling module. Is it really just a code style thing?
Here is good explanation: https://softwareengineering.stackexchange.com/a/187471
Short answer - there is no such difference.
For example:
if I write
import math
I am not good in programming. For a program to run it has to load the code to its memory and convert to 0s and 1s which a computer can understand?
So, will it load the entire math module when a reference to any function in that module is made in my program, or will it expand the module only once in the program? If it expands only once, I assume, the computer will load entire python file and all the modules it imports completely in memory? won't that cause a memory running out of space issue if I import too many native python code from the library?
Is that the reason some people say it is always good to import exact function in your program instead of wild cards?
Will it load the entire math module when a reference to any function
in that module is made in my program, or will it expand the module
only once in the program?
The math module will be loaded into memory once per Python program (or interpreter).
If it expands only once, I assume, the computer will load entire
python file and all the modules it imports completely in memory
Yes, in normal circumstances.
Won't that cause a memory running out of space issue if I import too
many native python code from the library?
No, not typically. Python modules would not put a dent in the memory of modern computers.
Is that the reason some people say it is always good to import exact
function in your program instead of wild cards?
No, the entire module will be loaded regardless if you use just one function in it. This is because that one function can rely on any other code in the module.
The reason it is advised to import specific functions is more of a best practice or recommendation to be explicit about what you are using from the module.
Also, the module may contain function names in it that are the same as ones you define yourself or are even in another imported module so you want to be careful not to import a bunch of names, especially if you are not going to use them.
Importing a python module means to load the namespace of what is available in that python module, into memory. Specifically, writing "import " tells python to look for a module with that name on your python path (typically a folder), and if it finds such an object, to then run that objects __init__.py file. This file is typically blank, meaning that python should simply load what is available in the module by reading through the files, but this file can also be customized.
Python automatically tracks what is loaded and doesn't re-load already loaded items. Hence, writing
import time
import itertools
import itertools as it
doesn't reload time if itertools also uses the time module and doesn't reload itertools if you rename it to it. In general, this is done very quickly; in fact, if the code is compiled (some modules are installed already compiled) it can be as fast as literally copying the bytes into memory (down to ns of time regardless of module size). Importing is one of the fastest commands you can do in python and is rarely the source of any speed issues. Copying and loading bytes into ram can be done millions of times a second and costs the computer nothing. It is when the computer has to perform calculations and compute that there is a concern.
Something like:
from itertools import combinations
does not load itertools any faster or slower than if you simply loaded the whole module (there are exceptions to this where some modules are so big that they are broken into sub-packages and so loading at the highest level doesn't load anything at all, for example, scipy, you have to specify the sub-package you want to load).
The reason it is recommended to not run from itertools import * is because this loads your namespace with dozens of functions which no one can track. Suddenly, you use the function combinations and no one that reads your code has any idea which module it came from. Hence, it is considered bad practice to * import. Ironically, some programming languages like C do nothing by * importing and it really is impossible to track where variables have come from.
I noticed many times that the import mod statement can be placed tightly before a call mod.something(). Though I noticed that usually developers put the import statement at the beginning of the source file. Is there a good reason for this?
I often use only a few functions from some module in particular place. It seems prettier to me to place the import statement tightly before the function call.
e.g.
# middle of the source file
import mod
mod.something()
What would you recommend and why?
One thing which can justify importing a module just before calling a function/using a class from that module is performance: sometimes initialization of a module can be expensive, because, for example, it involves loading and initializing a native library. If the code from a module is not always called, it can be a good idea to defer importing of that module until the last moment.
May as well move my comment here as an answer, though it feels a little redundant.
The PEP Style Guide calls for all imports to take place at the beginning of the module. This makes it easier for people to know what dependencies your module has, rather than having to dig through the entire source document.
As a caveat - on the rare occasion that an import would cause an error (circular imports are the best example of this) you can import immediately before you use some functionality. Generally speaking, however, this is bad form. If you are required to import somewhere other than the top of your module it usually indicates a design flaw.
I'm running a website using Django, and I import ipdb at the beginning of almost all of my scripts to make debugging easier. However, most of the time I never use the functions from the module (only when I'm debugging).
Just wondering, will this decrease my performance? It's just that when I want to create a breakpoint I prefer to write:
ipdb.set_trace()
as opposed to:
import ipdb; ipdb.set_trace()
But I've seen the second example done in several places, which makes me wonder if it's more efficient...
I just don't know how importing python modules relates to efficiency (assuming you're not using the module methods within your script).
As #wRAR mentioned, Loading a module may imply executing any amounts of code which can take any amount of time. On the other hand, the module will only be loaded once and any subsequent attempt to import will find the module present in os.sys.modules and reference to that.
In a Django environment in debuging mode, modules are removed from Django's AppCache and actually re-imported only when they are changed, which you will probably not do with ipdb, so in your case it should not be an issue.
However, in cases it would be an issue, there are some ways around it. Suppose you have a custom module that you use to load anyway, you can add a function to it that imports ipdb only when you require it:
# much used module: mymodule
def set_trace():
import ipdb
ipdb.set_trace()
in the module you want to use ipdb.set_trace:
import mymodule
mymodule.set_trace()
or, on top of your module, use the cross-module __debug__ variable:
if __debug__:
from ipdp import set_trace
else:
def set_trace(): return
Short answer: Not usually
Long answer:
It will take time to load the module. This may be noticeable if you are loading python off a network drive or other slow source. But if running directly off a hard drive you'll never notice.
As #wRar points out, importing a module can execute any amount of code. You can have whatever code you want executed at module startup. However, most modules avoid executing unreasonable amounts of code during startup. So that itself probably isn't a huge cause.
However, importing very large modules especially those that also result in importing a large number of c modules will take time.
So importing will take time, but only once per module imported. If you import modules at the top of your modules (as opposed to in functions) it only applies to startup time anyways. Basically, you aren't going to get much optimisation mileage out of avoiding importing modules.
Importing a module but not using it decreases (the system) performance:
It takes time to import the module
Imported modules use up memory
While the first point makes your program slower to start, the second point might make ALL your programs slower, depending on the total amount of memory you have on your system.
I'm just wondering, I often have really long python files and imports tend to stack quite quickly.
PEP8 says that the imports should always be written at the beginning of the file.
Do all the imported libraries get imported when calling a function coded in the file? Or do only the necessary libraries get called?
Does it make sense to worry about this? Is there no reason to import libraries within the functions or classes that need them?
Every time Python hits an import statement, it checks to see if that module has already been imported, and if not, imports it. So the imports at the top of your file will happen as soon as your file is run or imported by another module.
There is some overhead to this, so it's generally best to keep imports at the top of your file so that cost gets taken care of up front.
The best place for imports is at the top of your file. That documents the dependencies in one place and makes errors from their absence appear earlier. The import itself actually occurs at the time of the import statement, but this seldom matters much.
It is not typical that you have anything to gain by not importing a library until you are in a function or method that needs it. (There is never anything to gain by doing so inside the body of a class.) It is rare that you want optional dependencies and even rarer that this is the right technique to get them, though. Perhaps you can share a compelling use case?
Does it make sense to worry about
this?
No
There no reason to import libraries within the functions or classes that need them.
It's just slow because the import statement has to check to see if it's been imported once, and realize that it has been imported.
If you put this in a function that's called frequently, you can waste some time with all the import checking.
Imports happen when the module that contains the imports gets executed or imported, not when the functions are called.
Ordinarily, I wouldn't worry about it. If you are encountering slowdowns, you might profile to see if your problem is related to this. If it is, you can check to see if your module can divided up into smaller modules.
But if all the files are getting used by the same program, you'll just end up importing everything anyway.
If a function inside a module is the only one to import a given other module (say you have a function sending tweets, only if some configuration option is on), then it makes sense to import that specific module in the function.
Unless I see some profiling data proving otherwise, my guess is that the overhead of an import statement in a function is completely negligible.