I am a python beginner (python 2.7 and ironpython 2.7) and not a programmer.
From time to time, I find a code which starts like so:
import clr
clr.AddReferenceToFile("nameOfmodule")
from nameOfmodule import someMethod
What is the point of this?
Can't we just use:
from nameOfmodule import someMethod
?
I tried googling but have not really understand the explanation.
Thank you for the reply.
EDIT: the confusion is not between "from nameOfmodule import someMethod" and "import nameOfmodule.someMethod". I edited the code, so that now it makes more sense.
You don't need to import clr and AddReferenceToFile with recent versions of Python for .NET, but it still works for backwards compatibility so the examples you're looking at might still use it so they work for a larger number of people.
With recent versions, you can treat CLR assemblies as normal Python modules:
from nameOfModule import someMethod
clr is the Common Language Runtime, provided by Microsoft's .NET framework. So, in your example, the script uses clr so that it can refer to a component written in C# or Visual Basic and made into a library for use with something else.
http://pythonnet.github.io/ has some more information.
Are you using IronPython and .NET? If so the code in the first example is required to add a .NET assembly as reference in IronPython, so that Python can import it.
Otherwise you don't need to do it, and from nameOfmodule import someMethod will work.
Note, the code in your second example is incorrect for importing a function in a standard Python module. You can either:
>>> import nameOfmodule
>>> result = nameOfmodule.someMethod()
or
>>> from nameOfmodlue import someMethod
>>> result = someMethod()
but the following will give you an error:
>>> import nameOfmodule.someMethod
ImportError: No module named someMethod
In the second example you still have to use nameOfmodule.someMethod(...) when you want to call method, in the first you simply need someMethod(..). If you used import nameOfmodule.someMethod as sm that would make more sense.
If you like typing or you have another someMethod defined somewhere in your code use your second example.
An example using os.path:
In [3]: import os.path
In [4]: os.path.abspath(".")
Out[4]: '/home/foo'
In [5]: from os import path
In [6]: path.abspath(".")
Out[6]: '/home/foo'
In [7]: import os.path as op
In [8]: op.abspath(".")
Out[8]: '/home/foo'
The clr is only relevant if you are using Ironpython:
The import statement in Python is more analogous to a using statement in C#. you still need to load the relevant .dll assembly. C# does this at compile-time by using references; IronPython by default includes standard CLR references, which is why it is immediately possible to import System.Xml
However, if you want to load a .dll that is not included by default in IronPython, you must use clr.AddReference("myAssembly.dll") and then use the import statement to import the module.
There a a blog post here that goes into more detail
Related
I wrote a Python script which is executed via Jython 2.7. I need SQLite, so I decided to use sqlite3 for Jython (link) which is under /usr/local/lib/jython/Lib.
ghidra_batch.py
import sys
sys.path.append("/usr/local/lib/jython/Lib")
sys.path.append("/path/to/my/project/directory")
import sqlite3
I created another file where I define some functions for my database:
db.py
import platform
import sys
if platform.python_implementation == 'Jython':
sys.path.append("/usr/local/lib/jython/Lib")
sys.path.append("/path/to/my/project/directory")
import sqlite3
def open_db():
[some code]
def create_schema():
[some code]
Note: I check Python implementation because this script is run also via CPython. I append the path only when run via Jython to make it find its sqlite3 module, in case of CPython standard sqlite3 module is used.
Now my problem happens when I import open_db() in ghidra_batch.py:
from db import open_db
The result is the following:
ImportError: cannot import name open_db
Thanks for your help.
As a general rule: when working with Python, when something isn't not what you're expecting, simply print it.
Your from db import open_db line which was triggering that exception "told" me that:
The db module (package) was found
It's not the one that you're expecting to be (your db.py)
That's why I suggested in my comment to print information about it (obviously, before the error is hit):
import db
print(db)
print(dir(db))
The output confirmed it. So, there is another db module which is imported before yours. I tried replicating your environment (installed Jython, but I wasn't able to install jython-sqlite3).
After a bit of research, I think it's [BitBucket]: Taro L. Saito/sqlite-jdbc/Source - sqlite-jdbc/src/main/java/org/sqlite/DB.java (sqlite-jdbc is a jython-sqlite3 dependency).
The reasonable way is to modify your module name to something else (e.g.: sqlite_db_wrapper.py), and also update the import statement(s).
As a(n other) general rule, don't give your modules (common) names that might conflict with modules from Python's library.
Is there any differences among the following two statements?
import os
import os as os
If so, which one is more preferred?
It is just used for simplification,like say
import random
print random.randint(1,100)
is same as:
import random as r
print r.randint(1,100)
So you can use r instead of random everytime.
The below syntax will help you in understanding the usage of using "as" keyword while importing modules
import NAMES as RENAME from MODULE searching HOW
Using this helps developer to make use of user specific name for imported modules.
Example:
import random
print random.randint(1,100)
Now I would like to introduce user specific module name for random module thus
I can rewrite the above code as
import random as myrand
print myrand.randint(1,100)
Now coming to your question; Which one is preferred?
The answer is your choice; There will be no performance impact on using "as" as part of importing modules.
Is there any differences among the following two statements?
No.
If so, which one is more preferred?
The first one (import os), because the second one does the exact same thing but is longer and repeats itself for no reason.
If you want to use name f for imported module foo, use
import foo as f
# other examples
import numpy as np
import pandas as pd
In your case, use import os
The import .... as syntax was designed to limit errors.
This syntax allows us to give a name of our choice to the package or module we are importing—theoretically this could lead to name clashes, but in practice the as syntax is used to avoid them.
Renaming is particularly useful when experimenting with different implementations of a module.
Example: if we had two modules ModA and ModB that had the same API we could write import ModA as MyMod in a program, and later on switch to using import MoB as MyMod.
In answering your question, there is no preferred syntax. It is all up to you to decide.
Is there an import utility, such as importlib, that allows programmatically importing a module with a qualified name, such as import numpy as np?
It seems that the plain importlib.import_module() and the imp.load_module functions do not offer this. And the Python docs page on import tools didn't reveal anything either.
Specifically, instead of just doing importlib.import_module("numpy") I'm looking for something that "just works" along the lines of importlib.import_module("numpy", as="np") making the name "np" now available as the imported module.
My use case:
I'm using the DirectView.sync_imports function from IPython.parallel to ensure that all compute engines have the needed modules imported on them locally. This way, a user can execute code in a controller IPython session, but the backend can execute it in parallel.
In this use case, since the code is often just interactive, exploratory scripting, it is important to match exactly whatever import statements and names are provided by the programmer.
So if the following is a function that the programmer would like to map to other engines and execute in parallel on a variety of arguments, the same qualified import would need to be available on all engines:
def my_func(x):
import numpy as np
return np.random.randint(x, size=(10,1))
To get 4 such results, in parallel, a user could do the following (assuming everything with ipcluster or ipcontroller / ipengines is configured and running):
from IPython.parallel import Client
rc = Client()
direct_view = rc[:]
# In parallel.
four_random_arrays = direct_view.map(my_func, [2, 3, 4, 5]).get()
IPython provides a nice context manager for syncing the imports:
# This works fine and syncs the import on all engines.
with direct_view.sync_imports():
import numpy as np
But suppose you only have access to a string: "numpy as np" and you want to use importlib to do this within that context manager block without trying to parse the strings manually and tinker with whatever entries are in the loaded modules dict.
It's probably needless to say, but using exec here is also completely untenable, because what if instead of a module string, someone accidentally passes in something nasty or insecure, like a shell call to rm -rf /. Leveraging import tools that don't allow for execution of arbitrary code is important -- making it untenable to roll one's own string-parsing exec sort of solution.
import_module returns a value (the module being imported) -- set whatever variable you want to the return value of that. For example:
import importlib
p = importlib.import_module('os.path')
p.join('/', 'blah')
would effectively be equivalent to
import os.path as p
p.join('/', 'blah')
So my first python program, I downloaded the macholib package from PYPI, then unzipped it and ran installed the setup.py using python install, now I'm trying to run a program where I first import the macholib, then when I try to access methods like Macho, it gives me an error saying macholib module has no attributes called Macho.
My understanding is macholib is a package and not a module or something, hence I cant use the contents of the package.
Please answer, I've wasted too much time on such a simple newbie issue.
running a mac with the latest version of python.
Code:
import sys
import macholib
MachO(DivXInstaller.dmg)
I tried macholib.MachO(DivXInstaller.dmg) and macholib.MachO.MachO(DivXInstaller.dmg)
Error for python test.py
Traceback (most recent call last):
File "test.py", line 3, in <module>
MachO(DivXInstaller.dmg)
NameError: name 'MachO' is not defined
You have this line in your code:
MachO(DivXInstaller.dmg)
However, the name MachO is not defined anywhere. So Python tells you this with the error
NameError: name 'MachO' is not defined
If you want to use the MachO module under the name MachO, you have to import it:
from macholib import MachO
This you have not done, which is the cause of your error. All you did was
import macholib
which gives you the name "macholib" that you can use. However, macholib contains mostly submodules which you can not access like that, so that is not particularly useful. If you don't want to pollute the namespace, you can import MachO as
import machlibo.MachO
Which gives you access to the MachO module as macholib.MachO
You haven't defined DivXInstaller.dmg either, so that's going to be your next error. I recommend that you go through a Python tutorial before you start programming in it.
An import statement defines a namespace. Hence, in order to use something defined in the module or package, you need to qualify it. In this case, the module itself is macholib.MachO but you need to dig deeper, to the actual class MachO which, slightly confusingly, has the same name, so you need to import macholib.MachO and use macholib.MachO.MachO (note that when I try to do this I get an error "DistributionNotFound: altgraph", however).
Note further that it takes a filename as an argument, which is a string. Moreover, you are actually creating an instance of a class, so you probably mean
mlib = macholib.MachO.MachO("DivXInstaller.dmg")
where now mlib is actually the object you need to manipulate with further calls...
Alternately you can do from macholib import MachO or even from macholib.MachO import MachO so you could use MachO.MachO(...) or MachO(...) directly.
This is so simple if you understand Python Packages vs. Modules and import vs. from import:
A Python module is simply a Python source file.
A Python package is simply a directory of Python module(s).
1- import:
import package1.package2.module1
To access module1 classes, functions or variables you should use the whole namespace: package1.package2.modlue1.class1
You cannot import a class or function this way: (wrong)
import package1.package2.class1
2- from ... import
Instead use "from" to import a single class, function or variable of a module:
from package1.package2.module1 import class1
no need to address the whole namespace: class1.method1() works now
Note that you cannot import a method of a class this way
Example:
datetime is a class of module datetime that has a method called utcnow(), to access utcnow() one could:
import datetime
datetime.datetime.utcnow()
Or
from datetime import datetime
datetime.utcnow()
I'm debugging from the python console and would like to reload a module every time I make a change so I don't have to exit the console and re-enter it. I'm doing:
>>> from project.model.user import *
>>> reload(user)
but I receive:
>>>NameError: name 'user' is not defined
What is the proper way to reload the entire user class? Is there a better way to do this, perhaps auto-updating while debugging?
Thanks.
As asked, the best you can do is
>>> from project.models.user import *
>>> import project # get module reference for reload
>>> reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
it would be better and cleaner if you used the user module directly, rather than doing import * (which is almost never the right way to do it). Then it would just be
>>> from project.models import user
>>> reload(user)
This would do what you want. But, it's not very nice. If you really need to reload modules so often, I've got to ask: why?
My suspicion (backed up by previous experience with people asking similar questions) is that you're testing your module. There are lots of ways to test a module out, and doing it by hand in the interactive interpreter is among the worst ways. Save one of your sessions to a file and use doctest, for a quick fix. Alternatively, write it out as a program and use python -i. The only really great solution, though, is using the unittest module.
If that's not it, hopefully it's something better, not worse. There's really no good use of reload (in fact, it's removed in 3.x). It doesn't work effectively-- you might reload a module but leave leftovers from previous versions. It doesn't even work on all kinds of modules-- extension modules will not reload properly, or sometimes even break horribly, when reloaded.
The context of using it in the interactive interpreter doesn't leave a lot of choices as to what you are doing, and what the real best solution would be. Outside it, sometimes people used reload() to implement plugins etc. This is dangerous at best, and can frequently be done differently using either exec (ah the evil territory we find ourselves in), or a segregated process.
For python3.4+, reload has been moved to the importlib module. you can use importlib.reload(). You can refer to this post.
>>> import importlib
>>> import project # get module reference for reload
>>> importlib.reload(project.models.user) # reload step 1
>>> from project.models.user import * # reload step 2
For python3 versions before 3.4, the module to import is imp (instead of importlib)
IPython can reload modules before executing every new line:
%load_ext autoreload
%autoreload 2
Where %autoreload 2reloads "all modules (except those excluded by %aimport) every time before executing the Python code typed."
See the docs:
https://ipython.org/ipython-doc/3/config/extensions/autoreload.html
You can't use reload() in a effective way.
Python does not provide an effective support for reloading or unloading of previously imported
modules; module references makes it impractical to reload a module because references could exist in many places of your program.
Python 3 has removed reload() feature entirely.
Unfortunately you've got to use:
>>> from project.model import user
>>> reload(user)
I don't know off the top of my head of something which will automatically reload modules at the interactive prompt… But I don't see any reason one shouldn't exist (in fact, it wouldn't be too hard to implement, either…)
Now, you could do something like this:
from types import ModuleType
import sys
_reload_builtin = reload
def reload(thing):
if isinstance(thing, ModuleType):
_reload_builtin(thing)
elif hasattr(thing, '__module__') and thing.__module__:
module = sys.modules[thing.__module__]
_reload_builtin(module)
else:
raise TypeError, "reload() argument must be a module or have an __module__"
You could also try twisted.python.rebuild.rebuild.
from test_reload import add_test
where test_reload is a module, and add_test is a function
if you changed the function add_test, of course you need to reload this function.
then you can do this:
import imp
imp.reload(test_reload)
from test_reload import add_test
this will refresh the function add_test.
so you need to add
imp.reload(test_reload)
from test_reload import add_test --add this line in your code
As of Python 3.4 you can use importlib.reload(module)
>>> from importlib import reload
>>> from project.model import user
>>> reload(user)