I am trying to import modules while running my main python script, using a smaller setup.py script. However the importlib command: importlib.util.spec_from_file_location(name, location) doesn't appear to be detecting my small python script. Presumably I'm not filling in the name or location fields correctly.
Example Script A (setup.py):
import os
import pandas as pd
print("success!") # So I can see it has run.
Example Script B (my_script.py):
import importlib
setup_path = ("/home/solebay/My Project Name/")
start_up_script = importlib.util.spec_from_file_location("setup.py", setup_path)
module = importlib.util.module_from_spec(start_up_script)
Running the above snippet returns:
AttributeError: 'NoneType' object has no attribute 'loader'
I subsequently investigated by running type(start_up_script) the result it gives is typeNone.
The paths are correct. I verified this by running the following:
"/home/solebay/My Project Name/"
sudo python3 "/home/solebay/My Project Name/setup.py"
These printed the messages is a directory and success! respectively.
Note: Maurice Meyer succeeded in answering my main question, and so I have marked it as correct. However, I have not achieved my main goal; namely importing modules via another script. So if that is your aim, this question might not be appropriate for you.
The location argument passed to spec_from_file_location has to be the full path to the python script:
import importlib.util
spec = importlib.util.spec_from_file_location(
name='something__else', # name is not related to the file, it's the module name!
location='/tmp/solebay/My Project Name/setup.py' # full path to the script
)
my_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(my_mod)
print(my_mod)
Out:
success!
<module 'something__else' from '/tmp/solebay/My Project Name/setup.py'>
Related
the main module sits within the runner package and executes stuff in the other packages. The main module can also Update the other packages and when that happens I want to reload them in order to get the new functions/modules that were added to those packages.
Project Structure
|--runner
|----main.py
|--core
|----module_1.py
|--configurations
|--utils
But that doesn't work.
I tried the following commands:
importlib.reload - only reloads a single module, using it recursively with sys.modules didn't add the new modules to the import tree. example: if after the update, "core" received a new module "module_new.py" and its imported in "module_1.py" it's not recognized after the reload.
I tried using IPython.lib.deepreload - it didn't work as well.
I've been stuck with this issue for some time, and haven't found any working solution yet.
Suggestions? Thanks
I fixed the issue by restarting the entire program using a while loop from an outer execution script.
Exit code 2: update required
Do
{
$process = Start-Process python -ArgumentList $CommandLine -verb RunAs -PassThru -WindowStyle Minimized -Wait
} WHILE ($process.ExitCode -eq 2)
Modules will be reloaded by import command if they are not in sys.modules dict
# import some standard (non-updatable) modules
import numpy as np
# save set of non-reloadable modules on first run,
# and delete reloadable modules on other runs
if 'init_modules' not in globals():
init_modules = set(sys.modules.keys())
else:
modules = list(sys.modules.keys())
for m in modules:
if m not in init_modules:
del(sys.modules[m])
# import reloadable packages and modules
import MyPackage
i'm using python 2.7 and trying to gather documentation for our testing project using pdoc.
pdoc is located here: D:\dev\Python27\Scripts
the regression project here: C:\views\md_LDB_RegressionTests_v03.1_laptop\mts\Tests\LDB\Regression\Tests
We are using proboscis for our tests and i'm trying to create html documentation for the separate group of tests, a separate python file in my case.
I run such command:
D:\dev\Python27\Scripts>python pdoc --html "C:\views\md_LDB_RegressionTests_v03.
1_laptop\mts\Tests\LDB\Regression\Tests\tests\check_system_management\check_capa
bilities_encoding_problems.py"
and get such answer:
Traceback (most recent call last):
File "pdoc", line 458, in <module>
module = imp.load_source('__pdoc_file_module__', fp, f)
File "C:\views\md_LDB_RegressionTests_v03.1_laptop\mts\Tests\LDB\Regression\Te
sts\tests\check_system_management\check_capabilities_encoding_problems.py", line
4, in <module>
from common.builders.system_request import default_create_system, create_cap
ability
ImportError: No module named common.builders.system_request
pdoc can't import the function from other modules in our regression...
The structure of our project looks like this:
-Tests (C:\views\md_LDB_RegressionTests_v03.1_laptop\mts\Tests\LDB\Regression\Tests)
-"common" package (with init file)
-"builders" packege
-system_request.py
-"test" package
-check_system_management package
-check_capabilities_encoding_problems.py - this is the file i want to get documentation to
Of course there are lots of other packages but im not sure if it makes sense to describe all the structure now
The import part of the check_capabilities_encoding_problems.py looks like this:
import urllib
from hamcrest import assert_that, all_of
from proboscis import test, before_class, after_class
from common.builders.system_request import default_create_system, create_capability
from common.entity.LDBChecks import LDBChecks
How can i point to pdoc where to look for the functions of other modules?
thank you!
You can set PYTHONPATH env variable. This is a path that say python where to find modules and packages by 3th party also you.
When using pdoc with my Spyder IDE, I use the following script to add a directory to pdoc path
import pdoc
libpath = r'C:\Path\To\Module'
pdoc.import_path.append(libpath)
mod = pdoc.import_module('ModuleName')
doc = pdoc.Module(mod)
string = doc.html()
The pdoc.import_path is a list of currently used paths to look for your module; pdoc.import_path equals sys.path in default. More info can be found in pdoc documentation.
pydoc and pdoc read your code!!!
if you will run it from the same directory pdoc3 --html . or pydoc -w . it should work if all the modules are in the same directory. but if they are not:
make sure your main module in each directory has it sys full path append to it (to the same directory).
sys.path.append("D:/Coding/project/....)
Relative path will not do the trick!
I was trying to create a class in python with 'RawIOBase' as given below.
try:
import io
except ImportError:
class Serial(PosixSerial, FileLike):
pass
else:
class Serial(PosixSerial, io.RawIOBase):
pass
I was trying to run this using Python 2.6, but it is displaying the error:
AttributeError: 'module' object has no attribute 'RawIOBase'
I found the reason for this error is one more directory with name 'io' exists and 'init.py' file exists in the directory. so when I print using print(io), it is displaying
<module 'io' from '/projects/phx/tools/io/__init__.pyc'>.
And I found that
<module 'io' from '/usr/lib/python2.6/io.pyc'>
should be the path. But io.py file doesn't exist in /usr/lib/python2.6 path. So even though I exported PYHTONPATH to /usr/lib/python2.6, still when I import io, it is referring the local one instead of standard io module.
Can you please let me know where can I find the io.py file so that this program will be working.
Try this to see what you are using right now:
import io
print io.__file__
# /usr/lib/python2.7/io.pyc
the .py file should be in the same directory
You can use also use locate in shell to find all io.py versions:
/usr/lib/python2.7/io.py
/usr/lib/python2.7/io.pyc
/usr/lib/python2.7/dist-packages/IPython/utils/io.py
/usr/lib/python2.7/dist-packages/IPython/utils/io.pyc
# etc...
Keep in mind, that it might depend on how are you executing python, using different python version or - for example - IPython, can yield different results.
This concerns the importing of my own python modules in a HTCondor job.
Suppose 'mymodule.py' is the module I want to import, and is saved in directory called a XDIR.
In another directory called YDIR, I have written a file called xImport.py:
#!/usr/bin/env python
import os
import sys
print sys.path
import numpy
import mymodule
and a condor submit file:
executable = xImport.py
getenv = True
universe = Vanilla
output = xImport.out
error = xImport.error
log = xImport.log
queue 1
The result of submitting this is that, in xImport.out, the sys.path is printed out, showing XDIR. But in xImport.error, there is an ImporError saying 'No module named mymodule'. So it seems that the path to mymodule is in sys.path, but python does not find it. I'd also like to mention that error message says that the ImportError originates from the file
/mnt/novowhatsit/YDIR/xImport.py
and not YDIR/xImport.py.
How can I edit the above files to import mymodule.py?
When condor runs your process, it creates a directory on that machine (usually on a local hard drive). It sets that as the working directory. That's probably the issue you are seeing. If XDIR is local to the machine where you are running condor_submit, then it's contents don't exist on the remote machine where the xImport.py is running.
Try using the .submit feature transfer_input_files mechanism (see http://research.cs.wisc.edu/htcondor/manual/v7.6/2_5Submitting_Job.html) to copy the mymodule.py to the remote machines.
I can import the maya module with ease through the Python 2.7 IDE, but when working with Nuke's script editor, I cannot import Maya and get a "No module named maya" error
Any help?
well if you want to import maya modules you can add the path of "E:\Program Files\Autodesk\Maya2013\Python\Lib\site-packages" to your sys.path in nuke,
here is an explanation how to do it ..
first test the paths if its their by using the code below
import sys
[each for each in sys.path]
now in this list you can either insert in the beginning or append the path of your maya modules at the end by this line of code(the path below is in my case will be different on your computer)
sys.path.append("E:/Program Files/Autodesk/Maya2013/Python/Lib/site-packages")
when you are adding the path check with your slash should be either // or / ...
then again test if the path is added by using the code i give you in the beginning...
In addition to using the system module, you can append a path with the nuke module itself:
nuke.pluginAddPath('C:/Program Files/Autodesk/Maya2016/Python/Lib/site-packages/')
And to test:
import maya