I need help to import a script, i did 2 codes, the first is a test with some prints and in the second i try to import them:
Code 1
# I make some print's to try import and show if it works
def first():
print('Test')
class phrase:
def second():
print('Hello')
def third():
print('World')
Code 2
import os
attempt = os.system(r"python C:\Users\Gabri\PycharmProjects\pythonProject\Imagens.py")
# Obviously isn't works =(
attempt.first()
But in code 2, when i did os.system(r"python C:\Users\Gabri\PycharmProjects\pythonProject\Imagens.py") nothing happen.
Someone can help me to import this code? ;-;
1° Code is in C:\Users\Gabri\PycharmProjects\pythonProject
2° in C:\Users\Gabri\PycharmProjects\pythonProject\Prática\Vamove
If you want to keep the files where they are,
You should by able to do this:
import importlib.util
spec = importlib.util.spec_from_file_location(
"name", "C:\\Users\\Gabri\\PycharmProjects\\pythonProject\\Imagens.py")
Imagens = importlib.util.module_from_spec(spec)
spec.loader.exec_module(Imagens)
And then run your commands like this:
Imagens.first()
The easiest is if you put both in the same folder and make that folder a python directory from which you can import your other code as modules. All you need for that is a file in the folder containing a blank __init__.py file. Then you can import it to another code file in the same folder using
from folder_name import Imagens
and you should be able to use Imagens functions like any other module
Ex: Imagens.first()
I am debugging code, which has this line:
run('python /home/some_user/some_repo/pyflights/usertools/import.py /home/some_user/some_repo/pyflights/config/index_import.conf flights.map --import')
run - is some analog of os.system
So, I want to run this code without using run function. I need to import my import.py file and run it with sys.args. But how can I do this?
from some_repo.pyflights.usertools import import
There is no way to import import because import is a keyword. Moreover, importing a python file is different from running a script because most scripts have a section
if __name__ == '__main__':
....
When the program is running as a script, the variable __name__ has value __main__.
If you are ready to call a subprocess, you can use
`subprocess.call(...)`
Edit: actually, you can import import like so
from importlib import import_module
mod = import_module('import')
however it won't have the same effect as calling the script. Notice that the script probably uses sys.argv, and this must be addressed too.
Edit: here is an ersatz that you can try if you really don't want a subprocess. I don't guarantee it will work
import shlex
import sys
import types
def run(args):
"""Runs a python program with arguments within the current process.
Arguments:
#args: a sequence of arguments, the first one must be the file path to the python program
This is not guaranteed to work because the current process and the
executed script could modify the python running environment in incompatible ways.
"""
old_main, sys.modules['__main__'] = sys.modules['__main__'], types.ModuleType('__main__')
old_argv, sys.argv = sys.argv, list(args)
try:
with open(sys.argv[0]) as infile:
source = infile.read()
exec(source, sys.modules['__main__'].__dict__)
except SystemExit as exc:
if exc.code:
raise RuntimeError('run() failed with code %d' % exc.code)
finally:
sys.argv, sys.modules['__main__'] = old_argv, old_main
command = '/home/some_user/some_repo/pyflights/usertools/import.py /home/some_user/some_repo/pyflights/config/index_import.conf flights.map --import'
run(shlex.split(command))
I have below py script to download the files from artifactory.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import tarfile
import urllib
from urllib import urlretrieve
import ConfigParser
Config = ConfigParser.ConfigParser()
Config.read('/vivek/release.conf')
code_version = Config.get('main', 'app_version')
os.chdir('/tmp/')
arti_st_url='http://repo.com/artifactory/libs-release- local/com/name/tgz/abc.ear/{0}/abc.ear-{0}.tar.gz'.format(code_version)
arti_st_name='abc.ear-{0}.tar.gz'.format(code_version)
arti_sl_url='http://repo.com/artifactory/libs-release- local/com/name/tgz/def.ear/{0}/def.ear-{0}.tar.gz'.format(code_version)
arti_sl_name='def.ear-{0}.tar.gz'.format(code_version)
urllib.urlretrieve(arti_st_url, arti_st_name)
urllib.urlretrieve(arti_sl_url, arti_sl_name)
oneEAR = 'abc.ear-{0}.tar.gz'.format(code_version)
twoEAR = 'def.ear-{0}.tar.gz'.format(code_version)
tar = tarfile.open(oneEAR)
tar.extractall()
tar.close()
tar1 = tarfile.open(twoEAR)
tar1.extractall()
tar1.close()
os.remove(oneEAR)
os.remove(twoEAR)
This script works perfectly, thanks to stackoverflow.
Here's the next question. There's a variable "protocol" in release.conf. If it's equal to "localcopy", there's an existing py script that does something. If the "protocol" is equal to "artifactory",
above script should be called and executed. How can I achieve it?
Note: I am a beginner in Python, but my tasks are not. So, please help me out guys.
You could simply use:
import os
os.system("script_path")
to execute the script file. But there should be a line called shebang in the very top of that script file, you want to execute. If your python interpreter would be in /usr/bin/python this would be:
#!/usr/bin/python
Assuming you are a Linux user.
In Windows shebang isn't supported. It determines what program to use running *.py file itself.
//Edit:
To call that two scripts depending on a property config value you could just make another script called for example runthis.py which contains instruction like:
protocol = Config.get('main', 'protocol')
if protocol == 'localcopy':
os.system('path_to_localcopy_script)
if protocol == 'antifactory':
os.system('path_to_other_script')
Dont forgot to import needed modules in that new script.
Then you just run script you just made.
That is one way to do this.
If you dont want to create additional script, then put that code you wrote in a function, like:
def main():
...
Your code
...
And on the very bottom of your script file write:
if __name__ = '__main__':
protocol = Config.get('main', 'app_version')
if protocol == 'localcopy':
main()
if protocol == 'antifactory':
os.system('path_to_other_script')
if __name__ = '__main__' would execute only if you run that script by yourself (not by call from an other sctipt for example)
I need to run a script foo.py, but I need to also insert some debugging lines to run before the code in foo.py. Currently I just put those lines in foo.py and I'm careful not to commit that to Git, but I don't like this solution.
What I want is a separate file bar.py that I don't commit to Git. Then I want to run:
python /somewhere/bar.py /somewhere_else/foo.py
What I want this to do is first run some lines of code in bar.py, and then run foo.py as __main__. It should be in the same process that the bar.py lines ran in, otherwise the debugging lines won't help.
Is there a way to make bar.py do this?
Someone suggested this:
import imp
import sys
# Debugging code here
fp, pathname, description = imp.find_module(sys.argv[1])
imp.load_module('__main__', fp, pathname, description)
The problem with that is that because it uses import machinery, I need to be on the same folder as foo.py to run that. I don't want that. I want to simply put in the full path to foo.py.
Also: The solution needs to work with .pyc files as well.
Python has a mechanism for running code at startup; the site module.
"This module is automatically imported during initialization."
The site module will attempt to import a module named sitecustomize before __main__ is imported.
It will also attempt to import a module named usercustomize if your environment instructs it to.
For example, you could put a sitecustomize.py file in your site-packages folder that contains this:
import imp
import os
if 'MY_STARTUP_FILE' in os.environ:
try:
file_path = os.environ['MY_STARTUP_FILE']
folder, file_name = os.path.split(file_path)
module_name, _ = os.path.splitext(file_name)
fp, pathname, description = imp.find_module(module_name, [folder])
except Exception as e:
# Broad exception handling since sitecustomize exceptions are ignored
print "There was a problem finding startup file", file_path
print repr(e)
exit()
try:
imp.load_module(module_name, fp, pathname, description)
except Exception as e:
print "There was a problem loading startup file: ", file_path
print repr(e)
exit()
finally:
# "the caller is responsible for closing the file argument" from imp docs
if fp:
fp.close()
Then you could run your script like this:
MY_STARTUP_FILE=/somewhere/bar.py python /somewhere_else/foo.py
You could run any script before foo.py without needing to add code to reimport __main__.
Run export MY_STARTUP_FILE=/somewhere/bar.py and not need to reference it every time
You can use execfile() if the file is .py and uncompyle2 if the file is .pyc.
Let's say you have your file structure like:
test|-- foo.py
|-- bar
|--bar.py
foo.py
import sys
a = 1
print ('debugging...')
# run the other file
if sys.argv[1].endswith('.py'): # if .py run right away
execfile(sys.argv[1], globals(), locals())
elif sys.argv[1].endswith('.pyc'): # if .pyc, first uncompyle, then run
import uncompyle2
from StringIO import StringIO
f = StringIO()
uncompyle2.uncompyle_file(sys.argv[1], f)
f.seek(0)
exec(f.read(), globals(), locals())
bar.py
print a
print 'real job'
And in test/, if you do:
$ python foo.py bar/bar.py
$ python foo.py bar/bar.pyc
Both, outputs the same:
debugging...
1
real job
Please also see this answer.
You probably have something along the lines of:
if __name__ == '__main__':
# some code
Instead, write your code in a function main() in foo and then do:
if __name__ == '__main__':
main()
Then, in bar, you can import foo and call foo.main().
Additionaly, if you need to change the working directory, you can use the os.chdir(path) method, e.g. os.chdir('path/of/bar') .
bar.py would have to behave like the Python interpreter itself and run foo.py (or foo.pyc, as you asked for that) as if it was the main script. This is surprisingly hard. My 90% solution to do that looks like so:
def run_script_as_main(cmdline):
# It is crucial to import locally what we need as we
# later remove everything from __main__
import sys, imp, traceback, os
# Patch sys.argv
sys.argv = cmdline
# Clear the __main__ namespace and set it up to run
# the secondary program
maindict = sys.modules["__main__"].__dict__
builtins = maindict['__builtins__']
maindict.clear()
maindict['__file__'] = cmdline[0]
maindict['__builtins__'] = builtins
maindict['__name__'] = "__main__"
maindict['__doc__'] = None
# Python prepends a script's location to sys.path
sys.path[0] = os.path.dirname(os.path.abspath(cmdline[0]))
# Treat everything as a Python source file, except it
# ends in '.pyc'
loader = imp.load_source
if maindict["__file__"].endswith(".pyc"):
loader = imp.load_compiled
with open(cmdline[0], 'rb') as f:
try:
loader('__main__', maindict["__file__"], f)
except Exception:
# In case of an exception, remove this script from the
# stack trace; if you don't care seeing some bar.py in
# traceback, you can leave that out.
ex_type, ex_value, ex_traceback = sys.exc_info()
tb = traceback.extract_tb(ex_traceback, None)[1:]
sys.stderr.write("Traceback (most recent call last):\n")
for line in traceback.format_list(tb):
sys.stderr.write(line)
for line in traceback.format_exception_only(ex_type, ex_value):
sys.stderr.write(line)
This mimics the default behavior of the Python interpreter
relatively closely. It sets system globals __name__, __file__,
sys.argv, inserts the script location into sys.path, clears the global namespace etc.
You would copy that code to bar.py or import it from somewhere and use it like so:
if __name__ == "__main__":
# You debugging setup goes here
...
# Run the Python program given as argv[1]
run_script_as_main(sys.argv[1:])
Then, given this:
$ find so-foo-bar
so-foo-bar
so-foo-bar/debugaid
so-foo-bar/debugaid/bar.py
so-foo-bar/foo.py
and foo.py looking like this:
import sys
if __name__ == "__main__":
print "My name is", __name__
print "My file is", __file__
print "My command line is", sys.argv
print "First 2 path items", sys.path[:2]
print "Globals", globals().keys()
raise Exception("foo")
... running foo.py via bar.py gives you that:
$ python so-foo-bar/debugaid/bar.py so-foo-bar/foo.py
My name is __main__
My file is so-foo-bar/foo.py
My command line is ['so-foo-bar/foo.py']
First 2 path items ['~/so-foo-bar', '/usr/local/...']
Globals ['__builtins__', '__name__', '__file__', 'sys', '__package__', '__doc__']
Traceback (most recent call last):
File "so-foo-bar/foo.py", line 9, in
raise Exception("foo")
Exception: foo
While I'd guess that run_script_as_main is lacking in some interesting details, it's pretty close to the way Python would run foo.py.
The foo.py need not be on the same folder as the main folder. Use
export PYTHONPATH=$HOME/dirWithFoo/:$PYTHONPATH
To add the path that foo resides to Python path. Now you can
from foo import myfunc
myfunc()
And make myfunc do whatever u want
Have you tried this?
bar.py
imp.load_source('__main__', '../../foo.py')
for me it runs whatever is in foo.py (before and inside main)
After, maybe what you do in bar.py might be more tricky that my basic test.
The good thing with load_source is that it imports the .pyc if it already exists or initiates a "compile" of the .py and then imports the .pyc.
This solution is what ended up working well for me:
#!/usr/bin/env python
import imp
import sys
import os.path
file_path = sys.argv.pop(1)
assert os.path.exists(file_path)
folder, file_name = os.path.split(file_path)
###############################################################################
# #
# Debugging cruft here
# #
###############################################################################
module_name, _ = os.path.splitext(file_name)
sys.path.append(folder)
imp.load_module('__main__', *imp.find_module(module_name))
I've been playing around with IPython.parallel and I wanted to use some custom modules of my own, but haven't been able to do it as explained on the cookbook using dview.sync_imports(). The only thing that has worked for me was something like
def my_parallel_func(args):
import sys
sys.path.append('/path/to/my/module')
import my_module
#and all the rest
and then in the main just to
if __name__=='__main__':
#set up dview...
dview.map( my_parallel_func, my_args )
The correct way to do this would in my opinion be something like
with dview.sync_imports():
import sys
sys.path.append('/path/to/my/module')
import my_module
but this throws an error saying there is no module named my_module.
So, what is the right way of doing it using dview.sync_imports()??
The problem is that you're changing the PYTHONPATH just in the local process running the Client, and not in the remote processes running in the ipcluster.
You can observe this behaviour if you run the next piece of code:
from IPython.parallel import Client
rc = Client()
dview = rc[:]
with dview.sync_imports():
import sys
sys.path[:] = ['something']
def parallel(x):
import sys
return sys.path
print 'Local: ', sys.path
print 'Remote: ', dview.map_sync(parallel, range(1))
Basically all the modules that you want to use with sync_imports must already be in the PYTHONPATH.
If it's not in the PYTHONPATH then you must add it to the path in the function that you execute remotely, and then import the module in the function.