I would like to know if it is possible to execute a python2 script from a python3 script.
I have a file written using py3 that must execute legacy code written in py2 to obtain dictionaries for processing within the initial file.
The line in py3 to call the mentioned py2 script is
exec(open('python2script.py').read())
The script runs without error until it begins processing python2script.py, at which point it crashes at the first difference with version3.
As the comments pointed out, exec() uses the current python implementation, so you can't execute python 2 code from python 3 using it.
Unless you port it, your best bet is simply to call it as a subprocess, using either os.system..:
./py3.py
#!/usr/bin/env python3
import os
print('running py2')
os.system('./py2.py')
print('done')
./py2.py
#!/usr/bin/env python2.7
print "hello from python2!"
Then (after making them both executable) run:
$ ./py3.py
Or alternatively you can use the more flexible subprocess, which allows you to pass data back and forward more easily using a serialising module such as json so that you can get your results from the python2 script in your python3 code:
./py3.py
#!/usr/bin/env python3
import json
from subprocess import PIPE, Popen
print('running py2')
py2_proc = Popen(['./py2.py'], stdout=PIPE)
# do not care about stderr
stdout, _ = py2_proc.communicate()
result = json.loads(stdout.decode())
print('value1 was %s, value2 was %s' % (result['value1'], result['value2']))
./py2.py
#!/usr/bin/env python2.7
import json
my_result = {
'value1': 1,
'value2': 3
}
print json.dumps(my_result)
Like that it may be easy to pack up the data you need and transport it over.
Note: I have used a very simple environment setup here using my system's python2.7 and python3. In the real world the most painful thing about getting this sort of thing to work properly is configuring the environment correctly. Perhaps, e.g., you are using virtual environments. Perhaps you are running as a user which doesn't have the right python2 version in their path. Perhaps you can't make the files executable and so have to specify the path to python in your subprocess / os.system call. There are many options and it is very complicated, but out of the scope of the question. You just have to read the doc pages very carefully and try a few things out!
Related
This question already has answers here:
Actual meaning of 'shell=True' in subprocess
(7 answers)
Closed 8 months ago.
I would like to run a shell script from Python 3 in Linux passing two arguments that contain file paths to two different files. The shell script then calls a programme written in Python 2.
In Python 3, I call the shell script like this:
import os
import sys
os.chmod('/path/to/sh', 0o777)
subprocess.call(['/path/to/sh', '/path/to/file1', '/path/to/file2'], shell=True)
My shell script looks like this:
#!/bin/sh
set -x
path1=$1
path2=$2
python2 path/to/programme "$path1" "$path2"
Now, the file paths are empty, and the shell script returns something like python2 path/to/programme '' ''. Does someone know how I could correctly pass the file paths so that the programme written in Python 2 can read them?
Or is there even an easier solution such as using subprocess to directly call the programme written in Python 2?
There is no need for the shell script. You can use subprocess to run python2 directly.
a.py
#!/usr/bin/env python3
import subprocess
subprocess.call(['python2', './b.py', 'foo', 'bar'])
b.py
#!/usr/bin/env python2
import sys
print sys.argv
Running ./a.py outputs ['./b.py', 'foo', 'bar'].
You could also try using past.translation instead:
The past module provides an experimental translation package to help with importing and using old Python 2 modules in a Python 3 environment.
shell=True is only needed if you do something like
subprocess.run("/path/to/sh /path/to/file1 /path/to/file2", shell=True)
where the shell will split the single string into arguments that will identify as the program name and its arguments. But you already have the program name and its arguments identified, so
subprocess.run(['/path/to/sh', '/path/to/file1', '/path/to/file2'])
is all you need.
By using a list and shell=True, you are essentially asking Python to execute
sh -c /path/to/sh /path/to/file1 /path/to/file2
which uses /path/to/file1 to set the value of $0, not $1, in the command to execute.
I can run one program by typing: python enable_robot.py -e in the command line, but I want to run it from within another program.
In the other program, I imported subprocess and had subprocess.Popen(['enable_robot', 'baxter_tools/scripts/enable_robot.py','-e']), but I get an error message saying something about a callback.
If I comment out this line, the rest of my program works perfectly fine.
Any suggestions on how I could change this line to get my code to work or if I shouldn't be using subprocess at all?
If enable_robot.py requires user input, probably it wasn't meant to run from another python script. you might want to import it as a module: import enable_robot and run the functions you want to use from there.
If you want to stick to the subprocess, you can pass input with communicate:
p = subprocess.Popen(['enable_robot', 'baxter_tools/scripts/enable_robot.py','-e'])
p.communicate(input=b'whatever string\nnext line')
communicate documentation, example.
Your program enable_robot.py should meet the following requirements:
The first line is a path indicating what program is used to interpret
the script. In this case, it is the python path.
Your script should be executable
A very simple example. We have two python scripts: called.py and caller.py
Usage: caller.py will execute called.py using subprocess.Popen()
File /tmp/called.py
#!/usr/bin/python
print("OK")
File /tmp/caller.py
#!/usr/bin/python
import subprocess
proc = subprocess.Popen(['/tmp/called.py'])
Make both executable:
chmod +x /tmp/caller.py
chmod +x /tmp/called.py
caller.py output:
$ /tmp/caller.py
$ OK
I would like to create a file that will be used as standard input for a python script, and invoke said script with subprocess.call.
When I do it directly in the command line it works fine:
The input file:
# test_input
1/2/3
The python script
# script.py
thisDate = input('Please enter date: ').rstrip()
The following command works just fine:
python script.py < test_input
But when I try to do the following from within another python script, it doesn't work. (from this)
outfile1 = open('test_input', 'w')
outfile1.write('1/2/3')
outfile1.close()
input1 = open('test_input')
subprocess.call(['python', 'script.py'], stdin=input1)
But then I get the following error:
>>>thisDate = input('Please enter date: ').rstrip()
>>>AttributeError: 'int' object has no attribute 'rstrip'
When I did some debugging, it seems that it is getting the integer 0 as the input.
What is causing the inconsistency here? Are the two methods not equivalent (evidently they are not, but why)? My ultimate goal is to perform the exact same task as the above command line version that worked.
Thank you
You are using input when it should be raw_input, input in python2 will eval the string. If you run the script with python3 it will work as is, for python2 change to raw_input.
Using check_call is usually a better approach and using with to open your files.
import subprocess
with open('test_input') as input1:
subprocess.check_call(['python3', 'script.py'], stdin=input1)
So chepner was correct. When I amended the following line:
subprocess.call(['python', 'script.py'], stdin=input1)
to:
subprocess.call(['python3', 'script.py'], stdin=input1)
it worked just fine.
(I am trying to do this in python3)
In the first instance, the file has two lines, and input() reads and parses the first line, which is a comment.
In the second case, the comment line is missing, so Python reads and parses a number.
You probably meant to use raw_input(), or run the script with Python 3.
(You probably also meant for the input file to end with a newline, and it doesn't really make sense to use subprocess.call() to run Python when you are already running Python.)
python script.py < test_input command should fail. You might mean: python3 script.py < test_input instead due to the difference between input() vs raw_input() on Python 2 as mentioned in other answers. python as a rule refers to Python 2 version.
if the parent script is run only using python3 then you could use sys.executable to run the child script using the same python version (the same executable):
#!/usr/bin/env python3
import subprocess
import sys
with open('test_input', 'rb', 0) as input_file:
subprocess.check_call([sys.executable or 'python3', 'script.py'],
stdin=input_file)
If the parent and the child may use different python versions then set the correct shebang in script.py e.g., #!/usr/bin/env python3 and run the script directly:
#!/usr/bin/env python
import subprocess
with open('test_input', 'rb', 0) as input_file:
subprocess.check_call(['./script.py'], stdin=input_file)
Here, the child script may choose its own python version. Make sure the script has executable permissions: chmod +x script.py. Note: Python Launcher for Windows understands the shebang syntax too.
Unrelated: use .communicate() instead of outfile1.write('1/2/3'):
#!/usr/bin/env python3
from subprocess import Popen, PIPE
with Popen(['./script.py'], stdin=PIPE, universal_newlines=True) as p:
p.communicate('1/2/3')
I have a fabric script called fwp.py that I run without calling it throug fab by using:
if __name__ == '__main__':
# imports for standalone mode only
import sys
import fabric.main
fabric.main.main(fabfile_locations=[__file__])
The thing is then have to call the script by calling fwp.py. I'd like to rename it as fwp to be able to call it as fwp. But doing that would result in
Fatal error: Couldn't find any fabfiles!
Is there a way to make Python/Fabric import this file, despite the lack of a ".py" extension?
To reiterate and clarify:
I'm not using the "fab" utility (e.g. as fab task task:parameter); just calling my script as fwp.py task task:parameter, and would like to be able to call it as fwp task task:parameter.
Update
It's not a duplicate of this question. The question is not "How to run a stand-alone fabric script?", but "How to do so, while having a script without a .py" extension.
EDIT: Original answer corrected
The fabric.main.main() function automatically adds .py to the end of supplied fabfile locations (see https://github.com/fabric/fabric/blob/master/fabric/main.py#L93). Unfortunately that function also uses Python's import machinery to load the file so it has to look like a module or package. Without reimplementing much of the fabric.main module I don't think it will be possible. You could try monkey-patching both fabric.main.find_fabfiles and fabric.main.load_fabfiles to make it work.
Origininal answer (wrong)
I can get this to work unaltered on a freshly installed fabric package. The following will execute with a filename fwp and executable permission on version 1.10.1, Python2.7. I would just try upgrading fabric.
#!/usr/bin/env python
from fabric.api import *
import fabric.main
def do():
local('echo "Hello World"')
if __name__ == '__main__':
fabric.main.main(fabfile_locations=[__file__])
Output:
$ ./fwp do
Hello World
Done
I have a python script test.py I'm working on. I'd like to make edits and re-run it. I'm using Terminal on OSX to run the script. I can't get the script to run a second time without quitting out of terminal and starting it again.
# test.py
print "Howdy"
Terminal window:
$ python
>>> import test
Howdy
>>> import test
>>>
Question 1: How do I get the script to run again?
Question 2: Or is python designed to work like this:
# test.py
def printStuff():
print "Howdy"
Terminal:
$ python
>>> import test
>>> test.printStuff()
Howdy
>>> test.printStuff()
Howdy
>>>
1: you can use reload(moduleName) to do what you're trying to do (but see below).
2: There's a few different patterns, but I typically have a main() function inside all of my modules that have a clear "start point", or else I'll just have a bunch of library functions. So more or less what you're thinking in your example. You're not really supposed to "do stuff" on import, unless it's setting up the module. Think of a module as a library, not a script.
If you want to execute it as a script (in which case you shouldn't be using import), then there's a couple options. You can use a shebang at the top of your python script (Should I put #! (shebang) in Python scripts, and what form should it take?) and execute it directly from the command line, or you can use the __main__ module in your script as an entry point (https://docs.python.org/2/library/main.html), and then call the script with python from the command line, e.g. python myscript.
Use Python's reload() method:
https://docs.python.org/2/library/functions.html#reload
You want to use the reload method
>>> import test
Howdy
>>> reload(test)
Howdy