I have two scripts, the main is in Python 3, and the second one is written in Python 2 (it also uses a Python 2 library).
There is one method in the Python 2 script I want to call from the Python 3 script, but I don't know how to cross this bridge.
Calling different python versions from each other can be done very elegantly using execnet. The following function does the charm:
import execnet
def call_python_version(Version, Module, Function, ArgumentList):
gw = execnet.makegateway("popen//python=python%s" % Version)
channel = gw.remote_exec("""
from %s import %s as the_function
channel.send(the_function(*channel.receive()))
""" % (Module, Function))
channel.send(ArgumentList)
return channel.receive()
Example: A my_module.py written in Python 2.7:
def my_function(X, Y):
return "Hello %s %s!" % (X, Y)
Then the following function calls
result = call_python_version("2.7", "my_module", "my_function",
["Mr", "Bear"])
print(result)
result = call_python_version("2.7", "my_module", "my_function",
["Mrs", "Wolf"])
print(result)
result in
Hello Mr Bear!
Hello Mrs Wolf!
What happened is that a 'gateway' was instantiated waiting
for an argument list with channel.receive(). Once it came in, it as been translated and passed to my_function. my_function returned the string it generated and channel.send(...) sent the string back. On other side of the gateway channel.receive() catches that result and returns it to the caller. The caller finally prints the string as produced by my_function in the python 3 module.
You could run python2 using subprocess (python module) doing the following:
From python 3:
#!/usr/bin/env python3
import subprocess
python3_command = "py2file.py arg1 arg2" # launch your python2 script
process = subprocess.Popen(python3_command.split(), stdout=subprocess.PIPE)
output, error = process.communicate() # receive output from the python2 script
Where output stores whatever python 2 returned
Maybe to late, but there is one more simple option for call python2.7 scripts:
script = ["python2.7", "script.py", "arg1"]
process = subprocess.Popen(" ".join(script),
shell=True,
env={"PYTHONPATH": "."})
I am running my python code with python 3, but I need a tool (ocropus) that is written with python 2.7. I spent a long time trying all these options with subprocess, and kept having errors, and the script would not complete. From the command line, it runs just fine. So I finally tried something simple that worked, but that I had not found in my searches online. I put the ocropus command inside a bash script:
#!/bin/bash
/usr/local/bin/ocropus-gpageseg $1
I call the bash script with subprocess.
command = [ocropus_gpageseg_path, current_path]
process = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
output, error = process.communicate()
print('output',output,'error',error)
This really gives the ocropus script its own little world, which it seems to need. I am posting this in the hope that it will save someone else some time.
It works for me if I call the python 2 executable directly from a python 3 environment.
python2_command = 'C:\Python27\python.exe python2_script.py arg1'
process = subprocess.Popen(python2_command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()
python3_command = 'python python3_script.py arg1'
process = subprocess.Popen(python3_command.split(), stdout=subprocess.PIPE)
output, error = process.communicate()
I ended up creating a new function in the python3 script, which wraps the python2.7 code. It correctly formats error messages created by the python2.7 code and is extending mikelsr's answer and using run() as recommended by subprocess docs.
in bar.py (python2.7 code):
def foo27(input):
return input * 2
in your python3 file:
import ast
import subprocess
def foo3(parameter):
try:
return ast.literal_eval(subprocess.run(
[
"C:/path/to/python2.7/python.exe", "-c", # run python2.7 in command mode
"from bar import foo27;"+
"print(foo27({}))".format(parameter) # print the output
],
capture_output=True,
check=True
).stdout.decode("utf-8")) # evaluate the printed output
except subprocess.CalledProcessError as e:
print(e.stdout)
raise Exception("foo27 errored with message below:\n\n{}"
.format(e.stderr.decode("utf-8")))
print(foo3(21))
# 42
This works when passing in simple python objects, like dicts, as the parameter but does not work for objects created by classes, eg. numpy arrays. These have to be serialized and re-instantiated on the other side of the barrier.
Note: This was happening when running my python 2.x s/w in the liclipse IDE.
When I ran it from a bash script on the command line it didn't have the problem.
Here is a problem & solution I had when mixing python 2.x & 3.x scripts.
I am running a python 2.6 process & needed to call/execute a python 3.6 script.
The environment variable PYTHONPATH was set to point to 2.6 python s/w, so it was choking on the followng:
File "/usr/lib64/python2.6/encodings/__init__.py", line 123
raise CodecRegistryError,\
This caused the 3.6 python script to fail.
So instead of calling the 3.6 program directly I created a bash script which nuked the PYTHONPATH environment variable.
#!/bin/bash
export PYTHONPATH=
## Now call the 3.6 python scrtipt
./36psrc/rpiapi/RPiAPI.py $1
Instead of calling them in python 3, you could run them in conda env batch by creating a batch file as below:
call C:\ProgramData\AnacondaNew\Scripts\activate.bat
C:\Python27\python.exe "script27.py"
C:\ProgramData\AnacondaNew\python.exe "script3.py"
call conda deactivate
pause
I recommend to convert the Python2 files to Python3:
https://pythonconverter.com/
Related
My perl script is at path:
a/perl/perlScript.pl
my python script is at path:
a/python/pythonScript.py
pythonScript.py gets an argument from stdin, and returns result to stdout. From perlScript.pl , I want to run pythonScript.py with the argument hi to stdin, and save the results in some variable. That's what I tried:
my $ret = `../python/pythonScript.py < hi`;
but I got the following error:
The system cannot find the path specified.
Can you explain the path can't be found?
The qx operator (backticks) starts a shell (sh), in which prog < input syntax expects a file named input from which it will read lines and feed them to the program prog. But you want the python script to receive on its STDIN the string hi instead, not lines of a file named hi.
One way is to directly do that, my $ret = qx(echo "hi" | python_script).
But I'd suggest to consider using modules for this. Here is a simple example with IPC::Run3
use warnings;
use strict;
use feature 'say';
use IPC::Run3;
my #cmd = ('program', 'arg1', 'arg2');
my $in = "hi";
run3 \#cmd, \$in, \my $out;
say "script's stdout: $out";
The program is the path to your script if it is executable, or perhaps python script.py. This will be run by system so the output is obtained once that completes, what is consistent with the attempt in the question. See documentation for module's operation.
This module is intended to be simple while "satisfy 99% of the need for using system, qx, and open3 [...]. For far more power and control see IPC::Run.
You're getting this error because you're using shell redirection instead of just passing an argument
../python/pythonScript.py < hi
tells your shell to read input from a file called hi in the current directory, rather than using it as an argument. What you mean to do is
my $ret = `../python/pythonScript.py hi`;
Which correctly executes your python script with the hi argument, and returns the result to the variable $ret.
The Some of the other answers assume that hi must be passed as a command line parameter to the Python script but the asker says it comes from stdin.
Thus:
my $ret = `echo "hi" | ../python/pythonScript.py`;
To launch your external script you can do
system "python ../python/pythonScript.py hi";
and then in your python script
import sys
def yourFct(a, b):
...
if __name__== "__main__":
yourFct(sys.argv[1])
you can have more informations on the python part here
I have seen plenty examples of running a python script from inside a bash script and either passing in variables as arguments or using export to give the child shell access, I am trying to do the opposite here though.
I am running a python script and have a separate file, lets call it myGlobalVariables.bash
myGlobalVariables.bash:
foo_1="var1"
foo_2="var2"
foo_3="var3"
My python script needs to use these variables.
For a very simple example:
myPythonScript.py:
print "foo_1: {}".format(foo_1)
Is there a way I can import them directly? Also, I do not want to alter the bash script if possible since it is a common file referenced many times elsewhere.
If your .bash file is formatted as you indicated - you might be able to just import it direct as a Python module via the imp module.
import imp
bash_module = imp.load_source("bash_module, "/path/to/myGlobalVariables.bash")
print bash_module.foo_1
You can also use os.environ:
Bash:
#!/bin/bash
# works without export as well
export testtest=one
Python:
#!/usr/bin/python
import os
os.environ['testtest'] # 'one'
I am very new to python, so I would welcome suggestions for more idiomatic ways to do this, but the following code uses bash itself to tell us which values get set by first calling bash with an empty environment (env -i bash) to tell us what variables are set as a baseline, then I call it again and tell bash to source your "variables" file, and then tell us what variables are now set. After removing some false-positives and an apparently-blank line, I loop through the "additional" output, looking for variables that were not in the baseline. Newly-seen variables get split (carefully) and put into the bash dictionary. I've left here (but commented-out) my previous idea for using exec to set the variables natively in python, but I ran into quoting/escaping issues, so I switched gears to using a dict.
If the exact call (path, etc) to your "variables" file is different than mine, then you'll need to change all of the instances of that value -- in the subprocess.check_output() call, in the list.remove() calls.
Here's the sample variable file I was using, just to demonstrate some of the things that could happen:
foo_1="var1"
foo_2="var2"
foo_3="var3"
if [[ -z $foo_3 ]]; then
foo_4="test"
else
foo_4="testing"
fi
foo_5="O'Neil"
foo_6='I love" quotes'
foo_7="embedded
newline"
... and here's the python script:
#!/usr/bin/env python
import subprocess
output = subprocess.check_output(['env', '-i', 'bash', '-c', 'set'])
baseline = output.split("\n")
output = subprocess.check_output(['env', '-i', 'bash', '-c', '. myGlobalVariables.bash; set'])
additional = output.split("\n")
# these get set when ". myGlobal..." runs and so are false positives
additional.remove("BASH_EXECUTION_STRING='. myGlobalVariables.bash; set'")
additional.remove('PIPESTATUS=([0]="0")')
additional.remove('_=myGlobalVariables.bash')
# I get an empty item at the end (blank line from subprocess?)
additional.remove('')
bash = {}
for assign in additional:
if not assign in baseline:
name, value = assign.split("=", 1)
bash[name]=value
#exec(name + '="' + value + '"')
print "New values:"
for key in bash:
print "Key: ", key, " = ", bash[key]
Another way to do it:
Inspired by Marat's answer, I came up with this two-stage hack. Start with a python program, let's call it "stage 1", which uses subprocess to call bash to source the variable file, as my above answer does, but it then tells bash to export all of the variables, and then exec the rest of your python program, which is in "stage 2".
Stage 1 python program:
#!/usr/bin/env python
import subprocess
status = subprocess.call(
['bash', '-c',
'. myGlobalVariables.bash; export $(compgen -v); exec ./stage2.py'
]);
Stage 2 python program:
#!/usr/bin/env python
# anything you want! for example,
import os
for key in os.environ:
print key, " = ", os.environ[key]
As stated in #theorifice answer, the trick here may be that such formatted file may be interpreted by both as bash and as python code. But his answer is outdated. imp module is deprecated in favour of importlib.
As your file has extension other than ".py", you can use the following approach:
from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader
spec = spec_from_loader("foobar", SourceFileLoader("foobar", "myGlobalVariables.bash"))
foobar = module_from_spec(spec)
spec.loader.exec_module(foobar)
I do not completely understand how this code works (where there are these foobar parameters), however, it worked for me. Found it here.
I have used both Python and C for a while. C is good in a way that i can use Windows cmd or anything like that to compile files and easily read command line arguments. However, the only thing that runs python that I know is IDLE which is like an interpreter and doesnt take command-line arguments and it's hard to work with. Is there anything like the C's cmd and a compiler for python 3.x?
Thanks
However, the only thing that runs python that I know is IDLE which is
like an interpreter
You can still call python helloworld.py from a command line
and doesnt take command-line arguments
It's possible to read commandline arguments from python helloworld.py Alex using:
import sys
name = sys.argv[1] # Gives "Alex", argv[0] would be "helloworld.py"
a compiler for python 3.x
py2exe supports Python 3
And finally if you're looking to call commands from your Python code, there is a module called subprocess
if i understand your question , you can do this in python by importing cmd , os
for example :
import os
import cmd
import readline
class Console(cmd.Cmd):
def __init__(self):
cmd.Cmd.__init__(self)
self.prompt = "=>> "
self.intro = "Welcome to console!" ## defaults to None
## Command definitions ##
def do_hist(self, args):
"""Print a list of commands that have been entered"""
print self._hist
def do_exit(self, args):
"""Exits from the console"""
return -1
## Command definitions to support Cmd object functionality ##
def do_EOF(self, args):
"""Exit on system end of file character"""
return self.do_exit(args)
def do_shell(self, args):
"""Pass command to a system shell when line begins with '!'"""
os.system(args)
def do_help(self, args):
"""Get help on commands
'help' or '?' with no arguments prints a list of commands for which help is available
'help <command>' or '? <command>' gives help on <command>
"""
## The only reason to define this method is for the help text in the doc string
cmd.Cmd.do_help(self, args)
## Override methods in Cmd object ##
def preloop(self):
"""Initialization before prompting user for commands.
Despite the claims in the Cmd documentaion, Cmd.preloop() is not a stub.
"""
cmd.Cmd.preloop(self) ## sets up command completion
self._hist = [] ## No history yet
self._locals = {} ## Initialize execution namespace for user
self._globals = {}
def postloop(self):
"""Take care of any unfinished business.
Despite the claims in the Cmd documentaion, Cmd.postloop() is not a stub.
"""
cmd.Cmd.postloop(self) ## Clean up command completion
print "Exiting..."
def precmd(self, line):
""" This method is called after the line has been input but before
it has been interpreted. If you want to modifdy the input line
before execution (for example, variable substitution) do it here.
"""
self._hist += [ line.strip() ]
return line
def postcmd(self, stop, line):
"""If you want to stop the console, return something that evaluates to true.
If you want to do some post command processing, do it here.
"""
return stop
def emptyline(self):
"""Do nothing on empty input line"""
pass
def default(self, line):
"""Called on an input line when the command prefix is not recognized.
In that case we execute the line as Python code.
"""
try:
exec(line) in self._locals, self._globals
except Exception, e:
print e.__class__, ":", e
if __name__ == '__main__':
console = Console()
console . cmdloop()
this example is for use command lines in python , however you can write your python code and call .py file in cmd by run this command :
python <file_name>.py
search more for other examples , also see official doc : cmd — Support for line-oriented command interpreters
You can use the python interpreter as a compiler too to compile your python programs.
Say you have a test.py file which you want to compile; then you can use python test.py to compile the file.
To be true, you are not actually compiling the file, you are executing it line by line (well, call it interpreting)
For command line arguments you can use sys.argv as already mentioned in the above answers.
Provided on how you have it installed, you can probably just run the python scripts as is, by typing the script file name, for example:
C:\> test.py
If you have a relatively recent python installation, this will be associated with the python launcher (py.exe) and be equivalent to running
C:\> py test.py
If you only have one version of python installed this will run it with this, but the python launcher supports multiple ways to customize how it behaves with multiple versions of python.
Additionally, and as stated above, you can run the script with just the python command as well. The main difference is that running it with the python command allows you to specify exactly which installation gets ran, using the script name alone (or the py.exe version), will allow the system to select which installation gets ran.
I'm using similar approach to call python function from my shell script:
python -c 'import foo; print foo.hello()'
But I don't know how in this case I can pass arguments to python script and also is it possible to call function with parameters from command line?
python -c 'import foo, sys; print foo.hello(); print(sys.argv[1])' "This is a test"
or
echo "Wham" | python -c 'print(raw_input(""));'
There's also argparse (py3 link) that could be used to capture arguments, such as the -c which also can be found at sys.argv[0].
A second library do exist but is discuraged, called getopt.getopt.
You don't want to do that in shell script.
Try this. Create a file named "hello.py" and put the following code in the file (assuming you are on unix system):
#!/usr/bin/env python
print "Hello World"
and in your shell script, write something lke this
#!/bin/sh
python hello.py
and you should see Hello World in the terminal.
That's how you should invoke a script in shell/bash.
To the main question: how do you pass arguments?
Take this simple example:
#!/usr/bin/env python
import sys
def hello(name):
print "Hello, " + name
if __name__ == "__main__":
if len(sys.argv) > 1:
hello(sys.argv[1])
else:
raise SystemExit("usage: python hello.py <name>")
We expect the len of the argument to be at least two. Like shell programming, the first one (index 0) is always the file name.
Now modify the shell script to include the second argument (name) and see what happen.
haven't tested my code yet but conceptually that's how you should go about
edit:
If you just have a line or two simple python code, sure, -c works fine and is neat. But if you need more complex logic, please put the code into a module (.py file).
You need to create one .py file.
And after you call it this way :
python file.py argv1 argv2
And after in your file, you have sys.argv list, who give you list of argvs.
I know that I can run a python script from my bash script using the following:
python python_script.py
But what about if I wanted to pass a variable / argument to my python script from my bash script. How can I do that?
Basically bash will work out a filename and then python will upload it, but I need to send the filename from bash to python when I call it.
To execute a python script in a bash script you need to call the same command that you would within a terminal. For instance
> python python_script.py var1 var2
To access these variables within python you will need
import sys
print(sys.argv[0]) # prints python_script.py
print(sys.argv[1]) # prints var1
print(sys.argv[2]) # prints var2
Beside sys.argv, also take a look at the argparse module, which helps define options and arguments for scripts.
The argparse module makes it easy to write user-friendly command-line interfaces.
Use
python python_script.py filename
and in your Python script
import sys
print sys.argv[1]
Embedded option:
Wrap python code in a bash function.
#!/bin/bash
function current_datetime {
python - <<END
import datetime
print datetime.datetime.now()
END
}
# Call it
current_datetime
# Call it and capture the output
DT=$(current_datetime)
echo Current date and time: $DT
Use environment variables, to pass data into to your embedded python script.
#!/bin/bash
function line {
PYTHON_ARG="$1" python - <<END
import os
line_len = int(os.environ['PYTHON_ARG'])
print '-' * line_len
END
}
# Do it one way
line 80
# Do it another way
echo $(line 80)
http://bhfsteve.blogspot.se/2014/07/embedding-python-in-bash-scripts.html
use in the script:
echo $(python python_script.py arg1 arg2) > /dev/null
or
python python_script.py "string arg" > /dev/null
The script will be executed without output.
I have a bash script that calls a small python routine to display a message window. As I need to use killall to stop the python script I can't use the above method as it would then mean running killall python which could take out other python programmes so I use
pythonprog.py "$argument" & # The & returns control straight to the bash script so must be outside the backticks. The preview of this message is showing it without "`" either side of the command for some reason.
As long as the python script will run from the cli by name rather than python pythonprog.py this works within the script. If you need more than one argument just use a space between each one within the quotes.
and take a look at the getopt module.
It works quite good for me!
Print all args without the filename:
for i in range(1, len(sys.argv)):
print(sys.argv[i])