Develop Reference

Pass a path-variable to a subprocess in Python [duplicate]

This question already has answers here:
How can I specify working directory for a subprocess
(2 answers)
Closed 8 months ago.
im new to subprocessing and I have a question.
I can't find a proper solution online.
I want my path to be in a variable --> that will be passed to a function --> that will be passed to a subprocess.
I'm not allowed to show my real code, but this simple example (that I just can't get to work) would help me a lot.
This code snippet should do:
Just take my path from a variable.
"cd" the path in CMD.
Open a file that is located in this path.
So far I tried:
import subprocess
test_path = "C:/randome_path/.."
def Test_Function(test_path):
subprocess.call("cd", test_path, shell = True)
subprocess.call("python file.py", shell = True)
Test_Function()
My ErrorMessage is:
TypeError: Test_Function() missing 1 required positional argument: 'test_path'
Thank you for your time!

First you need to pass a parameter to your function because that's how you declarerd:
Test_Function(test_path) # here the function call with parameter
or using the key-value "approach"
another_path = # ...
Test_Function(test_path=another_path)
Second: the command is expecting a string not a further parameter
subprocess.call(f"python file.py", shell=True, cwd=test_path)
Note 1 to execute such command python file.py it is assumed that python's path is declared in some environment variable
Note 2 that subprocess may have some different "behaviour" under Windows
Try without shell=True. Now the commands should be given as a list of strings
def Test_Function(test_path):
subprocess.call(["python", "file.py"], cwd=test_path)

subprocess "echo $ENVIRONMENT_VAR" outputs "$ENVIRONMENT_VAR" [duplicate]

This question already has answers here:
How to use an existing Environment variable in subprocess.Popen()
(3 answers)
Closed 4 years ago.
Writing a python program running in a linux environment.
I cannot use paramiko etc in this environment.
I have written a series of methods to interact with the command line the one with the issue...
import subprocess
def echo(self, echo_arg):
cmd = subprocess.Popen(["echo", echo_arg], stdout=subprocess.PIPE)
return cmd.communicate()[0]
In linux I have an envionment variable UPFW_WORK_PATH
when I later call...
self.echo("$UPFW_WORK_PATH")
the console output returned is literally :
$UPFW_WORK_PATH
however when I type into the terminal...
echo $UPFW_WORK_PATH
I am returned (not actual path names):
/example/file/path
What is causing this discrepancy between manually typing "echo" to the terminal and my python method calling echo by subprocess

When you run echo $x from the shell, it is the shell that expands the variable into its value. So if the value of x is 5, for example, the argument that echo receives is 5. It will never know about the variable.
So the solution is to retrieve the value of the environment variable in your python program and pass that value to echo:
import subprocess
import os
echo_arg = os.environ['UPFW_WORK_PATH']
cmd = subprocess.Popen(["echo", echo_arg], stdout=subprocess.PIPE)

Passing a Python Variable to Batch File [duplicate]

This question already has answers here:
Why does passing variables to subprocess.Popen not work despite passing a list of arguments?
(5 answers)
Closed 1 year ago.
I have a basic batch file that takes user input:
#echo off
set /p Thing= Type Something:
echo %Thing%
pause
However, I'd like to use a variable written in Python to pass into the batch file. Let's say just a string 'arg1' This is just a basic example, but I still cannot figure it out. The below code will run the batch process, but 'arg1' has no impact
import subprocess
filepath = r'C:\Users\MattR\Desktop\testing.bat'
subprocess.call([filepath, 'arg1'])
I have also tried p = subprocess.Popen([filepath, 'arg1']) but the batch file does not run in Python.
I have searched the web and SO, but none of the answers seem to work for me. Here are some links I've also tried: Example 1, Example 2. I've also tried others but they seem fairly specific to the user's needs.
How do I start passing Python variables into my batch files?

Your subprocess likely needs to run with a shell if you want bash to work properly
Actual meaning of 'shell=True' in subprocess
so
subprocess.Popen([filepath, 'arg1'], shell=True)
If you want to see the output too then:
item = subprocess.Popen([filepath, 'arg1'], shell=True, stdout=subprocess.PIPE)
for line in item.stdout:
print line
As a further edit here's a working example of what you're after:
sub.py:
import subprocess
import random
item = subprocess.Popen(["test.bat", str(random.randrange(0,20))] ,
shell=True, stdout=subprocess.PIPE)
for line in item.stdout:
print line
test.bat
#echo off
set arg1=%1
echo I wish I had %arg1% eggs!
running it:
c:\code>python sub.py
I wish I had 8 eggs!
c:\code>python sub.py
I wish I had 5 eggs!
c:\code>python sub.py
I wish I had 9 eggs!

Here is how I managed to call a variable from python to batch file.
First, make a python file like this:
import os
var1 = "Hello, world!"
os.putenv("VAR1", var1) #This takes the variable from python and makes it a batch one
Second, make your batch file, by going to the folder where you want your python program to work, then right-clicking in the map, then create new text file. In this text file, write whatever you want to do with the variable and make sure you call your variable using %...% like so:
echo %VAR1%
Save this file as a batch file like so: file>save as>name_of_file.bat then select: save as file: all files.
Then to call your batch file in python, write:
os.system("name_of_file.bat")
Make sure all these files are in the same map for them to work!
There you go, this worked for me, hopefully I can help some people with this comment, because I searched for so long to find how this works.
PS: I also posted on another forum, so don't be confused if you see this answer twice.

Import bash variables from a python script

I have seen plenty examples of running a python script from inside a bash script and either passing in variables as arguments or using export to give the child shell access, I am trying to do the opposite here though.
I am running a python script and have a separate file, lets call it myGlobalVariables.bash
myGlobalVariables.bash:
foo_1="var1"
foo_2="var2"
foo_3="var3"
My python script needs to use these variables.
For a very simple example:
myPythonScript.py:
print "foo_1: {}".format(foo_1)
Is there a way I can import them directly? Also, I do not want to alter the bash script if possible since it is a common file referenced many times elsewhere.

If your .bash file is formatted as you indicated - you might be able to just import it direct as a Python module via the imp module.
import imp
bash_module = imp.load_source("bash_module, "/path/to/myGlobalVariables.bash")
print bash_module.foo_1

You can also use os.environ:
Bash:
#!/bin/bash
# works without export as well
export testtest=one
Python:
#!/usr/bin/python
import os
os.environ['testtest'] # 'one'

I am very new to python, so I would welcome suggestions for more idiomatic ways to do this, but the following code uses bash itself to tell us which values get set by first calling bash with an empty environment (env -i bash) to tell us what variables are set as a baseline, then I call it again and tell bash to source your "variables" file, and then tell us what variables are now set. After removing some false-positives and an apparently-blank line, I loop through the "additional" output, looking for variables that were not in the baseline. Newly-seen variables get split (carefully) and put into the bash dictionary. I've left here (but commented-out) my previous idea for using exec to set the variables natively in python, but I ran into quoting/escaping issues, so I switched gears to using a dict.
If the exact call (path, etc) to your "variables" file is different than mine, then you'll need to change all of the instances of that value -- in the subprocess.check_output() call, in the list.remove() calls.
Here's the sample variable file I was using, just to demonstrate some of the things that could happen:
foo_1="var1"
foo_2="var2"
foo_3="var3"
if [[ -z $foo_3 ]]; then
foo_4="test"
else
foo_4="testing"
fi
foo_5="O'Neil"
foo_6='I love" quotes'
foo_7="embedded
newline"
... and here's the python script:
#!/usr/bin/env python
import subprocess
output = subprocess.check_output(['env', '-i', 'bash', '-c', 'set'])
baseline = output.split("\n")
output = subprocess.check_output(['env', '-i', 'bash', '-c', '. myGlobalVariables.bash; set'])
additional = output.split("\n")
# these get set when ". myGlobal..." runs and so are false positives
additional.remove("BASH_EXECUTION_STRING='. myGlobalVariables.bash; set'")
additional.remove('PIPESTATUS=([0]="0")')
additional.remove('_=myGlobalVariables.bash')
# I get an empty item at the end (blank line from subprocess?)
additional.remove('')
bash = {}
for assign in additional:
if not assign in baseline:
name, value = assign.split("=", 1)
bash[name]=value
#exec(name + '="' + value + '"')
print "New values:"
for key in bash:
print "Key: ", key, " = ", bash[key]
Another way to do it:
Inspired by Marat's answer, I came up with this two-stage hack. Start with a python program, let's call it "stage 1", which uses subprocess to call bash to source the variable file, as my above answer does, but it then tells bash to export all of the variables, and then exec the rest of your python program, which is in "stage 2".
Stage 1 python program:
#!/usr/bin/env python
import subprocess
status = subprocess.call(
['bash', '-c',
'. myGlobalVariables.bash; export $(compgen -v); exec ./stage2.py'
]);
Stage 2 python program:
#!/usr/bin/env python
# anything you want! for example,
import os
for key in os.environ:
print key, " = ", os.environ[key]

As stated in #theorifice answer, the trick here may be that such formatted file may be interpreted by both as bash and as python code. But his answer is outdated. imp module is deprecated in favour of importlib.
As your file has extension other than ".py", you can use the following approach:
from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader
spec = spec_from_loader("foobar", SourceFileLoader("foobar", "myGlobalVariables.bash"))
foobar = module_from_spec(spec)
spec.loader.exec_module(foobar)
I do not completely understand how this code works (where there are these foobar parameters), however, it worked for me. Found it here.

Linux shell source command equivalent in python [duplicate]

This question already has answers here:
Emulating Bash 'source' in Python
(6 answers)
Closed 5 years ago.
Converting a shell script to python and trying to find the best way to perform the following. I need this as it contains environment variables I need read.
if [ -e "/etc/rc.platform" ];
then
. "/etc/rc.platform"
fi
I have the 'if' converted but not sure how to handle the . "/etc/rc.platform" as source is a shell command. So far I have the following
if os.path.isfile("/etc/rc.platform"):
print "exists" <just to verify the if if working>
<what goes here to replace "source /etc/rc.platform"?>
I've looked at subprocess and execfile without success.
The python script will need to access the environment variables set by rc.platform

A somewhat hackish solution is to parse the env output:
newenv = {}
for line in os.popen('. /etc/rc.platform >&/dev/null; env'):
try:
k,v = line.strip().split('=',1)
except:
continue # bad line format, skip it
newenv[k] = v
os.environ.update(newenv)
Edit: fixed split argument, thanks to #l4mpi

(Here's a demonstration of the solution crayzeewulf described in his comment.)
If /etc/rc.platform only contains environment variables, you can read them and set them as env vars for your Python process.
Given this file:
$ cat /etc/rc.platform
FOO=bar
BAZ=123
Read and set environment variables:
>>> import os
>>> with open('/etc/rc.platform') as f:
... for line in f:
... k, v = line.split('=')
... os.environ[k] = v.strip()
...
>>> os.environ['FOO']
'bar'
>>> os.environ['BAZ']
'123'

Too much work for the return. Going to keep a small shell script to get all the env vars that we need and forget reading them into python.

Try this:
if os.path.exists ("/etc/rc.platform"):
os.system("/etc/rc.platform")

Since source is a shell builtin, you need to set shell=True when you invoke subprocess.call
>>> import os
>>> import subprocess
>>> if os.path.isfile("/etc/rc.platform"):
... subprocess.call("source /etc/rc.platform", shell=True)
I'm not sure what you're trying to do here, but I still wanted to mention this: /etc/rc.platform might export some shell functions to be used by other scripts in rc.d. Since these are shell functions, they would be exported only to the shell instance invoked by subprocess.call() and if you invoke another subprocess.call(), these functions would not be available since you're spawning a fresh new shell to invoke the new script.