I have a python script called script.py which I would like to run on all of the directories inside of a specific directory. What is the easiest way to do this using batch scripting?
My assumption is some usage of the FOR command, but I can't quite make any leads to solve the problem using the command...
Use the /d operator on for:
for /D %%d in (*) do script.py %%d
This assumes you only need to execute one command per directory (script.py %%d) if you need to execute more use braces (). Also I'm guessing there's an execution engine needed first, but not sure what it is for you.
A multi-line example:
for /D %%d in (%1) do (
echo processing %%d
script.py %%d
)
Building upon Rudu's answer, and assuming script.py is located in the same directory (say C:\Test) as the following batch script, you could also run script.py recursively on all the directories present in C:\Test:
#echo off
for /d /r %%d in (*) do (
echo Processing: %%d
script.py "%%d"
)
The FOR help available from cmd.exe does not seem to mention it supports multiple modifiers, but it works on Windows 7.
Related
Why does the return from running the --version check on Python 2 behave differently than that on Python 3? Below is an example of the two behaviors when writing their output to a text file and subsequently checking the contents of that file.
C:\Users\user1>C:\Python27\python.exe --version >> file2.txt
Python 2.7.13
C:\Users\user1>type file2.txt
C:\Users\user1>C:\Python38\python.exe --version >> file3.txt
C:\Users\user1>type file3.txt
Python 3.8.1
C:\Users\user1>
Is there any way to produce the same behavior as Python 3? I would like to use the output from the --version check command as part of a batch file to ensure the proper version is being used to execute a script.
Python 2 seems to output the version information to STDERR (handle 2) rather than STDOUT (handle 1), so change:
C:\Python27\python.exe --version >> file2.txt
to:
C:\Python27\python.exe --version 2> file2.txt
The redirection operator >> is used to append to a file; to (over-)write, use > instead.
To write both STDOUT and STDERR to the file, use this (replace ?? by 27 or 34):
C:\Python??\python.exe --version > "file.txt" 2>&1
The expression 2>&1 means to redirect handle 2 to the current target of handle 1, which is the given text file due to > "file.txt". Note that > is equivalent to 1>, since the default handle for output redirection is 1 (STDOUT). The order is important here (so 2>&1 > "file.txt" would fail).
This works also when appending to a file:
C:\Python??\python.exe --version >> "file.txt" 2>&1
To get the result directly into a variable without a (temporary) file, use a for /F loop – in cmd:
for /F "delims=" %V in ('C:\Python??\python.exe --version 2^>^&1') do set VAR=%V
and in a batch-file:
for /F "delims=" %%V in ('C:\Python??\python.exe --version 2^>^&1') do set VAR=%%V
Note the escaping using ^ that is necessary to avoid the redirection to be attempted too early.
To store only the pure version number into the variable without the word Python in front, replace delims= by tokens=2.
Let us even go a step further: You could let a batch-file search all available Python versions on its own. Given that the directory paths containing python.exe are listed in the system variable PATH, you could use where to get the full paths to the executables, then let for /F loop through them to get their exact versions (the version number is simply echoed out here just for demonstration):
#echo off
for /F "delims=" %%W in ('where python') do (
for /F "tokens=2" %%V in ('"%%W" --version 2^>^&1') do (
echo %%V
)
)
If the PATH variable does not contain the paths, you could alternatively search all paths C:\Python?? using for /D (here ?? is meant literally as wildcards); where just checks whether there is actually a file python.exe:
#echo off
for /D %%X in ("%SystemDrive%\Python??") do (
for /F "delims=" %%W in ('where "%%~X":python 2^> nul') do (
for /F "tokens=2" %%V in ('"%%W" --version 2^>^&1') do (
echo %%V
)
)
)
The Python 2 behavior was to send --version to stderr. This was changed to stdout in Python 3.4, but not backported, out of concern for being a breaking change:
https://bugs.python.org/issue18338
https://bugs.python.org/issue28160
https://docs.python.org/dev/whatsnew/3.4.html#changes-in-python-command-behavior
You can redirect stderr to ensure the same behavior, e.g. that Python MAJOR.MINOR.PATCH is written to file.
This is basically a duplicate of Change the current directory from a Bash script, except that I'm on windows and want to be able to change my command line directory from a powershell script.
I'm actually running a python script from the powershell file, parsing the output directory from that python script, and then using that output as the dir to change to, so if there's a way to do this from within the python script instead of the powershell file extra points!
To be 100% clear, yes I know about "cd" and "os.chdir", and no that's not what I want - that only changes the current directory for the script or batch file, NOT the command line that you are running from!
Code follows (batch file was my first attempt, I want it to be PS1 script):
sw.bat - parent dir added to sys $PATH
#echo off
set _PY_CMD_="python C:\code\switch\switch.py %*"
for /f "tokens=*" %%f in ('%_PY_CMD_%') do (
cd /d %%f
)
switch.py
import os
import argparse
LOCAL = r'C:\code\local'
def parse_args():
parser = argparse.ArgumentParser(description='Workspace manager tool.')
parser.add_argument('command', type=str, help='The command to execute, or workspace to switch to.')
return parser.parse_args()
def execute_command(command):
if command in ['l', 'local']:
print(LOCAL)
def main():
args = parse_args()
execute_command(args.command)
if __name__ == '__main__':
main()
If I got your question right, a simplistic approach would involve using [SS64]: FOR /F.
script.py:
target_dir = r"C:\Program Files"
print(target_dir)
script.bat:
#echo off
set _PY_CMD="e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" script.py
for /f "tokens=*" %%f in ('%_PY_CMD%') do (
pushd "%%f"
)
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057062514]> ver
Microsoft Windows [Version 10.0.18362.239]
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057062514]> script.bat
[cfati#CFATI-5510-0:C:\Program Files]>
[cfati#CFATI-5510-0:C:\Program Files]> popd
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057062514]>
Notes:
I used pushd (instead of cd) because simple popd would return to the original directory. The drawback is that one has to remember (the stack) how many times they called pushd in order to match the correspondent popds. If want to switch to cd, use it like: cd /d %%f
#EDIT0:
At the beginning, the question was for batch only, after that the PowerShell (PS) requirement (which can qualify as a different question) was added. Anyway, here's a simple (dummy) script that does the trick (I am not a PS expert).
script.ps1:
$PyCmdOutput = & 'e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe' 'script.py'
Set-Location $PyCmdOutput
Output:
PS E:\Work\Dev\StackOverflow\q057062514> .\script.ps1
PS C:\Program Files>
Try os.chdir in Python:
os.chdir(path)
Change the current working directory to path. Availability: Unix, Windows.
Here's what I would do if you want to do the directory changes purely in batch:
Create a home directory Var of the Batch file you are calling your Python from.
SET homeDir=%cd%
Then use cd to go into the desired directory. Once the program is in the directory, create a new command prompt using start "Title here (optional)" cmd.exe
Finally, return to the home directory with
cd "%homeDir%"
The script might look something like this:
#echo off
REM using cd with no arguments returns the current directory path
SET homeDir=%cd%
REM set home Directory
cd "%desiredDirectory%"
REM moved directory
start "thisIsAName" cmd.exe
REM created new "shell" if you will
cd "%homeDir%"
REM returned to home directory
pause
With the help of CristiFati, I got this working in a batch file. That solution is posted in the question. After some more googling, I've translated the batch file solution to a Powershell Script solution, which is below:
$command = "& python C:\code\switch\switch.py $args"
Invoke-Expression $command | Tee-Object -Variable out | Out-Null
set-location $out
No changes required to the Python file.
Note - this answer is provided for those who want to take in any number of arguments to the python script and suppress stdout when calling the python script.
I would like to build a python script, which can manipulate the state of it's calling bash shell, especially it's working directory for the beginning.
With os.chdir or os.system("ls ..") you can only change the interpreters path, but how can I apply the comments changes to the scripts caller then?
Thank you for any hint!
You can't do that directly from python, as a child process can never change the environment of its parent process.
But you can create a shell script that you source from your shell, i.e. it runs in the same process, and in that script, you'll call python and use its output as the name of the directory to cd to:
/home/choroba $ cat 1.sh
cd "$(python -c 'print ".."')"
/home/choroba $ . 1.sh
/home $
Below is the code in shell script
source /proj/common/tools/repo/etc/profile.d/repo.sh
repo project init $branch
repo project sync
source poky/fnc-init-build-env build
bitbake -g $image
I am trying to convert shell script into python script
a = subprocess.call("source /proj/common/tools/repo/etc/profile.d/repo.sh", shell=True)
b = subprocess.call("repo project init " + branch, shell=True)
b2 = subprocess.call("repo project sync", shell=True)
c = subprocess.call("source poky/fnc-init-build-env build", shell=True)
os.chdir("poky/build")
d = subprocess.call("bitbake -g " + image, shell=True)
But i am getting the following error
/bin/sh: bitbake: command not found
How to resolve this in python ?
you must add bitbake to path:
set Path=%path%;PathOfBitbake
run it on Command prompt of Windows then retry
The problem is, that you run subprocess.call(something, shell=True) several times and assume that the variables set in the first call are present in the later calls, which use shells independend from the earlier calls.
I would put the commands in a shell script and then run it with a single subprocess.call command. There seems to be no real point in converting it line by line into python by just running shell commands with the subprocess module.
When repo and bitbake are python programs there could be a point to import the relevant modules from them and run the corresponding python functions instead of the shell commands provided by their main method.
When using shell=True, the first list element is a script to run, and subsequent arguments are passed to that script.
Each subprocess.Popen() invocation starts a single shell; state configured in one isn't carried through to others, so the source command is useless unless you run the commands that depend on it within the same invocation.
script='''
branch=$1; shift # pop first argument off the list, assign to variable named branch
source /proj/common/tools/repo/etc/profile.d/repo.sh || exit
repo project init "$branch" || exit
repo project sync || exit
source poky/fnc-init-build-env build || exit
exec "$#" # use remaining arguments to form our command to run
'''
subprocess.call([
"bash", "-c", script, # start bash, explicitly: /bin/sh does not support "source"
"_", # becomes $0 inside the shell
branch, # becomes $1, which is assigned to branch and shifted off
"bitbake", "-g", image # these arguments become "$#" after the shift
])
Note the || exits -- you should generally have those on any command where you don't explicitly intend to ignore failures.
You need to add the path to 'bitbake', so it is found from your python script.
sys.path.append(your_path_there)
I have been trying to run multiple cmds to execute the same command with different arguments but cmd does not work. Here is the code to run the file
cd\
cd F:\A 5th Semester Data\CN\Project\
start cmd \k python test.py A 5000 configA.txt
The next statements will have B, 5001 configB.txt and so on.
Please Help in this matter.
Best Regards.
File and folder paths containing a space character must be enclosed in double quotes as explained in last paragraph of last help page output by running in a command prompt window cmd /?.
The help explains also parameter /K as used by you to keep the console window open after Windows command interpreter finished execution of python which means after Python finished execution of test.py.
And command CD requires the usage of parameter /D if the drive should be also switched as explained in help output by running in a command prompt window cd /?.
cd /D "F:\A 5th Semester Data\CN\Project"
start cmd /k python test.py A 5000 configA.txt
The help of command START is output on running in a command prompt window start /? which lists and explains also the parameter /Dpath.
So the two lines above could be also written as one-liner:
start "Python Processing" /D"F:\A 5th Semester Data\CN\Project" %SystemRoot%\System32\cmd.exe /K python.exe test.py A 5000 configA.txt
But it looks like what you really need is a batch file containing the lines:
cd /D "F:\A 5th Semester Data\CN\Project"
python.exe test.py A 5000 configA.txt
python.exe test.py B 5001 configB.txt
rem and so on
Or using a FOR loop and an environment variable with delayed expansion:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
cd /D "F:\A 5th Semester Data\CN\Project"
set "Number=5000"
for %%I in (A B C D E) do (
python.exe test.py %%I !Number! config%%I.txt
set /A Number+=1
)
endlocal
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cd /?
echo /?
endlocal /?
for /?
set /?
setlocal /?