I have a script that keeps on dumping some values to the screen. Lets call this script 1
I need the values that gets dumped to the screen to be processed by another python script.Lets call this script2. I Script1 cant be stopped
I currently use python 2.2 so it would be great if someone could provide an example without using subprocess
script 1 looks something like this
import sys
import os
i = 0
while 1:
print i
sys.stdout.write(str(i)+"\n")
i = i + 1
I need to write script2 such that every output generated by script1 is processed by script2
-- edit --
I need it to run on a legacy implementation ie no power shell just the command prompt available in Windows XP
I'm not going to say anything about code design here, because I really don't know what you are working on.
But for script2.py:
import sys
for line in sys.stdin:
print "From stdin:", line
And then run (at least on unix/linux, if you are on windows, google "piping in powershell"):
python script1.py | python script2.py
Or did you want to do it within one python script? You could re-pipe sys.stdout, by simply replacing it with a class that has all the necessary methods (write, etc.).
I guess you want to process line by line:
# script2.py
import sys
for line in sys.stdin:
print line # do something here with line
[edit] Finally, you simply pipe them:
python script1.py | python script2.py
Related
I've got a python script that is named code.py and this takes two command line arguments --par_value xx --out_dir /path/to/dir where par_value is a certain parameter and out_dir is the output directory. I want to run this script for different values of the input parameter.
So I want to use the array:
par_vector = np.array([1,2,3])
and have the output file for each named, say, "output_1", "output_2" and "output_3". So I'm trying to write a python script where I can create that array above and an array of output strings, say, called output_strings so I can automatically run something along the like:
for i,j in zip(par_vector, output_strings):
python code.py --par_value i --out_dir j
But this does not run so I'm unable to figure out how to make such an automation of my code work rather than repeatedly calling the script code.py from the terminal. Any advice is welcome!
It might be a bit convoluted, but a way you could do it is to generate either .bat or .sh files (depending on your operating system) and call them with Popen from the subprocess library.
from subprocess import Popen, PIPE
for i,j in zip(par_vector, output_strings):
cmd = """python code.py --par_value {} --out_dir {}""".format(i,j)
temp_filename = "{}{}.bat".format(i,) #swap .bat for .sh if not on Windows
with open(temp_filename, 'wb') as out_file:
out_file.write(cmd)
execution = Popen([temp_filename], stdout=PIPE, stderr=PIPE)
results = execution.communicate()
# add conditions based on the output generated by results if you want to use it to verify, etc.
os.remove(temp_filename)
Obviously this needs to be changed to match your exact needs for the file name and location, etc..
I would use os.system for this if you're OK with the fact that os.system() is blocking. This would look something like:
import os
for i,j in zip(par_vector, output_strings):
os.system(f"python code.py --par_value {i} --out_dir {j}")
I picked up a script (script1.py) from another dev which uses the flags module from absl. Basically it is ran as such:
python script1.py --input1 input1 --input2 input2
I currently have another script (script2.py) which generates that input1 and input2 for script1 to run. How can I actually pass the args over to script1 from within script2? I know I have to import script1 but how can I then point it to those inputs?
Use the Python subprocess module for this. I am assuming that you are using a version that is 3.5 or newer. In this case you can use the run function.
import subprocess
result = subprocess.run(
['python', 'script1.py', '--input1 input1', '--input2 input2'],
capture_output=True)
# Get the output as a string
output = result.stdout.decode('utf-8')
Some notes:
This example ignores the return code. The code should check result.returncode and take correct actions based on that.
If the output is not needed, the capture_output=True and the last line can be dropped.
The documentation for the subprocess module can be found here.
A alternative (better) solution
A better solution (IMHO) would be to change the called script to be a module with a single function that you call. Then python code in script1.py could probably be simplified quite a bit. The resulting code in your script would then be something similar to:
from script1 import my_function
my_function(input1, input2)
It could be that the script from the other dev already has a function you can call directly.
I would like to run my python script from the command line when supplies with some arguments. However, one of the arguments should be a list of options specific to a segment of the script.
Example:
MODULES_TO_INSTALL = ['sale','purchase','account_accountant',]
how can I do this: python fichier.py liste_modules_to_install
I've done something similar in the past. It might be easier if instead of sending them as a list already, you call your script like so,
python script.py module1 module2 ... moduleN
Then a simple line of code to read in these passed modules from command line would be,
import sys
MODULES_TO_INSTALL = sys.argv[1:]
In a linux terminal typing
python script.py
Will run run script.py and exit the python console, but what if I just want to run a part of the script and leave the console open? For example, run script.py until line 15 and leave the console open for further scripting. How would I do this?
Let's say it's possible, then with the console still open and script.py run until line 15, can I then from inside the console call line fragments from other py files?
...something like
python script.py 15 #(opens script and runs lines 1-15 and leaves console open)
Then having the console open, I would like to run lines 25-42 from anotherscript.py
>15 lines of python code run from script.py
> run('anotherscript.py', lines = 25-42)
> print "I'm so happy the console is still open so I can script some more")
I'm so happy the console is still open so I can script some more
>
Your best bet might be pdb, the Python debugger. You can start you script under pdb, set a breakpoint on line 15, and then run your script.
python -m pdb script.py
b 15 # <-- Set breakpoint on line 15
c # "continue" -> run your program
# will break on line 15
You can then inspect your variables and call functions. Since Python 3.2, you can also use the interact command inside pdb to get a regular Python shell at the current execution point!
If that fits your bill and you also like IPython, you can check out IPdb, which is a bit nicer than normal pdb, and drops you into an IPython shell with interact.
if you want to run script.py from line a to line b, simply use this bash snippet:
cat script.py|head -{a+b}|tail -{b-a}|python -i
replace {a+b} and {b-a} with their values
You could use the python -i option to leave the console open at the end of the script.
It lets your script run until it exits, and you can then examine variables, call any function and any Python code, including importing and using other modules.
Of course your script needs to exit first, either at the end or, if your goal is to debug that part of the script, you could add a sys.exit() or os._exit() call where you want it to stop (such as your line 15).
For instance:
import os
print "Script starting"
a=1
def f(x):
return x
print "Exiting on line 8"
os._exit(0) # to avoid the standard SystemExit exception
print "Code continuing"
Usage example:
python -i test_exit.py
Scrit starting
Exiting on line 8
>>> print a
1
>>> f(4)
4
>>>
You cannot do that directly but you can do something similar from inside Python console (or IDLE) with exec :
just open you favorite Python console
load wanted lines into a string and exec them :
script = 'script.py'
txt = ''
with open(script) as sc:
for i, line in enumerate(sc):
if i >= begline and i<= endline:
txt = txt + line
exec(txt)
You can even write your own partial script runner based on that code ...
EDIT
I must admit that above answer alone really deserved to be downvoted. It is technically correct and probably the one that most closely meet what you asked for. But I should have warned you that it is bad pratice. Relying on line numbers to load pieces of source files is error prone and you should avoid it unless you really know what you are doing and why you do it that way. Python debugger at least allows you to control what are the lines you are about to execute.
If you really have to use this solution be sure to always print and double check the lines that you are about to execute. IMHO it is always both simpler and safer to copy and past lines in an IDE like IDLE that is packaged into any standard Python installation.
When I run any Python script, I would like to see the script's filename appear in the Windows command line window's titlebar. For example, if I run a script called "mytest.py", I want to see "mytest" in the titlebar. I would like this to be automatic, so I don't have to add code to every one of my scripts.
Currently I'm attempting to do this with sitecustomize.py, because when Python is run, including from double-clicking a Python script, sitecustomize is imported before the script runs.
I've tried getting __main__'s __file__ and sys.argv, but sitecustomize doesn't see either:
file sitecustomize.py:
import __main__, sys
print "hasattr __main__.__file__:", hasattr(__main__, "__file__")
print "hasattr sys.argv:", hasattr(sys, "argv")
print "-" * 60
file mytest.py:
import sys
print "__file__ is:", __file__
print "sys.argv is:", sys.argv
raw_input() # don't end the script immediately
output:
hasattr __main__.__file__: False
hasattr sys.argv: False
------------------------------------------------------------
__file__ is: C:\Documents and Settings\Owner\Desktop\mytest.py
sys.argv is: ['C:\\Documents and Settings\\Owner\\Desktop\\mytest.py']
I'm glad you asked! I now have it working for my scripts, and it's pretty cool.
Here's the code:
import sys
import time
from ctypes import windll
class SetTitle(object):
def __del__(self):
time.sleep(1)
command = ' '.join(sys.argv)
windll.kernel32.SetConsoleTitleA(command)
sys.argv = SetTitle()
This is for Python 2.x -- for 3.x you need to change SetConsoleTitleA to SetConsoleTitleW (last letter changes from A to W).
How it works: since the sys.argv object does yet exist, I create an object and assign it to sys.argv; then, when Python assigns the actual argv to sys.argv, my object is tossed, and the __del__ method is called; the __del__ method is then able to access the real argv and set the title bar accordingly. I put the 1 second sleep in just to avoid any possible race conditions, but I'm not sure it's necessary. If you don't want to see all the command-line args, you can pre-process command any way you like.
My thanks to the folks on python-win32 mailing list, and Thomas Heller in particular, for helping with the 'set title' portion of this question.
When I run any Python script, I would
like to see the script's filename
appear in the Windows command line
window's titlebar. For example, if I
run a script called "mytest.py", I
want to see "mytest" in the titlebar.
I would like this to be automatic, so
I don't have to add code to every one
of my scripts.
I think you should add this functionality to all your scripts by a module and not by hacking it into sitecustomize.py. Also even if you still want to go the sitecustomize path you will need to pass __file__ from your script, which means you will not get around to add some code to all your scripts.
What you certainly can do is to put that code into a module and then import it in all your python scripts. Like I mentioned above, you need to pass __file__ from your main script otherwise you will get the modules filename. Also there is no need to import __main__ to retrieve __file__.