For a distributed system I have several processes that are required to be launched and I am working on (developing) almost all of them at the same time (modify here, adapt there, ...).
I usually run them in several terminal windows, but would prefer some kind of "process launcher" where I can configure the individual processes in a config file, and then preferrably get the log output, maybe colorized, in a single terminal window.
I started to write something on my own, but have not been happy with the result and wonder if there is not a dedicated tool for that purpose.
Reloading (on file change) would be cool, but I could also use watchmedo for that purpose.
Does someone here have a hint for me?
You may be interested in docker-compose, using it you will be able to predefine multiple python processes easily and run them all together, seeing the logs of each process live and colored, restarting all or part of them, etc...
Related
In jupyter, when an error occurs, we can continuously debug the error with the command %debug. I want to know if there is the similar way in running python script (.py) in the shell.
I know that I can use pdb to make some break points, but I just want to know the way without such a pre-processing (because re-running the code until the error costs a lot of time).
In general, no: it depends on "the shell" that you are running. Jupyter launches with a lot of instrumentation in support of its debugger, assuming that you're using Jupyter because you want those capabilities at the ready.
I presume that you're using some UNIX shell (since you mention pdb); implicitly loading superfluous software is antithetical to the UNIX philosophy.
I think that what you'll need is one of the "after" debugger modes, although that will still leave you without information from just before the error point: those packages cannot do much to trace the history of problem variables.
Let's first talk about what I'm trying to do (using Python 3.x):
As a part of a small VFX pipeline, I'm working on a tool which is supposed to launch programs (e.g. Blender) after setting a couple of environment variables. No issues with this part directly. The issue is the visibility of the console.
There are three requirements that I find conflicting with each other.
Requirement 1: The console of the tool itself will need to be hidden.
Requirement 2: A separate cmd/console needs to be launched for Blender, as the output of this tool needs to be accessible. (For if I work on a script in Blender I need to be able to see errors and prints.)
Requirement 3: The tool itself should be hidden once the Blender launches, and should come back if Blender is closed. (This is so that it can quickly be restarted in case of a crash. A lot of VFX software is incredibly unstable once you are working on heavier scenes.)
Individually I'm able to cover each of the requirements. I'm even able to cover 2 different ones at the same time, but I struggle meeting all 3 of them at once.
(I'm using Tkinter for the UI)
Option number 1:
command = "C:\Program Files\Blender Foundation\Blender 2.91\blender.exe"
ui.withdraw()
os.system(command)
ui.update()
ui.deiconify()
Using os.system will freeze the tool while Blender is running. It will unfreeze and thus also unhide as soon as Blender crashes. But I'm missing the console output. (The tool itself is launched using subprocess.Popen. This is on purpose as I won't need the console of the tool itself, I only want the console to show up once Blender is actually launched.)
Alternatively, I tried:
ui.withdraw()
os.system(f'start cmd /C {command}') # also tried the same command using subprocess.Popen
ui.update()
ui.deiconify()
Launching a console and executing Blender from within it gives me the desired console, but the tool doesn't wait anymore until Blender is closed. It's immediately back to being visible as it considers launching the cmd as the end of the command.
I could use the second method to start the tool itself and then use the first method within it. That solves both Requirement 3 & 4, but it also means the console will be visible before Blender is launched failing requirement #2.
PS: Apologies, if anything is chaotic or overcomplicated. I'm a VFX artist and not a programmer and I only use python now and then. I feel like my knowledge is very imbalanced. It's quite advanced in regards to some topics but has big gaps regarding others.
I have the situation that I want the job done from eclipse so I use eclipse's .launch configuration. I tried to make it run python directly but got error: error 193 (%1 is not a valid Win32 app). where %1 is probably my python script.
I decided to create a simple batch script that calls this big wild python animal.
I did a lot of combinations and found this the best (batch outputs some strings, runs python, waits for it, the outputs some strings again):
start /b /wait "Python_script.py" "%1" "%2" "%3" "%4" "%5"
It worked until python itself started to run exe file.
Once again I tried a lot of combinations:
os.system([exe, arg1, arg2, ...]) and
subprocess.call(..) and subprocess.check_output(..)
-> I either didn't see the output in eclipse console, or the output was delayed or there was only python / or only exe's output in console.
finally I used subprocess.Popen(...) and it's nearly allright - the only defect is that the output from python script don't wait for exe's process to finish, and when I use subprocess.Popen(...).wait() exe passes output to console but the WHOLE output from python script is delayed until the 'exe' terminates. I want to delay only the part of pythons script output that is written after the exe is called.
how to achieve this 'partly console output delay' is the main topic
advices on python and eclipse .launch configuration will be appreciated
general advices on how does the communication between this processes(?) work will be appreciated
Thanks!
It sounds to me like you have three different processes you're trying to get to work together, you've tried a whole bunch of stuff to get it working, and the code is complex enough that you can't easily post it here. That makes it pretty hard to get a good answer (Stack Overflow works much better with focused questions), but here's the general approach I'd take:
Does your script run if you try to run Python_script.py directly from the command prompt?
If it doesn't, then look into registering the .py file type in Windows.
If it does, then maybe Eclipse launch configurations don't support or don't properly support Windows registered file types. There should be no need to mess with batch files and start; just replace Python_script.py in your launch configuration with c:\Python27\python.exe Python_script.py (or similar).
Get your script working from a command prompt - able to run, with proper Python and subprocess output, and waiting for everything to terminate.
If things work from the command prompt and still don't work from Eclipse, then post a new question with a small snippet of code showing what you're trying and a description of what's wrong. subprocess.call, subprocess.check_output, and Popen all have different uses, so it's hard to give general advice besides just referring to the documentation.
I'm making a drawing program with python and pygame. I am trying to incorporate a script-fu thing in which the program opens a python live interpreter upon startup and allows the user to execute commands in the interpreter alongside the graphical interface.
My current strategy is to run the main loop inside its own thread, then have the application opened using a bash script that does 'python -i main.py'
Is this a safe/effective/ideal way of doing this? How can I use locks to ensure that commands coming in from the interpreter are executed between main loop iterations?
This is my first time using threads, so please explain to me like I am 7.
Thanks :)
The interpreter won't cooperate with locks you set (since it doesn't know about them). Thus, you cannot guarantee when the code entered by the user will execute.
Consider using the code module to build your own interactive console (it's really easy!). Then you can do the locking every time you go to execute user input.
Why are you using a third party live interpreter? Do you realize that pygame comes with one built in? The documentation is here. This will eliminate all of your problems quite easily.
I have got a list of files in txt files and I need to check them out in edit mode, and make some changes(there are word documents), and check them back in via WinCVS.
I know I can write tcl scripts or macro, or python scripts in wincvs shell but I have some problems with them.
I have installed TCL 8.5 and selected tcl DLL in Admin>Preferences, tcl is now available, but whenever I type and execute a tcl script, it says
can not find channel named "stdout"
Do you have any idea regarding this error?
Also, I cannot see admin macros, it says Shell is not available. I have installed the latest version of python and select related dll in preferences.
Could anyone give me a hint for checking a list of files via wincvs?
many thanks in advance,
regards
The problem is that Tcl's trying to build the standard file descriptors into available-by-default channels (i.e., stdin, stdout and stderr) but this goes wrong when they're not opened by default. That's the case on Windows when running disconnected (which is what happens inside GUI applications on that platform). When you're running with a full Tcl shell such as wish, this is worked around, but you're embedded so that's not going to work; the code to fix things isn't run because it's part of the shell startup and not the library initialization (after all, replacing a process-global resource like file descriptors is a little unfriendly for any library to do without the app or user asking it to!)
The simplest workaround is to not write to stdout – note that it's the default destination of the puts command, so you have to be careful – and to take care not to write to stderr either, as that's probably under the same restrictions (which means that you've got to be careful how you trap errors, especially while testing your script).