Develop Reference

Python in Execute Process Task

I have written a python code, then ran it through SSIS in the Execute Process Task, because I want to schedule it with table refresh dependency(which is not possible through windows scheduler or Alteryx).
The code reads and writes to SQL management studio to translate via Google API (pip install googletrans==4.0.0-rc1), with no output or input through SSIS. the input and output are done through sqlacademy in python.
The codes runs perfectly fine in Jupyter notebook, and I saved it to be .py file, but I face a lot of error in the Execute Process Task in SSIS.
Here are a couple of the solutions I tried:
in EXCUTABLE (in command prompt written where python copied .exe path)
in Arguments C:......\file.py
in WorkingDirectory C:......\ (only removing the file name while keeping the backslash
no white spaces a solution from stackflow)
Error:
"C:\Users\me\AppData\Local\Programs\Python\Python311\python.exe" "UC2.py" at "C:\Users\me\Downloads\folder\test", The process exit code was "1" while the expected was "0".
in EXCUTABLE (in command prompt written where python copied .exe path)
in Arguments 'file.py' (file name in quataion a solution from stackflow)
in WorkingDirectory C:......\ (only removing the file name while keeping the backslash
no white spaces a solution from stackflow)
Error:
The process exit code was "2" while the expected was "0"
move the .exe to a folder were no other files exists.
The process exit code was "-1073741515" while the expected was "0"
I also tried opening a new project.
then I tried right click change success value no error when running in SSIS, but the following error pops up:
enter image description here
everything even the sql server is installed on the remote desktop.

HelloWorld.err.py
A trivial python script that will write a string to standard out, another to standard error and the exit with return code 1.
import sys
print("Hello world")
print("Egads, something bad happened!", file=sys.stderr)
exit(1)
SSIS package
A sample SSIS package would look something like the following. An Execute Process Task has a successor of a Script Task.
Double click the precedent constraint (the line between the two) and change that from the default of Success to Completion
EPT Python
An Execute Package Task that is configured as shown
Executable: C:\Program Files\Python39\python.exe
Arguments: C:\ssisdata\SO_75361110.err.py
StandardOutputVariable: User::StdOut
StandardErrorVariabele: User::StdErr
SCR Echo Back
This is a generic Script Task that I use and abuse to dump the values of SSIS Variables to the output log/results tab. It's easy, lightweight and let's you print values without popups or any hard work.
All you need to do is add the Variable(s) in the readonly/readWrite collection. In this case, I selected User::StdOut and User::StdErr
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
namespace ST_ede4b38a087c4724a71149ffc5ed6be9
{
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public void Main()
{
bool fireAgain = false;
foreach (Variable item in Dts.Variables)
{
Dts.Events.FireInformation(0, "SCR Echo Back", string.Format("{0}->{1}", item.QualifiedName, item.Value), "", 0, ref fireAgain);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
}
}
Results
When my package runs, as "expected" the SuccessValue from my run will be 1 given that I use exit(1) from the python script.
Let's see what the Output tab looks like
SSIS package "C:\Users\bfellows\source\repos\SO_Trash\SO_Trash\SO_75361110.dtsx" starting.
Error: 0xC0029151 at EPT Python the thing, Execute Process Task: In Executing "C:\Program Files\Python39\python.exe" "C:\ssisdata\SO_75361110.err.py" at "", The process exit code was "1" while the expected was "0".
Task failed: EPT Python the thing
Warning: 0x80019002 at SEQC Do the thing: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
Information: 0x0 at SCR Echo Back, SCR Echo Back: User::StdErr->Egads, something bad happened!
Information: 0x0 at SCR Echo Back, SCR Echo Back: User::StdOut->Hello world
Warning: 0x80019002 at SO_75361110: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "C:\Users\bfellows\source\repos\SO_Trash\SO_Trash\SO_75361110.dtsx" finished: Failure.
The program '[68044] DtsDebugHost.exe: DTS' has exited with code 0 (0x0).
Of note, are these two lines
Information: 0x0 at SCR Echo Back, SCR Echo Back: User::StdErr->Egads, something bad happened!
Information: 0x0 at SCR Echo Back, SCR Echo Back: User::StdOut->Hello world
That's our Script Task echoing back the value of our Standard Out and Standard Error text streams. My suspicion is that given a "normal" run is 10 minutes and the runs from SSIS are measured in seconds, the process execution is throwing errors to the caller but nobody is listening. This will add a listener and give you better information to diagnose your root cause.

How to hide bash output in terminal based on a condition using Python

I am trying to execute this bash command n python and try to evaluate the output, based on the output, I want to show my own-defined outputs.
The bash command I am trying to execute using the python is:
kubectl get pods --field-selector=status.phase=Failed -n kube-system
everything looks really good and only problem I am having is,
This outputs No resources found, means there are no resources matching the given criteria (i.e status.phase=Succeeded), but that is fine, but the problem is It prints in the terminal. What I want to do is, prints my own-defined output when the actual output is No resources found, but I can't do that since it already prints that output in the terminal. I even can't use the status_code to check, it always gives 0 even after the resources are found or not (which is correct) since code has executed successfully. Is there a way that I can filter the output during the executing of the bash command and gives a condition based on the output to print my own-defined output?
Here is my code snippet:
def subprocess_execute_arr(self):
output = subprocess.call(self)
return output
cmd_failed = ["kubectl", "get", "pods", "--field-selector=status.phase=Failed", "-n", "kube-system"]
failed = execute.subprocess_execute_arr(cmd_failed) //this is where it prints the output in the terminal
output:
No resources found.
PS: Here output is not an error, command has executed correctly but I don't need to print this output.
Any idea how to solve this?

What does it mean when Click exits with exit code 2?

I am currently testing a Click CLI application and get result.exit_code == 2. Why does that happen?

This appears to indicate a usage error:
An internal exception that signals a usage error. This typically aborts any further handling.
This is consistent with Click's own tests, e.g.
https://github.com/pallets/click/blob/123dd717439d8620d8d6be5574d2c9f007952326/tests/test_arguments.py#L82
https://github.com/pallets/click/blob/123dd717439d8620d8d6be5574d2c9f007952326/tests/test_arguments.py#L190
https://github.com/pallets/click/blob/123dd717439d8620d8d6be5574d2c9f007952326/tests/test_arguments.py#L201
https://github.com/pallets/click/blob/123dd717439d8620d8d6be5574d2c9f007952326/tests/test_formatting.py#L157
https://github.com/pallets/click/blob/123dd717439d8620d8d6be5574d2c9f007952326/tests/test_formatting.py#L177
https://github.com/pallets/click/blob/123dd717439d8620d8d6be5574d2c9f007952326/tests/test_formatting.py#L193

I ran
result = runner.invoke(cli, ['sync'])
instead of
result = runner.invoke(cli, ['--debug', 'sync'])
So you need to specify the flag as entered via CLI, not only pass the parameters consumed by the function if you use #click.option.
Additionally, the I made a typo for one of the flags.
How to debug
Look at the parameters you pass to runner.invoke (simplest: print it)
Execute it via CLI (e.g. cli(['--debug', 'sync']))
In my case this gave me the message
Error: no such option: --sync Did you mean --syncs?

Get the output from python tests in ruby script

I have a Python project with some tests in different folders and files. I wanted to write a script which executes all tests for the project and outputs some results. I want it to be a ruby script (because I know Ruby more than Python and for now I enjoy it more) and I was thinking to get the output from the tests and to parse them with ruby and to output something like "48 tests run in total, all ok" instead of the the output from python.
Long story short - I want I way to get the output from python test_something.py in a variable or in a file and I want nothing from it on the screen.
Here are my tries:
tests = Dir.glob("**/test_*")
wd = Dir.pwd
output = ''
tests.each do |test|
Dir.chdir(File.dirname(test))
# output += `python #{File.basename(test)}`
# system("python #{File.basename(test)} >> f.txt")
Dir.chdir(wd)
end
I tried both things which are commented, but both of them print the result on the standard exit and in the first one output variable is empty, in the second one the file is created but is empty again :(
Any ideas? Thank you very much in advance! :)

The test framework might have send the result to STDERR. Try use Open3.capture3 to capture the standard error.
require 'open3'
...
stdout, stderr, status = Open3.capture3(%{python "#{File.basename(test)}"})
and write the standard output and standard error to the destination:
File.write("f.txt", stdout + stderr)
You may check status.success? to see if you write the external command right. However, the test framework may return non-zero exit code on failed tests. In that case, you should check the stderr to see the actual error output.

Use Open3.capture2 as below:
output, _ = Open3.capture2("python #{File.basename(test)")
To write output to a file do as below:
File.write("f.txt", output)

Can the system trace function be set in MicroFocus Cobol?

Does MicroFocus Cobol or any other, have a feature equivalent to Python's sys.settrace()?
The function passed as a parameter to such a tracing function, would be called after the execution of each line of the source code.

It's not an exact equivalent, but you can use READY TRACE for debugging. Enable it with the TRACE compiler directive.

OpenCOBOL supports
-ftrace Generate trace code
- Executed SECTION/PARAGRAPH
-ftraceall Generate trace code
- Executed SECTION/PARAGRAPH/STATEMENTS
- Turned on by -debug
cobc command line options. This isn't quite the same as the Python point of view, but outputs a tracer round on entry to sections, paragraphs and sentences when enabled. No doubt other compilers will have something equivalent. Along with the READY TRACE, debugging and >>D other debugging features like those allowed with DECLARATIVES. http://opencobol.add1tocobol.com/#declaratives
procedure division.
declaratives.
handle-errors section.
use after standard error procedure on filename-1.
handle-error.
display "Something bad happened with " filename-1 end-display.
.
helpful-debug section.
use for debugging on main-file.
help-me.
display "Just touched " main-file end-display.
.
end declaratives.