I am trying to use the Process() robot framework library to launch and track processes.
https://robot-framework.readthedocs.io/en/v3.0.3/_modules/robot/libraries/Process.html
After I launch my process I am unable to use the get_process_id() method. I wrote a simple example using notepad.exe below
path = "C:\\WINDOWS\\system32"
Process().start_process('notepad.exe',shell=False, cwd=path)
var = Process().get_process_id()
BuiltIn().log_to_console(var)
This gives me the error of "No active process."
Alternatively, using handles as explained in the documentation
path = "C:\\WINDOWS\\system32"
handle = Process().start_process('notepad.exe',shell=False,cwd=path)
var = Process().get_process_id(handle)
BuiltIn().log_to_console(var)
I get the error "Non-existing index or alias '1'."
When you do Process().get_process_id(), you are creating a new instance of the library. This instance doesn't know about any processes started by the previous instance of the library.
You need to get a single instance of the library, and use that consistently.
processLib = Process()
processLib.start_process(...)
var = processLib.get_process_id()
The best thing to do is try to get a reference to an existing process library using BuiltIn().get_library_instance, and only create a new one if it doesn't exist.
Related
There is a python script start_test.py.
There is a second python script siple_test.py.
# pseudo code:
start_test.py --calls--> subprocess(python.exe simple_test.py, args_simple_test[])
The python interpreter for both scripts is the same. So instead of opening a new instance, I want to run simple_test.py directly from start_test.py. I need to preserve the sys.args environment. A nice to have would be to actually enter following code section in simple_test.py:
# file: simple_test.py
if __name__ == '__main__':
some_test_function()
Most important is, that the way should be a universal one, not depending on the content of the simple_test.py.
This setup would provide two benefits:
The call is much less resource intensive
The whole stack of simple_test.py can be debugged with pycharm
So, how do I execute the call of a python script, from a python script, without starting a new subprocess?
"Executing a script" is a somewhat blurry term.
Typically the if __name__== "__main__": part does the argument (sys.argv) decoding and then calls a worker function with explicit parameters. For clarity: It should not do anything else, since this additional work can't be called without creating a new process causing all the overhead you are trying to avoid.
You simply bypass that and call this implementing routine directly.
So you end up with start_test.py containing something like:
from simple_test import worker
# ...
worker(typed_arg1, typed_arg2)
I have a simple SQL file that I'd like to read and execute using a Python Script Snap in SnapLogic. I created an expression library file to reference the Redshift account and have included it as a parameter in the pipeline.
I have the code below from another post. Is there a way to reference the pipeline parameter to connect to the Redshift database, read the uploaded SQL file and execute the commands?
fd = open('shared/PythonExecuteTest.sql', 'r')
sqlFile = fd.read()
fd.close()
sqlCommands = sqlFile.split(';')
for command in sqlCommands:
try:
c.execute(command)
except OperationalError, msg:
print "Command skipped: ", msg
You can access pipeline parameters in scripts using $_.
Let's say, you have a pipeline parameter executionId. Then to access it in the script you can do $_executionId.
Following is a test pipeline.
With the following pipeline parameter.
Following is the test data.
Following is the script
# Import the interface required by the Script snap.
from com.snaplogic.scripting.language import ScriptHook
import java.util
class TransformScript(ScriptHook):
def __init__(self, input, output, error, log):
self.input = input
self.output = output
self.error = error
self.log = log
# The "execute()" method is called once when the pipeline is started
# and allowed to process its inputs or just send data to its outputs.
def execute(self):
self.log.info("Executing Transform script")
while self.input.hasNext():
try:
# Read the next document, wrap it in a map and write out the wrapper
in_doc = self.input.next()
wrapper = java.util.HashMap()
wrapper['output'] = in_doc
wrapper['output']['executionId'] = $_executionId
self.output.write(in_doc, wrapper)
except Exception as e:
errWrapper = {
'errMsg' : str(e.args)
}
self.log.error("Error in python script")
self.error.write(errWrapper)
self.log.info("Finished executing the Transform script")
# The Script Snap will look for a ScriptHook object in the "hook"
# variable. The snap will then call the hook's "execute" method.
hook = TransformScript(input, output, error, log)
Output:
Here, you can see that the executionId was read from the pipeline parameters.
Note: Accessing pipeline parameters from scripts is a valid scenario but accessing other external systems from the script is complicated (because you would need to load the required libraries) and not recommended. Use the snaps provided by SnapLogic to access external systems. Also, if you want to use other libraries inside scripts, try sticking to Javascript instead of going to python because there are a lot of open source CDNs that you can use in your scripts.
Also, you can't access any configured expression library directly from the script. If you need some logic in the script, you would keep it in the script and not somewhere else. And, there is no point in accessing account names in the script (or mappers) because, even if you know the account name, you can't use the credentials/configurations stored in that account directly; that is handled by SnapLogic. Use the provided snaps and mappers as much as possible.
Update #1
You can't access the account directly. Accounts are managed and used internally by the snaps. You can only create and set accounts through the accounts tab of the relevant snap.
Avoid using script snap as much as possible; especially, if you can do the same thing using normal snaps.
Update #2
The simplest solution to this requirement would be as follows.
Read the file using a file reader
Split based on ;
Execute each SQL command using the Generic JDBC Execute Snap
I am trying to use ctypes to access a dynamic library in python.
So far, I have created a bunch of structures in python to reflect the structure types in the library. I am now trying to test these structures to see if I have translated everything correctly. I currently have something like the following set up
import ctypes as ct
class MyStruct(ct.Structure)
''' put the actual implementation here '''
pass
if __name__ == "__main__":
mylib = ct.CDLL('/path/to/mylib.dylib')
loadFile = mylib.loadFile
loadFile.restype = ct.Pointer(MyStruct)
parent = MyStruct()
file = b"/path/to/file"
child = loadFile(file, ct.byref(parent))
print(child)
Now, for some reason this segfaults inside the dynamic library when I try to run this code. I am trying to determine where (and why) this happens. I have compiled both the library and my python executable with debug info and am now trying to use gdb to locate the problem. I can start my program and break at a line inside loadFile very nicely
gdb b loading.c:172
gdb r
and gdb breaks. Now, there is another function call (this function is also part of the dynamic library) on line 172 in loading.c that I want to step into. I tried to break inside this other function
gdb b foo.c:636
but gdb keeps giving me
Cannot access memory at address 0x2860
errors. Further, if I just break at line 172 and then use stepi then I can't get any debug info. Is there a way to get into this other function or is it a limitation of gdb when using dynamic libraries.
I am new to Jmeter. My HTTP request sampler call looks like this
Path= /image/**image_id**/list/
Header = "Key" : "Key_Value"
Key value is generated by calling a python script which uses the image_id to generate a unique key.
Before each sampler I wanted to generate the key using python script which will be passed as a header to the next HTTP Request sampler.
I know I have to used some kind of preprocessor to do that. Can anyone help me do it using a preprocessor in jmeter.
I believe that Beanshell PreProcessor is what you're looking for.
Example Beanshell code will look as follows:
import java.io.BufferedReader;
import java.io.InputStreamReader;
Runtime r = Runtime.getRuntime();
Process p = r.exec("/usr/bin/python /path/to/your/script.py");
p.waitFor();
BufferedReader b = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
StringBuilder response = new StringBuilder();
while ((line = b.readLine()) != null) {
response.append(line);
}
b.close();
vars.put("ID",response.toString());
The code above will execute Python script and put it's response into ID variable.
You will be able to refer it in your HTTP Request as
/image/${ID}/list/
See How to use BeanShell: JMeter's favorite built-in component guide for more information on Beanshell scripting in Apache JMeter and a kind of Beanshell cookbook.
You can also put your request under Transaction Controller to exclude PreProcessor execution time from load report.
A possible solution posted by Eugene Kazakov here:
JSR223 sampler has good possibility to write and execute some code,
just put jython.jar into /lib directory, choose in "Language" pop-up
menu jython and write your code in this sampler.
Sadly there is a bug in Jython, but there are some suggestion on the page.
More here.
You can use a BSF PreProcessor.
First download the Jython Library and save to your jmeter's lib directory.
On your HTTP sampler add a BSF PreProcessor, choose as language Jython and perform your needed magic to obtain the id, as an example I used this one:
import random
randImageString = ""
for i in range(16):
randImageString = randImageString + chr(random.randint(ord('A'),ord('Z')))
vars.put("randimage", randImageString)
Note the vars.put("randimage",randImageString") which will insert the variable available later to jmeter.
Now on your test you can use ${randimage} when you need it:
Now every Request will be different changing with the value put to randimage on the Python Script.
I'm using pydaemon ( http://www.python.org/dev/peps/pep-3143/ ) to make a friendly daemon. How do I give it a name? by default it's called 'python' but I want something more meaningful.
Changing the process name cannot be done from plain Python, and pydaemon is 100% Python. You need a C-level library like py-setproctitle to do that. Then, simply add the following to your main method:
try:
import setproctitle
setproctitle.setproctitle('my-awesome-program')
except:
pass # Ignore errors, since this is only cosmetic