I'm working with IronPython in a C#/.Net Core 3.1 project and I need to be able to validate a script before executing it in a production environment.
I have found this solution, creating my custom Microsoft.Scripting.Hosting.ErrorListener implementation:
public class IronPythonListener : ErrorListener
{
public List<ValidationError> Errors = new List<ValidationError>();
public override void ErrorReported(ScriptSource source, string message, SourceSpan span, int errorCode, Severity severity)
{
Errors.Add(new ValidationError
{
Message = message,
ErrorCode = errorCode,
Severity = severity,
Span = span
});
}
}
and then passing an instance of it to the Microsoft.Scripting.Hosting.ScriptSource.Compile(ErrorListener) method:
IronPythonListener listener = new IronPythonListener();
ScriptEngine engine = Python.CreateEngine();
ScriptSource scriptSource = engine.CreateScriptSourceFromString(script, SourceCodeKind.AutoDetect);
CompiledCode code = scriptSource.Compile(listener);
In listener.Errors list I find all the compilation errors.
This solution works, but for my purposes it is not complete, because for example:
if the passed script is something like my_var = 5 + "some text", listener.Errors is empty and the script is considered valid even though it cannot be executed (in fact, it throws a unsupported operand type(s) for +: 'int' and 'str' exception)
if the passed script contains calls to undefined functions (e.g. length(my_string) instead of len(my_string)), it is considered valid
Another thing that seems strange to me is that all the errors I can find are of type Severity.FatalError (e.g. passing my_var = 6 +), but I'm not able to find any Severity.Error or Severity.Warning.
Is there a way to improve the validation, without executing my compiled script?
Thanks in advance for any help, unfortunately I cannot find so much documentation about this.
Edit: I found some online validators (e.g. https://repl.it/languages/python3 or http://pep8online.com/) and they also don't provide a complete python validation (in the former one, validation is handled better by the IDE, but the 5 + "some text" error is detected only at execution time). Of course I can try to execute the script and catch the exception when listener.Errors is empty, but it would be better to avoid this.
Edit 2: I tried also this solution, using a separated python script to validate mine, but I have the same issues with undefined functions and wrong operators usage.
Related
I am a very new user to Quickfix Python. I want to send a QuoteRequest. My function to create a new quote request message is as below:
import quickfix as fix
def create_quote_request():
message = fix.Message()
header = message.getHeader()
header.setField(fix.MsgType(fix.MsgType_QuoteRequest))
gp = fix.Group()
gp.setField(fix.Symbol("GBPUSD"))
gp.setField(fix.Side(fix.Side_BUY))
gp.setField(fix.Account("TestAcc"))
gp.setField(fix.Currency("GBP"))
message.addGroup(gp)
fix.Session.sendToTarget(message, self.sessionID)
When I execute the code, I am getting error as below:
NotImplementedError: Wrong number r type of arguments for overloaded function 'new_group'.
Possible C/C++ prototypes are:
FIX::Group::Group(int, int)
FIX::Group::Group(int, int, int const[])
FIX::Group::Group(int, int, message_order const &)
FIX::Group::Group(FIX::Group const &)
I did read the documentation and found that the Group object requires arguments
Group(int field, int delim)
Not sure what to pass the values for field and delim. Appreciate your response and help.
not sure I´m on time to help. But I will try. I think #JimmyNJ is giving you proper answer.
According to official doc you can easily do it, but you don´t have to use a generic group, you have to use specific group available to your kind of message.
As far as I know, you should program something like this.
from quickfix44 import QuoteRequest
import quickfix as fix
message = QuoteRequest()
group = QuoteRequest.NoRelatedSym()
group.setField(fix.Symbol("GBPUSD"))
group.setField(fix.Side(fix.Side_BUY))
group.setField(fix.Account("TestAcc"))
group.setField(fix.Currency("GBP"))
message.addGroup(group)
I´m assuming you want to add these 4 tags into NoRelatedSym group, based in their tag numbers. I´ve also used FIX 44 version, maybe you are using a different version, but main idea is the same.
I try to download a bingads report using python SDK, but I keep getting an error says: "Type not found: 'Aggregation'" after submitting a report request. I've tried all 4 options mentioned in the following link:
https://github.com/BingAds/BingAds-Python-SDK/blob/master/examples/v13/report_requests.py
Authentication process prior to request works just fine.
I execute the following:
report_request = get_report_request(authorization_data.account_id)
reporting_download_parameters = ReportingDownloadParameters(
report_request=report_request,
result_file_directory=FILE_DIRECTORY,
result_file_name=RESULT_FILE_NAME,
overwrite_result_file=True, # Set this value true if you want to overwrite the same file.
timeout_in_milliseconds=TIMEOUT_IN_MILLISECONDS
)
output_status_message("-----\nAwaiting download_report...")
download_report(reporting_download_parameters)
after a careful debugging, it seems that the program fails when trying to execute a command within "reporting_service_manager.py". Here is workflow:
download_report(self, download_parameters):
report_file_path = self.download_file(download_parameters)
then:
download_file(self, download_parameters):
operation = self.submit_download(download_parameters.report_request)
then:
submit_download(self, report_request):
self.normalize_request(report_request)
response = self.service_client.SubmitGenerateReport(report_request)
SubmitGenerateReport starts a sequence of events ending with a call to "_SeviceCall.init" function within "service_client.py", returning an exception "Type not found: 'Aggregation'"
try:
response = self.service_client.soap_client.service.__getattr__(self.name)(*args, **kwargs)
return response
except Exception as ex:
if need_to_refresh_token is False \
and self.service_client.refresh_oauth_tokens_automatically \
and self.service_client._is_expired_token_exception(ex):
need_to_refresh_token = True
else:
raise ex
Can anyone shed some light? .
Thanks
Please be sure to set Aggregation e.g., as shown here.
aggregation = 'Daily'
If the report type does not use aggregation, you can set Aggregation=None.
Does this help?
This may be a bit late 2 months after the fact but maybe this will help someone else. I had the same error (though I suppose it may not be the same issue). It does look like you did what I did (and I'm sure others will as well): copy-paste the Microsoft example code and tried to run it only to find that it didn't work.
I spent quite some time trying to debug the issue and it looked to me like the XML wasn't being searched correctly. I was using suds-py3 for the script at the time so I tried suds-community and everything just worked after that.
I also re-read the Bing Ads API walkthrough for getting started again and found that they recommend suds-jurko instead.
Long story short: If you want to use the bingads API don't use suds-py3, use either suds-community (which I can confirm works for everything I've used the API for) or suds-jurko (which is the one recommended by Microsoft).
I am trying to read my FIX logs and to parse them with the cracker that I wrote in python. However, this does not work because in my cracker I have calls like message.getHeader() which are QuickFix methods. They unsurprisingly return an error:
AttributeError: 'str' object has no attribute 'getHeader'
The logs are all strings, but the cracker is embedded within QuickFix and so uses QF methods. Is there any way to take the string and transform it into a QF message so I can just call crack(message) on it, or do I have to rewrite my cracker for this special case?
The following code should work.
import quickfix as qfix
import quickfix44 as q44
message = q44.ExecutionReport()
message.setString(input_string, True, qfix.DataDictionary('CustomDictionary.xml'))
'message' object will be updated in place and you should be able to use it as a quickfix Message object
The way that I do it in C# is to read tag 35 to see what message type I need to create. Then, I create that message type, and use the setString method to populate it. Something like this:
if (line.Contains("35=8"))
{
message = new QuickFix44.ExecutionReport();
}
else if(line.Contains("35=AS"))
{
message = new QuickFix44.AllocationReport();
}
. . . . and so on
message.setString(line, true, dictionary);
application.fromApp(message, sessionId); //The cracker takes care of it from here
where dictionary is my data dictionary. Is a similar method available in the Python bindings?
The following also works and doesn't require prior knowledge of message type.
import quickfix as fix
string = "8=FIX.4.49=24735=s34=549=sender52=20060319-09:08:20.88156=target22=840=244=948=ABC55=ABC60=20060319-09:08:19548=184214549=2550=0552=254=1453=2448=8447=D452=4448=AAA35777447=D452=338=954=2453=2448=8447=D452=4448=aaa447=D452=338=910=056"
data_dictionary = fix.DataDictionary("FIX44.xml")
message = fix.Message(string, data_dictionary, True)
print(message.toXML())
I'm having quite a bit of trouble getting started with scripting in Mule. To be quite honest, I'm falling at the first hurdle - I can't find anything in the documentation which tells me how to access the payload or how to return data to my flow.
I'm using Jython 2.5 and Mule 3.4.
My flow is extremely simple: it takes some text from an Ajax source and simply echoes it. At the moment the Python script does nothing (as I cannot figure out how to get it to do something with the payload).
<flow name="Python Script" doc:name="Python Script">
<ajax:connector name="connector-ajax" serverUrl="http://192.168.0.1:8000" resourceBase="C:\mule\workspace\scripting\src\main\app\docroot" doc:name="Ajax" />
<scripting:component doc:name="Python">
<scripting:script engine="jython" file="C:\mule\workspace\scripting\src\main\app\python\myscript.py"/>
</scripting:component>
<echo-component doc:name="Echo"/>
</flow>
I have read through the Script Component Reference and the Scripting Module Reference - the module reference appears to have some relevant information but I can't figure out how to use it in Python.
I have also read through an article about 'Mule Punching' which seems like it would have answered my question if I was running version 2 of Mule. I attempted to use the same techniques in my Mule 3 project but it did not work.
Edit 24/07/2012
Using #ppiixx's response, I have got a little bit further with Python scripting.
Just having a single line of code, for example return len(payload) causes the Jython interpreter to throw an error as return cannot exist outside of a function. Fair enough, that's standard.
However, with the code
def main():
return len(payload)
main()
I get an error saying that 'No serializer can be found for class org.mule.transport.NullPayload'.
The log is below:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Started app 'python-test' +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
INFO 2013-07-24 09:16:08,821 [[python-test].flow.stage1.02] org.mule.component.simple.LogComponent:
********************************************************************************
* Message received in service: flow. Content is: '{NullPayload}' *
********************************************************************************
ERROR 2013-07-24 09:16:08,849 [[python-test].flow.stage1.02] org.mule.exception.CatchMessagingExceptionStrategy:
********************************************************************************
Message : No serializer found for class org.mule.transport.NullPayload and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) ) (org.codehaus.jackson.map.JsonMappingException)
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. No serializer found for class org.mule.transport.NullPayload and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) ) (org.codehaus.jackson.map.JsonMappingException)
org.codehaus.jackson.map.ser.impl.UnknownSerializer:52 (null)
2. No serializer found for class org.mule.transport.NullPayload and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) ) (org.codehaus.jackson.map.JsonMappingException) (org.mule.api.transformer.TransformerException)
org.mule.module.json.transformers.ObjectToJson:107 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transformer/TransformerException.html)
This leads me to think that using return on its own isn't enough to return data to the flow.
I've read through the scripting module reference, and while it gives examples in Groovy, it does not give examples in Python so I'm not sure where I'm going wrong.
Have a look at the 'Script Context Bindings' section here.
Basically a number of variables are available in the script context including: message,payload and log.
To return data using the python engine you set the result variable.
result = len(payload)
There is a example in the mule github.
I am having an intermittent error causing my Python module to crash, and I'm assuming it's because of a memory error occurring by not getting the refcounts correct in the c code. I have a bit of code that gets a response at a random time from a remote location. Based on the data received, it needs to update a data variable which I should have access to in Python. What's the best way to accomplish this? The following code runs most of the time, and it works correctly when it does, but when it doesn't it crashes Python (bringing up the visual studio debug box). Thanks.
if (event == kResponseEvent) {
list = PyList_New(0);
for (i = 0; i < event->count; i++) {
PyList_Append(list, Py_BuildValue("{s:i, s:s}",
"id", event->id,
"name", event->name));
}
PyModule_AddObject(module, "names", list);
}
PyModule_AddObject() steals a reference. As such, you should not be decrefing list after.
PyList_New() can return NULL to indicate an error, which you aren't checking for. Py_BuildValue() can return NULL to indicate an error, which you aren't checking for. PyList_Append() can return -1 to indicate an error, which you're also not checking for. PyList_Append() doesn't steal the reference, so you're leaking the reference to the dict returned by Py_BuildValue(). The latter may be causing you to run out of memory, which can cause Py_BuildValue() or PyList_Append() to fail, and your failure to handle the error can cause a crash.
(Something else can also cause Py_BuildValue() or PyList_Append() to fail, but that's hard to guess at from just this snippet.)