I am trying to Ingest data in Attivio (Active Intelligence Engine) using suds in python. The document id, Field name are ingested successfully. But fieldValue value is not getting populated. Here's the python code:
from suds.client import Client
url = "http://localhost:17000/ws/bean.attivioIngestWebService/com.attivio.webservice.service.IngestApi?wsdl"
client = Client(url)
cfg = client.factory.create('sessionConfig')
cfg.commitInterval = 1
sessionId = client.service.connect(cfg)
doc = client.factory.create('attivioDocument')
doc._id = "Doc1"
text = client.factory.create('Field')
text._name = "text"
textval = client.factory.create('fieldValue')
textval.value = "Test document text"
text.values = [textval]
doc.fields = [text]
print doc
try:
client.service.feed(sessionId, [doc])
client.service.commit(sessionId)
except Exception as e:
print e
UPDATE:
See the difference between the .net and python SOAP below. Because <value> is defined as anyType, suds doesn't put the type information on it and AIE can't handle it correctly. This maybe the root of problem. Any idea how to fix it?
.net
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<feed xmlns="http://webservice.attivio.com/">
<sessionId xmlns="">704#10.7.3.7_r437gs-f879-4c58-9da5-45erf7as88ex</sessionId>
<docs readOnly="false" id="dotnet" xmlns="">
<fields name="title">
<values>
**<value xsi:type="xsd:string">Hello dot net</value>**
</values>
</fields>
</docs>
</feed>
</s:Body>
</s:Envelope>POST /ws/bean.attivioIngestWebService/com.attivio.webservice.service.IngestApi HTTP/1.1
python
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://webservice.attivio.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns0:Body>
<ns1:feed>
<sessionId>401#192.168.1.100_fnwsf5b5218-9159-4a8f-987a</sessionId>
<docs id="Doc1">
<fields name="text">
<values>
**<value>Test document text</value>**
</values>
<metadata/>
</fields>
<mode/>
</docs>
</ns1:feed>
</ns0:Body>
</SOAP-ENV:Envelope>
Related
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult>true</LoginResult>
<aSessionID>AF-6A-51-FD-E6-8D-C8-12-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-CA</aSessionID>
</LoginResponse>
</soap:Body>
</soap:Envelope>
This xml format coming form sope api i want to read xml aSessionID form this. Please help me to do this in python
list_test.xml:
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult>true</LoginResult>
<aSessionID>AF-6A-51-FD-E6-8D-C8-12-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-CA</aSessionID>
<aSessionID>54F-6A-51-FD-E6-8D-C8-45-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-65</aSessionID>
</LoginResponse>
</soap:Body>
</soap:Envelope>
and then:
from xml.dom import minidom
doc = minidom.parse("list_test.xml")
sessionList = doc.getElementsByTagName('aSessionID')
for sess in sessionList:
print(sess.firstChild.nodeValue)
OUTPUT:
AF-6A-51-FD-E6-8D-C8-12-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-CA
54F-6A-51-FD-E6-8D-C8-45-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-65
EDIT:
To read the xml from a string rather than the file, you may use:
minidom.parseString(xml_str)
Hence:
from xml.dom import minidom
xml_str = '''<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult>true</LoginResult>
<aSessionID>AF-6A-51-FD-E6-8D-C8-12-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-CA</aSessionID>
<aSessionID>54F-6A-51-FD-E6-8D-C8-45-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-65</aSessionID>
</LoginResponse>
</soap:Body>
</soap:Envelope>'''
doc = minidom.parseString(xml_str)
sessionList = doc.getElementsByTagName('aSessionID')
for sess in sessionList:
print(sess.firstChild.nodeValue)
OUTPUT:
AF-6A-51-FD-E6-8D-C8-12-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-CA
54F-6A-51-FD-E6-8D-C8-45-AB-7E-C1-BD-50-7A-43-D0-AA-27-15-65
I'm having issues using suds-jurko (a fork of suds) to handle SOAP services and am running in some issues which seem related to the presence of nested complex types in the wsdl.
Here's the offending service as defined in the wsdl:
<?xml version="1.0" encoding="UTF-8"?>
<definitions name="FormHandler" targetNamespace="http://grid.agnis.net/FormHandler">
<import namespace="http://security.introduce.cagrid.nci.nih.gov/ServiceSecurity" location="FormHandler?wsdl=ServiceSecurity.wsdl">
</import>
<types>
<schema attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://grid.agnis.net/FormHandler">
<import namespace="gme://forms.AGNIS/2.0/net.agnis.forms" schemaLocation="FormHandler?xsd=net.agnis.forms.xsd"/>
<element name="SubmitFormRevisionRequest">
<complexType>
<sequence>
<element name="formRevision">
<complexType>
<sequence>
<element maxOccurs="1" minOccurs="1" ref="ns0:FormRevision"/>
So there is basically a <ns0:FormRevision> element nested inside a <formRevision> (which is taken from the current namespace)
I should be getting this:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="gme://forms.AGNIS/2.0/net.agnis.forms" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://grid.agnis.net/FormHandler" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns1:Body>
<ns2:SubmitFormRevisionRequest>
<ns2:formRevision>
<ns0:FormRevision>
<ns0:form publicId="4637831" version="1.0">
<ns0:originator uniqueName="cibmtr_center_number:XXX"/>
</ns0:form>
But when I print out the envelope, I get the following output:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="gme://forms.AGNIS/2.0/net.agnis.forms" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://grid.agnis.net/FormHandler" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns1:Body>
<ns2:SubmitFormRevisionRequest>
<ns2:formRevision>
<form publicId="4637831" version="1.0">
<originator uniqueName="cibmtr_center_number:XXX"/>
</form>
Notice the missing <ns0:FormRevision> element ? (along with the namespaces :ns0 for the other elements)
Can anyone assist me in fixing this issue ?
Thanks !!
JP
I have a log file containing xml envelopes (2 types of xml structures: request and response). What i need to do is to parse this file, extract xml-s and put them into 2 arrays as strings (1st array for requests and 2nd array for responses), so i can parse them later.
Any ideas how can i achieve this in python ?
Snippet of log file to be parsed (log contains ):
2014-10-31 12:27:33,600 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] Sending BILL request
2014-10-31 12:27:33,601 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] <?xml version="1.0" encoding="UTF-8"?>
<request xmlns="XXX" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<transactionheader>
<username>XXX</username>
<password>XXX</password>
<time>31/10/2014 12:27:33</time>
<clientreferencenumber>123</clientreferencenumber>
<numberrequests>3</numberrequests>
<information>Description</information>
<postbackurl>http://localhost/status</postbackurl>
</transactionheader>
<transactiondetails>
<items>
<item id="1" client="XXX1" keyword="test"/>
<item id="2" client="XXX2" keyword="test"/>
<item id="3" client="XXX3" keyword="test"/>
</items>
</transactiondetails>
</request>
2014-10-31 12:27:34,487 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] Response code 200 for bill request
2014-10-31 12:27:34,489 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] <?xml version="1.0" encoding="UTF-8"?>
<response xmlns="XXX" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<serverreferencenumber>XXX123XXX</serverreferencenumber>
<clientreferencenumber>123</clientreferencenumber>
<information>Queued for Processing</information>
<status>OK</status>
</response>
Many thanks for reply!
Regards,
Robert
As both #Paco and #Lord_Gestalter suggested, you can use xml.etree and replace the non-XML elements from your file, something like this:
# I use re to substitute non-XML elements
import re
# then use xml module as a parser
import xml.etree.ElementTree as ET
# read your file and store in string 's'
with open('yourfilehere','r') as f:
s = f.read()
# then remove non-XML element with re
# I also remove <?xml ...?> part as your file consists of multiple xml logs
s = re.sub(r'<\?xml.*?>', '', ''.join(re.findall(r'<.*>', s)))
# wrap your s with a root element
s = '<root>'+s+'</root>'
# parse s with ElementTree
tree = ET.fromstring(s)
tree
<Element 'root' at 0x7f2ab877e190>
if you don't care about xml parser and just want 'request' & 'response' string, use re.search
with open('yourfilehere','r') as f:
s = f.read()
# put the string of both request and response into 'req' and 'res'
# or you need to construct a better re.search if you have multiple requests, responses
req = [re.search(r'<request.*\/request>', s).group()]
res = [re.search(r'<response.*\/response>', s).group()]
req
['<request xmlns="XXX" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><transactionheader><username>XXX</username><password>XXX</password><time>31/10/2014 12:27:33</time><clientreferencenumber>123</clientreferencenumber><numberrequests>3</numberrequests><information>Description</information><postbackurl>http://localhost/status</postbackurl></transactionheader><transactiondetails><items><item id="1" client="XXX1" keyword="test"/><item id="2" client="XXX2" keyword="test"/><item id="3" client="XXX3" keyword="test"/></items></transactiondetails></request>']
res
['<response xmlns="XXX" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><serverreferencenumber>XXX123XXX</serverreferencenumber><clientreferencenumber>123</clientreferencenumber><information>Queued for Processing</information><status>OK</status></response>']
I have a wsld definition that looks like
...
<sequence>
<element name="version" nillable="false" type="xsd:string"/>
<element name="payment" nillable="false" type="tns1:payment"/>
...
</sequence>
...
This is the xml log of the request that is sent
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://impl.ws.payline.experian.com" ...>
<SOAP-ENV:Header/>
<ns2:Body>
<ns0:doWebPaymentRequest>
<ns0:version>
<ns0:version>4</ns0:version>
<ns0:payment>
<ns1:amount>33300</ns1:amount>
...
</ns0:payment>
</ns0:version>
...
So suds encloses a payment object into version (a string), and it breaks the request.
Why is that ?? Any way to go around this ?
For those wondering :
suds seems buggy when it comes to setting objects on the fly. But it seems to work fine for sending bare xml.
So what I did is have an xml file and replace part of it with what I need, and send it
file: obj.xml
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://impl.ws.payline.experian.com" ...>
...
<xxx>REPLACE_ME</xxx>
...
and the script :
client = Client(url='file://path/to/.wsdl')
xml_request = open('/path/to/obj.xml', 'rb').read()
xml_request = xml_request.replace('REPLACE_ME', value)
result = client.service.TheService(__inject={'msg': xml_request})
SharePoint 2010 Lists Web Service UpdateList fails with "Attempted to use an object that has ceased to exist"...but it DOES exist.
I would like to update an attribute for existing fields. For demonstration purposes, it should be easy to update Description. Using and extending haufe.sharepoint 0.1.9, I am able to query/update items and delete fields. I am confident I am correctly addressing the list having observed changes during update and delete. I believe the field is also accurately addressed because a) it can be deleted and b) if I change the “Name” or “ID”, the error changes to “Field with that name was not found”.
Dumping the SOAP message from SUDS, I can show the different messages and results. Three test cases are shown below. The first is the failure. The second shows a mismatched name results in a different error. The third shows how to delete the field by Name.
Any ideas on where to go next? Might there be a special permission to update a field beyond being able to delete that same field? Although I am subsite owner, I am not a SharePoint admin or server admin. So, looking at logs or installing custom code is difficult/out. That’s why I’m using a Python web service approach. I am nearly completely stumped.
Thanks, Rob
Message: This should work to update the field “Description” but does not. In addition to using the field ID, I’ve tried the Name, DisplayName, and StaticName to no avail. MSFT reference is:
http://msdn.microsoft.com/en-us/library/lists.lists.updatelist%28v=office.12%29.aspx
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:updateFields>
<Fields>
<Method ID="1">
<Field ID="08d8fb05-0de8-4e19-988c-e204ade07f47" Description="new desc"/>
</Method>
</Fields>
</ns1:updateFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Fault is:
<soap:Fault>
<faultcode>soap:Server</faultcode>
<faultstring>Exception of type 'Microsoft.SharePoint.SoapServer.SoapServerException' was thrown.</faultstring>
<detail>
<errorstring xmlns="http://schemas.microsoft.com/sharepoint/soap/">Attempted to use an object that has ceased to exist. (Exception from HRESULT: 0x80030102 (STG_E_REVERTED))</errorstring>
<errorcode xmlns="http://schemas.microsoft.com/sharepoint/soap/">0x80030102</errorcode>
</detail>
</soap:Fault>
Message: Expecting not to find the field and did not. Basically, this proves an incorrect name results in a different error.
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:updateFields>
<Fields>
<Method ID="1">
<Field ID="q08d8fb05-0de8-4e19-988c-e204ade07f47" Description="new desc"/>
</Method>
</Fields>
</ns1:updateFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Message: Able to delete the field like this. This just proves the field can be manipulated in some way successfully.
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:deleteFields>
<Fields>
<Method ID="3">
<Field Name="myText"/>
</Method>
</Fields>
</ns1:deleteFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
This works
Progress made for updating and adding fields! "Fragile" comes to mind as a way to describe the Lists.asmx web service. Notably, Field attributes are order dependent; Type needs to come first - guess maybe there's a wacky "if-tree" in Microsoft's code. In addition, DisplayName is required followed by Name if there might be ambiguity using only DisplayName.
The key debugging trick models a programatically added field after a similar, GUI added, field. To discover proper parameters:
Create a similar field thru the GUI
Set suds logging to debug level.
Open list using haufe.sharepoint's "service = Connector(url, username, password, list_id)" and examine the returned SOAP message for the field's parameters.
Use these attributes to drive the experimentation keeping in mind some fields are read-only or otherwise not intended for external use as defined by Microsoft.
Following are four success examples for updating a text field, updating a calculated formula field, adding a text field, and adding a calculated field. Hopefully this is enough for me and others to build on. Note: haufe.sharepoint does not support these additional methods or return of the results. So...some hacking is required.
Update a text field
Type must be first attribute. Use DisplayName followed by Name to avoid ambiguity with other fields.
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:updateFields>
<Fields>
<Method ID="1">
<Field Type="Text" Name="myText" DisplayName="myText" Description="new desc"/>
</Method>
</Fields>
</ns1:updateFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Update a calculated field
Type must be first attribute. Use DisplayName first and then Name to avoid ambiguity with other fields.
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:updateFields>
<Fields>
<Method ID="1">
<Field Type="Calculated" DisplayName="myCalcAdd" Name="myCalcAdd" ResultType="Number" ReadOnly="TRUE">
<Formula>=Jan*0.5</Formula>
<FormulaDisplayNames>=Jan*0.5</FormulaDisplayNames>
<FieldRefs>
<FieldRef Name="Jan"/>
</FieldRefs>
</Field>
</Method>
</Fields>
</ns1:updateFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Add text field
Type must come first. DisplayName vs Name seems to be the more important of the two attributes.
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:newFields>
<Fields>
<Method ID="1">
<Field Type="Text" Name="myTextAdd" DisplayName="myTextAdd" Description="My first added field"/>
</Method>
</Fields>
</ns1:newFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Add calc field
Type must come first. DisplayName is needed while Name is not. ResultType is also required at least in the case below.
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<ns1:UpdateList>
<ns1:listName>D538A29D-6DD4-423A-9E7D-2697917BDA78</ns1:listName>
<ns1:newFields>
<Fields>
<Method ID="1" AddToView="">
<Field Type="Calculated" DisplayName="myCalcAdd" ResultType="Number">
<Formula>=Jan*0.5</Formula>
<FormulaDisplayNames>=Jan*0.5</FormulaDisplayNames>
<FieldRefs>
<FieldRef Name="Jan"/>
</FieldRefs>
</Field>
</Method>
</Fields>
</ns1:newFields>
</ns1:UpdateList>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>