Develop Reference

Append data dynamically to my xml SOAP message

I am calling an API by sending an xml request by doing a string formatting like this:
data = '''<?xml version="1.0" encoding="utf-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SOAP-ENV:Body>
<ns2:MultiAvailabilityRequest xmlns:m="http://www.derbysoft.com/doorway" Password="CoolJoe" Token="{token}" UserName="CoolJoe">
<ns2:MultiAvailabilityCriteria NumberOfUnits="{units}">
<ns2:StayDateRange CheckIn="2016-05-02" CheckOut="2016-05-04"/>
<ns2:GuestCounts>
<ns2:GuestCount AdultCount="{adultcount}"/>
</ns2:GuestCounts>
<ns2:HotelCodes>
<ns2:HotelCode>{hotelcode}</ns2:HotelCode>
</ns2:HotelCodes>
</ns2:MultiAvailabilityCriteria>
</ns2::MultiAvailabilityRequest>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>'''.format(token=token, units=units, adultcount=adultcount, hotelcode=hotelcode)
The above code is working fine and getting the value of different hotelcodes, token etc and showing the results based on them.
But, I have one more different requirement where the hotelcodes could be more than 1 (either 2,3 or more). And, the required xml will look like this:
data = '''<?xml version="1.0" encoding="utf-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SOAP-ENV:Body>
<ns2:MultiAvailabilityRequest xmlns:m="http://www.derbysoft.com/doorway" Password="CoolJoe" Token="{token}" UserName="CoolJoe">
<ns2:MultiAvailabilityCriteria NumberOfUnits="{units}">
<ns2:StayDateRange CheckIn="2016-05-02" CheckOut="2016-05-04"/>
<ns2:GuestCounts>
<ns2:GuestCount AdultCount="{adultcount}"/>
</ns2:GuestCounts>
<ns2:HotelCodes>
<ns2:HotelCode>{hotelcode1}</ns2:HotelCode>
<ns2:HotelCode>{hotelcode2}</ns2:HotelCode>
<ns2:HotelCode>{hotelcode3}</ns2:HotelCode>
</ns2:HotelCodes>
</ns2:MultiAvailabilityCriteria>
</ns2::MultiAvailabilityRequest>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>'''.format(token=token, units=units, adultcount=adultcount)
So, my question is: how do I check whether two hotelcodes are present or more than two. As you can see from second xml for each hotel code, a new line like this adds up:
<ns2:HotelCode>{hotelcode1}</ns2:HotelCode>
Any help would be appreciated. Thanks.

Basically you should split the process in two parts:
fill in the hotel codes (doesn't really matter if it's one or more):
hotelcode_string =''.join(['<ns2:HotelCode>{hotelcode}</ns2:HotelCode>'.format(hotelcode=code) for code in set([item["hotelcode"] for item in hotelcode])])
put the hotel code section in the xml:
data = '''.... <ns2:HotelCodes>{hotelcode_string}</ns2:HotelCodes>
...'''.format(token=token, units=units, adultcount=adultcount,hotelcode_string=hotelcode_string)

Python suds XML element from nested complex type is not created in SOAP envelope

I'm having issues using suds-jurko (a fork of suds) to handle SOAP services and am running in some issues which seem related to the presence of nested complex types in the wsdl.
Here's the offending service as defined in the wsdl:
<?xml version="1.0" encoding="UTF-8"?>
<definitions name="FormHandler" targetNamespace="http://grid.agnis.net/FormHandler">
<import namespace="http://security.introduce.cagrid.nci.nih.gov/ServiceSecurity" location="FormHandler?wsdl=ServiceSecurity.wsdl">
</import>
<types>
<schema attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://grid.agnis.net/FormHandler">
<import namespace="gme://forms.AGNIS/2.0/net.agnis.forms" schemaLocation="FormHandler?xsd=net.agnis.forms.xsd"/>
<element name="SubmitFormRevisionRequest">
<complexType>
<sequence>
<element name="formRevision">
<complexType>
<sequence>
<element maxOccurs="1" minOccurs="1" ref="ns0:FormRevision"/>
So there is basically a <ns0:FormRevision> element nested inside a <formRevision> (which is taken from the current namespace)
I should be getting this:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="gme://forms.AGNIS/2.0/net.agnis.forms" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://grid.agnis.net/FormHandler" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns1:Body>
<ns2:SubmitFormRevisionRequest>
<ns2:formRevision>
<ns0:FormRevision>
<ns0:form publicId="4637831" version="1.0">
<ns0:originator uniqueName="cibmtr_center_number:XXX"/>
</ns0:form>
But when I print out the envelope, I get the following output:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="gme://forms.AGNIS/2.0/net.agnis.forms" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://grid.agnis.net/FormHandler" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns1:Body>
<ns2:SubmitFormRevisionRequest>
<ns2:formRevision>
<form publicId="4637831" version="1.0">
<originator uniqueName="cibmtr_center_number:XXX"/>
</form>
Notice the missing <ns0:FormRevision> element ? (along with the namespaces :ns0 for the other elements)
Can anyone assist me in fixing this issue ?
Thanks !!
JP

I want to remove the curly braces and XML namspace using lxml and just report the tag name

So I have the following XML document It is much longer:
<?xml version ="1.0" encoding="UTF-8" standalone="no" ?>
<!DOCTYPE fmresultset PUBLIC "-//FMI//DTD fmresultset//EN" "http://localhost:16020/fmi/xml/fmresultset.dtd">
<fmresultset xmlns="http://www.filemaker.com/xml/fmresultset" version="1.0">
<error code="0">
</error>
<product build="11/11/2014" name="FileMaker Web Publishing Engine" version="13.0.5.518">
</product>
I use the following python to extract some of the tag names:
doc = etree.fromstring(resulttxt)
print( doc.attrib)
print(doc.tag)
print(doc[4][0][0].tag)
if(doc[4][0][0].tag == 'field'):
print 'hi'
What I'm getting though is:
{'version': '1.0'}
{http://www.filemaker.com/xml/fmresultset}fmresultset
{http://www.filemaker.com/xml/fmresultset}field
The xmlns doesn't show up as an attribute of the root tag but it is there.
And it is placed in front of each tag name which makes it difficult to loop through and use conditionals. I want doc.tag just to show the tag and not the namespace and the tag.
This is day 1 for me using this. could anyone help out?

You need to handle namespaces, in your case an empty one:
from lxml import etree as ET
data = """<?xml version ="1.0" encoding="UTF-8" standalone="no" ?>
<!DOCTYPE fmresultset PUBLIC "-//FMI//DTD fmresultset//EN" "http://localhost:16020/fmi/xml/fmresultset.dtd">
<fmresultset xmlns="http://www.filemaker.com/xml/fmresultset" version="1.0">
<error code="0">
</error>
<product build="11/11/2014" name="FileMaker Web Publishing Engine" version="13.0.5.518">
</product>
</fmresultset>
"""
namespaces = {
"myns": "http://www.filemaker.com/xml/fmresultset"
}
tree = ET.fromstring(data)
print tree.find("myns:product", namespaces=namespaces).attrib.get("name")
Prints:
FileMaker Web Publishing Engine

How to extract xml from log file to parse in python

I have a log file containing xml envelopes (2 types of xml structures: request and response). What i need to do is to parse this file, extract xml-s and put them into 2 arrays as strings (1st array for requests and 2nd array for responses), so i can parse them later.
Any ideas how can i achieve this in python ?
Snippet of log file to be parsed (log contains ):
2014-10-31 12:27:33,600 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] Sending BILL request
2014-10-31 12:27:33,601 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] <?xml version="1.0" encoding="UTF-8"?>
<request xmlns="XXX" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<transactionheader>
<username>XXX</username>
<password>XXX</password>
<time>31/10/2014 12:27:33</time>
<clientreferencenumber>123</clientreferencenumber>
<numberrequests>3</numberrequests>
<information>Description</information>
<postbackurl>http://localhost/status</postbackurl>
</transactionheader>
<transactiondetails>
<items>
<item id="1" client="XXX1" keyword="test"/>
<item id="2" client="XXX2" keyword="test"/>
<item id="3" client="XXX3" keyword="test"/>
</items>
</transactiondetails>
</request>
2014-10-31 12:27:34,487 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] Response code 200 for bill request
2014-10-31 12:27:34,489 INFO Recharger_MTelemedia2Channel [mbpa.module.mgw.mtelemedia.mtbilling.MTSender][] <?xml version="1.0" encoding="UTF-8"?>
<response xmlns="XXX" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<serverreferencenumber>XXX123XXX</serverreferencenumber>
<clientreferencenumber>123</clientreferencenumber>
<information>Queued for Processing</information>
<status>OK</status>
</response>
Many thanks for reply!
Regards,
Robert

As both #Paco and #Lord_Gestalter suggested, you can use xml.etree and replace the non-XML elements from your file, something like this:
# I use re to substitute non-XML elements
import re
# then use xml module as a parser
import xml.etree.ElementTree as ET
# read your file and store in string 's'
with open('yourfilehere','r') as f:
s = f.read()
# then remove non-XML element with re
# I also remove <?xml ...?> part as your file consists of multiple xml logs
s = re.sub(r'<\?xml.*?>', '', ''.join(re.findall(r'<.*>', s)))
# wrap your s with a root element
s = '<root>'+s+'</root>'
# parse s with ElementTree
tree = ET.fromstring(s)
tree
<Element 'root' at 0x7f2ab877e190>
if you don't care about xml parser and just want 'request' & 'response' string, use re.search
with open('yourfilehere','r') as f:
s = f.read()
# put the string of both request and response into 'req' and 'res'
# or you need to construct a better re.search if you have multiple requests, responses
req = [re.search(r'<request.*\/request>', s).group()]
res = [re.search(r'<response.*\/response>', s).group()]
req
['<request xmlns="XXX" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><transactionheader><username>XXX</username><password>XXX</password><time>31/10/2014 12:27:33</time><clientreferencenumber>123</clientreferencenumber><numberrequests>3</numberrequests><information>Description</information><postbackurl>http://localhost/status</postbackurl></transactionheader><transactiondetails><items><item id="1" client="XXX1" keyword="test"/><item id="2" client="XXX2" keyword="test"/><item id="3" client="XXX3" keyword="test"/></items></transactiondetails></request>']
res
['<response xmlns="XXX" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><serverreferencenumber>XXX123XXX</serverreferencenumber><clientreferencenumber>123</clientreferencenumber><information>Queued for Processing</information><status>OK</status></response>']

Unable to ingest data using python suds

I am trying to Ingest data in Attivio (Active Intelligence Engine) using suds in python. The document id, Field name are ingested successfully. But fieldValue value is not getting populated. Here's the python code:
from suds.client import Client
url = "http://localhost:17000/ws/bean.attivioIngestWebService/com.attivio.webservice.service.IngestApi?wsdl"
client = Client(url)
cfg = client.factory.create('sessionConfig')
cfg.commitInterval = 1
sessionId = client.service.connect(cfg)
doc = client.factory.create('attivioDocument')
doc._id = "Doc1"
text = client.factory.create('Field')
text._name = "text"
textval = client.factory.create('fieldValue')
textval.value = "Test document text"
text.values = [textval]
doc.fields = [text]
print doc
try:
client.service.feed(sessionId, [doc])
client.service.commit(sessionId)
except Exception as e:
print e
UPDATE:
See the difference between the .net and python SOAP below. Because <value> is defined as anyType, suds doesn't put the type information on it and AIE can't handle it correctly. This maybe the root of problem. Any idea how to fix it?
.net
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<feed xmlns="http://webservice.attivio.com/">
<sessionId xmlns="">704#10.7.3.7_r437gs-f879-4c58-9da5-45erf7as88ex</sessionId>
<docs readOnly="false" id="dotnet" xmlns="">
<fields name="title">
<values>
**<value xsi:type="xsd:string">Hello dot net</value>**
</values>
</fields>
</docs>
</feed>
</s:Body>
</s:Envelope>POST /ws/bean.attivioIngestWebService/com.attivio.webservice.service.IngestApi HTTP/1.1
python
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://webservice.attivio.com/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns0:Body>
<ns1:feed>
<sessionId>401#192.168.1.100_fnwsf5b5218-9159-4a8f-987a</sessionId>
<docs id="Doc1">
<fields name="text">
<values>
**<value>Test document text</value>**
</values>
<metadata/>
</fields>
<mode/>
</docs>
</ns1:feed>
</ns0:Body>
</SOAP-ENV:Envelope>