Develop Reference

Spyne custom XML response

I am using Spyne with Django CMS.
A web service is calling my system and I want to reply with the below.
Can I use Spyne for customize response? Or do I have to go through models?
Please advise.
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<tns:initTestQueryResponse xmlns:tns="http://test.com/interface/test/v2"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://test.com/interface/test/v2 testQuery.xsd"
xmlns:v21="http://test.com/model/generic-query/v2"
xmlns:v22="http://test.com/model/common/v2">
<tns:field key="ID" type="ID">
<v21:description lang="en">Identifier</v21:description>
</tns:field>
<tns:field key="CUSTOMER_NAME" type="TEXT">
<v21:description lang="en">Customer Name</v21:description>
<v21:layoutOptions bold="true" italic="false" direction="HORIZONTAL"/>
</tns:field>
<tns:section key="CUSTOMER">
<v21:description lang="en">Customer</v21:description>
</tns:section>
<tns:advancedQuery>
<tns:criteriaGroup key="CUSTOMER" operator="OR">
<v21:criterion key="ID" />
<v21:criterion key="CUSTOMER_NAME" />
</tns:criteriaGroup>
</tns:advancedQuery>
<tns:advanceQueryPerson>
<tns:criteriaGroup key="CUSTOMER" operator="OR">
<v21:criterion key="ID" />
<v21:criterion key="CUSTOMER_NAME" />
</tns:criteriaGroup>
</tns:advanceQueryPerson>
<tns:context>
<v22:status>OK</v22:status>
</tns:context>
</tns:initTestQueryResponse>
</soapenv:Body>
</soapenv:Envelope>
This the request
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<p:initTestQueryRequest xmlns:p="http://test.com/interface/test/v2"
xmlns:p1="http://test.com/model/common/v2"
xmlns:p2="http://test.com/model/generic-query/v2"
xmlns:p3="http://test.com/model/test/v2"
xmlns:p4="http://test.com/model/service-fault/v2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://test.com/interface/test/v2 ../service/test/v2/TestQuery.xsd "
xsi:type="anyType"/>
</soapenv:Body>
</soapenv:Envelope>

You have two options:
Use a bare method named initTestQuery whose return type should be a class named "initTestQueryResponse" whose namespace is "http://test.com/interface/test/v2". You need to return an initTestQueryResponse instance from the initTestQuery function.
Use a bare method named "initTestQuery" whose return type is AnyXml. You need to return an lxml.etree.Element() that contains the tags you need. Please consult the lxml documentation about how to do that.
If you want to "edit" requests after they are deserialized but before they are validated, you must subclass the protocol and override create_in_document.
class MyProtocol(Soap11):
def create_in_document(self, ctx, charset=None):
super(MyProt, self).create_in_document(ctx, charset=charset)
# Do whatever you want with ctx.in_document
app = Application(in_protocol=MyProtocol(...), ...)
I hope this helps.

Python Suds' MessagePlugin does not seem to be updating my output

I am trying to use suds to make a request to a remote SOAP service. I need to set the namespace on the parameters thus:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://myemployer.com/" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header>
<AuthHeader xmlns="http://myemployer.com/">
<Token>blah</Token>
</AuthHeader>
</SOAP-ENV:Header>
<ns1:Body>
<ns0:AwardBadgesBatch>
<ns0:badgeAwards>
<AwardBadge>
<EnterpriseID>jhoov11</EnterpriseID>
...
</AwardBadge>
</ns0:badgeAwards>
<ns0:includeSuccessStatus>false</ns0:includeSuccessStatus>
</ns0:AwardBadgesBatch>
</ns1:Body>
</SOAP-ENV:Envelope>
Note the 'ns0' on both the badgeAwards and includeSuccessStatus elements. I also need to not have prefixes on the stuff inside badgeAwards.
I am using this code to make the request:
from suds.client import Client
from suds.sax.element import Element
from suds.sax.attribute import Attribute
from suds.plugin import MessagePlugin
import logging
logging.getLogger('suds.client').setLevel(logging.DEBUG)
logging.getLogger('suds.transport').setLevel(logging.DEBUG)
logging.basicConfig(level=logging.DEBUG)
url='https://myemployer.com/BadgingServices.asmx?wsdl'
c = Client(url)
t = c.service.Authenticate('blah')
t = t.Authenticate.Status.Token
header_root = Element('AuthHeader')
header_token = Element('Token').setText(t)
header_attribute = Attribute('xmlns', "http://myemployer.com/")
soap_header = header_root.insert(header_token)
soap_header.append(header_attribute)
class NamespaceFixer(MessagePlugin):
''' must add namespace prefix to all parameters '''
def marshalled(self, context):
body = context.envelope[1]
for i in body[0]:
i.setPrefix('ns0')
print "******* element after setting prefix: %s ********" % i.plain()
nsf = NamespaceFixer()
c.set_options(soapheaders=soap_header, plugins=[nsf])
from suds.sax.element import Element
ba2 = Element('badgeAwards')
ba = Element('AwardBadge')
ba2.append(ba)
entid = Element('EnterpriseID').setText('jhoov11')
ba.append(entid)
...
includeSuccess = Element('includeSuccessStatus').setText('false')
c.service.AwardBadgesBatch(badgeAwards= ba2, includeSuccessStatus=includeSuccess)
Now when I do this, I see the xml text output before the marshall() call, and I see the output from that call:
******* element after setting prefix: <ns0:badgeAwards/> ********
******* element after setting prefix: <ns0:includeSuccessStatus/> ********
but the folks on the other end insist (insist!) that the prefix is not set on those elements when they receive it.
Two questions come to mind:
am I doing something wrong in my MessagePlugin?
is there a way to display the final modified XML text before it gets sent off? I think I'm setting it but there's only so much I can say to them when I can't see the final full text I'm sending.
EDIT:
c.last_sent().plain()
u'<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:ns0="http://pebble.searshc.com/" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header><AuthHeader xmlns="http://myemployer.com/"><Token>blah</Token></AuthHeader></SOAP-ENV:Header>
<ns1:Body><ns0:AwardBadgesBatch><ns0:badgeAwards/><ns0:includeSuccessStatus/></ns0:AwardBadgesBatch></ns1:Body></SOAP-ENV:Envelope>'
so badgeAwards and includeSuccessStatus are being sent empty. ???
EDIT: trying factory. This wsdl is odd (to me)--it defines the call itself as a type, thus:
<s:element name="AwardBadgesBatch">
<s:complexType>
<s:sequence>
<s:element minOccurs="0" maxOccurs="1" name="badgeAwards">
<s:complexType mixed="true">
<s:sequence>
<s:any/>
</s:sequence>
</s:complexType>
</s:element>
<s:element minOccurs="1" maxOccurs="1" name="includeSuccessStatus" type="s:boolean"/>
</s:sequence>
</s:complexType>
</s:element>
...
<wsdl:operation name="AwardBadgesBatch">
<soap:operation soapAction="http://pebble.searshc.com/AwardBadgesBatch" style="document"/>
<wsdl:input>
<soap:body use="literal"/>
<soap:header message="tns:AwardBadgesBatchAuthHeader" part="AuthHeader" use="literal"/>
</wsdl:input>
<wsdl:output>
<soap:body use="literal"/>
<soap:header message="tns:AwardBadgesBatchAuthHeader" part="AuthHeader" use="literal"/>
</wsdl:output>
</wsdl:operation>
It does not define an AwardBadge type. So I just make my AwardBadge thus:
ba = Element('AwardBadge')
entid = Element('EnterpriseID').setText('jhoov11')
ba.append(entid)
btype = Element('badgeTypeID').setText('30')
ba.append(btype)
startdate = Element('startDate').setText('2013-8-22')
ba.append(startdate)
enddate = Element('endDate').setText('9999-12-31')
ba.append(enddate)
isdeleted = Element('isDeleted').setText('0')
ba.append(isdeleted)
and I set the values on my call
call = c.factory.create('AwardBadgesBatch')
call.badgeAwards= [ba]
call.includeSuccessStatus= False
and I call the service
c.service.AwardBadgesBatch(call)
and get some errors. When I look at what I sent I see
c.last_sent().plain()
u'<?xml version="1.0" encoding="UTF-8"?><SOAP-ENV:Envelope xmlns:ns0="http://myemployer.com/" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header>
<AuthHeader xmlns="http://myemployer.com/"><Token>blah</Token></AuthHeader>
</SOAP-ENV:Header>
<ns1:Body>
<ns0:AwardBadgesBatch>
<ns0:badgeAwards>
<AwardBadge/>
<ns0:includeSuccessStatus>false</ns0:includeSuccessStatus>
</ns0:badgeAwards>
<ns0:includeSuccessStatus/>
</ns0:AwardBadgesBatch>
</ns1:Body>
</SOAP-ENV:Envelope>'
So, three problems:
AwardBadge is empty.
the includeSuccessStatus is duplicated, one inside badgeAwards, where it doesn't belong.
the includeSuccessStatus that's in the right place is empty.
It doesn't make a difference whether I set badgeAwards to ba or [ba].
Any more insight?

Suds provides factory for service types - client.service.factory.create(type_name). That gives you python object. Here is demonstration, may be you can adapt it to your case:
...
client = Client(...)
award = client.factory.create("ns0:AwardBadgesBatch")
award.includeSuccessStatus = True
...
Suds has two nice methods - client.last_sent() and client.last_received(). Using them after request you can see what exactly did you send and received. Also, you can know a lot about service printing suds client:
client = Client(...)
print client

I think it is a bug in suds 0.4. Any updates done in MessagePlugins are simply not applied and the original message is send instead.
You can fix it by patching client.py in suds:
--- python2.7/site-packages/suds/client.py Sun Jan 08 16:58:36 2017 +0100
+++ client.py Thu Oct 05 11:04:20 2017 +0200
## -631,7 +631,8 ##
else:
soapenv = soapenv.plain()
soapenv = soapenv.encode('utf-8')
- plugins.message.sending(envelope=soapenv)
+ ctx = plugins.message.sending(envelope=soapenv)
+ soapenv = ctx.envelope
request = Request(location, soapenv)
request.headers = self.headers()
reply = transport.send(request)
Another option might be to switch to maybe switch to the jurko fork of suds, which also fixes this bug. However, I haven't tried that option myself.
The problem is also mentioned in this question: Suds Client ignores modification done by plugins into sending method

python suds wrong namespace prefix in SOAP request

I use python/suds to implement a client and I get wrong namespace prefixes in the sent SOAP header for a spefic type of parameters defined by element ref= in the wsdl.
The .wsdl is referencing a data types .xsd file, see below. The issue is with the function GetRecordAttributes and its first argument of type gbt:recordReferences.
File: browse2.wsdl
<xsd:schema targetNamespace="http://www.grantadesign.com/10/10/Browse" xmlns="http://www.grantadesign.com/10/10/Browse" xmlns:gbt="http://www.grantadesign.com/10/10/GrantaBaseTypes" elementFormDefault="qualified" attributeFormDefault="qualified">
<xsd:import schemaLocation="grantabasetypes2.xsd" namespace="http://www.grantadesign.com/10/10/GrantaBaseTypes"/>
<xsd:element name="GetRecordAttributes">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="gbt:recordReferences">
</xsd:element>
Referenced File : grantabasetypes2.xsd
<element name="recordReferences">
<complexType>
<sequence>
<element name="record" minOccurs="0" maxOccurs="unbounded" type="gbt:MIRecordReference"/>
</sequence>
</complexType>
</element>
SOAP Request sent by suds:
<SOAP-ENV:Envelope xmlns:ns0="http://www.grantadesign.com/10/10/GrantaBaseTypes" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://www.grantadesign.com/10/10/Browse" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns1:Body>
<ns2:GetRecordAttributes>
<ns2:recordReferences>
<ns0:record>
</ns0:record>
</ns2:recordReferences>
</ns2:GetRecordAttributes>
</ns1:Body>
</SOAP-ENV:Envelope>
Problem : <ns2:recordReferences> has wrong prefix, should be <ns0:recordReferences> since it belongs to the namespace ...GrantaBaseTypes defined in the .xsd.
This happens for all arguments defined by ref= in the wsdl. How can this be automatically fixed?
Note: I checked that the "good" prefix is accepted by the service by manually sending the xml SOAP request via curl.
UPDATE
I meddled with SUDS source code and the following empirical fix forces all elements with ref= attribute to assume the ref-ed namespace (previously, they take on the schema root namespace or whatever tns is):
File: /suds/xsd/sxbase.py
class SchemaObject(object):
....
def namespace(self, prefix=None):
ns = self.schema.tns
#FIX BEGIN
if self.ref and self.ref in self.schema.elements.keys():
ns = self.ref
#FIX END
Works with my service, but I'm not sure if it'll break other things. I would prefer a smarter solution that does not change SUDS source code.
Thanks,
Alex

Write a Suds plugin to modify the XML before it is sent.
from suds.client import Client
from suds.plugin import MessagePlugin
class MyPlugin(MessagePlugin):
def marshalled(self, context):
#modify this line to reliably find the "recordReferences" element
context.envelope[1][0][0].setPrefix('ns0')
client = Client(WSDL_URL, plugins=[MyPlugin()])
Quoting Suds documentation:
marshalled()
Provides the plugin with the opportunity to inspect/modify the envelope Document before it is sent.

I had the exact same problem when using suds to access a BizTalk/IIS SOAP service.
From what I can tell from the WSDL it occurs when there is a "complexType" that is not part of the "targetNamespace" (it has it's own), which has a child that is also a complexType, but with no namespace set. In BizTalk this means that the child should belong to the same namespace as the parent, but Suds seem to think that it then should be part of the targetNamespace ....
The fix in the source-code solved the thing "correctly", but since I want to be able to upgrade without applying the fix every time I went for another solution....
My solution was to skip Suds and just copy the raw XML, use that as a template and copy the values into it ... Not beautiful, but at least simple.
The solution to add a plugin is in my opinion equally hardcoded and perhaps even harder to maintain.

You could build soap message yourself and use SoapClient to send the message :
sc = SoapClient(cli.service.XXXMethod.client,cli.service.XXXMethod.method)
sc.send(some_soap_doc)

I prefer regular expressions :)
import re
class EnvelopeFixer(MessagePlugin):
def sending(self, context):
# rimuovi i prefissi
context.envelope = re.sub( 'ns[0-9]:', '', context.envelope )
return context.envelope

Wrong (?) datatype generated with soaplib

I'm having a problem with soaplib.
I've the following function provided by a web service :
#soap(Integer, Integer, _returns=Integer)
def test(self, n1, n2):
return n1 + n2
The corresponding declaration for the datatypes in the generated WSDL file is
<xs:complexType name="test">
<xs:sequence>
<xs:element name="n1" type="xs:integer" minOccurs="0" nillable="true"/>
<xs:element name="n2" type="xs:integer" minOccurs="0" nillable="true"/>
</xs:sequence>
</xs:complexType>
<xs:complexType> name="testResponse">
<xs:sequence>
<xs:element name="testResult" type="xs:integer" minOccurs="0" nillable="true"/>
</xs:sequence>
</xs:complexType>
When I use some IDE (Visual Studio, PowerBuilder) to generate code from that WSDL file, whatever the IDE, it generates two classes for test and for testResponse, whose attributes are Strings.
Does anyone know if I can tweak my Python declaration so that I avoid complexType and obtain real Integer datatype on my client side?

I checked your code but i get same output. I am using suds to parse the values.
In [3]: from suds import client
In [4]: cl = client.Client('http://localhost:8080/?wsdl')
In [5]: cl.service.test(10,2)
Out[5]: 12
But when i check the type of that value.
In [6]: type(cl.service.test(10,2))
Out[6]: <class 'suds.sax.text.Text'>
So SOAPLIB will be return string but from type of that data you can convert it.
I check the response by writing this
#soap(_returns=Integer)
def test(self):
return 12
So i get in SOA Client Plugin of Firefox response as
<?xml version='1.0' encoding='utf-8'?>
<senv:Envelope
xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing"
xmlns:plink="http://schemas.xmlsoap.org/ws/2003/05/partner-link/"
xmlns:xop="http://www.w3.org/2004/08/xop/include"
xmlns:senc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:s12env="http://www.w3.org/2003/05/soap-envelope/"
xmlns:s12enc="http://www.w3.org/2003/05/soap-encoding/"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:senv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">
<senv:Body>
<tns:testResponse>
<tns:testResult>
12
</tns:testResult>
</tns:testResponse>
</senv:Body>
</senv:Envelope>
From XML you cant get raw integer data.

OK, not all datatypes of XSD are defined in soaplib.
Integer is defined in soaplib and is seen in the WSDL file as an integer, which the .NET framework (used by PowerBuilder) failed to understand.
Int is OK for .NET/PowerBuilder but soaplib isn't defined in soaplib.
Thus, I moved from soaplib to rpclib. Those libs are very close (one is a fork of the other).

Fought with the same thing, but couldn't move away from soaplib.
So, I monkeypatch it this way:
from soaplib.serializers.primitive import Integer
class BetterInteger(Integer):
__type_name__ = "int"
Integer = BetterInteger
And then move on with life.
However, the XSD spec defines both 'integer': "Represents a signed integer. Values may begin with an optional "+" or "-" sign. Derived from the decimal datatype." and 'int' "Represents a 32-bit signed integer in the range [-2,147,483,648, 2,147,483,647]. Derived from the long datatype."
So, the better solution is:
from soaplib.serializers.primitive import Integer
class Int32(Integer):
__type_name__ = "int"
And use your new 'Int32' class to type your input parameters.
[soaplib 1.0.0]

lxml parse xsd file without Schema URL

I am using lxml to parse an xsd file and am looking for an easy way to remove the URL namespace attached to each element name. Here's the xsd file:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" version="2.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="rootelement">
<xs:complexType>
<xs:choice maxOccurs="unbounded">
<xs:element minOccurs="1" maxOccurs="1" name="element1">
<xs:complexType>
<xs:all>
<xs:element name="subelement1" type="xs:string" />
<xs:element name="subelement2" type="xs:integer" />
<xs:element name="subelement3" type="xs:dateTime" />
</xs:all>
<xs:attribute name="id" type="xs:integer" use="required" />
</xs:complexType>
</xs:element>
</xs:choice>
<xs:attribute fixed="2.0" name="version" type="xs:decimal" use="required" />
</xs:complexType>
</xs:element>
</xs:schema>
and using this code:
from lxml import etree
parser = etree.XMLParser()
data = etree.parse(open("testschema.xsd"),parser)
root = data.getroot()
rootelement = root.getchildren()[0]
rootelementattribute = rootelement.getchildren()[0].getchildren()[1]
print "root element tags"
print rootelement[0].tag
print rootelementattribute.tag
elements = rootelement.getchildren()[0].getchildren()[0].getchildren()
elements_attribute = elements[0].getchildren()[0].getchildren()[1]
print "element tags"
print elements[0].tag
print elements_attribute.tag
subelements = elements[0].getchildren()[0].getchildren()[0].getchildren()
print "subelements"
print subelements
I get the following output
root element tags
{http://www.w3.org/2001/XMLSchema}complexType
{http://www.w3.org/2001/XMLSchema}attribute
element tags
{http://www.w3.org/2001/XMLSchema}element
{http://www.w3.org/2001/XMLSchema}attribute
subelements
[<Element {http://www.w3.org/2001/XMLSchema}element at 0x7f2998fb16e0>, <Element {http://www.w3.org/2001/XMLSchema}element at 0x7f2998fb1780>, <Element {http://www.w3.org/2001/XMLSchema}element at 0x7f2998fb17d0>]
I don't want "{http://www.w3.org/2001/XMLSchema}" to appear at all when I pull the tag data (altering the xsd file is not an option). The reason I need the xsd tag info is that I am using this to validate column names from a series of flat files. On the "element" level there are multiple elements that I'm pulling, as well as subelements, which I am using a dictionary to validate columns. Also, any suggestions on improving the code above would be greatly, such as a way to use fewer "getchildren" calls, or just make it more organized.

I'd use:
print elem.tag.split('}')[-1]
But you could also use the xpath function local-name():
print elem.xpath('local-name()')
As for fewer getchildren() calls: just leave them out. getchildren() is a deprecated way of making a list of the direct children (you should just use list(elem) instead if you actually want this).
You can iterate over, or use an index on an element directly. For example: rootelement[0] will give you the first child element of rootelement (but more efficient than if you were use rootelement.getchildren()[0], because this would act like list(rootelement) and create a new list first)

I wonder why etree.XMLParser(ns_clean=True) doesn't work. It had not worked for me so did it getting namespace from root.nsmap between brackets and replacing it with empty string
print rootelement[0].tag.replace('{%s}' %root.nsmap['xs'], '')

The easiest thing to do is to just use string slicing to remove namespace prefix:
>>> print rootelement[0].tag[34:]
complexType

If the URI might change in the future (for some unknown reason or you're truly paranoid), consider the following:
print "root element tags"
tag, nsmap, prefix = rootelement[0].tag, rootelement[0].nsmap, rootelement[0].prefix
tag = tag[len(nsmap[prefix]) + 2:]
print tag
This is a very unlikely case, but who knows?