TPA.wsdl https://pastebin.com/7DBhCHbv
DataService.xsd https://pastebin.com/AFhg64hH
from zeep import Client
import base64
from requests import Session
from zeep.wsse.username import UsernameToken
from zeep.transports import Transport
from zeep.exceptions import Fault
Username = '....'
Password = '....'
sendFile = 'V07_220110.ffdata'
session = Session()
session.verify = False
try:
wsdl = 'TPA.wsdl'
# initialize zeep client
client = Client(
wsdl=wsdl,
wsse=UsernameToken(Username, Password),
transport=Transport(session=session)
)
with open(sendFile, "rb") as pdf_file:
encoded_string = base64.b64encode(pdf_file.read())
with client.options(raw_response=True):
node = client.service.uploadEdasDraft(sendFile, encoded_string )
print(node.content)
except Fault as fault:
parsed_fault_detail = client.wsdl.types.deserialize(fault.detail[0])
Always getting Response
I got error ORA-31011: XML parsing failed
From SOAPUI everything sending ok with Enable MTOM settings
So how to make request to it, and how debug sending requests?
Based on that WSDL file, a code like this:
from zeep import Client
from zeep.wsse.username import UsernameToken
username = 'username'
password = 'p#$$word'
file_name = 'test.txt'
client = Client(wsdl = 'TPA.wsdl',
wsse = UsernameToken(username, password))
with open(file_name, "rb") as f:
content = f.read()
client.service.uploadEdasDraft(file_name, content)
should produce something like this:
<soap-env:Envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope">
<soap-env:Header>
<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken>
<wsse:Username>username</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">p#$$word</wsse:Password>
</wsse:UsernameToken>
</wsse:Security>
</soap-env:Header>
<soap-env:Body>
<ns0:uploadEdasDraft xmlns:ns0="http://types.data.external.ws.edas.sodra.epr.lt">
<fileName>test.txt</fileName>
<document>dGhpcyBpcyBhIHRlc3QNCnRoaXMgaXMgYSB0ZXN0DQp0aGlzIGlzIGEgdGVzdA0KdGhpcyBpcyBhIHRlc3QNCg==</document>
</ns0:uploadEdasDraft>
</soap-env:Body>
</soap-env:Envelope>
Since the document type is marked as xsd:base64Binary zeep should handle the base64 encoding for you (your code seems to be doing the encoding twice).
In the example above I'm using a text file, but I assume your file named V07_220110.ffdata is an XML file, since that's what this attribute says: xmime:expectedContentTypes="application/xml". Server will probably complain if you don't send a file with this content type. This may also be a possible cause for that "ORA-31011: XML parsing failed" message, together with the double encoding (server is expecting XML in the document but finds another base64 string).
From SOAPUI everything sending ok with Enable MTOM settings
When using MTOM, your file is not encoded within the SOAP message as text, but it's attached as binary next to it, and you get references to that part of the message. See an explanation here: How does MTOM work?
Your document element might change to something like:
<document>
<inc:Include href="cid:123456789" xmlns:inc="http://www.w3.org/2004/08/xop/include"/>
</document>
where 123456789 is a reference to the content id in the multipart message.
Does your call work from SoapUI only if you enable MTOM? Have you tried sending a base64 encoded file string within the SOAP message using SoapUI?
If your call only works with MTOM then you have a problem, because I'm not sure zeep can handle it out of the box.
The documentation (https://docs.python-zeep.org/en/master/attachments.html) mentions only a multipart response and how you can read the file from the response, but says nothing about making requests. See for example these items:
https://github.com/mvantellingen/python-zeep/issues/599
https://github.com/mvantellingen/python-zeep/pull/314
https://github.com/remaudcorentin-dev/python-zeep-adv
The last project might help you with some code samples (https://github.com/remaudcorentin-dev/python-zeep-adv/blob/master/src/zeep/transport_with_attach.py) but do consider the caveat:
This has been developed for a specific usage and this code should probably not be used (has it) for other puposes.
(or at your own risks ;) )
So it seems you might have to build your own multipart request from scratch or from that sample in the project.
[...] and how debug sending requests?
You might use the HistoryPlugin just to see what messages get exchanged between your client and server, but since you might need to see all of the request, I suggest Wireshark, TcpMon (old, defunct, but still useful), or SoapUI with TcpMon.
This might not be the answer you are looking for, but hope that at least it leaves you more equipped in figuring out how to make the call. Zeep unfortunately is a small fish client in a large pond of WS specifications.
Related
I'm trying to POST data to a URL using Pythons requests library.
If I try and do this by setting a multiline string variable which contains the post data in my script, everything works fine.
If I try to read in an external file with the same data in, the request fails on the application server I'm posting to, because it thinks there is invalid XML.
For example:
This works
starturl="http://myserver.example.com/location/where/I/post"
username=user
password=mypassword
# Set the XML data
xmldata="""<?xml version="1.0" encoding="utf-8"?>
(Lots more xml)
"""
# POST the job data
session = requests.Session()
request = session.post(starturl, auth=(username,password), data=xmldata, headers=post_headers)
Server side application processes the request just fine. However, if the only change I make is to read the xml data from an external file, this no longer works.
This does not work
xmlfile="/path/to/my/xmldata.xml"
xmldata = open(xmlfile,'r')
session = requests.Session()
request = session.post(start_url, auth=(username,password), data=xmldata.read(), headers=post_headers)
The server side application, then errors with:
"Data at the root level is invalid. Line 1, position 1"
When inspecting with wireshark I can see there is a difference in the request body of my POST. Three little dots are appearing from somewhere
When it works:
Content-Type: application/xml
Authorization: Basic c3BvdGFkbTpQQHNzdzByZA==
<?xml version="1.0" encoding="utf-8"?>
When it fails:
Content-Type: application/xml
Authorization: Basic c3BvdGFkbTpQQHNzdzByZA==
...<?xml version="1.0" encoding="utf-8"?>
I'm not sure what's causing the 3 leading dots to appear in the request body. I've inspected the source XML file, tried stripping newlines from it. Nothing seems to do the trick?
It's impossible to tell for sure without having your xml file, but you might have a BOM at the beginning of your file. Microsoft is notably (in)famous for insisting on putting useless BOM on all utf-8 files.
You can check the first three characters of your file for the codecs.BOM_UTF8 sequence ('\xef\xbb\xbf') and strip it out if it's there.
i'm testing out SUDS library and I'm trying to make a simple request to an endpoint but i get unusual output. Why?
from suds.client import Client
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.xsd.schema').setLevel(logging.DEBUG)
url = "http://xmlgw.companieshouse.gov.uk/v1-0/xmlgw/Gateway"
client = Client(url)
print client
Output:
Martynass-MacBook-Air:CH martynas$ python ch.py
DEBUG:suds.xsd.schema:loaded:
schema collection
Schema:0x109a7db90
(raw)
<schema/>
(model)
DEBUG:suds.xsd.schema:MERGED:
Schema:0x109a7db90
(raw)
<schema/>
(model)
You can't use suds for this servce, suds is based on SOAP, which is another web service protocol. What you can do is send an xml request and get a response.
import requests
target_url = "http://xmlgw.companieshouse.gov.uk/v1-0/xmlgw/Gateway"
headers={'Content-type': 'text/xml'}
print requests.post(target_url, data=xml, headers=headers).text
Where the xml is defined according to their schemes.
http://xmlgw.companieshouse.gov.uk/example_http.html
This is one examaple
xml = ('''
<GovTalkMessage xmlns="http://www.govtalk.gov.uk/schemas/govtalk/govtalkheader"
xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"
xmlns:gt="http://www.govtalk.gov.uk/schemas/govtalk/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.govtalk.gov.uk/schemas/govtalk/govtalkheader">
<EnvelopeVersion>1.0</EnvelopeVersion>
<Header>
<MessageDetails>
<Class>CompanyDetails</Class>
<Qualifier>request</Qualifier>
<TransactionID>14456553</TransactionID>
</MessageDetails>
<SenderDetails>
<IDAuthentication>
<SenderID>My_SenderID</SenderID>
<Authentication>
<Method>CHMD5</Method>
<Value>e999e113407884fa410fa2f53bc23952</Value>
</Authentication>
</IDAuthentication>
<EmailAddress>sometest#some.email.address</EmailAddress>
</SenderDetails>
</Header>
<GovTalkDetails>
<Keys/>
</GovTalkDetails>
<Body>
<CompanyDetailsRequest xmlns="http://xmlgw.companieshouse.gov.uk/v1-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlgw.companieshouse.gov.uk/v1-0/schema/CoDets.xsd">
<CompanyNumber>01002361</CompanyNumber>
<GiveMortTotals>1</GiveMortTotals>
</CompanyDetailsRequest>
</Body>
</GovTalkMessage>
''')
<Class>CompanyDetails</Class> What type of info you're getting. kinda what "function" to call
<Authentication>
<Method>CHMD5</Method>
<Value>e999e113407884fa410fa2f53bc23952</Value>
</Authentication>
</IDAuthentication>
Here you would put the login info i guess
<CompanyDetailsRequest xmlns="http://xmlgw.companieshouse.gov.uk/v1-0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlgw.companieshouse.gov.uk/v1-0/schema/CoDets.xsd">
<CompanyNumber>01002361</CompanyNumber>
<GiveMortTotals>1</GiveMortTotals>
</CompanyDetailsRequest>
The "function" call and it's parameters
Now this will give me a response telling me the authorization failed. So if you have an account there, this should work for you.
Here you can find the list of schemes they have for different types of request. Some of them have sample request to help you out.
http://xmlgw.companieshouse.gov.uk/v1-0/xmlgw/SchemaStatusOutput
Here is the complete guide of all their schemes.
http://xmlgw.companieshouse.gov.uk/data_usage_guide_dec_2013.pdf
There aren't any wsdl definitions for that site. Try something like
http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL
for your url
then you can try something like
client.service.GetWeatherInformation()
From the suds document, "You will need to know the url for WSDL for each service used."
An explicit example
from suds.client import Client
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.xsd.schema').setLevel(logging.DEBUG)
url = " http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL"
client = Client(url)
client.service.GetWeatherInformation()
Outputs a ton of data.
Suds does not make it easy to discover the service, it's better to first test a bit with soapui or generate a human-readable doc of the wsdl with this xslt : http://code.google.com/p/wsdl-viewer/ . So you know the structure of requests and replies, and which services are available.
Requests and responses in soap are xml trees, so once you get the result, you need to access the content of the xml tag that contains the information you're interested in. Here is an example that should work ( I don't have a username, but the result.Status.Success works ).
import suds
client = suds.client.Client("http://webservices.data-8.co.uk/companieshouse.asmx?WSDL")
result = client.service.GetCompanyDetails("username", "password", 1234)
print result.Status.Success
print result.Result.CompanyName
You can not make a request against .xsd. XSD is definition of the exchanged message. You must make a request against webservice Looking here you can find more info about that web service. But also there is pricing page indicating that you must pay to use their service. Probably when you pay you will get username and password to authenticate with the service.
I am trying to add an attachment to my timeline with the multipart encoding. I've been doing something like the following:
req = urllib2.Request(url,data={body}, header={header})
resp = urllib2.urlopen(req).read()
And it has been working fine for application/json. However, I'm not sure how to format the body for multipart. I've also used some libraries: requests and poster and they both return 401 for some reason.
How can I make a multipart request either with a libary(preferably a plug-in to urllib2) or with urllib2 itself (like the block of code above)?
EDIT:
I also would like this to be able to support the mirror-api "video/vnd.google-glass.stream-url" from https://developers.google.com/glass/timeline
For the request using poster library here is the code:
register_openers()
datagen, headers = multipart_encode({'image1':open('555.jpg', 'rb')})
Here it is using requets:
headers = {'Authorization' : 'Bearer %s' % access_token}
files = {'file': open('555.jpg', 'rb')}
r = requests.post(timeline_url,files=files, headers=headers)
Returns 401 -> header
Thank you
There is a working Curl example of a multipart request that uses the streaming video url feature here:
Previous Streaming Video Answer with Curl example
It does exactly what you are trying to do, but with Curl. You just need to adapt that to your technology stack.
The 401 you are receiving is going to prevent you even if you use the right syntax. A 401 response indicates you do not have authorization to modify the timeline. Make sure you can insert a simple hello world text only card first. Once you get past the 401 error and get into parsing errors and format issues the link above should be everything you need.
One last note, you don't need urllib2, the Mirror API team dropped a gem of a feature in our lap and we don't need to be bothered with getting the binary of the video, check that example linked above I only provided a URL in the multipart payload, no need to stream the binary data! Google does all the magic in XE6 and above for us.
Thanks Team Glass!
I think you will find this is simpler than you think. Try out the curl example and watch out for incompatible video types, when you get that far, if you don't use a compatible type it will appear not to work in Glass, make sure your video is encoded in a Glass friendly format.
Good luck!
How to add an attachment to a timeline with multipart encoding:
The easiest way to add attachments with multipart encoding to a timeline is to use the
Google APIs Client Library for Python. With this library, you can simple use the following example code provided in the Mirror API timeline insert documentation (click the Python tab under Examples).
from apiclient.discovery import build
service = build('mirror', 'v1')
def insert_timeline_item(service, text, content_type=None, attachment=None,
notification_level=None):
timeline_item = {'text': text}
media_body = None
if notification_level:
timeline_item['notification'] = {'level': notification_level}
if content_type and attachment:
media_body = MediaIoBaseUpload(
io.BytesIO(attachment), mimetype=content_type, resumable=True)
try:
return service.timeline().insert(
body=timeline_item, media_body=media_body).execute()
except errors.HttpError, error:
print 'An error occurred: %s' % error
You cannot actually use requests or poster to automatically encode your data, because these libraries encode things in multipart/form-data whereas Mirror API wants things in multipart/related.
How to debug your current error code:
Your code gives a 401, which is an authorization error. This means you are probably failing to include your access token with your requests. To include an access token, set the Authorization field to Bearer: YOUR_ACCESS_TOKEN in your request (documentation here).
If you do not know how to get an access token, the Glass developer docs has a page here explaining how to obtain an access token. Make sure that your authorization process requested the following scope for multipart-upload, otherwise you will get a 403 error. https://www.googleapis.com/auth/glass.timeline
This is how I did it and how the python client library does it.
from email.mime.multipart import MIMEMultipart
from email.mime.nonmultipart import MIMENonMultipart
from email.mime.image import MIMEImage
mime_root = MIMEMultipart('related', '===============xxxxxxxxxxxxx==')
headers= {'Content-Type': 'multipart/related; '
'boundary="%s"' % mime_root.get_boundary(),
'Authorization':'Bearer %s' % access_token}
setattr(mime_root, '_write_headers', lambda self: None)
#Create the metadata part of the MIME
mime_text = MIMENonMultipart(*['application','json'])
mime_text.set_payload("{'text':'waddup doe!'}")
print "Attaching the json"
mime_root.attach(mime_text)
if method == 'Image':
#DO Image
file_upload = open('555.jpg', 'rb')
mime_image = MIMENonMultipart(*['image', 'jpeg'])
#add the required header
mime_image['Content-Transfer-Encoding'] = 'binary'
#read the file as binary
mime_image.set_payload(file_upload.read())
print "attaching the jpeg"
mime_root.attach(mime_image)
elif method == 'Video':
mime_video = MIMENonMultipart(*['video', 'vnd.google-glass.stream-url'])
#add the payload
mime_video.set_payload('https://dl.dropboxusercontent.com/u/6562706/sweetie-wobbly-cat-720p.mp4')
mime_root.attach(mime_video)
Mark Scheel I used your video for testing purposes :) Thank you.
I'm trying to replace curl with Python & the requests library. With curl, I can upload a single XML file to a REST server with the curl -T option. I have been unable to do the same with the requests library.
A basic scenario works:
payload = '<person test="10"><first>Carl</first><last>Sagan</last></person>'
headers = {'content-type': 'application/xml'}
r = requests.put(url, data=payload, headers=headers, auth=HTTPDigestAuth("*", "*"))
When I change payload to a bigger string by opening an XML file, the .put method hangs (I use the codecs library to get a proper unicode string). For example, with a 66KB file:
xmlfile = codecs.open('trb-1996-219.xml', 'r', 'utf-8')
headers = {'content-type': 'application/xml'}
content = xmlfile.read()
r = requests.put(url, data=content, headers=headers, auth=HTTPDigestAuth("*", "*"))
I've been looking into using the multipart option (files), but the server doesn't seem to like that.
So I was wondering if there is a way to simulate curl -T behaviour in Python requests library.
UPDATE 1:
The program hangs in textmate, but throws an UnicodeEncodeError error on the commandline. Seems that must be the problem. So the question would be: is there a way to send unicode strings to a server with the requests library?
UPDATE 2:
Thanks to the comment of Martijn Pieters the UnicodeEncodeError went away, but a new issue turned up.
With a literal (ASCII) XML string, logging shows the following lines:
2012-11-11 15:55:05,154 INFO Starting new HTTP connection (1): my.ip.address
2012-11-11 15:55:05,294 DEBUG "PUT /v1/documents?uri=/example/test.xml HTTP/1.1" 401 211
2012-11-11 15:55:05,430 DEBUG "PUT /v1/documents?uri=/example/test.xml HTTP/1.1" 201 0
Seems the server always bounces the first authentication attempt (?) but then accepts the second one.
With a file object (open('trb-1996-219.xml', 'rb')) passed to data, the logfile shows:
2012-11-11 15:50:54,309 INFO Starting new HTTP connection (1): my.ip.address
2012-11-11 15:50:55,105 DEBUG "PUT /v1/documents?uri=/example/test.xml HTTP/1.1" 401 211
2012-11-11 15:51:25,603 WARNING Retrying (0 attempts remain) after connection broken by 'BadStatusLine("''",)': /v1/documents?uri=/example/test.xml
So, first attempt is blocked as before, but no second attempt is made.
According to Martijn Pieters (below), the second issue can be explained by a faulty server (empty line).
I will look into this, but if someone has a workaround (apart from using curl) I wouldn't mind hearing it.
And I am still surprised that the requests library behaves so differently for small string and file object. Isn't the file object serialized before it gets to the server anyway?
To PUT large files, don't read them into memory. Simply pass the file as the data keyword:
xmlfile = open('trb-1996-219.xml', 'rb')
headers = {'content-type': 'application/xml'}
r = requests.put(url, data=xmlfile, headers=headers, auth=HTTPDigestAuth("*", "*"))
Moreover, you were opening the file as unicode (decoding it from UTF-8). As you'll be sending it to a remote server, you need raw bytes, not unicode values, and you should open the file as a binary instead.
Digest authentication always requires you to make at least two request to the server. The first request doesn't contain any authentication data. This first request will fail with a 401 "Authorization required" response code and a digest challenge (called a nounce) to be used for hashing your password etc. (the exact details don't matter here). This is used to make a second request to the server containing your credentials hashed with the challenge.
The problem is in the this two step authentication: your large file was already send with the first unauthorized request (send in vain) but on the second request the file object is already at the EOF position. Since the file size was also send in the Content-length header of the second request, this causes the server to wait for a file that will never be send.
You could solve it using a requests Session and first make a simple request for authentication purposes (say a GET request). Then make a second PUT request containing the actual payload using the same digest challenge form the first request.
sess = requests.Session()
sess.auth = HTTPDigestAuth("*", "*")
sess.get(url)
headers = {'content-type': 'application/xml'}
with codecs.open('trb-1996-219.xml', 'r', 'utf-8') as xmlfile:
sess.put(url, data=xmlfile, headers=headers)
i used requests in python to upload an XML file using the commands.
first to open the file use open()
file = open("PIR.xsd")
fragment = file.read()
file.close()
copy the data of XML file in the payload of the requests and post it
payload = {'key':'PFAkrzjmuZR957','xmlFragment':fragment}
r = requests.post(URL,data=payload)
to check the html validation code
print (r.text)
I'm having trouble with a POST using httplib. Here is the code:
import base64
import urllib
import httplib
http = urllib3.PoolManager()
head = {"Authorization":"Basic %s" % base64.encodestring("foo:bar")}
fields = {"token":"088cfe772ce0b7760186fe4762843a11"}
conn = httplib.HTTPSConnection("foundation.iplantc.org")
conn.set_debuglevel(2)
conn.request('POST', '/auth-v1/renew', urllib.urlencode(fields), head)
print conn.getresponse().read()
conn.close()
The POST that comes out is correct. I know I started a telnet session and typing it in worked fine. Here it is:
'POST /auth-v1/renew HTTP/1.1\r\nHost: foundation.iplantc.org\r\nAccept-Encoding: identity\r\nContent-Length: 38\r\nAuthorization: Basic YXRlcnJlbDpvTnl12aesf==\n\r\n\r\ntoken=088cfe772ce0b7760186fe4762843a11'
But the response from the server is "token not found" when the python script sends it. BTW this does work fine with urllib3 (urllib2 shows the same error), which uses an multipart encoding, but I want to know what is going wrong with the above. I would rather not depend on yet another 3rd party package.
httplib doesn't automatically add a Content-Type header, you have to add it yourself.
(urllib2 automatically adds application/x-www-form-urlencoded as Content-Type).
But what's probably throwing the server off is the additional '\n' after your authorization header, introduced by base64.encodestring. You should better use base64.urlsafe_b64encode instead.