I want to use a WSDL SOAP based web service in Python. I have looked at the Dive Into Python code but the SOAPpy module does not work under Python 2.5.
I have tried using suds which works partly, but breaks with certain types (suds.TypeNotFound: Type not found: 'item').
I have also looked at Client but this does not appear to support WSDL.
And I have looked at ZSI but it looks very complex. Does anyone have any sample code for it?
The WSDL is https://ws.pingdom.com/soap/PingdomAPI.wsdl and works fine with the PHP 5 SOAP client.
I would recommend that you have a look at SUDS
"Suds is a lightweight SOAP python client for consuming Web Services."
There is a relatively new library which is very promising and albeit still poorly documented, seems very clean and pythonic: python zeep.
See also this answer for an example.
I recently stumbled up on the same problem. Here is the synopsis of my solution:
Basic constituent code blocks needed
The following are the required basic code blocks of your client application
Session request section: request a session with the provider
Session authentication section: provide credentials to the provider
Client section: create the Client
Security Header section: add the WS-Security Header to the Client
Consumption section: consume available operations (or methods) as needed
What modules do you need?
Many suggested to use Python modules such as urllib2 ; however, none of the modules work-at least for this particular project.
So, here is the list of the modules you need to get.
First of all, you need to download and install the latest version of suds from the following link:
pypi.python.org/pypi/suds-jurko/0.4.1.jurko.2
Additionally, you need to download and install requests and suds_requests modules from the following links respectively ( disclaimer: I am new to post in here, so I can't post more than one link for now).
pypi.python.org/pypi/requests
pypi.python.org/pypi/suds_requests/0.1
Once you successfully download and install these modules, you are good to go.
The code
Following the steps outlined earlier, the code looks like the following:
Imports:
import logging
from suds.client import Client
from suds.wsse import *
from datetime import timedelta,date,datetime,tzinfo
import requests
from requests.auth import HTTPBasicAuth
import suds_requests
Session request and authentication:
username=input('Username:')
password=input('password:')
session = requests.session()
session.auth=(username, password)
Create the Client:
client = Client(WSDL_URL, faults=False, cachingpolicy=1, location=WSDL_URL, transport=suds_requests.RequestsTransport(session))
Add WS-Security Header:
...
addSecurityHeader(client,username,password)
....
def addSecurityHeader(client,username,password):
security=Security()
userNameToken=UsernameToken(username,password)
timeStampToken=Timestamp(validity=600)
security.tokens.append(userNameToken)
security.tokens.append(timeStampToken)
client.set_options(wsse=security)
Please note that this method creates the security header depicted in Fig.1. So, your implementation may vary depending on the correct security header format provided by the owner of the service you are consuming.
Consume the relevant method (or operation) :
result=client.service.methodName(Inputs)
Logging:
One of the best practices in such implementations as this one is logging to see how the communication is executed. In case there is some issue, it makes debugging easy. The following code does basic logging. However, you can log many aspects of the communication in addition to the ones depicted in the code.
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
logging.getLogger('suds.transport').setLevel(logging.DEBUG)
Result:
Here is the result in my case. Note that the server returned HTTP 200. This is the standard success code for HTTP request-response.
(200, (collectionNodeLmp){
timestamp = 2014-12-03 00:00:00-05:00
nodeLmp[] =
(nodeLmp){
pnodeId = 35010357
name = "YADKIN"
mccValue = -0.19
mlcValue = -0.13
price = 36.46
type = "500 KV"
timestamp = 2014-12-03 01:00:00-05:00
errorCodeId = 0
},
(nodeLmp){
pnodeId = 33138769
name = "ZION 1"
mccValue = -0.18
mlcValue = -1.86
price = 34.75
type = "Aggregate"
timestamp = 2014-12-03 01:00:00-05:00
errorCodeId = 0
},
})
Zeep is a decent SOAP library for Python that matches what you're asking for: http://docs.python-zeep.org
Right now (as of 2008), all the SOAP libraries available for Python suck. I recommend avoiding SOAP if possible. The last time we where forced to use a SOAP web service from Python, we wrote a wrapper in C# that handled the SOAP on one side and spoke COM out the other.
I periodically search for a satisfactory answer to this, but no luck so far. I use soapUI + requests + manual labour.
I gave up and used Java the last time I needed to do this, and simply gave up a few times the last time I wanted to do this, but it wasn't essential.
Having successfully used the requests library last year with Project Place's RESTful API, it occurred to me that maybe I could just hand-roll the SOAP requests I want to send in a similar way.
Turns out that's not too difficult, but it is time consuming and prone to error, especially if fields are inconsistently named (the one I'm currently working on today has 'jobId', JobId' and 'JobID'. I use soapUI to load the WSDL to make it easier to extract endpoints etc and perform some manual testing. So far I've been lucky not to have been affected by changes to any WSDL that I'm using.
It's not true SOAPpy does not work with Python 2.5 - it works, although it's very simple and really, really basic. If you want to talk to any more complicated webservice, ZSI is your only friend.
The really useful demo I found is at http://www.ebi.ac.uk/Tools/webservices/tutorials/python - this really helped me to understand how ZSI works.
If you're rolling your own I'd highly recommend looking at http://effbot.org/zone/element-soap.htm.
SOAPpy is now obsolete, AFAIK, replaced by ZSL. It's a moot point, because I can't get either one to work, much less compile, on either Python 2.5 or Python 2.6
#!/usr/bin/python
# -*- coding: utf-8 -*-
# consume_wsdl_soap_ws_pss.py
import logging.config
from pysimplesoap.client import SoapClient
logging.config.dictConfig({
'version': 1,
'formatters': {
'verbose': {
'format': '%(name)s: %(message)s'
}
},
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'verbose',
},
},
'loggers': {
'pysimplesoap.helpers': {
'level': 'DEBUG',
'propagate': True,
'handlers': ['console'],
},
}
})
WSDL_URL = 'http://www.webservicex.net/stockquote.asmx?WSDL'
client = SoapClient(wsdl=WSDL_URL, ns="web", trace=True)
client['AuthHeaderElement'] = {'username': 'someone', 'password': 'nottelling'}
#Discover operations
list_of_services = [service for service in client.services]
print(list_of_services)
#Discover params
method = client.services['StockQuote']
response = client.GetQuote(symbol='GOOG')
print('GetQuote: {}'.format(response['GetQuoteResult']))
Related
TL;DR
I have a VSCode extension acting as a client for an LSP server written in Python (using the pygls library) and I can't seem to get basic requests sent to my LSP server from the extension.
The Longer Version
I'm working on an LSP server for a custom YAML-based language and have run into some issues I can't seem to resolve. Specifically, our tool is written in Python, so I'm using pygls to ease the creation of the language server and am creating a VSCode extension to handle the client side of things.
At this stage, my goal is to get a very basic hover functionality to work where I've hard-coded the hover text to be displayed to the user. Unfortunately, I have yet to be able to get this to work as it doesn't seem like my client is correctly sending the request to the server, or my server is not correctly handling it.
To try and resolve this, I have:
Looked through several examples of extensions to see how my VSCode extension differs (to name a few):
Microsoft/vscode-extension-samples: lsp-sample
openlawlibrary/pygls: json-extension
Eugleo/magic-racket
Gone through the LSP Specification to know which parameters need to be passed for the Hover request.
Gone through the VSCode Langage Server Extension Guide
Looked through the vscode-Languageserver-node code base for hints as to why my hover handler (in the server) is never getting called, etc.
Some things I noticed that were different with what I'm doing versus what others have:
Many extensions are using TypeScript to write the server; I'm using Python.
Some extensions do not add the call to client.start() to the context.subscriptions; I do.
Bonus: I get the sense that I should be sending the initialize request and the initialized notification before expecting any of the hovering to work but, from what I've seen, no other extension I've come across explicitly sends either of those. (Additionally, just because I was curious, I tried and it still didn't provide any different results.)
At this point, I'm really not sure where I'm going wrong - any insights/pointers are greatly appreciated. Thanks!
The relevant parts of my server implementation are as follows:
server.py
# imports elided
# we start the server a bit differently, but the essence is this:
server = LanguageServer()
server.start_ws("127.0.0.1", 8080)
#server.feature(methods.HOVER)
async def handle_hover(ls: LanguageServer, params: HoverParams):
"""Handle a hover event."""
logger.info(f"received hover request\nparams are: {params.text_document.uri}")
ls.show_message("received hover request")
ls.show_message(f"file: {params.text_document.uri}; line: {params.position.line}; character: {params.position.character}")
return Hover(contents="Hello from your friendly AaC LSP server!")
The relevant parts of the client (I think, I don't use VSCode so I may be missing something) are as follows:
AacLanguageServer.ts
// imports elided
// extension.ts (not this file) contains a call to the startLspClient(...) function
export class AacLanguageServerClient {
private static instance: AacLanguageServerClient;
private aacLspClient!: LanguageClient;
// certain checks are elided for brevity
private startLspClient(context: ExtensionContext, aacPath: string, host: string, port: number): void {
if (this.aacLspClient) { return; }
this.aacLspClient = new LanguageClient(
"aac",
"AaC Language Client",
this.getServerOptions(aacPath, "start-lsp", "--host", host, "--port", `${port}`),
this.getClientOptions(),
);
this.aacLspClient.trace = Trace.Verbose;
context.subscriptions.push(this.aacLspClient.start());
this.registerHoverProvider(context);
}
private async registerHoverProvider(context: ExtensionContext): Promise<void> {
const client = this.aacLspClient;
context.subscriptions.push(languages.registerHoverProvider({ scheme: "file", language: "aac", pattern: "**/*.yaml" }, {
provideHover(document, position, token): ProviderResult<Hover> {
window.showInformationMessage(
`File: ${document.uri.path}; Line: ${position.line}; Character: ${position.character}`
);
return client.sendRequest("textDocument/hover", {
textDocument: document,
position: position,
}, token);
}
}));
}
private getServerOptions(command: string, ...args: any[]): ServerOptions {
return {
args,
command,
};
}
private getClientOptions(): LanguageClientOptions {
return {
documentSelector: [
{ scheme: "file", language: "aac", pattern: "**/*.aac" },
{ scheme: "file", language: "aac", pattern: "**/*.yaml" },
],
diagnosticCollectionName: "aac",
outputChannelName: "Architecture-as-Code",
synchronize: {
fileEvents: workspace.createFileSystemWatcher("**/.clientrc"),
}
};
}
}
When I debug this in VSCode, I can see that the server is started in the session but if I hover over any symbol in the shown file, nothing happens. Can someone
Finally, all the code (in it's current state as of the time of posting this question) can be found in this repository, if a more complete code reference is desired. The LSP server-related code is located in the python/src/aac/lang/server.py file; and the client-related code is located in the vscode_extension/src/AacLanguageServer.ts file.
NOTE: This is still an incomplete answer! That's why I haven't accepted it.
I will update this when I figure out how to get the TCP and WebSockets servers working, but apparently a "good-enough" way to get the VSCode extension and my LSP server to communicate is by using the pygls I/O server for everything regardless of being in development mode.
Just creating a new document and setting the language to 'aac' wasn't enough.
But saving it with the extension .aac made the server active and "data" and "import" were suggested, so I tried to hover a word, and look - I got a response:
And the output to the channel:
By the way, I think that I found a bug. Shouldn't the error messages in lines 39-40 in AacLanguageServerClient.ts use variable name instead of item?
assertTrue(item.length > 0, `Cannot start Language Server; '${name}' is not configured!`);
assertTrue(fs.existsSync(item), `Cannot use ${name} as it does not exist!`);
I am using python with jira package and writing a simple script that will create or update all the existing issues for a project on my company's server.
Creating multiple issues via python is very fast and i can create 100 issues within 30 or seconds. But the problem is when i want to update those issues. When i update issues it takes a very long time probably 4 or 5 minutes for updating 100 issues. I am getting the InsecureRequestWarnings warning. I tried to disable warnings as well but still the program is very slow when it comes to updating the issues. How can i make updating issues faster?
NOTE: Each issue update takes more than 3.1 secs.
from jira import *
import urllib3
urllib3.disable_warnings() #Comment this to see warnings
options = {'server': 'Company Server', 'verify': False}
jira = JIRA(options, basic_auth=('username', "password"))
nameOfProjects = "Project name from jira"
issuesJira = jira.search_issues(jql_str='project=
"{}"'.format(nameOfProjects),fields='summary, key,type,status', startAt=0,
maxResults=1000)
test = 0
for issue in issuesJira:
issue.update(notify=False, fields={
'summary' :'some Text',
'description': 'some Text- ' +str(test),
'priority': {"name": 'High'},
'components': [{'name': 'TestMode'}],
"issuetype": {"name": 'Requirement'},
'fixVersions': [{'name': 'test'}]})
print('issue is updated-', test)
test = test +1
print('END')
I'm also experiencing this issue.
It seems there may be a bug in the python API that adds a 4s sleep in the update method.
https://github.com/pycontribs/jira/issues/622
Since this bug has been open for almost 2 years, I think it's time for a fork!
Is username in the first user directory that is configured in Jira user management? That can affect authentication times.
Does the project have a complex permission scheme or issue security scheme?
What changes if you only update one field such as Summary? Does that change the timing?
I would expect about 1s/issue update in many installations
I am consuming a SOAP webservice with suds (0.4). The WSDL I am using throws an error
>>> import uuid
>>> from suds.client import Client
>>> wsdl = 'https://toolkit.dnb.com/locked/resource/docs/TransactionAndMessageDefinition/ProductList.wsdl'
>>> client = Client(wsdl)
The service I am consuming expects one parameter productListRequest, which is a complex type where you put UserId, Password and a complex type of ProductListInput.
I fill these with:
>>> productListRequest = client.factory.create('productListRequest')
>>> productListRequest.UserId = 'myusername'
>>> productListRequest.Password = 'mypassword'
>>> productListRequest.TRNUID = uuid.uuid4()
>>> ProductListInput = client.factory.create('ProductListInput')
>>> ProductListInput.DnB_DUNS_Number = ''
>>> ProductListInput.Product = 'Product Name'
>>> ProductListInput.CountryCode = 'IT'
>>> ProductListInput.TradeUp = ''
>>> ProductListInput.Reason = ''
>>> productListRequest.ProductListInput = ProductListInput
But whenever I am calling the service:
>>> print client.service.ws_ProductList(productListRequest)
I get Type not found: 'ArrayOfUPD_FLDSItem'
I am really stuck here. I have googled this error for 2 days and honestly I do not know what to do! Maybe someone with a deeper understanding of WSDL and suds can help.
So my questions:
Is this WSDL, which I am consuming proper defined? (If it is proper defined, I will report it to the suds
maintainers)
If this WSDL is not proper defined, is there a workaround (e.g.
suds schema doctor) to fix it on suds site?
Is there a alternative Python library, which I should use?
Suds is currently the best choice for WSDL consumption in Python. Unfortunately WSDL itself is such a complex mess that making good out of it is difficult.
Luckily Suds come with extensive logging capabilities which you can use to debug the problem and this is the first step of solving it. This earlier question answers how to enable it:
How can I output what SUDs is generating/receiving?
However, giving a complete answer for the type error would require seeing extensive logging output and/or source code, so I suggest you somehow try to narrow down the problem. To make the problem ultimately solvable a sample (non-working) schema and Python code would be nice.
(The error might hint that there is some subschema / external schema defined / missing which Suds cannot load for reason X)
At first:
It does not make sense to call the product list without a DUNS-Number. The transaction gives all avaliable products to a given DUNS. If the DUNS number is left empty, you will only get a field list of the product you stated (assuming you put a valid product name into your call, not "product name").
BUT:
Even by putting all parameters in, I ran into the same problem and was not able to solve it, either.
Check with DnB and make them correct the WSDL - their WSDLs are quite buggy: Note that they have simply forgotten a whole transaction in the WSDL implementation (prodOrderRequest_3 for retrieving data from the toolkit archive)
My solution is to use the XML-Version of the Toolkit for this and the other mentioned transaction. Unfortunately.
Couple months ago I have asked the same question but in the context of older version of ZSI (How to display outcoming and incoming SOAP message for ZSI.ServiceProxy in Python?). Now, in the new version of ZSI 2.1 there is no tacefile parameter). I tried to find a documentation for the new version but I faild. Does anyone know how to display the SOAP messages generated and received by ZSI 2.1? Thank you in advance :-)
For debugging I have found less interfering solution using wireshark to trace the TCP packages. It looks like that:
I had this same problem. My workaround was to modify the dispatch.py file that comes with ZSI.
I created a logging function (logmessage) for my app that would store SOAP messages into a database and then added that function where necessary. I do not recall the ZSI version I was using however. You should be able to find these functions pretty easily in the code though. I ave approximate L numbers since i made other edits
in Dispatch.py file in your site-packages directory
L156 - logs SOAP responses
def _Dispatch(tons-of-args, **kw):
#several lines of code edited here#
#several lines of code edited here#
#several lines of code edited here#
sw = SoapWriter(nsdict=nsdict)
sw.serialize(result, tc)
logmessage( str(sw), 1, kw['request'].get_remote_host() ) #LOGGING HERE
L168 - logs SOAP errors
def _ModPythonSendFault(f, **kw):
logmessage( str(f.AsSOAP()), 1, kw['request'].get_remote_host() ) #LOGGING ADDED HERE
_ModPythonSendXML(f.AsSOAP(), 500, **kw)
L277 - logs requests
def AsHandler(request=None, modules=None, **kw):
'''Dispatch from within ModPython.'''
a = request.read(-1)
logmessage( a, 0, request.get_remote_host() ) #LOGGING ADDED HERE
ps = ParsedSoap(a)
kw['request'] = request
_Dispatch(ps, modules, _ModPythonSendXML, _ModPythonSendFault, **kw)
currently I'm interfacing the Twitter API using the OAuth protocol and writing the code in Python. As most of the users out there, I think the toughest part of the specs is dealing with signatures.
After wandering around the web in search for a solution, I decided to go for my custom code, so as to have a better understanding of what is going on.
For the sake of other users, I'm posting here a very simple and short implementation of the SHA1 signature specs in Python:
import hmac
from hashlib import sha1
from urllib import quote, urlencode
from base64 import b64encode
from urlparse import urlparse
def sign_request_sha1(url,method,data,secret=""):
pu = urlparse(urlparse(url).geturl())
normUrl = "%s://%s%s%s" % (
pu.scheme,
pu.hostname,
"" if not pu.port or {"http":80,"https":443}[pu.scheme] == pu.port else ":%d" % pu.port,
pu.path,
)
names = data.keys()
names.sort()
sig = "%s&%s&%s" % (
method.upper(),
quote(normUrl,''),
quote("&".join(["%s=%s" % (k,quote(data[k].encode('utf-8'),'')) for k in names]),''),
)
key = "%s&%s" % (quote(CONSUMER_SECRET.encode('utf-8'),''),secret)
return b64encode(hmac.new(key,sig,sha1).digest())
The input parameters to the function are:
url: the url you are going to call for the specific OAuth request.
method: this must be "GET" or "POST" depending on how you're going to issue your request.
data: a dictionary containing all the parameters of the requests, including any custom argument but excluding the "oauth_signature" one (for obvious reasons).
secret: a secret token you received in the initial phase of the protocol.
I tested it with Twitter and it seems to work but I'd like to receive some comments about mistakes, improvements and so on.
Lastly, here you find a piece of code calling the code for the initial "request token" phase:
from random import getrandbits
from base64 import b64encode
from time import time
def twitter_request_token(req,callback,errback):
req_url="http://twitter.com:80/oauth/request_token"
data = { \
"oauth_consumer_key" : CONSUMER_KEY,
"oauth_nonce" : b64encode("%0x" % getrandbits(256))[:32],
"oauth_timestamp" : str(int(time())),
"oauth_signature_method" : "HMAC-SHA1",
"oauth_version" : "1.0",
"oauth_callback" : "http://localhost:8080/",
}
data["oauth_signature"] = sign_request_sha1(req_url,"GET",data)
Thank you.
My knee-jerk reaction to this is If You're Typing The Letters A-E-S Into Your Code, You're Doing It Wrong. Or, as redditor khafra recently reminded us of the Sicilian's version:
Haha.. you fool! You fell victim to one of the classic blunders. The most famous is: Never get involved in a land war in Asia. But only slightly less famous is this: Never attempt to roll your own crypto when there's a well-tested library that'll do it better!
I mean, I get it. The first time I looked at it, oauth.py didn't impress me either. There's been a lot of work on it since and it's looking better, but there still appear to be no tests, so I don't know. Anyway, tests or no tests, it's been reviewed and used by more people than your code has.
But that's just me being uptight on the subject of crypto code reuse and doesn't really help you in figuring out the protocol machinery. It looks okay to me, but I haven't had my head in the OAuth spec too much lately.
Just use some more lines for that pu.port business; having a conditional if expression, an or expression, and the {}[] construct all in one line is really hard to read.
If you really want code review by people who are familiar with the protocol, you're probably better off asking the mailing list. And if you can offer them an alternate API that will make the code in their repository more appealing to new users, that'll be good for everyone.