I have a weird problem with SUDS in python.
I am writing an application, that uses a SOAP service. And for approximately 2 months everything was going fine. All of a sudden, it stopped working. Here is the weird part though.
If I run the unittest for the suds manager I wrote, everything goes well and the test passes like usual. If I run the code in the GUI application it fails, with the following traceback:
ERROR:suds.client:<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns0="http://service.skip.fiskistofa.is/" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns1:Body>
<ns0:getLandings>
<from>2013-10-11</from>
<to>2013-10-01</to>
</ns0:getLandings>
</ns1:Body>
</SOAP-ENV:Envelope>
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 808, in __bootstrap_inner
self.run()
File "/home/fintor/bin/python/aflafrettir/gui/gui.py", line 332, in run
landings = self.manager.get_landings(self.date_from, self.date_to)
File "/home/fintor/bin/python/aflafrettir/landings/manager.py", line 66, in get_landings
return self.call_method('getLandings', date_from, date_to)
File "/home/fintor/bin/python/aflafrettir/landings/manager.py", line 49, in call_method
return getattr(self.client.service, method)(*args)
File "/home/fintor/.virtenvs/aflafrettir/lib/python2.7/site-packages/suds/client.py", line 542, in __call__
return client.invoke(args, kwargs)
File "/home/fintor/.virtenvs/aflafrettir/lib/python2.7/site-packages/suds/client.py", line 602, in invoke
result = self.send(soapenv)
File "/home/fintor/.virtenvs/aflafrettir/lib/python2.7/site-packages/suds/client.py", line 649, in send
result = self.failed(binding, e)
File "/home/fintor/.virtenvs/aflafrettir/lib/python2.7/site-packages/suds/client.py", line 702, in failed
r, p = binding.get_fault(reply)
File "/home/fintor/.virtenvs/aflafrettir/lib/python2.7/site-packages/suds/bindings/binding.py", line 265, in get_fault
raise WebFault(p, faultroot)
WebFault: Server raised fault: 'Couldn't create SOAP message due to exception: XML reader error: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character 'C' (code 67) in prolog; expected '<'
at [row,col {unknown-source}]: [1,1]'
The code that fails, is running in a thread within the GUI application(PySide), the relevant code is here:
class Worker(Thread):
signal = AflafrettirSignal()
def __init__(self, queue, manager, date_from, date_to):
super(Worker, self).__init__()
self.queue = queue
self.manager = manager
self.date_from = date_from
self.date_to = date_to
self.daemon = True
def run(self):
landings = self.manager.get_landings(self.date_from, self.date_to)
for l in landings:
tmp = Landings()
tmp.set_variable(l)
tmp.calc_total_catch()
self.queue.put(tmp)
self.signal.work_done.emit(True)
and it's called in the gui like so:
manager = LandingsManager()
manager.set_credentials(self.username, self.password)
manager.get_client()
worker = Worker(self.queue, manager, date_from, date_to)
worker.start()
Finally, here is the code from the unittest module, which is working fine:
manager = LandingsManager()
manager.set_credentials(usern, passw)
manager.get_client()
data = manager.get_landings('2013-10-01', '2013-10-11')
I have verified that the username and password is the same when I set the credentials as well as the date from and date to when I call the get_landings method.
Like I said, it is a weird problem, and I am not sure what causes this to fail in one place, and work in another. I hope that someone can enlighten me regarding the cause.
EDIT:
I figured out what was wrong. The password contained a carriage return, and the SOAP service did not like that.
Related
I am using telenium to automate the test of kivy application.
https://github.com/tito/telenium/blob/master/README.md
def enable_server():
def start_server():
os.system('python server.py')
t1 = threading.Thread(target=start_server, daemon=True)
t1.start()
My Skeleton of the telenium testcases looks like this.
class UITestCase(TeleniumTestCase):
cmd_entrypoint = [" main.py"]
def first_test(self):
"""code to test"""
def second_test(self):
enable_server()
"""code to test"""
def third_test(self):
enable_server()
"""code to test"""
Since two of tests need the enable_server() the application does not execute the third_test completely and fails. I am not sure why this is happening.
Error:
Traceback (most recent call last):
File "node_sim.py", line 63, in <module>
loop.run_until_complete(start_server)
File "/home/user/.pyenv/versions/3.7.3/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
return future.result()
File "/home/user/.pyenv/versions/3.7.3/lib/python3.7/asyncio/tasks.py", line 603, in _wrap_awaitable
return (yield from awaitable.__await__())
File "/home/user/.pyenv/versions/3.7.3/lib/python3.7/site-packages/websockets/legacy/server.py", line 1071, in __await_impl__
server = await self._create_server()
File "/home/user/.pyenv/versions/3.7.3/lib/python3.7/asyncio/base_events.py", line 1378, in create_server
% (sa, err.strerror.lower())) from None
OSError: [Errno 98] error while attempting to bind on address ('0.0.0.0', 5000): address already in use
You should put enable_server in init method. It will run before every UITestCase.
Each new TeleniumTestCase will close and start the application, so you always run from a clean app. If you always need to do something before starting the test, you can overload the init. This will be executed once before any tests in the class starts:
doc
Im working on a slack bot using the new slack 2.0 python library. I am new to python decorators and I suspect that is part of my problem.
Here is my code...
#!/opt/rh/rh-python36/root/usr/bin/python
import os
import slack
# instantiate Slack client
slack_token = os.environ['SLACK_BOT_TOKEN']
rtmclient = slack.RTMClient(token=slack_token)
webclient = slack.WebClient(token=slack_token)
# get the id of my user
bot_id = webclient.auth_test()['user_id']
print('Bot ID: {0}'.format(bot_id))
def get_user_info(user_id):
user_info = webclient.users_info(user=user_id)['ok']
return user_info
#slack.RTMClient.run_on(event='message')
def parse_message(**payload):
data = payload['data']
user_id = data['user']
print(get_user_info(user_id))
rtmclient.start()
It outputs the Bot ID(using the webclient) when started but then crashes with RuntimeError: This event loop is already running when I make another call to webclient.
[root#slackbot-01 bin]# scl enable rh-python36 /root/slackbot/bin/slackbot.py
Bot ID: UBT547D31
Traceback (most recent call last):
File "/root/slackbot/bin/slackbot.py", line 24, in <module>
rtmclient.start()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 197, in start
return self._event_loop.run_until_complete(future)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/asyncio/base_events.py", line 467, in run_until_complete
return future.result()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 339, in _connect_and_read
await self._read_messages()
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 390, in _read_messages
await self._dispatch_event(event, data=payload)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 440, in _dispatch_event
self._execute_in_thread(callback, data)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/rtm/client.py", line 465, in _execute_in_thread
future.result()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/root/slackbot/bin/slackbot.py", line 22, in parse_message
print(get_user_info(user_id))
File "/root/slackbot/bin/slackbot.py", line 15, in get_user_info
user_info = webclient.users_info(user=user_id)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/web/client.py", line 1368, in users_info
return self.api_call("users.info", http_verb="GET", params=kwargs)
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/slack/web/base_client.py", line 154, in api_call
return self._event_loop.run_until_complete(future)
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/asyncio/base_events.py", line 454, in run_until_complete
self.run_forever()
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/asyncio/base_events.py", line 408, in run_forever
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
The really confusing part to me is that if I comment out the line that makes the first call to webclient.auth_test(), I have no issues at all. My call to webclient.users_info() works every time rtmclient sends me data.
#!/opt/rh/rh-python36/root/usr/bin/python
import os
import slack
# instantiate Slack client
slack_token = os.environ['SLACK_BOT_TOKEN']
rtmclient = slack.RTMClient(token=slack_token)
webclient = slack.WebClient(token=slack_token)
# get the id of my user
#bot_id = webclient.auth_test()['user_id']
#print('Bot ID: {0}'.format(bot_id))
def get_user_info(user_id):
user_info = webclient.users_info(user=user_id)['ok']
return user_info
#slack.RTMClient.run_on(event='message')
def parse_message(**payload):
data = payload['data']
user_id = data['user']
print(get_user_info(user_id))
rtmclient.start()
[root#slackbot-01 bin]# scl enable rh-python36 /root/slackbot/bin/slackbot.py
True
True
^C[root#slackbot-01 bin]#
I need to get the bot id so that I can make sure it doesnt answer it's own messages. I don't why my code doesnt work after I get the bot id outside of the parse message function with a decorator.
What am I doing wrong here?
The python event loop is a tricky thing to program libraries around and there are some issues with the way the event queue is managed in the 2.0 version of SlackClient. It looks like some improvements were made with 2.1 but it appears to be a work in progress, and I still encounter this. I'd expect there will be future updates to make it more robust.
In the meantime, the following code at the top of your file (use pip to install) usually resolves it for me:
import nest_asyncio
nest_asyncio.apply()
Keep in mind this will alter the way the rest of your application is handling the event queue, if that's a factor.
If you're using RTM, the RTMClient creates a WebClient for you. The handle for it should be getting passed to you in the payload when you handle an event. You can check your ID by looking for the 'open' event which is always dispatched after RTM successfully connects and doing the lookup inside your 'open' event handler.
I'm building an asynchronous HTTP request class with urllib2. The code looks like this:
import urllib2, threading, datetime
class ExportHandler(urllib2.HTTPHandler):
def http_response(self, link, actualdate, nowdate ):
link += "&export=csv"
export = urllib2.urlopen( link )
for link, actualdate in commissionSummaryLinks:
o = urllib2.build_opener(ExportHandler())
t = threading.Thread(target=o.open, args=(link, actualdate, datetime.datetime.now().strftime("%Y%m%d%H%M%S")))
t.start()
print "I'm asynchronous!"
t.join()
print "ending threading"
Suffice it to say commissionSummaryLinks IS populated and actualdate is a date time.datetime.strptime() object.
Anyway, I'm receiving an error from all the issued threads that looks like this:
Exception in thread Thread-9:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
self.__target(*self.__args, **self.__kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 402, in open
req = meth(req)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1123, in do_request_
'Content-length', '%d' % len(data))
TypeError: object of type 'datetime.date' has no len()
I'm running this on OS X (if it matters). Can anyone tell me what the problem here is?
When you instantiate your thread, you need to provide the arguments to o.open, not arguments to the http_response method of ExportHandler.
In this case, o.open has the following method signature:
open(self, fullurl, data=None, timeout=<object object>) method of urllib2.OpenerDirector instance
My guess is that you should only need to set args=(link,).
If you still need to use those other arguments, you'll probably want to modify the constructor of ExportHandler to take the other arguments you need, and then use them as appropriate in the http_response method. Take a look at the Python Class tutorial for more information on defining a class constructor.
Now I am writing simple OpenERP module for my customer. I use SUDS to connect with bank, to get Statements.
I wrote xml request, that works without problem. I get response from bank, that also looks ok. Problem is that response from bank use type, that is not defined in WSDL (I wrote to bank support, that they have a bug).
Traceback (most recent call last):
File "example3.py", line 112, in <module>
wiadomosc = client.service.GetStatement(__inject={'msg': xml})
File "/usr/lib/python2.7/dist-packages/suds/client.py", line 542, in __call__
return client.invoke(args, kwargs)
File "/usr/lib/python2.7/dist-packages/suds/client.py", line 773, in invoke
return self.send(msg)
File "/usr/lib/python2.7/dist-packages/suds/client.py", line 647, in send
result = self.succeeded(binding, reply.message)
File "/usr/lib/python2.7/dist-packages/suds/client.py", line 684, in succeeded
reply, result = binding.get_reply(self.method, reply)
File "/usr/lib/python2.7/dist-packages/suds/bindings/binding.py", line 156, in get_reply
result = self.replycomposite(rtypes, nodes)
File "/usr/lib/python2.7/dist-packages/suds/bindings/binding.py", line 230, in replycomposite
sobject = unmarshaller.process(node, resolved)
File "/usr/lib/python2.7/dist-packages/suds/umx/typed.py", line 66, in process
return Core.process(self, content)
File "/usr/lib/python2.7/dist-packages/suds/umx/core.py", line 48, in process
return self.append(content)
File "/usr/lib/python2.7/dist-packages/suds/umx/core.py", line 63, in append
self.append_children(content)
File "/usr/lib/python2.7/dist-packages/suds/umx/core.py", line 140, in append_children
cval = self.append(cont)
File "/usr/lib/python2.7/dist-packages/suds/umx/core.py", line 61, in append
self.start(content)
File "/usr/lib/python2.7/dist-packages/suds/umx/typed.py", line 80, in start
raise TypeNotFound(content.node.qname())
suds.TypeNotFound: Type not found: 'ns29:BkToCstmrStmt'
And types, they provide
....
ns37:BaselineStatus3Code
ns32:BatchBookingIndicator
ns33:BatchBookingIndicator
ns36:BatchBookingIndicator
ns30:BatchInformation1
ns31:BatchInformation2
ns29:BatchInformation2
ns39:BilateralLimitDetails3
ns27:BkToCstmrCardRptType
ns27:BkTxCdType
ns27:BookgDtType
ns10:BookingDate
ns4:Bool
ns16:Bool
ns2:Bool
ns15:Bool
ns17:BouldingNumber
....
So there is no BkToCstmrStmt. How can I make suds to just get response from server, and not analyse it? Just build tree?
Thank you
Still I don't know what is wrong with my code. I solved it in that way:
class MessageInterceptor(suds.plugin.MessagePlugin):
def __init__(self, *args, **kwargs):
self.message = None
def received(self, context):
#recieved xml as a string
#print "%s bytes received" % len(context.reply)
self.message = context.reply
#clean up reply to prevent parsing
context.reply = ""
return context
message_interceptor = MessageInterceptor()
client = Client('https://some-adress-to?wsdl',plugins=[message_interceptor])
So now I can call client method
xml = Raw("""
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
<soapenv:Header>
...
</soapenv:Header>
<soapenv:Body>
<urn1:GetStatement>
...
</urn1:GetStatement>
</soapenv:Body>
</soapenv:Envelope>
""")
response = client.service.GetStatement(__inject={'msg': xml})
Now suds thinks that got nothing from server. But message we recieved is stored in
message_interceptor.message
Now to get normal dict object from message I do it like that:
import xmltodict
message_interceptor.message = message_interceptor.message.replace('ns17:','')
message_interceptor.message = message_interceptor.message.replace('ns40:','')
message_interceptor.message = message_interceptor.message.replace('soap:','')
response = xmltodict.parse(message_interceptor.message)['Envelope']['Body']['GetStatementResponse']['Document']
Now I can use response as normal response from suds.
I'm running a Flask app and internally using a library written in Node.js, which I access through ZeroRPC (the actual node process is managed by Circus). This works fine on its own; I can unit test with no issues. But when starting the Flask app as a listening process, and calling into a REST api which calls this libary, the program throws an exception when trying to start the process. The code to start the service is as follows:
from circus.watcher import Watcher
from circus.arbiter import ThreadedArbiter
from circus.util import (DEFAULT_ENDPOINT_DEALER, DEFAULT_ENDPOINT_SUB,
DEFAULT_ENDPOINT_MULTICAST)
class Node(object):
{... omitted code that initializes self._arbiter and self._client ...}
def start(self):
if self._arbiter and self._client:
return
port = 'ipc:///tmp/inlinejs_%s' % os.getpid()
args = 'lib/server.js --port %s' % port
watcher = Watcher('node', '/usr/local/bin/node', args,
working_dir=INLINEJS_DIR)
self._arbiter = ThreadedArbiter([watcher], DEFAULT_ENDPOINT_DEALER,
DEFAULT_ENDPOINT_SUB, multicast_endpoint=DEFAULT_ENDPOINT_MULTICAST)
self._arbiter.start()
self._client = zerorpc.Client()
self._client.connect(port)
This function returns, but shortly afterwards in a separate thread, I get this error:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/python/lib/python2.7/site-packages/circus/_patch.py", line 21, in _bootstrap_inner
self.run()
File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 647, in run
return Arbiter.start(self)
File "/python/lib/python2.7/site-packages/circus/util.py", line 319, in _log
return func(self, *args, **kw)
File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 456, in start
self.initialize()
File "/python/lib/python2.7/site-packages/circus/util.py", line 319, in _log
return func(self, *args, **kw)
File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 427, in initialize
self.evpub_socket.bind(self.pubsub_endpoint)
File "socket.pyx", line 432, in zmq.core.socket.Socket.bind (zmq/core/socket.c:4022)
File "checkrc.pxd", line 21, in zmq.core.checkrc._check_rc (zmq/core/socket.c:5838)
ZMQError: Address already in use
I have no idea why this is happening, especially since it doesn't happen in unit tests. Can anyone shed any light?
In general, when you get this type of "Address in use" error, it means that your program is trying to bind on an IP port number but something else got there first.
I am not familiar with this library, but since the error is caused by "evpub_socket.bind", I am going to guess that you have a conflict with the port number specified by the constant DEFAULT_ENDPOINT_SUB. From the circus source code I see these constants:
DEFAULT_ENDPOINT_DEALER = "tcp://127.0.0.1:5555"
DEFAULT_ENDPOINT_SUB = "tcp://127.0.0.1:5556"
DEFAULT_ENDPOINT_STATS = "tcp://127.0.0.1:5557"
Check your system (netstat) and see if any process is listening on ports 5555, 5556, 5557. Or perhaps you are running this program twice and you forgot about the first one.