Develop Reference

How to set explicit variables in an URL - python

In the following code snippet at the bottom, how do I explicitly set {0} and {1} to variables like:
0=host-123
1=128.24.22.21
such that I can keep the URL with the variables and call those variables elsewhere without actually explicitly setting them as host-123 and 128.24.22.21 elsewhere ?
def getipam(host, neighboraddr, token):
"""Fetch IPAM data for host and neighbor ip"""
url = 'https://ipam.app.secretcdn.net/v1/hosts/{0}/bgp/neighbors/neighbor/{1}'.format(host, neighboraddr)
response = requests.get(url, headers={'x-api-rw': token})
results = response.json()
return results

0/1/integers are immutable in python. You cannot change the value. You might want to call it by some other name like x or y something and then you can assign it

mock xmlrpc.client method python

I am in the process of learning unit testing, however I am struggling to understand how to mock functions for unit testing. I have reviewed many how-to's and examples but the concept is not transferring enough for me to use it on my code. I am hoping getting this to work on a actual code example I have will help.
In this case I am trying to mock isTokenValid.
Here is example code of what I want to mock.
<in library file>
import xmlrpc.client as xmlrpclib
class Library(object):
def function:
#...
AuthURL = 'https://example.com/xmlrpc/Auth'
auth_server = xmlrpclib.ServerProxy(AuthURL)
socket.setdefaulttimeout(20)
try:
if pull == 0:
valid = auth_server.isTokenValid(token)
#...
in my unit test file I have
import library
class Tester(unittest.TestCase):
#patch('library.xmlrpclib.ServerProxy')
def test_xmlrpclib(self, fake_xmlrpclib):
assert 'something'
How would I mock the code listed in 'function'? Token can be any number as a string and valid would be a int(1)

First of all, you can and should mock xmlrpc.client.ServerProxy; your library imports xmlrpc.client as a new name, but it is still the same module object so both xmlrpclib.ServerProxy in your library and xmlrpc.client.ServerProxy lead to the same object.
Next, look at how the object is used, and look for calls, the (..) syntax. Your library uses the server proxy like this:
# a call to create an instance
auth_server = xmlrpclib.ServerProxy(AuthURL)
# on the instance, a call to another method
valid = auth_server.isTokenValid(token)
So there is a chain here, where the mock is called, and the return value is then used to find another attribute that is also called. When mocking, you need to look for that same chain; use the Mock.return_value attribute for this. By default a new mock instance is returned when you call a mock, but you can also set test values.
So to test your code, you'd want to influence what auth_server.isTokenValid(token) returns, and test if your code works correctly. You may also want to assert that the correct URL is passed to the ServerProxy instance.
Create separate tests for different outcomes. Perhaps the token is valid in one case, not valid in another, and you'd want to test both cases:
class Tester(unittest.TestCase):
#patch('xmlrpc.client.ServerProxy')
def test_valid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response for a valid token
mock_auth_server.isTokenValid.return_value = 1
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')
#patch('xmlrpc.client.ServerProxy')
def test_invalid_token(self, mock_serverproxy):
# the ServerProxy(AuthURL) return value
mock_auth_server = mock_serverproxy.return_value
# configure a response; now testing for an invalid token instead
mock_auth_server.isTokenValid.return_value = 0
# now run your library code
return_value = library.Library().function()
# and make test assertions
# about the server proxy
mock_serverproxy.assert_called_with('some_url')
# and about the auth_server.isTokenValid call
mock_auth_server.isTokenValid.assert_called_once()
# and if the result of the function is expected
self.assertEqual(return_value, 'expected return value')

There are many mock attributes to use, and you can change your patch decorator usage a little as follows:
class Tester(unittest.TestCase):
def test_xmlrpclib(self):
with patch('library.xmlrpclib.ServerProxy.isTokenValid') as isTokenValid:
self.assertEqual(isTokenValid.call_count, 0)
# your test code calling xmlrpclib
self.assertEqual(isTokenValid.call_count, 1)
token = isTokenValid.call_args[0] # assume this token is valid
self.assertEqual(isTokenValid.return_value, 1)
You can adjust the code above to satisfy your requirements.

How to deal with globals in modules?

I try to make a non blocking api calls for OpenWeatherMap, but my problem is:
When i was doing tests on the file, and run it, the global api was taking effect, but when importing the function, global dont work anymore, and api dident change: api = ""?
Just after declaring the function i put global api, and then when I use print 'The API link is: ' + api I get the exact api, but global dident took effect!
Here is the code: https://github.com/abdelouahabb/tornadowm/blob/master/tornadowm.py#L62
What am I doing wrong?
When I import the file:
from tornadowm import *
forecast('daily', q='london', lang='fr')
The API link is: http://api.openweathermap.org/data/2.5/forecast/daily?lang=fr&q=london
api
Out[5]: ''
When executing the file instead of importing it:
runfile('C:/Python27/Lib/site-packages/tornadowm.py', wdir='C:/Python27/Lib/site-packages')
forecast('daily', q='london', lang='fr')
The API link is: http://api.openweathermap.org/data/2.5/forecast/daily?lang=fr&q=london
api
Out[8]: 'http://api.openweathermap.org/data/2.5/forecast/daily?lang=fr&q=london'
Edit: here is the code, if the Git got updated:
from tornado.httpclient import AsyncHTTPClient
import json
import xml.etree.ElementTree as ET
http_client = AsyncHTTPClient()
url = ''
response = ''
args = []
link = 'http://api.openweathermap.org/data/2.5/'
api = ''
result = {}
way = ''
def forecast(way, **kwargs):
global api
if way in ('weather', 'forecast', 'daily', 'find'):
if way == 'daily':
way = 'forecast/daily?'
else:
way += '?'
for i, j in kwargs.iteritems():
args.append('&{0}={1}'.format(i, j))
a = ''.join(set(args))
api = (link + way + a.replace(' ', '+')).replace('?&', '?')
print 'The API link is: ' + api
def handle_request(resp):
global response
if resp.error:
print "Error:", resp.error
else:
response = resp.body
http_client.fetch(api, handle_request)
else:
print "please put a way: 'weather', 'forecast', 'daily', 'find' "
def get_result():
global result
if response.startswith('{'):
print 'the result is JSON, stored in the variable result'
result = json.loads(response)
elif response.startswith('<'):
print 'the result is XML, parse the result variable to work on the nodes,'
print 'or, use response to see the raw result'
result = ET.fromstring(response)
else:
print '''Sorry, no valid response, or you used a parameter that is not compatible with the way!\n please check http://www.openweathermap.com/api for more informations''

It's the side effect of using global.
When you do from tornadowm import * your forecast() function is, we could say metaphorically, "on its own" and is not "hard-linked" to your global space anymore.
Why? Because any effect you make on your global api will "end" with your function, and the definition of api = "" in your global space will take precedence.
Also, as a side note, it's not considered a good practice to use from something import *. You should do from tornadowm import forecast or even better, import tornadown and then use tornadowm.forecast().
OR
Even better, I just noticed your forecast() function doesn't return anything. Which technically makes it not a function anymore, but a procedure (a procedure is like a function but it returns nothing, it just "does" stuff).
Instead of using a global, you should define api in this function and then return api from it. Like this:
def forecast(blablabla):
api = "something"
blablabla
return api
And then
import tornadowm
api = tornadown.forecast(something)
And you're done.

Globals are global only to the module they're defined in. So, normally, you would expect tornadowm.api to be changed when you call forecast, but not api in some other namespace.
The import * is contributing to your understanding of the problem. This imports api (among other names) into the importing namespace. This means that api and tornadowm.api initially point to the same object. But these two names are not linked in any way, and so calling forecast() changes only tornadowm.api and now the two names point to different objects.
To avoid this, don't use import *. It is bad practice anyway and this is just one of the reasons. Instead, import tornadowm and access the variable in the importing module as tornadowm.api.

I'm afraid this is because global is coupled within module, by the time you from tornadowm import * you have imported the api name, but the global api won't take any effects within another module.

Passing a variable to a JSON web service

I am accessing the class from the code api_service.py, which can be found here. When I call the first function, I have no problem, because no variables are passed:
from api_service import ApiService
import json
def main():
api_key = *removed*
access_token = *removed*
calling = ApiService(api_key,access_token)
survey_list = calling.get_survey_list()
But when I use the same type of routine as above to call a function from ApiService that requires a variable, I'm told that I should pass an object.
survey_details = calling.get_survey_details("1234")
survey_details = json.loads(json.dumps(survey_details))
print survey_details
The specific error message:
{u'status': 3, u'errmsg': u"Value '1234' for field '_data' is not of type object"}
Details for the get_survey_details aspect of the SurveyMonkey API are here, although I think a python-guru can solve this without knowing about the API.

This is a javascript/json object:
{field:'value'}
You have passed a string which, doesn't count as an "object" for these purposes.
Note that the error message is being generated by the service you are accessing. This question would be better directed to the creator of the service.

How to pass SOAP headers into python SUDS that are not defined in WSDL file

I have a camera on my network which I am trying to connect to with suds but suds doesn't send all the information needed. I need to put extra soap headers not defined in the WSDL file so the camera can understand the message. All the headers are contained in a SOAP envelope and then the suds command should be in the body of the message.
I have checked the suds website
and it says to pass in the headers like so: (This passes in the element as a header but I have an envelope so I'm not sure how to input this)
from suds.sax.element import Element
client = client(url)
ssnns = ('ssn', 'http://namespaces/sessionid')
ssn = Element('SessionID', ns=ssnns).setText('123')
client.set_options(soapheaders=ssn)
result = client.service.addPerson(person)
Now, I am not sure how I would implement this. Say for example, I have the below header:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP
ENC="http://www.w3.org/2003/05/soap-encoding"
<wsa:MessageID SOAP-ENV:mustUnderstand="true">urn:uuid:43268c01-f09c6</wsa:MessageID>
<SOAP-ENV:Header>
Using this or a similar example does anyone know how I would pass a valid SOAP message to the targeted service?
Thanks

I have worked out how to enter in new headers and namespaces in suds.
As stated above you create an Element and pass it in as a soapheader as so:
from suds.sax.element import Element
client = client(url)
ssnns = ('ssn', 'http://namespaces/sessionid')
ssn = Element('SessionID', ns=ssnns).setText('123')
client.set_options(soapheaders=ssn)
result = client.service.addPerson(person)
But if you would like to add a namespace I have found adding a prefix seem's to do the trick. So when you create one of the elements you add addPrefix. I'm not sure if this was the way it was intended to be done but it work's.
ssn = Element('SessionID', ns=ssnns).setText('123').addPrefix(p='SOAP-ENC', u='http://www.w3.org/2003/05/soap-encoding')
The p = 'SOAP-ENC' can be any prefix eg. wsa and the u = http://address is the address of the namespace.
A complete script that would run could be:
#!/usr/local/bin/python2.6
import suds
#import logging
from suds.client import Client
from suds.sax.element import Element
from suds.sax.attribute import Attribute
from suds.xsd.sxbasic import Import
def absoluteMove():
# connects to WSDL file and stores location in variable 'client'
client = Client('http://10.10.10.10/p.wsdl')
client.options.location = 'http://10.10.10.10:32963'
# Create the header
wsans = ('wsa', 'http://schemas.xmlsoap.org/ws/2004/08/addressing')
mustAttribute = Attribute('SOAP-ENV:mustUnderstand', 'true')
n1s = ('SOAP-ENC', 'http://www.w3.org/2003/05/soap-encoding')
msgId = Element('Element').addPrefix(p='SOAP-ENC', u='http://www.w3.org/2003/05/soap-encoding')
msgId2 = Element('Address', ns=wsans).setText('http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous')
msgId1 = Element('ReplyTo', ns=wsans).insert(msgId2)
msgId1.append(mustAttribute)
msgId3 = Element('To', ns=wsans).setText('http://10.10.10.10:32954')
msgId3.append(mustAttribute)
client.set_options(soapheaders=[msgId, msgId1, msgId3, msgId2])
# Create 'token' object to pass as an argument using the 'factory' namespace
token = client.factory.create('ns4:ReferenceToken')
# Create 'dest' object to pass as an argument and values passed to this object
dest = client.factory.create('ns4:PTZVector')
dest.PanTilt._x=1
dest.PanTilt._y=4.9
dest.Zoom._x=1
# Create 'speed' object to pass as an argument and values passed to this object
speed = client.factory.create('ns4:PTZSpeed')
speed.PanTilt._x=0
speed.PanTilt._y=0
speed.Zoom._x=1
# 'AbsoluteMove' method invoked passing in the new values entered in the above objects
try:
result = client.service.AbsoluteMove(token, dest, speed)
print "absoluteMove result ", result
return result
except suds.WebFault, e:
print "suds.WebFaults caught: "
print e
if __name__ == '__main__': result = absoluteMove()
This moves the camera. To change the type of soap-envelope check my next question.
You can add logging into this script whci allow's you to check what xml command you have sent which is handy:
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
The location can be put into the script as an option if the location is not in the wsdl file.