Develop Reference

Using python and suds, data not read by server side because element is not defined as an array

I am a very inexperienced programmer with no formal education. Details will be extremely helpful in any responses.
I have made several basic python scripts to call SOAP APIs, but I am running into an issue with a specific API function that has an embedded array.
Here is a sample excerpt from a working XML format to show nested data:
<bomData xsi:type="urn:inputBOM" SOAP-ENC:arrayType="urn:bomItem[]">
<bomItem>
<item_partnum></item_partnum>
<item_partrev></item_partrev>
<item_serial></item_serial>
<item_lotnum></item_lotnum>
<item_sublotnum></item_sublotnum>
<item_qty></item_qty>
</bomItem>
<bomItem>
<item_partnum></item_partnum>
<item_partrev></item_partrev>
<item_serial></item_serial>
<item_lotnum></item_lotnum>
<item_sublotnum></item_sublotnum>
<item_qty></item_qty>
</bomItem>
</bomData>
I have tried 3 different things to get this to work to no avail.
I can generate the near exact XML from my script, but a key attribute missing is the 'SOAP-ENC:arrayType="urn:bomItem[]"' in the above XML example.
Option 1 was using MessagePlugin, but I get an error because my section is like the 3 element and it always injects into the first element. I have tried body[2], but this throws an error.
Option 2 I am trying to create the object(?). I read a lot of stack overflow, but I might be missing something for this.
Option 3 looked simple enough, but also failed. I tried setting the values in the JSON directly. I got these examples by an XML sample to JSON.
I have also done a several other minor things to try to get it working, but not worth mentioning. Although, if there is a way to somehow do the following, then I'm all ears:
bomItem[]: bomData = {"bomItem"[{...,...,...}]}
Here is a sample of my script:
# for python 3
# using pip install suds-py3
from suds.client import Client
from suds.plugin import MessagePlugin
# Config
#option 1: trying to set it as an array using plugin
class MyPlugin(MessagePlugin):
def marshalled(self, context):
body = context.envelope.getChild('Body')
bomItem = body[0]
bomItem.set('SOAP-ENC:arrayType', 'urn:bomItem[]')
URL = "http://localhost/application/soap?wsdl"
client = Client(URL, plugins=[MyPlugin()])
transact_info = {
"username":"",
"transaction":"",
"workorder":"",
"serial":"",
"trans_qty":"",
"seqnum":"",
"opcode":"",
"warehouseloc":"",
"warehousebin":"",
"machine_id":"",
"comment":"",
"defect_code":""
}
#WIP - trying to get bomData below working first
inputData = {
"dataItem":[
{
"fieldname": "",
"fielddata": ""
}
]
}
#option 2: trying to create the element here and define as an array
#inputbom = client.factory.create('ns3:inputBOM')
#inputbom._type = "SOAP-ENC:arrayType"
#inputbom.value = "urn:bomItem[]"
bomData = {
#Option 3: trying to set the time and array type in JSON
#"#xsi:type":"urn:inputBOM",
#"#SOAP-ENC:arrayType":"urn:bomItem[]",
"bomItem":[
{
"item_partnum":"",
"item_partrev":"",
"item_serial":"",
"item_lotnum":"",
"item_sublotnum":"",
"item_qty":""
},
{
"item_partnum":"",
"item_partrev":"",
"item_serial":"",
"item_lotnum":"",
"item_sublotnum":"",
"item_qty":""
}
]
}
try:
response = client.service.transactUnit(transact_info,inputData,bomData)
print("RESPONSE: ")
print(response)
#print(client)
#print(envelope)
except Exception as e:
#handle error here
print(e)
I appreciate any help and hope it is easy to solve.

I have found the answer I was looking for. At least a working solution.
In any case, option 1 worked out. I read up on it at the following link:
https://suds-py3.readthedocs.io/en/latest/
You can review at the '!MessagePlugin' section.
I found a solution to get message plugin working from the following post:
unmarshalling Error: For input string: ""
A user posted an example how to crawl through the XML structure and modify it.
Here is my modified example to get my script working:
#Using MessagePlugin to modify elements before sending to server
class MyPlugin(MessagePlugin):
# created method that could be reused to modify sections with similar
# structure/requirements
def addArrayType(self, dataType, arrayType, transactUnit):
# this is the code that is key to crawling through the XML - I get
# the child of each parent element until I am at the right level for
# modification
data = transactUnit.getChild(dataType)
if data:
data.set('SOAP-ENC:arrayType', arrayType)
def marshalled(self, context):
# Alter the envelope so that the xsd namespace is allowed
context.envelope.nsprefixes['xsd'] = 'http://www.w3.org/2001/XMLSchema'
body = context.envelope.getChild('Body')
transactUnit = body.getChild("transactUnit")
if transactUnit:
self.addArrayType('inputData', 'urn:dataItem[]', transactUnit)
self.addArrayType('bomData', 'urn:bomItem[]', transactUnit)

Add Entries Python-LDAP

I'm trying to add entries with python ldap. I'm getting a naming convention error. My code is
import ldap
import ldap.modlist as modlist
LOGIN = ""
PASSWORD = ''
LDAP_URL = "ldap://127.0.0.1:389"
user='grant'
l = ldap.initialize(LDAP_URL)
l.bind(LOGIN, PASSWORD)
dn="ou=Enki Users,dc=enki,dc=local"
attrs = {}
attrs['objectclass'] = ['top','organizationalRole','simpleSecurityObject']
attrs['cn'] = 'test'
attrs['userPassword'] = 'test'
attrs['description'] = 'User object for replication using slurpd'
# Convert our dict to nice syntax for the add-function using modlist-module
ldif = modlist.addModlist(attrs)
# Do the actual synchronous add-operation to the ldapserver
l.add_s(dn,ldif)
# Its nice to the server to disconnect and free resources when done
l.unbind_s()
The error is:
ldap.NAMING_VIOLATION: {'info': "00002099: NameErr: DSID-0305109C, problem 2005 (NAMING_VIOLATION), data 0, best match of:\n\t'dc=enki,dc=local'\n", 'desc': 'Naming violation'}
The code that runs but doesn't insert the user into the correc organizational unit is the following code. However even though it runs I can't find the user in active directory. Please help me find whats wrong. I'm basically making a django webform for user management.
import ldap
import ldap.modlist as modlist
LOGIN = ""
PASSWORD = ''
LDAP_URL = "ldap://127.0.0.1:389"
user='grant'
l = ldap.initialize(LDAP_URL)
l.bind(LOGIN, PASSWORD)
dn="cn=test,ou=Enki Users,dc=enki,dc=local"
attrs = {}
attrs['objectclass'] = ['top','organizationalRole','simpleSecurityObject']
attrs['cn'] = 'test'
attrs['userPassword'] = 'test'
attrs['description'] = 'User object for replication using slurpd'
# Convert our dict to nice syntax for the add-function using modlist-module
ldif = modlist.addModlist(attrs)
# Do the actual synchronous add-operation to the ldapserver
l.add_s(dn,ldif)
# Its nice to the server to disconnect and free resources when done
l.unbind_s()

I speculate (but have not tested to prove it) that the root cause of your error is that your entry does not contain a "naming attribute" that matches the leftmost attribute in the DN of your entry, which in your case is ou=Enki Users. To add this naming attribute to the entry, you can add the following line in the part of your code that populates the attrs dict.
attrs['ou'] = 'Enki Users'

How Do I Suppress or Disable Warnings in reSTructuredText?

I'm working on a CMS in Python that uses reStructuredText (via docutils) to format content. Alot of my content is imported from other sources and usually comes in the form of unformatted text documents. reST works great for this because it makes everything look pretty sane by default.
One problem I am having, however, is that I get warnings dumped to stderr on my webserver and injected into my page content. For example, I get warnings like the following on my web page:
System Message: WARNING/2 (, line 296); backlink
My question is: How do I suppress, disable, or otherwise re-direct these warnings?
Ideally, I'd love to write these out to a log file, but if someone can just tell me how to turn off the warnings from being injected into my content then that would be perfect.
The code that's responsible for parsing the reST into HTML:
from docutils import core
import reSTpygments
def reST2HTML( str ):
parts = core.publish_parts(
source = str,
writer_name = 'html')
return parts['body_pre_docinfo'] + parts['fragment']

def reST2HTML( str ):
parts = core.publish_parts(
source = str,
writer_name = 'html',
settings_overrides={'report_level':'quiet'},
)
return parts['body_pre_docinfo'] + parts['fragment']

It seems the report_level accept string is an old version. Now, the below is work for me.
import docutils.core
import docutils.utils
from pathlib import Path
shut_up_level = docutils.utils.Reporter.SEVERE_LEVEL + 1
docutils.core.publish_file(
source_path=Path(...), destination_path=Path(...),
settings_overrides={'report_level': shut_up_level},
writer_name='html')
about level
# docutils.utils.__init__.py
class Reporter(object):
# system message level constants:
(DEBUG_LEVEL,
INFO_LEVEL,
WARNING_LEVEL,
ERROR_LEVEL,
SEVERE_LEVEL) = range(5)
...
def system_message(self, level, message, *children, **kwargs):
...
if self.stream and (level >= self.report_level # self.report_level was set by you. (for example, shut_up_level)
or self.debug_flag and level == self.DEBUG_LEVEL
or level >= self.halt_level):
self.stream.write(msg.astext() + '\n')
...
return msg
According to the above code, you know that you can assign the self.report_level (i.e. settings_overrides={'report_level': ...}) let the warning not show.
and I set it to SERVER_LEVEL+1, so it will not show any error. (you can set it according to your demand.)

How to pass SOAP headers into python SUDS that are not defined in WSDL file

I have a camera on my network which I am trying to connect to with suds but suds doesn't send all the information needed. I need to put extra soap headers not defined in the WSDL file so the camera can understand the message. All the headers are contained in a SOAP envelope and then the suds command should be in the body of the message.
I have checked the suds website
and it says to pass in the headers like so: (This passes in the element as a header but I have an envelope so I'm not sure how to input this)
from suds.sax.element import Element
client = client(url)
ssnns = ('ssn', 'http://namespaces/sessionid')
ssn = Element('SessionID', ns=ssnns).setText('123')
client.set_options(soapheaders=ssn)
result = client.service.addPerson(person)
Now, I am not sure how I would implement this. Say for example, I have the below header:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:SOAP
ENC="http://www.w3.org/2003/05/soap-encoding"
<wsa:MessageID SOAP-ENV:mustUnderstand="true">urn:uuid:43268c01-f09c6</wsa:MessageID>
<SOAP-ENV:Header>
Using this or a similar example does anyone know how I would pass a valid SOAP message to the targeted service?
Thanks

I have worked out how to enter in new headers and namespaces in suds.
As stated above you create an Element and pass it in as a soapheader as so:
from suds.sax.element import Element
client = client(url)
ssnns = ('ssn', 'http://namespaces/sessionid')
ssn = Element('SessionID', ns=ssnns).setText('123')
client.set_options(soapheaders=ssn)
result = client.service.addPerson(person)
But if you would like to add a namespace I have found adding a prefix seem's to do the trick. So when you create one of the elements you add addPrefix. I'm not sure if this was the way it was intended to be done but it work's.
ssn = Element('SessionID', ns=ssnns).setText('123').addPrefix(p='SOAP-ENC', u='http://www.w3.org/2003/05/soap-encoding')
The p = 'SOAP-ENC' can be any prefix eg. wsa and the u = http://address is the address of the namespace.
A complete script that would run could be:
#!/usr/local/bin/python2.6
import suds
#import logging
from suds.client import Client
from suds.sax.element import Element
from suds.sax.attribute import Attribute
from suds.xsd.sxbasic import Import
def absoluteMove():
# connects to WSDL file and stores location in variable 'client'
client = Client('http://10.10.10.10/p.wsdl')
client.options.location = 'http://10.10.10.10:32963'
# Create the header
wsans = ('wsa', 'http://schemas.xmlsoap.org/ws/2004/08/addressing')
mustAttribute = Attribute('SOAP-ENV:mustUnderstand', 'true')
n1s = ('SOAP-ENC', 'http://www.w3.org/2003/05/soap-encoding')
msgId = Element('Element').addPrefix(p='SOAP-ENC', u='http://www.w3.org/2003/05/soap-encoding')
msgId2 = Element('Address', ns=wsans).setText('http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous')
msgId1 = Element('ReplyTo', ns=wsans).insert(msgId2)
msgId1.append(mustAttribute)
msgId3 = Element('To', ns=wsans).setText('http://10.10.10.10:32954')
msgId3.append(mustAttribute)
client.set_options(soapheaders=[msgId, msgId1, msgId3, msgId2])
# Create 'token' object to pass as an argument using the 'factory' namespace
token = client.factory.create('ns4:ReferenceToken')
# Create 'dest' object to pass as an argument and values passed to this object
dest = client.factory.create('ns4:PTZVector')
dest.PanTilt._x=1
dest.PanTilt._y=4.9
dest.Zoom._x=1
# Create 'speed' object to pass as an argument and values passed to this object
speed = client.factory.create('ns4:PTZSpeed')
speed.PanTilt._x=0
speed.PanTilt._y=0
speed.Zoom._x=1
# 'AbsoluteMove' method invoked passing in the new values entered in the above objects
try:
result = client.service.AbsoluteMove(token, dest, speed)
print "absoluteMove result ", result
return result
except suds.WebFault, e:
print "suds.WebFaults caught: "
print e
if __name__ == '__main__': result = absoluteMove()
This moves the camera. To change the type of soap-envelope check my next question.
You can add logging into this script whci allow's you to check what xml command you have sent which is handy:
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
The location can be put into the script as an option if the location is not in the wsdl file.

Google Analytics and Python

I'm brand new at Python and I'm trying to write an extension to an app that imports GA information and parses it into MySQL. There is a shamfully sparse amount of infomation on the topic. The Google Docs only seem to have examples in JS and Java...
...I have gotten to the point where my user can authenticate into GA using SubAuth. That code is here:
import gdata.service
import gdata.analytics
from django import http
from django import shortcuts
from django.shortcuts import render_to_response
def authorize(request):
next = 'http://localhost:8000/authconfirm'
scope = 'https://www.google.com/analytics/feeds'
secure = False # set secure=True to request secure AuthSub tokens
session = False
auth_sub_url = gdata.service.GenerateAuthSubRequestUrl(next, scope, secure=secure, session=session)
return http.HttpResponseRedirect(auth_sub_url)
So, step next is getting at the data. I have found this library: (beware, UI is offensive) http://gdata-python-client.googlecode.com/svn/trunk/pydocs/gdata.analytics.html
However, I have found it difficult to navigate. It seems like I should be gdata.analytics.AnalyticsDataEntry.getDataEntry(), but I'm not sure what it is asking me to pass it.
I would love a push in the right direction. I feel I've exhausted google looking for a working example.
Thank you!!
EDIT: I have gotten farther, but my problem still isn't solved. The below method returns data (I believe).... the error I get is: "'str' object has no attribute '_BecomeChildElement'" I believe I am returning a feed? However, I don't know how to drill into it. Is there a way for me to inspect this object?
def auth_confirm(request):
gdata_service = gdata.service.GDataService('iSample_acctSample_v1.0')
feedUri='https://www.google.com/analytics/feeds/accounts/default?max-results=50'
# request feed
feed = gdata.analytics.AnalyticsDataFeed(feedUri)
print str(feed)

Maybe this post can help out. Seems like there are not Analytics specific bindings yet, so you are working with the generic gdata.

I've been using GA for a little over a year now and since about April 2009, i have used python bindings supplied in a package called python-googleanalytics by Clint Ecker et al. So far, it works quite well.
Here's where to get it: http://github.com/clintecker/python-googleanalytics.
Install it the usual way.
To use it: First, so that you don't have to manually pass in your login credentials each time you access the API, put them in a config file like so:
[Credentials]
google_account_email = youraccount#gmail.com
google_account_password = yourpassword
Name this file '.pythongoogleanalytics' and put it in your home directory.
And from an interactive prompt type:
from googleanalytics import Connection
import datetime
connection = Connection() # pass in id & pw as strings **if** not in config file
account = connection.get_account(<*your GA profile ID goes here*>)
start_date = datetime.date(2009, 12, 01)
end_data = datetime.date(2009, 12, 13)
# account object does the work, specify what data you want w/
# 'metrics' & 'dimensions'; see 'USAGE.md' file for examples
account.get_data(start_date=start_date, end_date=end_date, metrics=['visits'])
The 'get_account' method will return a python list (in above instance, bound to the variable 'account'), which contains your data.

You need 3 files within the app. client_secrets.json, analytics.dat and google_auth.py.
Create a module Query.py within the app:
class Query(object):
def __init__(self, startdate, enddate, filter, metrics):
self.startdate = startdate.strftime('%Y-%m-%d')
self.enddate = enddate.strftime('%Y-%m-%d')
self.filter = "ga:medium=" + filter
self.metrics = metrics
Example models.py: #has the following function
import google_auth
service = googleauth.initialize_service()
def total_visit(self):
object = AnalyticsData.objects.get(utm_source=self.utm_source)
trial = Query(object.date.startdate, object.date.enddate, object.utm_source, ga:sessions")
result = service.data().ga().get(ids = 'ga:<your-profile-id>', start_date = trial.startdate, end_date = trial.enddate, filters= trial.filter, metrics = trial.metrics).execute()
total_visit = result.get('rows')
<yr save command, ColumnName.object.create(data=total_visit) goes here>