So I'm working with a block of code which communicates with the Flickr API.
I'm getting a 'syntax error' in xml.parsers.expat.ExpatError (below). Now I can't figure out how it'd be a syntax error in a Python module.
I saw another similar question on SO regarding the Wikipedia API which seemed to return HTML intead of XML. Flickr API returns XML; and I'm also getting the same error when there shouldn't be a response from Flickr (such as flickr.galleries.addPhoto)
CODE:
def _dopost(method, auth=False, **params):
#uncomment to check you aren't killing the flickr server
#print "***** do post %s" % method
params = _prepare_params(params)
url = '%s%s/%s' % (HOST, API, _get_auth_url_suffix(method, auth, params))
payload = 'api_key=%s&method=%s&%s'% \
(API_KEY, method, urlencode(params))
#another useful debug print statement
#print url
#print payload
return _get_data(minidom.parse(urlopen(url, payload)))
TRACEBACK:
Traceback (most recent call last):
File "TESTING.py", line 30, in <module>
flickr.galleries_create('test_title', 'test_descriptionn goes here.')
File "/home/vlad/Documents/Computers/Programming/LEARNING/curatr/flickr.py", line 1006, in galleries_create
primary_photo_id=primary_photo_id)
File "/home/vlad/Documents/Computers/Programming/LEARNING/curatr/flickr.py", line 1066, in _dopost
return _get_data(minidom.parse(urlopen(url, payload)))
File "/usr/lib/python2.6/xml/dom/minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "/usr/lib/python2.6/xml/dom/expatbuilder.py", line 928, in parse
result = builder.parseFile(file)
File "/usr/lib/python2.6/xml/dom/expatbuilder.py", line 207, in parseFile
parser.Parse(buffer, 0)
xml.parsers.expat.ExpatError: syntax error: line 1, column 62
(Code from http://code.google.com/p/flickrpy/ under New BSD licence)
UPDATE:
print urlopen(url, payload) == <addinfourl at 43340936 whose fp = <socket._fileobject object at 0x29400d0>>
Doing a urlopen(url, payload).read() returns HTML which is hard to read in a terminal :P but I managed to make out a 'You are not signed in.'
The strange part is that Flickr shouldn't return anything here, or if permissions are a problem, it should return a 99: User not logged in / Insufficient permissions error as it does with the GET function (which I'd expect would be in valid XML).
I'm signed in to Flickr (in the browser) and the program is properly authenticated with delete permissions (dangerous, but I wanted to avoid permission problems.)
SyntaxError normally means an error in Python syntax, but I think here that expatbuilder is overloading it to mean an XML syntax error. Put a try:except block around it, and print out the contents of payload and to work out what's wrong with the first line of it.
My guess would be that flickr is rejecting your request for some reason and giving back a plain-text error message, which has an invalid xml character at column 62, but it could be any number of things. You probably want to check the http status code before parsing it.
Also, it's a bit strange this method is called _dopost but you seem to actually be sending an http GET. Perhaps that's why it's failing.
This seems to fix my problem:
url = '%s%s/?api_key=%s&method=%s&%s'% \
(HOST, API, API_KEY, method, _get_auth_url_suffix(method, auth, params))
payload = '%s' % (urlencode(params))
It seems that the API key and method had to be in the URL not in the payload. (Or maybe only one needed to be there, but anyways, it works :-)
Related
I'm trying to use SharePlum which is a Python module for SharePoint but when I try to connect to my SharePoint, SharePlum raises me this error:
Traceback (most recent call last):
File "C:/Users/me/Desktop/Sharpoint/sharpoint.py", line 13, in site = Site(sharepoint_url, auth=auth)
File "C:\Users\me\AppData\Local\Programs\Python\Python36\lib\site-packages\shareplum\shareplum.py", line 46, in init self.users = self.GetUsers()
File "C:\Users\me\AppData\Local\Programs\Python\Python36\lib\site-packages\shareplum\shareplum.py", line 207, in GetUsers raise Exception("Can't get User Info List")
Exception: Can't get User Info List
Here is the very short code that I have written:
auth = HttpNtlmAuth(username, password)
site = Site(sharepoint_url, auth=auth)
This error seems to indicate bad username/password but I'm pretty sure that the one I have are correct...
Ok, it seems that I found the solution for my problem, it's about the Sharepoint URL that I gave.
If we take this example : https://www.mysharepoint.com/Your/SharePoint/DocumentLibrary
You have to remove the last part : /DocumentLibrary.
Why remove this part precisely ?
In fact, when you go deep enough in your Sharepoint, your url will look like something like : https://www.mysharepoint.com/Your/SharePoint/DocumentLibrary/Forms/AllItems.aspx?RootFolder=%2FYour%2FSharePoint%2DocumentLibrary%2FmyPersonnalFolder&FolderCTID=0x0120008BBC54784D92004D1E23F557873CC707&View=%7BE149526D%2DFD1B%2D4BFA%2DAA46%2D90DE0770F287%7D
You can see that the right of the path is in RootFolder=%2FYour%2FSharePoint%2DocumentLibrary%2Fmy%20personnal%20folder and not in the "normal" URL anymore (if it were, it will be like that https://www.mysharepoint.com/Your/SharePoint/DocumentLibrary/myPersonnalFolder/).
What you have to remove is the end of the "normal" URL so in this case, /DocumentLibrary.
So my correct Sharepoint URL to input in SharePlum will be https://www.mysharepoint.com/Your/SharePoint/
I'm pretty new to Sharepoint so I'm not really sure that this I the right answer to this problem for the others persons, may someone who know Sharepoint better than me can confirm ?
I know this is not actual solution for your problem and I would add just comment but it was too long so I will post as answer.
I can't replicate your issue, but by looking into source code of shareplum.py you can see why program throws the error. In line 196 of shareplum.py there is if clause (if response.status_code == 200:) which checks if the request to access your sharepoint url was successful (than it has status code 200) and if request failed (than it has some other status code) than it throws exception (Can't get User Info List). If you want to find out more about your problem go to your shareplum.py file ("C:\Users\me\AppData\Local\Programs\Python\Python36\lib\site-packages\shareplum\shareplum.py") and add this line print('{} {} Error: {} for url: {}'.format(response.status_code, 'Client'*(400 <= response.status_code < 500) + 'Server'*(500 <= response.status_code < 600), response.reason, response.url)) before line 207 ('raise Exception("Can't get User Info List")'). Then your shareplum.py should look like this:
# Parse Response
if response.status_code == 200:
envelope = etree.fromstring(response.text.encode('utf-8'))
listitems = envelope[0][0][0][0][0]
data = []
for row in listitems:
# Strip the 'ows_' from the beginning with key[4:]
data.append({key[4:]: value for (key, value) in row.items() if key[4:]})
return {'py': {i['ImnName']: i['ID']+';#'+i['ImnName'] for i in data},
'sp': {i['ID']+';#'+i['ImnName'] : i['ImnName'] for i in data}}
else:
print('{} {} Error: {} for url: {}'.format(response.status_code, 'Client'*(400 <= response.status_code < 500) + 'Server'*(500 <= response.status_code < 600), response.reason, response.url))
raise Exception("Can't get User Info List")
Now just run your program again and it should print out why it isn't working.
I know it is best not to change files in Python modules, but if you know what you change then there is no problem so when you are finished just delete the added line.
Also when you find out status code you can search it online, just type it in google or search on List_of_HTTP_status_codes.
I'm on Python 3.5.1, using requests, the relevant part of the code is as follows:
req = requests.post(self.URL, data={"username": username, "password": password})
self.cookies = {"MOODLEID1_": req.cookies["MOODLEID1_"], "MoodleSession": req.cookies["MoodleSession"]}
self.URL has the correct page, and the POST is working as intended, I did some print to check that, and it passed.
My output:
Traceback (most recent call last):
File "D:/.../main.py", line 14, in <module>
m.login('first.last', 'pa$$w0rd!')
File "D:\...\moodle2.py", line 14, in login
self.cookies = {"MOODLEID1_": req.cookies["MOODLEID1_"], "MoodleSession": req.cookies["MoodleSession"]}
File "D:\...\venv\lib\site-packages\requests\cookies.py", line 287, in __getitem__
return self._find_no_duplicates(name)
File "D:\...\venv\lib\site-packages\requests\cookies.py", line 345, in _find_no_duplicates
raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
KeyError: "name='MOODLEID1_', domain=None, path=None"
I'm trying to debug during runtime to check what req.cookies has. But what I get is surprising, at least for me. If you put a breakpoint on self.cookies = {...} and run [(c.name, c.value, c.domain) for c in req.cookies] I get an empty list, like there isn't any cookie in there.
The site does send cookies, checking with a Chrome extension, I found 2, "MOODLEID1_" and "MoodleSession", so why I'm not getting them?
The response doesn't appear to contain any cookies. Look for one or more Set-Cookie headers in req.headers.
Cookies stored in a browser are there because a response included a Set-Cookie header for each of those cookies. You'll have to find what response the server sets those cookies with; apparently it is not this response.
If you need to retain those cookies (once set) across requests, do use a requests.Session() object; this'll retain any cookies returned by responses and send them out again as appropriate with new requests.
This question already has an answer here:
Python GAE urlfetch credentials
(1 answer)
Closed 7 years ago.
GAE Python URL Fetch throws InvalidURLError while the same URL works perfectly with Postman ( Google Chrome App ).
CODE
url = "https://abcdefgh:28dfd95928dfd95928dfd95928dfd95928dfd95928dfd959#twilix.exotel.in/v1/Accounts/abcdefgh/Sms/send"
form_fields = {
"From": "08039511111",
"To": "+919844100000",
"Body": "message for you"
}
form_data = urllib.urlencode (form_fields)
try:
result = urlfetch.fetch(url=url,
payload=form_data,
method=urlfetch.POST,
headers={'Content-Type': 'application/x-www-form-urlencoded' }
)
logging.info ("result = ["+repr (result)+"] ")
except Exception:
logging.error ("Exception. ["+traceback.format_exc ()+"] ")
OUTPUT LOGS
2016-01-21 15:48:23.368 +0530 E Exception. [
Traceback (most recent call last): File "main.py", line 27, in get method=urlfetch.POST,
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 271, in fetch return rpc.get_result()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result return self.__get_result_hook(self)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 389, in _get_fetch_result 'Invalid request URL: ' + url + error_detail) InvalidURLError: Invalid request URL: https://abcdefgh:28dfd95928dfd95928dfd95928dfd95928dfd95928dfd959#twilix.exotel.in/v1/Accounts/abcdefgh/Sms/send ]
For security purpose, I have replaced sensitive text in the URL with similar different characters.
The code indicates an INVALID_URL RPC error code was received from the urlfetch service.
The most common occurence seems to be due to the URL length limit (check if your unedited URL hits that): Undocumented max length for urlfetch URL?
A long time ago it was also seen for very slow URLs (in Go land, but I suspect the urlfetch service itself is the same serving all language sandboxes) - unsure if this still stands, I also see a DEADLINE_EXCEEDED error code as well which might have been introduced specifically for such case in the meantime): Google App Engine Go HTTP request to a slow page
The failure might also be related to incorrect parsing of the rather unusual "host" portion of your URL foo:blah#hostname. Check if it you're getting the same error if dropping the foo:blah# portion. If it's indeed the case you might want to file an issue with Google - the URL seems valid, works with curl as well.
I found the problem and the solution.
We need to specify the HTTP auth info using headers.
urlfetch.make_fetch_call ( rpc,
url,
method = urlfetch.POST,
headers = { "Authorization" : "Basic %s" % base64.b64encode ( URL_USERNAME+":"+URL_PASSOWRD ) },
)
Courtesy
https://stackoverflow.com/a/8454580/1443563 by raugfer
I am using Python's AWeber API (https://github.com/aweber/AWeber-API-Python-Library), and I frequently get these exceptions. I have no idea why this happens. Any ideas?
File "/<path>/aweber_api/entry.py", line 160, in __getattr__
return self._child_collection(attr)
File "/<path>/aweber_api/entry.py", line 151, in _child_collection
self._child_collections[attr] = self.load_from_url(url)
File "/<path>/aweber_api/base.py", line 38, in load_from_url
response = self.adapter.request('GET', url)
File "/<path>/aweber_api/oauth.py", line 60, in request
'{0}: {1}'.format(error_type, error_msg))
APIException: UnauthorizedError: Combination of nonce, timestamp, and consumer_key must be unique. https://labs.aweber.com/docs/troubleshooting#unauthorized
The error message is actually due to OAuth. You are sending the same request multiple times. You need to generate your request again (even if the same command and parameters) to get a new timestamp and nonce.
This is an OAuth measure to ensure it isn't dealing with the exact same request multiple times. e.g. your program actually sends the command twice at the exact same time.
I'm trying to write a Python client for a a WSDL service. I'm using the Suds library to handle the SOAP messages.
When I try to call the service, I get a Suds exception: <rval /> not mapped to message part. If I set the retxml Suds option I get XML which looks OK to me.
Is the problem with the client code? Am I missing some flag which will allow Suds to correctly parse the XML? Alternatively, the problem could be with the server. Is the XML not structured correctly?
My code is a follows (method names changed):
c = Client(url)
p = c.factory.create('MyParam')
p.value = 100
c.service.run(p)
This results in a Suds exception:
File "/home/.../test.py", line 38, in test
res = self.client.service.run(p)
File "/usr/local/lib/python2.6/dist-packages/suds-0.3.9-py2.6.egg/suds/client.py", line 539, in __call__
return client.invoke(args, kwargs)
File "/usr/local/lib/python2.6/dist-packages/suds-0.3.9-py2.6.egg/suds/client.py", line 598, in invoke
result = self.send(msg)
File "/usr/local/lib/python2.6/dist-packages/suds-0.3.9-py2.6.egg/suds/client.py", line 627, in send
result = self.succeeded(binding, reply.message)
File "/usr/local/lib/python2.6/dist-packages/suds-0.3.9-py2.6.egg/suds/client.py", line 659, in succeeded
r, p = binding.get_reply(self.method, reply)
File "/usr/local/lib/python2.6/dist-packages/suds-0.3.9-py2.6.egg/suds/bindings/binding.py", line 151, in get_reply
result = self.replycomposite(rtypes, nodes)
File "/usr/local/lib/python2.6/dist-packages/suds-0.3.9- py2.6.egg/suds/bindings/binding.py", line 204, in replycomposite
raise Exception('<%s/> not mapped to message part' % tag)
Exception: <rval/> not mapped to message part
The returned XML (modified to remove customer identifiers)
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body>
<ns2:getResponse xmlns:ns2="http://api.xxx.xxx.com/api/">
<rval xmlns="http://xxx.xxx.xxx.com/api/">
<ns2:totalNumEntries>
2
</ns2:totalNumEntries>
<ns2:entries>
<ns2:id>
1
</ns2:id>
</ns2:entries>
<ns2:entries>
<ns2:id>
2
</ns2:id>
</ns2:entries>
</rval>
</ns2:getResponse>
</S:Body>
</S:Envelope>
Possible dup of What does suds mean by "<faultcode/> not mapped to message part"?
Here is my answer from that question:
I had a similar issue where the call was successful, and Suds crashed on parsing the response from the client. The workaround I used was to use the Suds option to return raw XML and then use BeautifulSoup to parse the response.
Example:
client = Client(url)
client.set_options(retxml=True)
soapresp_raw_xml = client.service.submit_func(data)
soup = BeautifulStoneSoup(soapresp_raw_xml)
value_i_want = soup.find('ns:NewSRId')
This exception actually means that the answer from SOAP-service contains tag <rval>, which doesn't exist in the WSDL-scheme of the service.
Keep in mind that the Suds library caches the WSDL-scheme, that is why the problem may occur if the WSDL-scheme was changed recently. Then the answers match the new scheme, but are verified by the suds-client with the old one. In this case rm /tmp/suds/* will help you.