Develop Reference

Exception: TypeError(string indices must be integers)

I have written the below python function (a snippet of the full code) to work in AWS Lambda. The purpose of it is to take a GeoJSON from an S3 bucket and parse it accordingly.
Once parsed, it is placed back into JSON format (data) and then should be inserted into the specified database using
bulk_item['uuid'] = str(uuid.uuid4())
bulk_item['name'] = feature_name
bulk_item['type'] = feature_type
bulk_item['info'] = obj
bulk_item['created'] = epoch_time
bulk_item['scope'] = 2
data = json.dumps(bulk_item)
print(data)
self.database.upsert_record(self.organisation, json_doc=data)
except Exception as e:
print(f'Exception: {e.__class__.__name__}({e})')
The db_access file in which the above is relating to is another python script. The function upsert_record is as below:
def upsert_record(self, organisation,
json_doc={}):
My code is working perfectly until I try to upsert it into the database. Once this line is gotten to, it throws the error
Traceback (most recent call last):
File "/var/task/s3_asset_handler.py", line 187, in process_incoming_file
self.database.upsert_record(self.organisation, json_doc=data)
File "/opt/python/database_access.py", line 1218, in upsert_record
new_uuid = json_doc['uuid']
TypeError: string indices must be integers
I can't seem to figure out the issue at all

You are trying to get an element from a JSON object, but passing a string.
The
data = json.dumps(bulk_item)
creates a string representing the object.
Try using bulk_item on it's own.

How to use get_attachment call upon QueryResult (python cloudant) ?

I've been trying to get attachment image data from documents in Cloudant.
I can successfully do it once a document is selected (direct extract with _id, etc).
Now trying to do it in combination with "query" operation using selector, I run into trouble.
Here is my code.
targetName="chibika33"
targetfile="chibitest.png"
#--------------------------------------------------
# get all the documents with the specific nameField
#--------------------------------------------------
myDatabase.create_query_index(fields = ['nameField'])
selector = {'nameField': {'$eq': targetName}}
docs = myDatabase.get_query_result(selector)
#--------------------------------------------------
# get the attachment files to them, save it locally
#--------------------------------------------------
count = 0
for doc in docs:
count=count+1
result_filename="result%03d.png"%(count)
dataContent = doc.get_attachment(targetfile, attachment_type='binary')
dataContentb =base64.b64decode(dataContent)
with open(result_filename,'wb') as output:
output.write(dataContentb)
Causes error as;
Traceback (most recent call last):
File "view8.py", line 44, in <module>
dataContent = doc.get_attachment(targetfile, attachment_type='binary')
AttributeError: 'dict' object has no attribute 'get_attachment'
So far, I've been unable to find any API for converting dict to document object in the python-cloudant-document...[python-cloudant document]: http://python-cloudant.readthedocs.io/en/latest/index.html
Any advise would be highly appreciated.

The returned structure from get_query_result(...) isn't an array of documents.
Try:
resp = myDatabase.get_query_result(selector)
for doc in resp['docs']:
# your code here
See the docs at:
http://python-cloudant.readthedocs.io/en/latest/database.html#cloudant.database.CloudantDatabase.get_query_result

Cannot encode object: pymongo.cursor.Cursor object at

I am trying to retrieve an audio file stored in MongoDB when above error is thrown.
The code is as follows:
elif json_data != None and 'retriever' in json_data:
query_param = json_data['retriever']
data = db.soundData
x = data.find({'name': query_param})
y = data.find({'data': x})
return Response(y, mimetype='audio/mp3')
Under name I have the name of the file and under data is audio file itself.
As I am new to pymongo can somebody point to where an error could be coming from?

First of all, you need not be saving your file itself in mongo what you should be saving is the filename and the file itself is better off on the file system.
The error appears because, both x and y are indeed mongodb cursors rather than the data that you expect. You should be using find_one instead.
find_one(filter=None, *args, **kwargs) Get a single document from the
database.
All arguments to find() are also valid arguments for find_one(),
although any limit argument will be ignored. Returns a single
document, or None if no matching document is found.
y = data.find_one({'data': x})
You can make your code a bit more concise with
y = data.find_one({'data': {'name': query_param}})

Can't find SalesForce Object

I'm having some trouble with SalesForce, I've never used it before so I'm not entirely sure what is going wrong here. I am using the simple_salesforce python module. I have successfully pulled data from SalesForce standard objects, but this custom object is giving me trouble. My query is
result = sf.query("Select Name from Call_Records__c")
which produces this error:
Traceback (most recent call last):
File "simple.py", line 15, in <module>
result = sf.query("Select Name from Call_Records__c")
File "/usr/local/lib/python2.7/dist-packages/simple_salesforce/api.py", line 276, in query
_exception_handler(result)
File "/usr/local/lib/python2.7/dist-packages/simple_salesforce/api.py", line 634, in _exception_handler
raise exc_cls(result.url, result.status_code, name, response_content)
simple_salesforce.api.SalesforceMalformedRequest: Malformed request https://sandbox.company.com/services/data/v29.0/query/?q=Select+Name+from+Call_Records__c. Response content: [{u'errorCode': u'INVALID_TYPE', u'message': u"\nSelect Name from Call_Records__c\n ^\nERROR at Row:1:Column:18\nsObject type 'Call_Records__c' is not supported. If you are attempting to use a custom object, be sure to append the '__c' after the entity name.
Please reference your WSDL or the describe call for the appropriate names."}]
I've tried it with and without the __c for both the table name and the field name, still can't figure this out. Anything blatantly wrong?

Make sure your result is Call_Records__c/CallRecords__c
Call_Records__c result = sf.query("Select Name from Call_Records__c")
Or
CallRecords__c result = sf.query("Select Name from CallRecords__c")

Try using -
result = sf.query("Select Name from CallRecords__c")

Read BSON file in Python?

I want to read a BSON format Mongo dump in Python and process the data. I am using the Python bson package (which I'd prefer to use rather than have a pymongo dependency), but it doesn't explain how to read from a file.
This is what I'm trying:
bson_file = open('statistics.bson', 'rb')
b = bson.loads(bson_file)
print b[0]
But I get:
Traceback (most recent call last):
File "test.py", line 11, in <module>
b = bson.loads(bson_file)
File "/Library/Python/2.7/site-packages/bson/__init__.py", line 75, in loads
return decode_document(data, 0)[1]
File "/Library/Python/2.7/site-packages/bson/codec.py", line 235, in decode_document
length = struct.unpack("<i", data[base:base + 4])[0]
TypeError: 'file' object has no attribute '__getitem__'
What am I doing wrong?

I found this worked for me with a mongodb 2.4 BSON file and PyMongo's 'bson' module:
import bson
with open('survey.bson','rb') as f:
data = bson.decode_all(f.read())
That returned a list of dictionaries matching the JSON documents stored in that mongo collection.
The f.read() data looks like this in a BSON:
>>> rawdata[:100]
'\x04\x01\x00\x00\x12_id\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02_type\x00\x07\x00\x00\x00simple\x00\tchanged\x00\xd0\xbb\xb2\x9eI\x01\x00\x00\tcreated\x00\xd0L\xdcfI\x01\x00\x00\x02description\x00\x14\x00\x00\x00testing the bu'

The documentation states :
> help(bson.loads)
Given a BSON string, outputs a dict.
You need to pass a string. For example:
> b = bson.loads(bson_file.read())

loads expects a string (that's what the 's' stands for), not a file. Try reading from the file, and passing the result to loads.