Develop Reference

Exception: TypeError(string indices must be integers)

I have written the below python function (a snippet of the full code) to work in AWS Lambda. The purpose of it is to take a GeoJSON from an S3 bucket and parse it accordingly.
Once parsed, it is placed back into JSON format (data) and then should be inserted into the specified database using
bulk_item['uuid'] = str(uuid.uuid4())
bulk_item['name'] = feature_name
bulk_item['type'] = feature_type
bulk_item['info'] = obj
bulk_item['created'] = epoch_time
bulk_item['scope'] = 2
data = json.dumps(bulk_item)
print(data)
self.database.upsert_record(self.organisation, json_doc=data)
except Exception as e:
print(f'Exception: {e.__class__.__name__}({e})')
The db_access file in which the above is relating to is another python script. The function upsert_record is as below:
def upsert_record(self, organisation,
json_doc={}):
My code is working perfectly until I try to upsert it into the database. Once this line is gotten to, it throws the error
Traceback (most recent call last):
File "/var/task/s3_asset_handler.py", line 187, in process_incoming_file
self.database.upsert_record(self.organisation, json_doc=data)
File "/opt/python/database_access.py", line 1218, in upsert_record
new_uuid = json_doc['uuid']
TypeError: string indices must be integers
I can't seem to figure out the issue at all

You are trying to get an element from a JSON object, but passing a string.
The
data = json.dumps(bulk_item)
creates a string representing the object.
Try using bulk_item on it's own.

Amazon Neptune on submitting query: AttributeError: 'str' object has no attribute 'source_instructions'

I have the following code running on AWS lambda, but getting the following error.
Error
[ERROR] AttributeError: 'str' object has no attribute 'source_instructions'
Traceback (most recent call last):
File "/var/task/gremlin_python/driver/driver_remote_connection.py", line 56, in submit
    result_set = self._client.submit(bytecode, request_options=self._extract_request_options(bytecode))
  File "/var/task/gremlin_python/driver/driver_remote_connection.py", line 81, in _extract_request_options
    options_strategy = next((x for x in bytecode.source_instructionsEND RequestId: 4ee8073c-e941-43b3-8014-8717893b3188
Source code
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
def test_neptune(host):
remoteConn = DriverRemoteConnection('wss://{}:8182/gremlin','g'.format(host))
query = "g.V().groupCount().by(label).unfold().project('label','count').by(keys).by(values)"
response = remoteConn.submit(query)
print("response-> {}" .format(response))
# iterate repsonse
# go thru label
for set_item in response:
for item in set_item:
print("item-> item: {}".format(item))
remoteConn.close()
test_neptune()

Your DriverRemoteConnection call is wrong. You have:
remoteConn = DriverRemoteConnection('wss://{}:8182/gremlin','g'.format(host))
So you are sending {} as the hostname, and passing 'g' as a second parameter, which is probably where the error comes from. I don't know what you intended the 'g' for, but you probably want:
remoteConn = DriverRemoteConnection('wss://{}:8182/gremlin'.format(host))

If you send the query as a text string you need to create the Client object differently or write the query as in-line Python. There are two examples at (1) and (2) that show each option. The error you are seeing is because the server is trying to find Gremlin bytecode in the packet sent but only found a string (which does not have a source_instructions method).
Using a DriverRemoteConnection you can use a Python line of code such as:
result = (g.V().groupCount().
by(label).
unfold().
project('label','count').
by(keys).
by(values).
next())
If you actually want/need to send the query as a string instead of bytecode, please see my answer to this question
https://github.com/krlawrence/graph/blob/master/sample-code/basic-client.py
https://github.com/krlawrence/graph/blob/master/sample-code/glv-client.py

Access JSON file data using preset query - Python

I am reading a json file with dictionary and values, but I am battling to use a variable as a query item when searching the json file.
x = value_cloud = "%s%s%s" % (["L1_METADATA_FILE"],["IMAGE_ATTRIBUTES"],["CLOUD_COVER"])
for meta in filelist(dir):
with open (meta) as data_file:
data = json.load(data_file)
cloud = str(data[x])
The error I get is:
Traceback (most recent call last):
File "E:\SAMPLE\Sample_Script_AWS\L8_TOA_using_gdal_rasterio.py", line 96, in <module>
cloud = str(data[x])
KeyError: "['L1_METADATA_FILE']['IMAGE_ATTRIBUTES']['CLOUD_COVER']"
What I actually want is to search the json file for the key in the variable...
The keys do exist in the json file because when I run the following I get the correct output.
cloud = str(data["L1_METADATA_FILE"]["IMAGE_ATTRIBUTES"]["CLOUD_COVER"])
print cloud
My knowledge of python is sketchy, and I am passing the variable through as a string and not an expression or object and therefore it gives me that error. What is the correct way to create the variable and call the keys that I want.
Thanks in advance!

Your key ends up including the brackets in the string, which which where the error comes from. If you use each key in its own variable, like this:
x, y, z = "L1_METADATA_FILE", "IMAGE_ATTRIBUTES" , "CLOUD_COVER"
and then:
cloud = str(data[x][y][z])
it should avoid any errors.

Can't find SalesForce Object

I'm having some trouble with SalesForce, I've never used it before so I'm not entirely sure what is going wrong here. I am using the simple_salesforce python module. I have successfully pulled data from SalesForce standard objects, but this custom object is giving me trouble. My query is
result = sf.query("Select Name from Call_Records__c")
which produces this error:
Traceback (most recent call last):
File "simple.py", line 15, in <module>
result = sf.query("Select Name from Call_Records__c")
File "/usr/local/lib/python2.7/dist-packages/simple_salesforce/api.py", line 276, in query
_exception_handler(result)
File "/usr/local/lib/python2.7/dist-packages/simple_salesforce/api.py", line 634, in _exception_handler
raise exc_cls(result.url, result.status_code, name, response_content)
simple_salesforce.api.SalesforceMalformedRequest: Malformed request https://sandbox.company.com/services/data/v29.0/query/?q=Select+Name+from+Call_Records__c. Response content: [{u'errorCode': u'INVALID_TYPE', u'message': u"\nSelect Name from Call_Records__c\n ^\nERROR at Row:1:Column:18\nsObject type 'Call_Records__c' is not supported. If you are attempting to use a custom object, be sure to append the '__c' after the entity name.
Please reference your WSDL or the describe call for the appropriate names."}]
I've tried it with and without the __c for both the table name and the field name, still can't figure this out. Anything blatantly wrong?

Make sure your result is Call_Records__c/CallRecords__c
Call_Records__c result = sf.query("Select Name from Call_Records__c")
Or
CallRecords__c result = sf.query("Select Name from CallRecords__c")

Try using -
result = sf.query("Select Name from CallRecords__c")

Updating a Spreadsheet in Google Docs

I have a python script that takes a generated CSV and uploads it to Google Docs. It can upload it just fine, put I cannot seem to get it to replace the data, it returns an error I cannot find reference to.
Le Code:
import gdata.auth
import gdata.docs
import gdata.docs.service
import gdata.docs.data
import gdata.docs.client
email = 'admin#domain.com'
CONSUMER_KEY='domain.com'
CONSUMER_SECRET='blah54545blah'
ms_client = gdata.docs.client.DocsClient('Domain_Doc_Upload')
ms_client.auth_token = gdata.gauth.TwoLeggedOAuthHmacToken(CONSUMER_KEY, CONSUMER_SECRET, email)
url = 'http://docs.google.com/feeds/documents/private/full/sd01blahgarbage'
ms = gdata.data.MediaSource(file_path="C:\\people.csv", content_type='text/csv')
csv_entry2 = ms_client.Update(url, ms)
It returns:
Traceback (most recent call last):
File "so_test.py", line 19, in <module>
csv_entry2 = ms_client.Update(ms, url)
File "build\bdist.win-amd64\egg\gdata\client.py", line 717, in update
AttributeError: 'MediaSource' object has no attribute 'to_string'
I cannot find anything about the 'to_string' attribute, so I am lost on the trace. ANy help, much appreciated.

I took a look at the docs and it looks like the Update method takes (entry, ms) where entry needs to be a gdata.docs.data.DocsEntry object. You should be able to get the DocsEntry object by getting a feed from your client.
feed = client.GetDocList()