Develop Reference

constructing a message format from the fetchall result in python

*New to Programming
Question: I need to use the below "Data" (two rows as arrays) queried from sql and use it to create the message structure below.
data from sql using fetchall()
Data = [[100,1,4,5],[101,1,4,6]]
##expected message structure
message = {
"name":"Tom",
"Job":"IT",
"info": [
{
"id_1":"100",
"id_2":"1",
"id_3":"4",
"id_4":"5"
},
{
"id_1":"101",
"id_2":"1",
"id_3":"4",
"id_4":"6"
},
]
}
I tried to create below method to iterate over the rows and then input the values, this is was just a starting, but this was also not working
def create_message(data)
for row in data:
{
"id_1":str(data[0][0],
"id_2":str(data[0][1],
"id_3":str(data[0][2],
"id_4":str(data[0][3],
}
Latest Code
def create_info(data):
info = []
for row in data:
temp_dict = {"id_1_tom":"","id_2_hell":"","id_3_trip":"","id_4_clap":""}
for i in range(0,1):
temp_dict["id_1_tom"] = str(row[i])
temp_dict["id_2_hell"] = str(row[i+1])
temp_dict["id_3_trip"] = str(row[i+2])
temp_dict["id_4_clap"] = str(row[i+3])
info.append(temp_dict)
return info

Edit: Updated answer based on updates to the question and comment by original poster.
This function might work for the example you've given to get the desired output, based on the attempt you've provided:
def create_info(data):
info = []
for row in data:
temp_dict = {}
temp_dict['id_1_tom'] = str(row[0])
temp_dict['id_2_hell'] = str(row[1])
temp_dict['id_3_trip'] = str(row[2])
temp_dict['id_4_clap'] = str(row[3])
info.append(temp_dict)
return info
For the input:
[[100, 1, 4, 5],[101,1,4,6]]
This function will return a list of dictionaries:
[{"id_1_tom":"100","id_2_hell":"1","id_3_trip":"4","id_4_clap":"5"},
{"id_1_tom":"101","id_2_hell":"1","id_3_trip":"4","id_4_clap":"6"}]
This can serve as the value for the key info in your dictionary message. Note that you would still have to construct the message dictionary.

python trading bot( paper trading)

So I am writing a trading bot in python. It is more for fun and I just started. Every method works alone, so I excluded them here, to not give you 300lines of Code. I also exclude the hole analyze method, since even if I clear the rest of the method, the same Error appears. When I use analyze just once, it doesn't do anything but also gets no error and when I use it in a loop I get an error: 
Exception has occurred: KeyError
'result' (mark get_crypto_data)
This doesn't make sense to me, since if I print get_crypto_data it works just fine.
def get_crypto_data(pair,since):
return api.query_public("OHLC", data = {"pair" : pair, "since" : since})["result"][pair] #array of prices (60sek)
def analyze(pair,since):
data = get_crypto_data(pair[0]+pair[1], since)
if __name__ == "__main__":
api = krakenex.API()
api.load_key("KrakenKey.txt")
pair = ('XETH' , 'ZEUR') # Currency pair
since = str(int(time.time() - 3600))
while True:
analyze(pair,since)
The data structure receiving from the API looks like this(example)(without indents):
{
"error": [ ],
"result": {
"XXBTZUSD": [
[
1616662740,
"52591.9",
"52599.9",
"52591.8",
"52599.9",
"52599.1",
"0.11091626",
5
],
[
1616662800,
"52600.0",
"52674.9",
"52599.9",
"52665.2",
"52643.3",
"2.49035996",
30
],
[
1616662860,
"52677.7",
"52686.4",
"52602.1",
"52609.5",
"52634.5",
"1.25810675",
20
],
[
1616662920,
"52603.9",
"52627.5",
"52601.2",
"52616.4",
"52614.0",
"3.42391799",
23
],
[
1616662980,
"52601.2",
"52601.2",
"52599.9",
"52599.9",
"52599.9",
"0.43748934",
7
]
],
"last": 1616662920
}
}

Context
A KeyError in Python is raised when you try to search for an item in an object that doesn't exist. For example, if you make a request:
response = requests.get(url).json()
response['nonexistent']
# KeyError raised as nonexistent doesn't exist in the object
With that in mind, a KeyError when you make an API call to receive this object:
api.query_public("OHLC", data = {"pair" : pair, "since" : since})
We can infer that for whatever reason, ["result"] is not a key in the object above. To debug the issue, follow the steps below.
Debugging
Make the API call and save the response to a variable. Then print the variable along with its type to understand how you can interact with it.
response = api.query_public("OHLC", data = {"pair" : pair, "since" : since})
print(response, type(response))
If it's in a String format (or another standard convertible format), you can use the inbuilt json library to convert it to a dictionary object you can call as you did in your example.
import json
response = api.query_public("OHLC", data = {"pair" : pair, "since" : since})
response = json.loads(response)
Otherwise, given the structure of the output you displayed, it may be wise to convert the response to a string and then follow step 2.
import json
# Get the response
response = api.query_public("OHLC", data = {"pair" : pair, "since" : since})
# Convert to string
response = str(response)
# Convert to dictionary using JSON
response = json.loads(response)
# Call the data you want
data = response["result"]["XXBTZUSD"]

Python writing AVRO timestamp-millis: datum.astimezone(tz=timezones.utc) AttributeError: 'int' object has no attribute 'astimezone'

Environment: python 3.8.4 under Windows 10.
I am trying to write avro from dictionary.
Dictionary contains timestamp
def get_dict(self):
return {"msg_header": {...
"msg_timestamp": int(datetime.datetime.timestamp(datetime.datetime.now())),
...},
...
}
Avro schema:
{
"name" : "msg_timestamp",
"type" : {
"type" : "long",
"logicalType" : "timestamp-millis"
}
Writing Avro:
writer = avro.io.DatumWriter(schema)
bytes_writer = io.BytesIO()
encoder = avro.io.BinaryEncoder(bytes_writer)
writer.write(get_dict(), encoder) #----------Exception here
raw_bytes = bytes_writer.getvalue()
Exception:
...
line
581, in write_timestamp_millis_long
datum = datum.astimezone(tz=timezones.utc)
AttributeError: 'int' object has no attribute 'astimezone'
Any ideas how to fix it? I do not want conversion to UTC. Even if I set system TZ to UTC, it does not help, method astimezone(tz=timezones.utc) is being executed causing exception

Finally I ended up using fastavro, it provides more verbose exception messages rather than "the datum ... is not an example of the schema...", it works without any issue and process a list of messages.
import fastavro
...
parsed_schema = fastavro.schema.load_schema(schema_path)
...
def get_dict(self):
return {"msg_header": {...
"msg_timestamp": int(datetime.datetime.timestamp(datetime.datetime.now())),
...},
...
}
...
#in a for loop do something and Prepare list of messages
messages.append(get_dict().copy())
...
bytes_writer = io.BytesIO()
fastavro.writer(bytes_writer, parsed_schema, messages)
raw_bytes = bytes_writer.getvalue()

msg_timestamp is defined as a logical type so that means your dictionary should contain a datetime and the library will automatically convert it to a long when serializing and back to a datetime when deserializing.
So instead of:
"msg_timestamp": int(datetime.datetime.timestamp(datetime.datetime.now()))
You just want to do:
"msg_timestamp": datetime.datetime.now(tz=datetime.timezone.utc)
If instead you want to turn it to an int yourself and not have the library do it, don't use logicalType.

how can i take a specific element from this list in python?

I'm working with the Microsoft Azure face API and I want to get only the glasses response.
heres my code:
########### Python 3.6 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64, requests, json
###############################################
#### Update or verify the following values. ###
###############################################
# Replace the subscription_key string value with your valid subscription key.
subscription_key = '(MY SUBSCRIPTION KEY)'
# Replace or verify the region.
#
# You must use the same region in your REST API call as you used to obtain your subscription keys.
# For example, if you obtained your subscription keys from the westus region, replace
# "westcentralus" in the URI below with "westus".
#
# NOTE: Free trial subscription keys are generated in the westcentralus region, so if you are using
# a free trial subscription key, you should not need to change this region.
uri_base = 'https://westcentralus.api.cognitive.microsoft.com'
# Request headers.
headers = {
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': subscription_key,
}
# Request parameters.
params = {
'returnFaceAttributes': 'glasses',
}
# Body. The URL of a JPEG image to analyze.
body = {'url': 'https://upload.wikimedia.org/wikipedia/commons/c/c3/RH_Louise_Lillian_Gish.jpg'}
try:
# Execute the REST API call and get the response.
response = requests.request('POST', uri_base + '/face/v1.0/detect', json=body, data=None, headers= headers, params=params)
print ('Response:')
parsed = json.loads(response.text)
info = (json.dumps(parsed, sort_keys=True, indent=2))
print(info)
except Exception as e:
print('Error:')
print(e)
and it returns a list like this:
[
{
"faceAttributes": {
"glasses": "NoGlasses"
},
"faceId": "0f0a985e-8998-4c01-93b6-8ef4bb565cf6",
"faceRectangle": {
"height": 162,
"left": 177,
"top": 131,
"width": 162
}
}
]
I want just the glasses attribute so it would just return either "Glasses" or "NoGlasses"
Thanks for any help in advance!

I think you're printing the whole response, when really you want to drill down and get elements inside it. Try this:
print(info[0]["faceAttributes"]["glasses"])
I'm not sure how the API works so I don't know what your specified params are actually doing, but this should work on this end.
EDIT: Thank you to #Nuageux for noting that this is indeed an array, and you will have to specify that the first object is the one you want.

I guess that you can get few elements in that list, so you could do this:
info = [
{
"faceAttributes": {
"glasses": "NoGlasses"
},
"faceId": "0f0a985e-8998-4c01-93b6-8ef4bb565cf6",
"faceRectangle": {
"height": 162,
"left": 177,
"top": 131,
"width": 162
}
}
]
for item in info:
print (item["faceAttributes"]["glasses"])
>>> 'NoGlasses'

Did you try:
glasses = parsed[0]['faceAttributes']['glasses']

This looks more like a dictionary than a list. Dictionaries are defined using the { key: value } syntax, and can be referenced by the value for their key. In your code, you have faceAttributes as a key that for value contains another dictionary with a key glasses leading to the last value that you want.
Your info object is a list with one element: a dictionary. So in order to get at the values in that dictionary, you'll need to tell the list where the dictionary is (at the head of the list, so info[0]).
So your reference syntax will be:
#If you want to store it in a variable, like glass_var
glass_var = info[0]["faceAttributes"]["glasses"]
#Or if you want to print it directly
print(info[0]["faceAttributes"]["glasses"])
What's going on here? info[0] is the dictionary containing several keys, including faceAttributes,faceId and faceRectangle. faceRectangle and faceAttributes are both dictionaries in themselves with more keys, which you can reference to get their values.
Your printed tree there is showing all the keys and values of your dictionary, so you can reference any part of your dictionary using the right keys:
print(info["faceId"]) #prints "0f0a985e-8998-4c01-93b6-8ef4bb565cf6"
print(info["faceRectangle"]["left"]) #prints 177
print(info["faceRectangle"]["width"]) #prints 162
If you have multiple entries in your info list, then you'll have multiple dictionaries, and you can get all the outputs as so:
for entry in info: #Note: "entry" is just a variable name,
# this can be any name you want. Every
# iteration of entry is one of the
# dictionaries in info.
print(entry["faceAttributes"]["glasses"])
Edit: I didn't see that info was a list of a dictionary, adapted for that fact.

Getting certain information from string

I'm new to python as was wondering how I could get the estimatedWait and routeName from this string.
{
"lastUpdated": "07:52",
"filterOut": [],
"arrivals": [
{
"routeId": "B16",
"routeName": "B16",
"destination": "Kidbrooke",
"estimatedWait": "due",
"scheduledTime": "06: 53",
"isRealTime": true,
"isCancelled": false
},
{
"routeId":"B13",
"routeName":"B13",
"destination":"New Eltham",
"estimatedWait":"29 min",
"scheduledTime":"07:38",
"isRealTime":true,
"isCancelled":false
}
],
"serviceDisruptions":{
"infoMessages":[],
"importantMessages":[],
"criticalMessages":[]
}
}
And then save this to another string which would be displayed on the lxterminal of the raspberry pi 2. I would like only the 'routeName' of B16 to be saved to the string. How do I do that?

You just have to deserialise the object and then use the index to access the data you want.
To find only the B16 entries you can filter the arrivals list.
import json
obj = json.loads(json_string)
# filter only the b16 objects
b16_objs = filter(lambda a: a['routeName'] == 'B16', obj['arrivals'])
if b16_objs:
# get the first item
b16 = b16_objs[0]
my_estimatedWait = b16['estimatedWait']
print(my_estimatedWait)

You can use string.find() to get the indices of those value identifiers
and extract them.
Example:
def get_vaules(string):
waitIndice = string.find('"estimatedWait":"')
routeIndice = string.find('"routeName":"')
estimatedWait = string[waitIndice:string.find('"', waitIndice)]
routeName = string[routeIndice:string.find('"', routeIndice)]
return estimatedWait, routeName
Or you could just deserialize the json object (highly recommended)
import json
def get_values(string):
jsonData = json.loads(string)
estimatedWait = jsonData['arrivals'][0]['estimatedWait']
routeName = jsonData['arrivals'][0]['routeName']
return estimatedWait, routeName
Parsing values from a JSON file using Python?