Develop Reference

Best way to refactor nested 'with patch' statements

I am trying to patch out the Azure Digital Twin API in my code. Currently I have achieved a way which works but is probably not the most Pythonic by nesting with patch statements.
What is the best way to rewrite this such that I can use it in multiple test functions and change the return values if needed?
def test_create_digital_twin(self):
with patch("endpoints.digital_twin.ClientSecretCredential"):
with patch("azure_digital_twin.create_digital_twin.DigitalTwinsClient.query_twins",) as mock_query:
with patch("azure_digital_twin.create_digital_twin.DigitalTwinsClient.upsert_digital_twin") as mock_upsert_twin:
with patch("azure_digital_twin.create_digital_twin.DigitalTwinsClient.upsert_relationship") as mock_upsert_relationship:
mock_query.return_value = []
mock_upsert_twin.return_value = {
"$dtId":"spc-1",
"$etag":"random",
"$model":"dtmi:digitaltwins:rec_3_3:core:Asset;1"
}
mock_upsert_relationship.return_value = {
"$relationshipId":"spc-1-hasPart-spc-2",
"$sourceId":"spc-1",
"$targetId" : "spc-2",
"$relationshipName":"hasPart"
}
response = self.client.post(
endpoint,
params={"node" : "spc-1"},
)
assert response.status_code == status.HTTP_201_CREATED

You might use an ExitStack from the contextlib module.
from contextlib import ExitStack
def test_create_digital_twin(self):
with ExitStack() as es:
def make_azure_patch(x):
return es.enter_context(patch(f'azure_digital_twin.create_digital_twin.DigitalTwinsClient.{x}'))
es.enter_context("endpoints.digital_twin.ClientSecretCredential"))
mock_query = make_patch("query_twins")
mock_upsert_twin = make_patch("upsert_digital_twin")
mock_upsert_relationship = make_patch("upsert_relationship")
mock_query.return_value = []
mock_upsert_twin.return_value = {
"$dtId":"spc-1",
"$etag":"random",
"$model":"dtmi:digitaltwins:rec_3_3:core:Asset;1"
}
mock_upsert_relationship.return_value = {
"$relationshipId":"spc-1-hasPart-spc-2",
"$sourceId":"spc-1",
"$targetId" : "spc-2",
"$relationshipName":"hasPart"
}
response = self.client.post(
endpoint,
params={"node" : "spc-1"},
)
assert response.status_code == status.HTTP_201_CREATED
make_azure_patch is just a helper function to reduce the length of the lines creating three of the individual patches.

constructing a message format from the fetchall result in python

*New to Programming
Question: I need to use the below "Data" (two rows as arrays) queried from sql and use it to create the message structure below.
data from sql using fetchall()
Data = [[100,1,4,5],[101,1,4,6]]
##expected message structure
message = {
"name":"Tom",
"Job":"IT",
"info": [
{
"id_1":"100",
"id_2":"1",
"id_3":"4",
"id_4":"5"
},
{
"id_1":"101",
"id_2":"1",
"id_3":"4",
"id_4":"6"
},
]
}
I tried to create below method to iterate over the rows and then input the values, this is was just a starting, but this was also not working
def create_message(data)
for row in data:
{
"id_1":str(data[0][0],
"id_2":str(data[0][1],
"id_3":str(data[0][2],
"id_4":str(data[0][3],
}
Latest Code
def create_info(data):
info = []
for row in data:
temp_dict = {"id_1_tom":"","id_2_hell":"","id_3_trip":"","id_4_clap":""}
for i in range(0,1):
temp_dict["id_1_tom"] = str(row[i])
temp_dict["id_2_hell"] = str(row[i+1])
temp_dict["id_3_trip"] = str(row[i+2])
temp_dict["id_4_clap"] = str(row[i+3])
info.append(temp_dict)
return info

Edit: Updated answer based on updates to the question and comment by original poster.
This function might work for the example you've given to get the desired output, based on the attempt you've provided:
def create_info(data):
info = []
for row in data:
temp_dict = {}
temp_dict['id_1_tom'] = str(row[0])
temp_dict['id_2_hell'] = str(row[1])
temp_dict['id_3_trip'] = str(row[2])
temp_dict['id_4_clap'] = str(row[3])
info.append(temp_dict)
return info
For the input:
[[100, 1, 4, 5],[101,1,4,6]]
This function will return a list of dictionaries:
[{"id_1_tom":"100","id_2_hell":"1","id_3_trip":"4","id_4_clap":"5"},
{"id_1_tom":"101","id_2_hell":"1","id_3_trip":"4","id_4_clap":"6"}]
This can serve as the value for the key info in your dictionary message. Note that you would still have to construct the message dictionary.

how do I merge two list[dict] and order properly by its ID?

I am trying to merge two list[dict] into the following format and order both techniques, and subtechnique, but I am not sure how to properly join them without getting affected other techniques
techniques = [{
"technique_id":"T1548",
"technique":"demo",
"url":"url",
"tactic":[
"demo",
"demo"
]
}]
subtechniques = [{
"technique_id":"T1548.002",
"technique":"demo",
"url":"url"
}]
def merge_techniques(techniques, subtechniques):
change_list = []
for x in techniques:
for y in subtechniques:
if x['technique_id'] == y['technique_id'].split('.')[0]:
print(x)
print(y)
return change_list
merge_techniques(techniques, subtechniques)
desired output
{
"technique_id":"T1548",
"technique":"dmep",
"url":"https://xxxxxxxxxxxx",
"tactic":[
"xxxxxxxxxxxx",
"xxxxxxxxxxxx"
],
"subtechnique": [
{
"technique_id":"T1548.002",
"technique":"demo",
"url":"url"
}
]
}

If you don't want to create a new object, you can discard copy operations.
import copy
def merge_techniques(techniques, subtechniques):
result = []
# create technique_id dictionary
techniques_dict = dict()
for tech in techniques:
# create new object
tech_copy = copy.copy(tech)
techniques_dict[tech['technique_id']] = tech_copy
result.append(tech_copy)
# visit all sub techniques
for subtech in subtechniques:
tech_id = subtech['technique_id'].split('.')[0]
# search by tech_id
if tech_id not in techniques_dict:
# if not found
print('Tech %s is not found'%(tech_id))
continue
# get tech by tech_id
tech = techniques_dict[tech_id]
# create subtechnique array if not existed
if 'subtechnique' not in tech:
tech['subtechnique'] = []
# copy subtech object
tech['subtechnique'].append(copy.copy(subtech))
return result

Getting certain information from string

I'm new to python as was wondering how I could get the estimatedWait and routeName from this string.
{
"lastUpdated": "07:52",
"filterOut": [],
"arrivals": [
{
"routeId": "B16",
"routeName": "B16",
"destination": "Kidbrooke",
"estimatedWait": "due",
"scheduledTime": "06: 53",
"isRealTime": true,
"isCancelled": false
},
{
"routeId":"B13",
"routeName":"B13",
"destination":"New Eltham",
"estimatedWait":"29 min",
"scheduledTime":"07:38",
"isRealTime":true,
"isCancelled":false
}
],
"serviceDisruptions":{
"infoMessages":[],
"importantMessages":[],
"criticalMessages":[]
}
}
And then save this to another string which would be displayed on the lxterminal of the raspberry pi 2. I would like only the 'routeName' of B16 to be saved to the string. How do I do that?

You just have to deserialise the object and then use the index to access the data you want.
To find only the B16 entries you can filter the arrivals list.
import json
obj = json.loads(json_string)
# filter only the b16 objects
b16_objs = filter(lambda a: a['routeName'] == 'B16', obj['arrivals'])
if b16_objs:
# get the first item
b16 = b16_objs[0]
my_estimatedWait = b16['estimatedWait']
print(my_estimatedWait)

You can use string.find() to get the indices of those value identifiers
and extract them.
Example:
def get_vaules(string):
waitIndice = string.find('"estimatedWait":"')
routeIndice = string.find('"routeName":"')
estimatedWait = string[waitIndice:string.find('"', waitIndice)]
routeName = string[routeIndice:string.find('"', routeIndice)]
return estimatedWait, routeName
Or you could just deserialize the json object (highly recommended)
import json
def get_values(string):
jsonData = json.loads(string)
estimatedWait = jsonData['arrivals'][0]['estimatedWait']
routeName = jsonData['arrivals'][0]['routeName']
return estimatedWait, routeName
Parsing values from a JSON file using Python?

Python utility for parsing blocks?

I have a file that starts something like:
databaseCons = {
main = {
database = "readable_name",
hostname = "hostdb1.serv.com",
instances= {
slaves = {
conns = "8"
}
}
maxconns = "5",
user = "user",
pass = "pass"
}
}
So, what I'd like to do is parse this out into a dict of sub-dicts, something like:
{'main': {'database': 'readable_name', 'hostname': 'hostdb1.serv.com', 'maxconns': '5', 'instances': {'slave': {'maxCount': '8'}}, 'user': 'user', 'pass': 'pass'}}
I think the above makes sense... but please feel free to edit this if it doesn't. Basically I want the equivalent of:
conns = '8'
slave = dict()
slave['maxCount'] = conns
instances = dict()
instances['slave'] = slave
database = 'readable_name'
hostname = 'hostdb1.serv.com'
maxconns = '5'
user = 'user'
pas = 'pass'
main = dict()
main['database'] = database
main['hostname'] = hostname
main['instances'] = instances
main['maxconns'] = maxconns
main['user'] = user
main['pass'] = pas
databaseCons = dict()
databaseCons['main'] = main
Are there any modules out there that can handle this sort of parsing? Even what I've suggested above looks messy.. there's got to be a better way I'd imagine.

Here is a pyparsing parser for your config file:
from pyparsing import *
def to_dict(t):
return {k:v for k,v in t}
series = Forward()
struct = Suppress('{') + series + Suppress('}')
value = quotedString.setParseAction(removeQuotes) | struct
token = Word(alphanums)
assignment = Group(token + Suppress('=') + value + Suppress(Optional(",")))
series << ZeroOrMore(assignment).setParseAction(to_dict)
language = series + stringEnd
def config_file_to_dict(filename):
return language.parseFile(filename)[0]
if __name__=="__main__":
from pprint import pprint
pprint(config_file_to_dict('config.txt'))