get a part from a dictionary - python

i'm trying to get the pulse as an output for the given url using this code
from OTXv2 import OTXv2
from OTXv2 import IndicatorTypes
otx = OTXv2("my_key")
test=otx.get_indicator_details_full(IndicatorTypes.DOMAIN, "google.com")
and when i print test i become this output:
{'general': {'sections': ['general', 'geo', 'url_list', 'passive_dns', 'malware', 'whois', 'http_scans'], 'whois': 'http://whois.domaintools.com/google.com', 'alexa': 'http://www.alexa.com/siteinfo/google.com', 'indicator': 'google.com', 'type': 'domain', 'type_title': 'Domain', 'validation': [{'source': 'ad_network', 'message': 'Whitelisted ad network domain www-google-analytics.l.google.com', 'name': 'Whitelisted ad network domain'}, {'source': 'akamai', 'message': 'Akamai rank: #3', 'name': 'Akamai Popular Domain'}, {'source': 'alexa', 'message': 'Alexa rank: #1', 'name': 'Listed on Alexa'}, {'source': 'false_positive', 'message': 'Known False Positive', 'name': 'Known False Positive'}, {'source': 'majestic', 'message': 'Whitelisted domain google.com', 'name': 'Whitelisted domain'}, {'source': 'whitelist', 'message': 'Whitelisted domain google.com', 'name': 'Whitelisted domain'}], 'base_indicator': {'id': 12915, 'indicator': 'google.com', 'type': 'domain', 'title': '', 'description': '', 'content': '', 'access_type': 'public', 'access_reason': ''}, 'pulse_info': {'count': 0, 'pulses': [], 'references': [], 'related': {'alienvault': {'adversary': [], 'malware_families': [], 'industries': []}, 'other': {'adversary': [], 'malware_families': [], 'industries': []}}}, 'false_positive':...
i want to get only the part 'count': 0 in pulse_info
i tried using test.values() but it's like i have many dictionaries together
any idea how can i solve that?
Thank you

print(test["general"]["pulse_info"]["count"])

Related

JIRA JQL Query - Results

I'm writing a JQL query to fetch only Service Requests. I'm not able to fetch the Service Request field name from the below json using JQL Query. Any help will be appreciated
'jql': 'key=ITSM-1917' => This is working .. But I'm trying to fetch based on issuetype.name='Service Request'
{'expand': 'names,schema', 'startAt': 0, 'maxResults': 100, 'total': 1, 'issues': [{'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '373234', 'self': '', 'key': 'ITSM-1917', 'fields': {'issuetype': {'self': '', 'id': '10300', 'description': 'Created by JIRA Service Desk.', 'iconUrl': '', '**name': 'Service Request**', 'subtask': False, 'avatarId': 11006}, 'assignee': {'self': '', 'name': 'IT Service Management', 'key': 'JIRAUSER10945', 'emailAddress': '', 'avatarUrls': {'48x48': 'https://www.gravatar.com/avatar/067a17d84b041546f0f658bd011bc3ba?d=mm&s=48', '24x24': 'https://www.gravatar.com/avatar/067a17d84b041546f0f658bd011bc3ba?d=mm&s=24', '16x16': 'https://www.gravatar.com/avatar/067a17d84b041546f0f658bd011bc3ba?d=mm&s=16', '32x32': 'https://www.gravatar.com/avatar/067a17d84b041546f0f658bd011bc3ba?d=mm&s=32'}, 'displayName': 'IT Service Management', 'active': True, 'timeZone': 'America/New_York'}, 'created': '2022-11-23T01:34:11.000-0500', 'status': {'self': '', 'description': 'A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, certified, or are closed.', 'iconUrl': '', 'name': 'Resolved', 'id': '5', 'statusCategory': {'self': '', 'id': 3, 'key': 'done', 'colorName': 'green', 'name': 'Done'}}}}]}
You have to use the following JQL instead:
issuetype = "Service Request"
Please check the Jira search documentation. If you meant anything else, be more specific.

How get extract unique dictionary from list of dictionary with preference of value

I have dictionary below
test = [ { 'id': '195', 'Name': 'i', 'Email': 'chdtn#gmail.com', 'role': 'Product' },
{ 'id': '219', 'Name': 'umar', 'Email': 'ddhi#gmail.com', 'role': 'Product' },
{ 'id': '74', 'Name': 'Are', 'Email': 'ddhit#gmail.com', 'role': 'Tester' },
{ 'id': '24', 'Name': 'Mee', 'Email': 'huul#gmail.com', 'role': 'Tester' },
{ 'id': '230', 'Name': 'abc', 'Email': 'deyan#gmail.com', 'role': 'Tester' },
{ 'id': '220', 'Name': 'Sc', 'Email': 'deyan#gmail.com', 'role': 'Product' },
{ 'id': '230', 'Name': 'Sn', 'Email': 'deyan#gmail.com', 'role': 'Tester' } ]
I need to extract unique email from above list dict
I need to give give role preference Product then to Tester
My Code is below
dict([(d['Email'], d) for d in test]).values()
My Out:
dict_values([{'id': '195', 'Name': 'i', 'Email': 'chdtn#gmail.com', 'role': 'Product'},
{'id': '219', 'Name': 'umar', 'Email': 'ddhi#gmail.com', 'role': 'Product'},
{'id': '74', 'Name': 'Are', 'Email': 'ddhit#gmail.com', 'role': 'Tester'},
{'id': '24', 'Name': 'Mee', 'Email': 'huul#gmail.com', 'role': 'Tester'},
{'id': '230', 'Name': 'Sn', 'Email': 'deyan#gmail.com', 'role': 'Tester'}])
Here in my out
{'id': '230', 'Name': 'Sn', 'Email': 'deyan#gmail.com', 'role': 'Tester'}
has to replace with
{ 'id': '220', 'Name': 'Sc', 'Email': 'deyan#gmail.com', 'role': 'Product' }
because "Product" have higher preference.
How to update my code? dict([(d['Email'], d) for d in test]).values()
Here is in case you would like to insist on using dictionaries.
We go from one row to another. Check if the email is already in the new dictionary as key.
If not, we add this as a new one.
If so, we check our new row. If our new role is "product", we will delete what was already in the dictionary, and add the new row.
new_dict = {}
for row in test:
if row["Email"] not in new_dict.keys():
new_dict.update({row["Email"]: row})
else:
if row["role"]=="Product":
new_dict.pop(row["Email"])
new_dict.update({row["Email"]: row})
Perhaps you could try it with two loops; once to get the unique emails, and second time to make sure to prioritize "Product".
It wasn't clear what happens if there is no "Product" for duplicate "Emails", so in the loop below, the first email is selected in that case.
tmp = {}
for d in test:
tmp.setdefault(d['Email'], []).append(d)
out = []
for k, lst in tmp.items():
if len(lst) == 1:
out.append(lst[0])
else:
for d in lst:
if d['role'] == 'Product':
out.append(d)
break
else:
out.append(lst[0])
Output:
[{'id': '195', 'Name': 'i', 'Email': 'chdtn#gmail.com', 'Account': 'Product'},
{'id': '219', 'Name': 'umar', 'Email': 'ddhi#gmail.com', 'Account': 'Product'},
{'id': '74', 'Name': 'Are', 'Email': 'ddhit#gmail.com', 'role': 'Tester'},
{'id': '24', 'Name': 'Mee', 'Email': 'huul#gmail.com', 'role': 'Tester'},
{'id': '220', 'Name': 'Sc', 'Email': 'deyan#gmail.com', 'role': 'Product'}]
Make it to a data frame and drop_duplicates by Email after sorting the column role.
test = [ { 'id': '195', 'Name': 'i', 'Email': 'chdtn#gmail.com', 'role': 'Product' },
{ 'id': '219', 'Name': 'umar', 'Email': 'ddhi#gmail.com', 'role': 'Product' },
{ 'id': '74', 'Name': 'Are', 'Email': 'ddhit#gmail.com', 'role': 'Tester' },
{ 'id': '24', 'Name': 'Mee', 'Email': 'huul#gmail.com', 'role': 'Tester' },
{ 'id': '230', 'Name': 'abc', 'Email': 'deyan#gmail.com', 'role': 'Tester' },
{ 'id': '220', 'Name': 'Sc', 'Email': 'deyan#gmail.com', 'role': 'Product' },
{ 'id': '230', 'Name': 'Sn', 'Email': 'deyan#gmail.com', 'role': 'Tester' } ]
df = pd.DataFrame(test)
df1 = df.sort_values(by = ["Email", "role"], ascending = True)
res_df = df1.drop_duplicates(["Email"])
output_list = []
for i in res_df.values :
output_list.append(dict([("id", i[0]), ("Name", i[1]), ("Email", i[2]), ("role", i[3])]))
> output_list
[{'id': '195', 'Name': 'i', 'Email': 'chdtn#gmail.com', 'role': 'Product'},
{'id': '219', 'Name': 'umar', 'Email': 'ddhi#gmail.com', 'role': 'Product'},
{'id': '74', 'Name': 'Are', 'Email': 'ddhit#gmail.com', 'role': 'Tester'},
{'id': '220', 'Name': 'Sc', 'Email': 'deyan#gmail.com', 'role': 'Product'},
{'id': '24', 'Name': 'Mee', 'Email': 'huul#gmail.com', 'role': 'Tester'}]

How to Match two APIs to update one API dataset using Python

I want to be able to GET information from API 1 and match it with API 2 and be able to update API 2's information with API 1. I am trying to figure out the most efficient/automated way to accomplish this as it also needs to be updated at a interval of every 10 minutes
I can query and get the results from API 1 this is my code and what my code looks like.
import json
import requests
myToken = '52c32f6588004cb3ab33b0ff320b8e4f'
myUrl = 'https://api1.com/api/v1/devices.json'
head = {'Authorization': 'Token {}'.format(myToken)}
response = requests.get(myUrl, headers=head)
r = json.loads(response.content)
r
The payload looks like this from API 1
{ "device" : {
"id": 153,
"battery_status" : 61,
"serial_no": "5QBYGKUI05",
"location_lat": "-45.948917",
"location_lng": "29.832179",
"location_address": "800 Laurel Rd, Lansdale, PA 192522,USA"}
}
I want to be able to take this information and match by "serial_no" and update all the other pieces of information for the corresponding device in API 2
I query the data for API 2 and this is what my code looks like
params = {
"location":'cf6707e3-f0ae-4040-a184-737b21a4bbd1',
"dateAdded":'ge:11/23/2020'}
url = requests.get('https://api2.com/api/assets',auth=('api2', '123456'), params=params)
r = json.loads(url.content)
r['items']
The JSON payload looks like this
[{'id': '064ca857-3783-460e-a7a2-245e054dcbe3',
'name': 'Apple Laptop 1',
'model': {'id': '50f5993e-2abf-49c8-86e0-8743dd58db6f',
'name': 'MacBook Pro'},
'manufacturer': {'id': 'f56244e2-76e3-46da-97dd-f72f92ca0779',
'name': 'APPLE'},
'room': {'id': '700ff2dc-0118-46c6-936a-01f0fa88c620',
'name': 'Storage Room 1',
'thirdPartyId': ''},
'location': {'id': 'cf6707e3-f0ae-4040-a184-737b21a4bbd1',
'name': 'Iron Mountain',
'thirdPartyId': ''},
'position': 'NonMounted',
'containerAsset': {'id': '00000000-0000-0000-0000-000000000000',
'name': None},
'baseAsset': {'id': '064ca857-3783-460e-a7a2-245e054dcbe3',
'name': 'Apple Laptop 1'},
'description': None,
'status': {'id': 'df9906d8-2856-45e3-9cba-bd7a1ac4971f',
'name': 'Production'},
'serialNumber': '5QBYGKUI06',
'tagNumber': None,
'alternateTagNumber': None,
'verificationStatus': {'id': 'cb3560a9-eef5-47b9-b033-394d3a09db18',
'name': 'Verified'},
'requiresRFID': False,
'requiresHangTag': False,
'bottomPosition': 0.0,
'leftPosition': 0.0,
'rackPosition': 'Front',
'labelX': None,
'labelY': None,
'verifyNameInRear': False,
'verifySerialNumberInRear': False,
'verifyBarcodeInRear': False,
'isNonDataCenter': False,
'rotate': False,
'customer': {'id': '00000000-0000-0000-0000-000000000000', 'name': None},
'thirdPartyId': '',
'temperature': None,
'dateLastScanned': None,
'placement': 'Floor',
'lastScannedLabelX': None,
'lastScannedLabelY': None,
'userDefinedValues': [{'userDefinedKeyId': '79e77a1e-4030-4308-a8ff-9caf40c04fbd',
'userDefinedKeyName': 'Longitude ',
'value': '-75.208917'},
{'userDefinedKeyId': '72c8056e-9b7d-40ac-9270-9f5929097e82',
'userDefinedKeyName': 'Address',
'value': '800 Laurel Rd, New York ,NY 19050, USA'},
{'userDefinedKeyId': '31aeeb91-daef-4364-8dd6-b0e3436d6a51',
'userDefinedKeyName': 'Battery Level',
'value': '67'},
{'userDefinedKeyId': '22b7ce4f-7d3d-4282-9ecb-e8ec2238acf2',
'userDefinedKeyName': 'Latitude',
'value': '35.932179'}]}
The documentation provided by API 2 tells me they only support PUT for updates as of right now but I would also want to know how I would do this using PATCH as it will be available in the future. So the data payload that I need to successful PUT is this
payload = {'id': '064ca857-3783-460e-a7a2-245e054dcbe3',
'name': 'Apple Laptop 1',
'model': {'id': '50f5993e-2abf-49c8-86e0-8743dd58db6f',
'name': 'MacBook Pro'},
'manufacturer': {'id': 'f56244e2-76e3-46da-97dd-f72f92ca0779',
'name': 'APPLE'},
'room': {'id': '700ff2dc-0118-46c6-936a-01f0fa88c620',
'name': 'Storage Room 1',
'thirdPartyId': ''},
'status': {'id': 'df9906d8-2856-45e3-9cba-bd7a1ac4971f',
'name': 'Production'},
'serialNumber': '5QBYGKUI06',
'verificationStatus': {'id': 'cb3560a9-eef5-47b9-b033-394d3a09db18',
'name': 'Verified'},
'requiresRFID': 'False',
'requiresHangTag': 'False',
'userDefinedValues': [{'userDefinedKeyId': '79e77a1e-4030-4308-a8ff-9caf40c04fbd',
'userDefinedKeyName': 'Longitude ',
'value': '-75.248920'},
{'userDefinedKeyId': '72c8056e-9b7d-40ac-9270-9f5929097e82',
'userDefinedKeyName': 'Address',
'value': '801 Laurel Rd, New York, Ny 192250, USA'},
{'userDefinedKeyId': '31aeeb91-daef-4364-8dd6-b0e3436d6a51',
'userDefinedKeyName': 'Battery Level',
'value': '67'},
{'userDefinedKeyId': '22b7ce4f-7d3d-4282-9ecb-e8ec2238acf2',
'userDefinedKeyName': 'Latitude',
'value': '29.782177'}]}
So apart of this is figuring out how I can query the json data portions that I need for the update
I am able to update the information using this line
requests.put('https://api2.com/api/assets/064ca857-3783-460e-a7a2-245e054dcbe3',auth=('API2', '123456'), data=json.dumps(payload))
but I need for it to dynamically update so I don't think the hard coded id parameter in the line will be efficient in a automation/efficiency standpoint. If anybody has any ideas, resources to point me in the right direction to know more about this process (I don't really know what it is even called) would be greatly appreciated.
Not entirely sure what you are trying to do here, but if you want to pull information nested in the responses you can do this.
Serial number from API 1
r['device']['serial_no']
Serial number for API 2
either r[0]['serialNumber'] or r['items'][0]['serialNumber'] depending on what you are showing
To modify the payload serial number, for example
payload['serialNumber'] = '123456abcdef'

How to display azure resources in tabular form python SDK

I created a ResourceManagementClient using the azure python sdk:
resource_client = ResourceManagementClient(service_credential, subscription_id)
The output produced via...
for item in resource_client.resource_groups.list():
print(item)
...is difficult to read and its not in valid json format.
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/azureMaps', 'name': 'azureMaps', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499748>, 'location': 'westus', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/speechToText', 'name': 'speechToText', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba4997f0>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/neo4j-rg', 'name': 'neo4j-rg', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499b00>, 'location': 'westus', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/things', 'name': 'things', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499a90>, 'location': 'westus', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/appInsights', 'name': 'appInsights', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499a20>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/computerVision', 'name': 'computerVision', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499c50>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/dataBricks', 'name': 'dataBricks', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499be0>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/databricks-rg-databricksDEV-v6oygdyuh3sii', 'name': 'databricks-rg-databricksDEV-v6oygdyuh3sii', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499d30>, 'location': 'westus2', 'managed_by': '/subscriptions/mySub/resourceGroups/dataBricks/providers/Microsoft.Databricks/workspaces/databricksDEV', 'tags': {}}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/cool', 'name': 'cool', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499cf8>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/wsFunctionApp', 'name': 'wsFunctionApp', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499e48>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/azureStorage', 'name': 'azureStorage', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499dd8>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/azureSQL', 'name': 'azureSQL', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499d68>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/azureFunction', 'name': 'azureFunction', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499f28>, 'location': 'westus2', 'managed_by': None, 'tags': None}
{'additional_properties': {}, 'id': '/subscriptions/mySub/resourceGroups/NetworkWatcherRG', 'name': 'NetworkWatcherRG', 'properties': <azure.mgmt.resource.resources.v2018_05_01.models.resource_group_properties_py3.ResourceGroupProperties object at 0x7f0fba499f98>, 'location': 'westus2', 'managed_by': None, 'tags': None}
Can someone help me pretty print this output into something like a display() table?
Not sure why you want this, you can print all the properties you want of the item directly.
Something like:
print(item.id)
print(item.location)
print(item.tags)
Or if you want a valid json, you could custom the properties you want like below, the output is a valid json:
import json
result = []
for item in resource_client.resource_groups.list():
additional_properties = item.additional_properties
id = item.id
location = item.location
tags = item.tags
type = item.type
row = {"additional_properties": additional_properties, "id": id, "location": location, "tags": tags, "type": type}
result.append(row)
data = json.dumps(result)
print(data)
If you want to make the output easy to read, you could use the JSON Formatter Tool.

From MongoDB convert from dictionary to row with Pandas

This is a test coming from MongoDB, I need to convert to MySQL. But! Sometimes there is more then one "agents", if that's the case I need each agent on their own row and that agent should have the same "display_name". For example Walter should have Gloria on one row and Barb on next and both have Walt Mosley under "display_name".
[{'name': 'Loomis, Gloria',
'primaryemail': 'gloria#gmail.com',
'primaryphone': '212-382-1121'},
{'name': 'Hogson, Barb',
'primaryemail': 'bho124#aol.com',
'primaryphone': ''}]
I've tried this but it just splits out the key/values.
a,b,c = [[d[e] for d in test] for e in sorted(test[0].keys())]
print(a,b,c)
This is the original JSON format:
{'_id': ObjectId('58e6ececafb08d6'),
'item_type': 'Contributor',
'role': 0,
'short_bio': 'Walter Mosley (b. 1952)',
'firebrand_id': 1588,
'display_name': 'Walter Mosley',
'first_name': 'Walter',
'last_name': 'Mosley',
'slug': 'walter-mosley',
'updated': datetime.datetime(2020, 1, 7, 8, 17, 11, 926000),
'image': 'https://s3.amazonaws.com/8588-book-contributor.jpg',
'social_media_name': '',
'social_media_link': '',
'website': '',
'agents': [{'name': 'Loomis, Gloria',
'primaryemail': 'gloria#gmail.com',
'primaryphone': '212-382-1121'},
{'name': 'Hogson, Barb',
'primaryemail': 'bho124#aol.com',
'primaryphone': ''}],
'estates': [],
'deleted': False}
If you've an array of dictionaries from your JSON file, try this :
JSON input :
inputJSON = [{'item_type': 'Contributor',
'role': 0,
'short_bio': 'Walter Mosley (b. 1952)',
'firebrand_id': 1588,
'display_name': 'Walter Mosley',
'first_name': 'Walter',
'last_name': 'Mosley',
'slug': 'walter-mosley',
'image': 'https://s3.amazonaws.com/8588-book-contributor.jpg',
'social_media_name': '',
'social_media_link': '',
'website': '',
'agents': [{'name': 'Loomis, Gloria',
'primaryemail': 'gloria#gmail.com',
'primaryphone': '212-382-1121'},
{'name': 'Hogson, Barb',
'primaryemail': 'bho124#aol.com',
'primaryphone': ''}],
'estates': [],
'deleted': False}]
Code :
import copy
finalJSON = []
for each in inputJSON:
for agnt in each.get('agents'):
newObj = copy.deepcopy(each)
newObj['agents'] = agnt
finalJSON.append(newObj)
print(finalJSON)

Categories

Resources