I have a json file with set of data with repeating fields. I need to remove an entry with specific data of a field.
Json file:
[
{
"targets": [
"172.17.1.199"
],
"labels": {
"__meta_netbox_pop": "st-1742",
"__snmp_module__": "arista_sw"
}
},
{
"targets": [
"172.17.1.51"
],
"labels": {
"__meta_netbox_pop": "st-1754",
"__snmp_module__": "arista_sw"
}
}
]
The json file goes on and but this is an example of the whole json file.
I need to remove an entry of targets with its labels given a data of the target's IP.
Input:
172.17.1.51
expected output:
[
{
"targets": [
"172.17.1.199"
],
"labels": {
"__meta_netbox_pop": "st-1742",
"__snmp_module__": "arista_sw"
}
}
]
Using jq:
$ jq --arg ip 172.17.1.51 'map(select(.targets | contains([$ip]) | not ))' input.json
[
{
"targets": [
"172.17.1.199"
],
"labels": {
"__meta_netbox_pop": "st-1742",
"__snmp_module__": "arista_sw"
}
}
]
If I understand the question correctly, this should do the trick:
The target parameter is the IP address you want to remove, while the file_path is the path of the json file
edit: also, don't forget to import json or else it won't work
def remove_obj(target, file_path):
with open(file_path, "r") as data_file:
data = json.load(data_file)
for obj in data:
if target in obj["targets"]:
data.remove(obj)
with open(file_path, "w") as data_file:
json.dump(data, data_file, indent=4)
Related
I am having json file which around 600MB. The structure of the json file is,
[
{
"metadata":{
"batchSize":100,
"totalRecords":"1000",
"batchIndex":1
},
"notificationData":[
{
"brandId":"A",
"sourceUniqueId":"12345",
"contentType":"PDF",
"transactionId":"cABCD",
"batchId":"ABC_1",
}
]
},
{
"metadata":{
"batchSize":100,
"totalRecords":"1000",
"batchIndex":2
},
"notificationData":[
{
"brandId":"B",
"sourceUniqueId":"789",
"contentType":"PDF1",
"transactionId":"XYZ",
"batchId":"XYZ_1",
}
]
}
]
Now, Output I want the notificationData array of provided batchIndex. I am able to find specific batchIndex But how to get notificationData of that batchIndex. BatchIndex is unique. And after getting notificationData stops the execution. No need to parse complete file.
f = open('batchIndex.json')
data = ijson.items(f,'item.metadata')
jsons = (o for o in data if o['batchIndex'] == input_batchIndex)
for j in jsons:
print(j)
I am trying to get a specific key vulnerabilityID and vulnerabilitySoln from this json file. This is towards the bottom of the json script.
{
"vulnerabilities": [
{
"labels": [
{
"labelKey": "com.acme.image.source",
"labelValue": "http://github.com/base-image.git"
},
{
"labelKey": "BA",
"labelValue": "abc"
},
{
"labelKey": "OwnerContact",
"labelValue": "john.doe#gmail.com"
}
],
"repositoryDigest": [
{
"digest": "9089",
"repository": "frost/micro",
"registry": "artifactory.acme.com"
}
],
"artifactorySha": "4324",
"imageID": "er34342",
"createdTimestamp": "2022-10-17",
"engineeringEID": "afd343",
"vulnerabilities": [
{
"test1": "12444",
"test2": "433w",
"vulnerabilityID": "12345",
"vulnerabilitySoln": "http://website.com"
}
]
}
]
}
Here is the code that i attempted
import json
import smtplib
with open("dummy.json") as json_file:
data = json.load(json_file)
def vulnData():
for i in data["vulnerabilities"]:
vulnID = (i["vulnerabilities"]["vulnerabilityID"])
vulnSoln = (i["vulnerabilities"]["vulnerabilitySoln"])
print(vulnID, vulnSoln)
vulnData()
This is the error output I am getting -- TypeError: list indices must be integers or slices, not str. Please suggest any ideas how i can get specific value.
vulnID = (i["vulnerabilities"]["vulnerabilityID"])
TypeError: list indices must be integers or slices, not str
There are also two levels of lists in there:
def vulnData():
for i in data["vulnerabilities"]:
vuln = i["vulnerabilities"][0]
vulnID = vuln["vulnerabilityID"]
vulnSoln = vuln["vulnerabilitySoln"])
print(vulnID, vulnSoln)
I've been trying to figure out a way to store proxy data in a json form, i know the easier way is to just take each proxy from the text box and save it to the file and then to access it i would just load the information from the file but i want to have groups that work with different types of IP's. Say for example one group uses the proxy IP from a certain provider and another group would use an IP from a different one, i would need to store the IP's in their respected groups which is why i think i need to create a json file to store each of the proxies in their own json array. What i'm having trouble with is adding the IP's to the json array as i am trying to loop over a transfer file with the IP's in them and then add it to the json array. As of now i tried this,
def save_proxy():
proxy = pooled_data_default.get('1.0', 'end-2c')
transfer_file = open('proxies.txt', 'w')
transfer_file.write(proxy)
transfer_file.close()
transfer_file1 = open('proxies.txt', 'r')
try:
with open('proxy_groups.txt', 'r+') as file:
proxy_group = json.load(file)
except:
proxy_group = []
total = []
for line in transfer_file1:
line = transfer_file1.readline().strip()
total.append(line)
proxy_group.append({
'group_name': pool_info.get(),
'proxy': [{
'proxy': total,
}]
}),
with open('proxy_groups.txt', 'w') as outfile:
json.dump(proxy_group, outfile, indent=4)
This doesn't work but it was my attempt at taking each line from the file and adding it to the json array dynamically. Any help is appreciated.
EDIT: this is what is being outputted:
[
{
"group_name": "Defualt",
"proxy": [
{
"proxy": [
"asdf",
""
]
}
]
}
]
This was the input
wdsa
asdf
sfs
It seems that it is only selecting the middle one of the 3. I thought that printing the list of them would work but it is still printing the middle and then a blank space at the end.
An example of my data is the input to the text box may be
wkenwwins:1000:username:password
uwhsuh:1000:username:password
2ewswsd:1000:username:password
gfrfccv:1000:username:password
the selected group which i may want to save this to could be called 'Default'. I select default and then clicking save should add these inputs to the seperate txt sheet called 'proxies.txt', which it does. From the text sheet i then want to loop through each line and append each line to the json data. Which it doesnt do, here it was i expect it to look like in json data
[
{
"group_name": "Defualt",
"proxy": [
{
"proxy": [
'ewswsd:1000:username:password',
'wkenwwins:1000:username:password',
'uwhsuh:1000:username:password'
]
}
]
}
]
So then say if i made 2 groups the json data txt file should look like this:
[
{
"group_name": "Defualt",
"proxy": [
{
"proxy": [
'ewswsd:1000:username:password',
'wkenwwins:1000:username:password',
'uwhsuh:1000:username:password'
]
}
]
}
]
[
{
"group_name": "Test",
"proxy": [
{
"proxy": [
'ewswsd:1000:username:password',
'wkenwwins:1000:username:password',
'uwhsuh:1000:username:password'
]
}
]
}
]
This is so i can access each group by only calling the group name.
You can simplify the save_proxy() as below:
def save_proxy():
proxy = pooled_data_default.get('1.0', 'end-1c')
# save the proxies to file
with open('proxies.txt', 'w') as transfer_file:
transfer_file.write(proxy)
# load the proxy_groups if exists
try:
with open('proxy_groups.txt', 'r') as file:
proxy_group = json.load(file)
except:
proxy_group = []
proxy_group.append({
'group_name': pool_info.get(),
'proxy': proxy.splitlines()
})
with open('proxy_groups.txt', 'w') as outfile:
json.dump(proxy_group, outfile, indent=4)
The output file proxy_groups.txt would look like below:
[
{
"group_name": "default",
"proxy": [
"wkenwwins:1000:username:password",
"uwhsuh:1000:username:password"
]
}
]
I'm trying to parse more than 100 json files, but i do not need all the info.
i only need to parse the first set of the 'coordinates', the CSV already have printed URL and URL type, but i cannot print the first set of coordinates.
this is a section of the Json file
{
"type":"featureCollection",
"features" : [
{
"type": "feature",
"geometry": {
"type": "multilinestring",
"coordinates":[
[
[
148.9395348,
-21.3292286
],
[
148.93963,
-21.33001
],
[
148.93969,
-21.3303
]
]
]
},
"properties":{
"url" :"www.thiswebpageisfake.com",
"url_type":"fake"
},
"information":{
"timestamp":"10/10/19"
}
}]
}
i'm using python 2.7, i have tried creating an array for coordinates but i have a type error
import os
import csv
import json
import sys
reload(sys)
file_path = 'C:\\Users\\user\\Desktop\\Python\\json'
dirs = os.listdir(file_path)
file_out = 'C:\\Users\\user\\output.csv'
f = csv.writer(open(file_out, "wb+"))
f.writerow(
['url','url_type','lat','long'])
for file in dirs:
json_dict = json.loads(open(os.path.join(file_path, file)).read())
print file
for key in json_dict['features']:
for key1 in key:
description = key['properties']['description']
if description is None:
description = 'null'
array = ()
array = (key['geometry']['type']['coordinates'])
f.writerow([file,
key['properties']['url'],
key['properties']['url_type'],
array[1]
])
print 'completed'
Firstly, it looks like your second loop is supposed to be nested in the first, otherwise you'll do nothing with all json files except the last and only end up processing one file.
Secondly, your array should be defined as array = (key['geometry']['coordinates']), as 'coordinates' is not contained in 'type'.
json file
[{"Attachment": [
{
"Page:": [
{
"Path": "a\\b\\c.pdf", #field to be extracted
"PageID": 1
}
]
}
],
"ID": 13221}]
I tried the following but getting the TypeError: list indices must be integers, not str
with open(file) as f:
d = json.load(f)
print(d[0]['Attachment']['Page']['Path'])
d[0]['Attachment'] is a list, so is d[0]['Attachment'][0]['Page:'].
with open(file) as f:
d = json.load(f)
print(d[0]['Attachment'][0]['Page:'][0]['Path'])
will do the job.