Develop Reference

Convert pandas data frame to JSON with strings separated

I have a pandas.dataframe named 'df' with the following format:
group_name
Positive_Sentiment
Negative_Sentiment
group1
helpful, great support
slow customer service, weak interface, bad management
I would like to convert this dataframe to a JSON file with the following format:
[{
"Group Name": "group1",
"Postive Sentiment": [
"helpful",
"great support"
],
"Negative Sentiment": [
"slow customer service",
"weak interface",
"bad management"
]
}
]
So far I have used this:
import json
b = []
for i in range(len(df)):
x={}
x['Group Name']=df.iloc[i]['group_name']
x['Positive Sentiment']= [df.iloc[i]['Positive_Sentiment']]
x['Negative Sentiment']= [df.iloc[i]['Negative_Sentiment']]
b.append(x)
##Export
with open('AnalysisResults.json', 'w') as f:
json.dump(b, f, indent = 2)
This results in:
[{
"Group Name": "group1",
"Postive Sentiment": [
"helpful,
great support"
],
"Negative Sentiment": [
"slow customer service,
weak interface,
bad UX"
]
}
]
You can see it is quite close. The crucial difference is the double-quotes around the ENTIRE contents of each row (e.g., "helpful, great support") instead of each comma-separated string in the row (e.g., "helpful", "great support"). I would like double-quotes around each string.

You can apply split(",") to your columns:
from io import StringIO
import pandas as pd
import json
inp = StringIO("""group_name Positive_Sentiment Negative_Sentiment
group1 helpful, great support slow customer service, weak interface, bad management
group2 great, good support interface meeeh, bad management""")
df = pd.read_csv(inp, sep="\s{2,}")
def split_and_strip(sentiment):
[x.strip() for x in sentiment.split(",")]
df["Positive_Sentiment"] = df["Positive_Sentiment"].apply(split_and_strip)
df["Negative_Sentiment"] = df["Negative_Sentiment"].apply(split_and_strip)
print(json.dumps(df.to_dict(orient="record"), indent=4))
# to save directly to a file:
with open("your_file.json", "w+") as f:
json.dump(df.to_dict(orient="record"), f, indent=4)
Output:
[
{
"group_name": "group1",
"Positive_Sentiment": [
"helpful",
"great support"
],
"Negative_Sentiment": [
"slow customer service",
"weak interface",
"bad management"
]
},
{
"group_name": "group2",
"Positive_Sentiment": [
"great",
"good support"
],
"Negative_Sentiment": [
"interface meeeh",
"bad management"
]
}
]

Remove specific data from json file in linux

I have a json file with set of data with repeating fields. I need to remove an entry with specific data of a field.
Json file:
[
{
"targets": [
"172.17.1.199"
],
"labels": {
"__meta_netbox_pop": "st-1742",
"__snmp_module__": "arista_sw"
}
},
{
"targets": [
"172.17.1.51"
],
"labels": {
"__meta_netbox_pop": "st-1754",
"__snmp_module__": "arista_sw"
}
}
]
The json file goes on and but this is an example of the whole json file.
I need to remove an entry of targets with its labels given a data of the target's IP.
Input:
172.17.1.51
expected output:
[
{
"targets": [
"172.17.1.199"
],
"labels": {
"__meta_netbox_pop": "st-1742",
"__snmp_module__": "arista_sw"
}
}
]

Using jq:
$ jq --arg ip 172.17.1.51 'map(select(.targets | contains([$ip]) | not ))' input.json
[
{
"targets": [
"172.17.1.199"
],
"labels": {
"__meta_netbox_pop": "st-1742",
"__snmp_module__": "arista_sw"
}
}
]

If I understand the question correctly, this should do the trick:
The target parameter is the IP address you want to remove, while the file_path is the path of the json file
edit: also, don't forget to import json or else it won't work
def remove_obj(target, file_path):
with open(file_path, "r") as data_file:
data = json.load(data_file)
for obj in data:
if target in obj["targets"]:
data.remove(obj)
with open(file_path, "w") as data_file:
json.dump(data, data_file, indent=4)

Filter JSON data into a CSV list

My goal is to take the host names out of this JSON file (output.json) and put them into a CSV list (newoutput.csv) using python, with the end result looking like:
TheHost1, TheHost2, TheHost3
There are a couple hundred entrys, with the hostnames under "specific_data.data.hostname"
Here is a snippet of what the output.json file looks like:
[
{
"adapter_list_length": 3,
"adapters": [
...
],
"internal_id": "...",
"labels": [
"...",
"..."
],
"specific_data.data.hostname": [
"TheHost1"
],
"specific_data.data.last_seen": "...",
"specific_data.data.network_interfaces.ips": [
"...",
"...",
"..."
],
"specific_data.data.network_interfaces.mac": [
"..."
],
"specific_data.data.os.type": [
"..."
]
},
{
"adapter_list_length": 3,
"adapters": [
"...",
"....",
"...",
"..."
],
"internal_id": "...",
"labels": [
"...",
"Router"
],
"specific_data.data.hostname": [
"TheHost2"
],
"specific_data.data.last_seen": "...",
"specific_data.data.network_interfaces.ips": [
I'm new to python, and any help would really be appreciated.

You can iterate the list of dictionaries and extend a "hosts" list with the list of hosts found in each record. When you have all of the values, build the comma separated string.
import json
# python 2.7 needs different open
import codecs
filename = "output.json"
# json is usually utf-8 encoded but this is not 100% guaranteed
data = json.loads(codecs.open(filename, encoding="utf-8"))
hosts = []
for record in data:
if "specific_data.data.hostname" in record:
hosts.extend(record["specific_data.data.hostname"])
# assuming hosts are all ascii
with open("hostnames.csv", "w") as fileobj:
fileobj.write(",".join(hosts))
fileobj.write("\n")

The following sets the csv variable to a string with comma-separated hostnames.
import json
object = json.loads(snippet)
csv = ', '.join(x['specific_data.data.hostname'][0] for x in object)

Replacing words in JSON file using Python

smpl.json file:
[
{
"add":"dtlz",
"emp_details":[
[
"Shubham",
"ksing.shubh#gmail.com",
"intern"
],
[
"Gaurav",
"gaurav.singh#cobol.in",
"developer"
],
[
"Nikhil",
"nikhil#geeksforgeeks.org",
"Full Time"
]
]
}
]
Python file:
import json
with open('smpl.json', 'r') as file:
json_data = json.load(file)
for item in json_data["emp_details"]:
if item[''] in ['Shubham']:
item[''] = 'Indra'
with open('zz_smpl.json', 'w') as file:
json.dump(json_data, file, indent=4)
Since I'm having trouble with the code. Any help would be great.
Looking forward for your help.Thanks in advance!!!

1st, you need to understand list/arrays and maps data structures, and how they are represented by JSON. Seriously, you must understand those data structures in order to use JSON.
An empty array a1
a1 = []
Array with 3 integers
a2 = [1, 2, 3]
To address the 2nd value
a2[0] is 1st value
a2[1] is 2nd value
In python, to subset a2 into 2nd and 3rd value
a3 = a2[1:]
Maps/dicts are containers of key:value pairs.
And empty map (called a dict in python)
d1 = {}
Maps with 2 pairs
d2 = { 'name' : 'Chandra Gupta Maurya' , 'age' : 2360 }
d3 = { 'street' : 'ashoka' , 'location' : 'windsor place' , 'city' : 'delhi' }
such that value of
d2['name'] is 'Chandra Gupta Maurya'
An array of two maps. When you do this in python (and javaScript)
ad1 = [ d2, d3 ]
you are equivalently doing this:
ad1 = [
{ 'name' : 'Chandra Gupta Maurya' , 'age' : 2360 } ,
{ 'street' : 'ashoka' , 'location' : 'windsor place' , 'city' : 'delhi' }
]
so that ad1[0] is
{ 'name' : 'Chandra Gupta Maurya' , 'age' : 2360 }
Obviously "emp_details" is in position 0 of an array
json_data[0]['emp_details']
json_data[0]['emp_details'] itself is the key to an array of maps.
>>> json.dumps (json_data[0]["emp_details"] , indent=2)
produces
'[\n [\n "Shubham",\n "ksing.shubh#gmail.com",\n "intern"\n ],\n [\n "Gaurav",\n "gaurav.singh#cobol.in",\n "developer"\n ],\n [\n "Nikhil",\n "nikhil#geeksforgeeks.org",\n "Full Time"\n ]\n]'
and
>>> print ( json.dumps (json_data[0]["emp_details"], indent=2) )
produces
[
[
"Shubham",
"ksing.shubh#gmail.com",
"intern"
],
[
"Gaurav",
"gaurav.singh#cobol.in",
"developer"
],
[
"Nikhil",
"nikhil#geeksforgeeks.org",
"Full Time"
]
]
Therefore,
>>> json_data[0]["emp_details"][1]
['Gaurav', 'gaurav.singh#cobol.in', 'developer']
Then you might wish to do the replacement
>>> json_data[0]["emp_details"][1][2] = 'the rain in maine falls plainly insane'
>>> json_data[0]["emp_details"][1][1] = "I'm sure the lure in jaipur pours with furore"
>>> print ( json.dumps (json_data, indent=2) )
produces
[
{
"add": "dtlz",
"emp_details": [
[
"Shubham",
"ksing.shubh#gmail.com",
"intern"
],
[
"Gaurav",
"I'm sure the lure in jaipur pours with furore",
"the rain in maine falls plainly insane"
],
[
"Nikhil",
"nikhil#geeksforgeeks.org",
"Full Time"
]
]
}
]

There are 2 problems with your code.
First, the JSON contains an array as the root. Therefore you need to get emp_details property of the first item:
for item in json_data[0]["emp_details"]:
Then in item variable, you need to check the item at index zero:
if item[0] in ['Shubham']:
Here is the full working code:
import json
with open('smpl.json', 'r') as file:
json_data = json.load(file)
for item in json_data[0]["emp_details"]:
if item[0] in ['Shubham']:
item[0] = 'Indra'
with open('zz_smpl.json', 'w') as file:
json.dump(json_data, file, indent=4)
The working repl.it link: https://repl.it/#HarunYlmaz/python-json-write

Here's a more generic solution where outermost json array could have multiple entries (dictionaries):
import json
with open('test.json', 'r') as file:
json_data = json.load(file)
for item in json_data:
for emp in item['emp_details']:
if emp[0] in ['Shubham']:
emp[0] = 'Indra'
with open('zz_smpl.json', 'w') as file:
json.dump(json_data, file, indent=4)

How can i read through each line of a file and append it to a json file in python?

I've been trying to figure out a way to store proxy data in a json form, i know the easier way is to just take each proxy from the text box and save it to the file and then to access it i would just load the information from the file but i want to have groups that work with different types of IP's. Say for example one group uses the proxy IP from a certain provider and another group would use an IP from a different one, i would need to store the IP's in their respected groups which is why i think i need to create a json file to store each of the proxies in their own json array. What i'm having trouble with is adding the IP's to the json array as i am trying to loop over a transfer file with the IP's in them and then add it to the json array. As of now i tried this,
def save_proxy():
proxy = pooled_data_default.get('1.0', 'end-2c')
transfer_file = open('proxies.txt', 'w')
transfer_file.write(proxy)
transfer_file.close()
transfer_file1 = open('proxies.txt', 'r')
try:
with open('proxy_groups.txt', 'r+') as file:
proxy_group = json.load(file)
except:
proxy_group = []
total = []
for line in transfer_file1:
line = transfer_file1.readline().strip()
total.append(line)
proxy_group.append({
'group_name': pool_info.get(),
'proxy': [{
'proxy': total,
}]
}),
with open('proxy_groups.txt', 'w') as outfile:
json.dump(proxy_group, outfile, indent=4)
This doesn't work but it was my attempt at taking each line from the file and adding it to the json array dynamically. Any help is appreciated.
EDIT: this is what is being outputted:
[
{
"group_name": "Defualt",
"proxy": [
{
"proxy": [
"asdf",
""
]
}
]
}
]
This was the input
wdsa
asdf
sfs
It seems that it is only selecting the middle one of the 3. I thought that printing the list of them would work but it is still printing the middle and then a blank space at the end.
An example of my data is the input to the text box may be
wkenwwins:1000:username:password
uwhsuh:1000:username:password
2ewswsd:1000:username:password
gfrfccv:1000:username:password
the selected group which i may want to save this to could be called 'Default'. I select default and then clicking save should add these inputs to the seperate txt sheet called 'proxies.txt', which it does. From the text sheet i then want to loop through each line and append each line to the json data. Which it doesnt do, here it was i expect it to look like in json data
[
{
"group_name": "Defualt",
"proxy": [
{
"proxy": [
'ewswsd:1000:username:password',
'wkenwwins:1000:username:password',
'uwhsuh:1000:username:password'
]
}
]
}
]
So then say if i made 2 groups the json data txt file should look like this:
[
{
"group_name": "Defualt",
"proxy": [
{
"proxy": [
'ewswsd:1000:username:password',
'wkenwwins:1000:username:password',
'uwhsuh:1000:username:password'
]
}
]
}
]
[
{
"group_name": "Test",
"proxy": [
{
"proxy": [
'ewswsd:1000:username:password',
'wkenwwins:1000:username:password',
'uwhsuh:1000:username:password'
]
}
]
}
]
This is so i can access each group by only calling the group name.

You can simplify the save_proxy() as below:
def save_proxy():
proxy = pooled_data_default.get('1.0', 'end-1c')
# save the proxies to file
with open('proxies.txt', 'w') as transfer_file:
transfer_file.write(proxy)
# load the proxy_groups if exists
try:
with open('proxy_groups.txt', 'r') as file:
proxy_group = json.load(file)
except:
proxy_group = []
proxy_group.append({
'group_name': pool_info.get(),
'proxy': proxy.splitlines()
})
with open('proxy_groups.txt', 'w') as outfile:
json.dump(proxy_group, outfile, indent=4)
The output file proxy_groups.txt would look like below:
[
{
"group_name": "default",
"proxy": [
"wkenwwins:1000:username:password",
"uwhsuh:1000:username:password"
]
}
]