Develop Reference

Python lambda expression - string indices must be integers

I am trying to use data from one json file to update another. In trying to be efficient for searching, I'm attempting to use a lambda expression to locate the correct record to update.
The goal is to be able to most efficiently update the "PreviousMappings" key.
Code:
for p in Path('jobs/data').glob('*.json'):
with open(p, 'r') as f:
print('Loaded - ', f.name)
jdata = json.load(f)
for j in jdata['Mappings']:
jc = j['Job Code']
with open('newdata/jobs.json', 'r+') as s:
print('Loaded - ', s.name)
data = json.load(s)
found = list(filter(lambda x:x['Jobcode'] == jc, data)) #throws exception
JSON:
{
"JobCodes": [
{
"Bid Code": "123-456",
"Description": "JOB DESCRIPTION",
"Jobcode": "ABC123",
"PreviousMappings": ""
}
]
}

This does what you're asking, but you might consider a different approach.
data = json.load( open('newdata/jobs.json', 'r') )
for p in Path('jobs/data').glob('*.json'):
with open(p, 'r') as f:
print('Loaded - ', f.name)
jdata = json.load(f)
for j in jdata['Mappings']:
jc = j['Job Code']
for k in data:
if k['Jobcode'] == jc:
k['PreviousMappings'] = "something"
break
json.dump( open('newdata/jobs.json','w'), data )
If you have a LOT of files, you might consider building an index, so you can do a direct lookup. For example (untested):
data = json.load( open('newdata/jobs.json', 'r') )
dindex = {}
for k in data:
dindex[k['Jobcode']] = k
Now you don't have to search -- Python will do it:
for j in jdata['Mappings']:
jc = j['Job Code']
if jc in dindex:
dindex[jc]['PreviousMappings'] = "something"

Handle index position in Python script to delete json objects from json file - Resolved

I have a file (my_file.json) has contents as below;
[
{
"use":"abcd",
"contact":"xyz",
"name":"my_script.py",
"time":"11:22:33"
},
{
"use":"abcd"
"contact":"xyz",
"name":"some_other_script.py",
"time":"11:22:33"
},
{
"use":"apqwkndf",
"contact":"xyz",
"name":"my_script.py",
"time":"11:22:33"
},
{
"use":"kjdshfjkasd",
"contact":"xyz",
"name":"my_script.py",
"time":"11:22:33"
}
]
I used following python code to delete the objects that has "name":"my_script.py",
#!/bin/usr/python
impoty json
obj = json.load(open("my_file.json"))
index_list = []
for i in xrange(len(obj)):
if obj[i]["name"] == ["my_script.py"]
index_list.append(i)
for x in range(len(index_list)):
obj.pop(index_list[x])
open("output_my_file.json","w".write(json.dumps(obj, indent=4, separators=(',',': ')))
but it seems I am stuck, because after popping an index the index position in actual obj gets changed, which leads to wrong index deletion or sometimes pop index gets out of range. Any other solution?

Try popping in reverse order:
for x in reversed(range(len(index_list))):

This will create a new list and assign only those without "name": "my_script.py" to the new list.
obj = [i for i in obj if i["name"] != "my_script.py"]

import json
with open('my_file.json') as f:
data = json.load(f)
data = [item for item in data if item.get('name') != 'my_script.py']
with open('output_my_file.json', 'w') as f:
json.dump(data, f, indent=4)

Try:
import json
json_file = json.load(open("file.json"))
for json_dict in json_file:
json_dict.pop("name",None)
print(json.dumps(json_file, indent=4))
You don't need the last line where it says 'json.dumps' I just have it there so it looks more readable when printed.

As a general rule of thumb, you usually don't want to ever change an iterable while iterating over it.
I suggest you save the elements you do want in the first loop:
import json
with open('path/to/file', 'r') as f:
data = json.load(f)
items_to_keep = []
for item in data:
if item['name'] != 'my_script.py':
items_to_keep.append(item)
with open('path/to/file', 'w') as f:
json.dump(items_to_keep, f, ...)
The filtering can be reduced into a single line (called list-comprehension)
import json
with open('path/to/file', 'r') as f:
data = json.load(f)
items_to_keep = [item for item in data if item['name'] != 'my_script.py']
with open('path/to/file', 'w') as f:
json.dump(items_to_keep, f, ...)

Dump content of JSON in specific format

I am need to actually create a Windows INI using Python scipt file of below format:
AGENTIP = 1.2.3.4,
VARFILE = C:\Users\output\temp.out
INFOFILE= C:\Users\output\info.out
SYNTEST = Run:Level1/Get
CMDMODE = RUNTESTSUITE
And Below is my Python code where I have the data in JSON string and then dump the content in a file:
def change_test_details(self, ver, level, grp):
data = {"AGENTIP" : "1.2.3.4", "VARFILE" : "C:\\Users\\output\\temp.out", "INFOFILE" : "C:\\Users\\output\\info.out", "SYNTEST" :"Run:Level1/Get", "CMDMODE" :"RUNTESTSUITE"}
data["SYNTEST"] = ver + ":" + level + "/" + grp
with open("a.txt", 'w') as outfile:
json.dump(data, outfile,indent=2)
When the method is called with below param:
"BETA" "Level5" "Set"
The final output if the file is
{
"AGENTIP": "1.2.3.4",
"VARFILE": "C:\\Users\\output\\temp.out",
"INFOFILE": "C:\\Users\\output\\info.out",
"SYNTEST": "\"BETA\":\"Level5\"/\"Set\"",
"CMDMODE": "RUNTESTSUITE"
}
There is '{ .. }' braces and extra double quotes and '\' and expected value of SYNTEST should be BETA:Level5/Set ?
How can change the JSON string to required format?

json.dump() prints an object as a json formatted string. So when you dump that into outfile you get a json string in the file, which is how it's supposed to behave.
What you want is to iterate over the items and print them according to the format you want.
def change_test_details(ver, level, grp):
data = {"AGENTIP" : "1.2.3.4", "VARFILE" : "C:\\Users\\output\\temp.out", "INFOFILE" : "C:\\Users\\output\\info.out", "SYNTEST" :"Run:Level1/Get", "CMDMODE" :"RUNTESTSUITE"}
data["SYNTEST"] = ver + ":" + level + "/" + grp
with open("a.txt", 'w') as outfile:
for k, v in data.items():
outfile.write(f'{k} = {v}\n')
change_test_details("BETA", "Level5","Set")
When you run this, a.txt looks something like:
AGENTIP = 1.2.3.4
VARFILE = C:\Users\output\temp.out
INFOFILE = C:\Users\output\info.out
SYNTEST = BETA:Level5/Set
CMDMODE = RUNTESTSUITE

how to Adding list values into a JSON file

How to add list values into a JSON file. Here is my code :
import os,string
drives_a = [chr(x) + ':' for x in range(65, 90) if os.path.exists(chr(x) + ':')]

Does this work for you?
import json
lst = [1,2,3]
file_name = 'test.json'
with open(file_name, 'w') as f:
f.write(json.dumps(lst))

How to convert csv to json in python?

I'm very new to programming, have been learning python from past 3/4 weeks and
this is one of the assignments given.
Input
A, B, C, D
1, 2, 3, 4
5, 6, 7, 8
Output
{{A:"1", B:"2", C:"3", D:"4"}, {A:"5", B:"6", C:"7", D:"8"}}
I've been trying with the code as:
import csv
import json
csvfile = open('test.csv','r')
jsonfile = open('test.json','w')
x = ("a","b","c","d")
reader = csv.DictReader(csvfile, x)
for row in reader:
json.dump(row, jsonfile)
The output for this code comes as below:
{"a": "1", "null": ["5", "6", "7", "8", "9"], "c": "3", "b": "2", "d": "4"}
Can anyone help me on this?

Dump after processing whole rows.
import csv
import json
with open('test.csv') as f:
reader = csv.DictReader(f)
rows = list(reader)
with open('test.json', 'w') as f:
json.dump(rows, f)

For those who like one-liners:
import csv
import json
json_data = [json.dumps(d) for d in csv.DictReader(open('file.csv'))]
Checkout this fiddle for a working example:
https://pyfiddle.io/fiddle/5992b8f4-552f-4970-91b6-a52cdee16ebc/?i=true

import csv
import json
# Constants to make everything easier
CSV_PATH = './csv.csv'
JSON_PATH = './json'
# Reads the file the same way that you did
csv_file = csv.DictReader(open(CSV_PATH, 'r'))
# Created a list and adds the rows to the list
json_list = []
for row in csv_file:
json_list.append(row)
# Writes the json output to the file
file(JSON_PATH, 'w').write(json.dumps(json_list))

Convert CSV to Json Python
import csv
import urllib2
url = '<YOURCSVURL>'
response = urllib2.urlopen(url)
cr = csv.reader(response)
line = {}
data = []
for index, row in enumerate(cr):
if index:
for index, col in enumerate(row):
line[name[index]] = col
data.append(line.copy())
else:
name = row
print data

You can attempt it using this code :
def inputfunction(lists):
tmpdict = {}
for element_index in range(len(lists)):
tmpdict[headers[elementindex]] = lists[element_index]
return tmpdict
def run(filename):
filelist = [eachline.split(',') for eachline in open(inputfile,'r')]
headers = filelist[0]
values = filelist[1:]
finallist = []
for lists in values:
finallist.append(inputfunction(lists))
return finallist