I have written a python code to convert csv file into json file. But the output is not the same as I desired. please look and suggest modifications.
Below is the expected json file.
[
{
"id": "1",
"MobileNo": "923002546363"
},
{
"id": "2",
"MobileNo": "923343676143"
}
]
below is the code that I have written in python.
import csv, json
def csv_to_json(csvFilePath, jsonFilePath):
jsonArray = []
#read csv file
with open(csvFilePath, encoding='utf-8') as csvf:
#load csv file data using csv library's dictionary reader
csvReader = csv.DictReader(csvf)
#convert each csv row into python dict
for row in csvReader:
#add this python dict to json array
jsonArray.append(row)
#convert python jsonArray to JSON String and write to file
with open(jsonFilePath, 'w', encoding='utf-8') as jsonf:
jsonString = json.dumps(jsonArray, indent=4)
jsonf.write(jsonString)
csvFilePath = r'my_csv_data.csv'
jsonFilePath = r'data.json'
csv_to_json(csvFilePath, jsonFilePath)
As your post doesn't provide current output, I just created a csv file to run your code:
id,MobileNo
1,923002546363
2,923343676143
3,214134367614
And works just fine:
[
{
"id": "1",
"MobileNo": "923002546363"
},
{
"id": "2",
"MobileNo": "923343676143"
},
{
"id": "3",
"MobileNo": "214134367614"
}
]
Check if your csv file isn't corrupted. And if possible, edit your post with current output and your csv file.
Related
Hello I show you my problem's :
I right that for convert my csv in Json. But the résult is not exactly what I Want .
main.py
import csv
filename ="forcebrute.csv"
# opening the file using "with"
# statement
with open(filename, 'r') as data:
for line in csv.DictReader(data):
print(line)
csv
name;price;profit
Action-1;20;5
Action-2;30;10
Action-3;50;15
Action-4;70;20
Action-5;60;17
result i have:
{'name;price;profit': 'Action-1;20;5'}
{'name;price;profit': 'Action-2;30;10'}
{'name;price;profit': 'Action-3;50;15'}
{'name;price;profit': 'Action-4;70;20'}
{'name;price;profit': 'Action-5;60;17'}
And I would like this result:
You will need to specify the column delimiter then you can use json.dumps() to give you the required output format
import csv
import json
with open('forcebrute.csv') as data:
print(json.dumps([d for d in csv.DictReader(data, delimiter=';')], indent=2))
Output:
[
{
"name": "Action-1",
"price": "20",
"profit": "5"
},
{
"name": "Action-2",
"price": "30",
"profit": "10"
},
{
"name": "Action-3",
"price": "50",
"profit": "15"
},
{
"name": "Action-4",
"price": "70",
"profit": "20"
},
{
"name": "Action-5",
"price": "60",
"profit": "17"
}
]
You will need to use Dictreader from the csv library to read the contents of the CSV file and then convert the contents to a list before using json.dumps to turn the data into JSON.
import csv
import json
filename ="forcebrute.csv"
# Open the CSV file and read the contents into a list of dictionaries
with open(filename, 'r') as f:
reader = csv.DictReader(f, delimiter=';')
csv_data = list(reader)
# Convert the data to a JSON string and print it to the console
json_data = json.dumps(csv_data)
print(json_data)
An easy approach would be using pandas, also quite fast with large csv files. It might need some tweaking but you get the point.
import pandas as pd
import json
df = pd.read_csv(filename, sep = ';')
data = json.dumps(df.to_dict('records'))
I'm trying to iterate over a JSON file and write specific key values to a new JSON file:
def get_rubrik_failed_archives_main():
with open("get_failed_archives.json") as json_file:
json_data = json.load(json_file)
for archive_data in json_data["data"]:
dictionary = {
"objectName": archive_data["latestEvent"]["objectName"],
"time": archive_data["latestEvent"]["time"],
"eventType": archive_data["latestEvent"]["eventType"],
"eventStatus": archive_data["latestEvent"]["eventStatus"]
}
with open("rubrik_failed_archives.json", "w") as file:
json.dump(dictionary, file, indent=4, sort_keys=True)
The problem is that I cannot seem to write multiple objects into the JSON file, as I only get one object:
{
"eventStatus": "Failure",
"eventType": "Archive",
"objectName": "Template",
"time": "2022-08-21T16:09:31.863Z"
}
How do I write a for loop so that all of the needed key values get written into the new JSON file?
It appears that new data cannot be updated in dictionary.
So, The answer I came up with is json_data.update(dictionary)to add to the for loop.
def get_rubrik_failed_archives_main():
with open("get_failed_archives.json") as json_file:
json_data = json.load(json_file)
for archive_data in json_data["data"]:
dictionary = {
"objectName": archive_data["latestEvent"]["objectName"],
"time": archive_data["latestEvent"]["time"],
"eventType": archive_data["latestEvent"]["eventType"],
"eventStatus": archive_data["latestEvent"]["eventStatus"]
}
json_data.update(dictionary)
with open("rubrik_failed_archives.json", "w") as file:
json.dump(dictionary, file, indent=4, sort_keys=True)
I don't know if it will be solved because I can't check it, but I hope it helps you.
I am trying to delete an element in a json file,
here is my json file:
before:
{
"names": [
{
"PrevStreak": false,
"Streak": 0,
"name": "Brody B#3719",
"points": 0
},
{
"PrevStreak": false,
"Streak": 0,
"name": "XY_MAGIC#1111",
"points": 0
}
]
}
after running script:
{
"names": [
{
"PrevStreak": false,
"Streak": 0,
"name": "Brody B#3719",
"points": 0
}
]
}
how would I do this in python? the file is stored locally and I am deciding which element to delete by the name in each element
Thanks
I would load the file, remove the item, and then save it again. Example:
import json
with open("filename.json") as f:
data = json.load(f)
f.pop(data["names"][1]) # or iterate through entries to find matching name
with open("filename.json", "w") as f:
json.dump(data, f)
You will have to read the file, convert it to python native data type (e.g. dictionary), then delete the element and save the file. In your case something like this could work:
import json
filepath = 'data.json'
with open(filepath, 'r') as fp:
data = json.load(fp)
del data['names'][1]
with open(filepath, 'w') as fp:
json.dump(data, fp)
Try this:
# importing the module
import ast
# reading the data from the file
with open('dictionary.txt') as f:
data = f.read()
print("Data type before reconstruction : ", type(data))
# reconstructing the data as a dictionary
a_dict = ast.literal_eval(data)
{"names":[a for a in a_dict["names"] if a.get("name") !="XY_MAGIC#1111"]}
import json
with open("test.json",'r') as f:
data = json.loads(f.read())
names=data.get('names')
for idx,name in enumerate(names):
if name['name']=='XY_MAGIC#1111':
del names[idx]
break
print(names)
In order to read the file best approach would be using the with statement after which you could just use pythons json library and convert json string to python dict. once you get dict you can access the values and do your operations as required. you could convert it as json using json.dumps() then save it
This does the right thing useing the python json module, and prettyprints the json back to the file afterwards:
import json
jsonpath = '/path/to/json/file.json'
with open(jsonpath) as file:
j = json.loads(file.read())
names_to_remove = ['XY_MAGIC#1111']
for element in j['names']:
if element['name'] in names_to_remove:
j['names'].remove(element)
with open(jsonpath, 'w') as file:
file.write(json.dumps(j, indent=4))
I have a JSON file with 10000 data entries like below in a file.
{
"1":{
"name":"0",
"description":"",
"image":""
},
"2":{
"name":"1",
"description":"",
"image":""
},
...
}
I need to write each entry in this object into its own file.
For example, the output of each file looks like this:
1.json
{
"name": "",
"description": "",
"image": ""
}
I have the following code, but I'm not sure how to proceed from here. Can anyone help with this?
import json
with open('sample.json', 'r') as openfile:
# Reading from json file
json_object = json.load(openfile)
You can use a for loop to iterate over all the fields in the outer object, and then create a new file for each inner object:
import json
with open('sample.json', 'r') as input_file:
json_object = json.load(input_file)
for key, value in json_object.items():
with open(f'{key}.json', 'w') as output_file:
json.dump(value, output_file)
I'm exploring the avro file format and am currently struggling to append data. I seem to overwrite in each run. I found an existing thread here, saying I should not pass in a schema in order to "append" to existing file without overwriting. Even my lint gives this clue: If the schema is not present, presume we're appending.. However, If I try to declare DataFileWriter as DataFileWriter(open("users.avro", "wb"), DatumWriter(), None) then the code wont run.
Simply put, how do I append values to an existing avro files without writing over existing content.
schema = avro.schema.parse(open("user.avsc", "rb").read()
writer = DataFileWriter(open("users.avro", "wb"), DatumWriter(), schema)
print("start appending")
writer.append({"name": "Alyssa", "favorite_number": 256})
writer.append({"name": "Ben", "favorite_number": 12, "favorite_color": "blue"})
writer.close()
print("write successful!")
# Read data from an avro file
with open('users.avro', 'rb') as f:
reader = DataFileReader(open("users.avro", "rb"), DatumReader())
users = [user for user in reader]
reader.close()
print(f'Schema {schema}')
print(f'Users:\n {users}')
I'm not sure how to do it with the standard avro library, but if you use fastavro it can be done. See the example below:
from fastavro import parse_schema, writer, reader
schema = {
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": ["int", "null"]},
{"name": "favorite_color", "type": ["string", "null"]}
]
}
parsed_schema = parse_schema(schema)
records = [
{"name": "Alyssa", "favorite_number": 256},
{"name": "Ben", "favorite_number": 12, "favorite_color": "blue"},
]
# Write initial 2 records
with open("users.avro", "wb") as fp:
writer(fp, schema, records)
# Append third record
with open("users.avro", "a+b") as fp:
writer(fp, schema, [{"name": "Chris", "favorite_number": 1}])
# Read all records
with open("users.avro", "rb") as fp:
for record in reader(fp):
print(record)
The solution to skip the schema is correct, but only once you have the Avro file set up with the correct schema.
In your code, what you have incorrectly, was to put wb instead of ab in the file open mode.
Putting None or no argument at all in DataFileWriter should not matter and the code should run.
This reproducible code initializes the file in the correct schema. It does not matter if it is ab or wb mode, just write to an empty file with a schema and close it.
writer = DataFileWriter(open("reproducible.avro", "ab+"), DatumWriter(), schema)
writer.close()
Now to write the actual records in the append mode (so no re-reading the file!), you can skip the schema while in the ab mode:
for i in range(3):
writer = DataFileWriter(open("reproducible.avro", "ab+"), DatumWriter())
writer.append(db_entry)
writer.close()
Finally, read the entire file:
reader = DataFileReader(open("reproducible.avro", "rb"), DatumReader())
for data in reader:
print(data)
reader.close()
Works for me on Windows, with Python 3.9.13 and avro library 1.11.1.
For full reproducible example, please begin with:
import avro.schema
from avro.datafile import DataFileReader, DataFileWriter
from avro.io import DatumReader, DatumWriter
import json
schema = {
"type": "record",
"name": "recordName",
"fields": [
{
"name": "id",
"type": "string"
}
]
}
schema = avro.schema.parse(json.dumps(schema))
db_entry = {
"id": "random_id"
}