When I print these strings
print query, netinfo
I get output below, which is fine. How do i take these strings and put them into a CSV file into a single row?
8.8.8.8 [{'updated': '2012-02-24T00:00:00', 'handle': 'NET-8-0-0-0-1', 'description': 'Level 3 Communications, Inc.', 'tech_emails': 'ipaddressing#level3.com', 'abuse_emails': 'abuse#level3.com', 'postal_code': '80021', 'address': '1025 Eldorado Blvd.', 'cidr': '8.0.0.0/8', 'city': 'Broomfield', 'name': 'LVLT-ORG-8-8', 'created': '1992-12-01T00:00:00', 'country': 'US', 'state': 'CO', 'range': '8.0.0.0 - 8.255.255.255', 'misc_emails': None}, {'updated': '2014-03-14T00:00:00', 'handle': 'NET-8-8-8-0-1', 'description': 'Google Inc.', 'tech_emails': 'arin-contact#google.com', 'abuse_emails': 'arin-contact#google.com', 'postal_code': '94043', 'address': '1600 Amphitheatre Parkway', 'cidr': '8.8.8.0/24', 'city': 'Mountain View', 'name': 'LVLT-GOGL-8-8-8', 'created': '2014-03-14T00:00:00', 'country': 'US', 'state': 'CA', 'range': None, 'misc_emails': None}]
I have tried hobbling this together but it's all jacked up. I could use some help on how to use the csv module.
writer=csv.writer(open('dict.csv', 'ab'))
for key in query:
writer.writerow(query)
You can put your variables in a tuple and write to csv file :
import csv
from operator import itemgetter
with open('ex.csv', 'wb') as csvfile:
spamreader = csv.writer(csvfile, delimiter=' ')
spamreader.writerow((query, netinfo))
Note: if you are in python 3 use following code :
import csv
from operator import itemgetter
with open('ex.csv', 'w',newline='') as csvfile:
spamreader = csv.writer(csvfile, delimiter=' ')
spamreader.writerow((query, netinfo))
Related
This question already has answers here:
CSV file written with Python has blank lines between each row
(11 answers)
Closed last year.
I have a below code which is creating a csv file:
import csv
# my data rows as dictionary objects
mydict = [{'branch': 'COE', 'cgpa': '9.0', 'name': 'Nikhil', 'year': '2'},
{'branch': 'COE', 'cgpa': '9.1', 'name': 'Sanchit', 'year': '2'},
{'branch': 'IT', 'cgpa': '9.3', 'name': 'Aditya', 'year': '2'},
{'branch': 'SE', 'cgpa': '9.5', 'name': 'Sagar', 'year': '1'},
{'branch': 'MCE', 'cgpa': '7.8', 'name': 'Prateek', 'year': '3'},
{'branch': 'EP', 'cgpa': '9.1', 'name': 'Sahil', 'year': '2'}]
# field names
fields = ['name', 'branch', 'year', 'cgpa']
# name of csv file
filename = "university_records.csv"
# writing to csv file
with open(filename, 'w') as csvfile:
# creating a csv dict writer object
writer = csv.DictWriter(csvfile, fieldnames=fields)
# writing headers (field names)
writer.writeheader()
# writing data rows
writer.writerows(mydict)
Running the above code is giving below excel sheet
It contains blank rows as well. How can I remove these blank rows. Thanks
You should create a dataframe with your dict, and then just use
pd.to_csv(name_of_dataframe, sep=your_columns_sep)
Adding the newline='' in the with open ... does the trick:
import csv
my_dict = [{'branch': 'COE', 'cgpa': '9.0', 'name': 'Nikhil', 'year': '2'},
{'branch': 'COE', 'cgpa': '9.1', 'name': 'Sanchit', 'year': '2'},
{'branch': 'IT', 'cgpa': '9.3', 'name': 'Aditya', 'year': '2'},
{'branch': 'SE', 'cgpa': '9.5', 'name': 'Sagar', 'year': '1'},
{'branch': 'MCE', 'cgpa': '7.8', 'name': 'Prateek', 'year': '3'},
{'branch': 'EP', 'cgpa': '9.1', 'name': 'Sahil', 'year': '2'}]
fields = ['name', 'branch', 'year', 'cgpa']
filename = "foo_bar.csv"
with open(filename, 'w', newline='') as csv_file:
writer = csv.DictWriter(csv_file, fieldnames=fields)
writer.writeheader()
writer.writerows(my_dict)
Why does the the resulting .CSV file have interspersed blank lines?
# https://www.tutorialspoint.com/How-to-save-a-Python-Dictionary-to-CSV-file
import csv
csv_columns = ['No','Name','Country'] # These are dictionary keys, they map to columns in the CSV file
dict_data = [
{'No': 1, 'Name': 'Alex', 'Country': 'India'},
{'No': 2, 'Name': 'Ben', 'Country': 'USA'},
{'No': 3, 'Name': 'Shri Ram', 'Country': 'India'},
{'No': 4, 'Name': 'Smith', 'Country': 'USA'},
{'No': 5, 'Name': 'Yuva Raj', 'Country': 'India'},
]
csv_file = "NamesExample.csv" #relative path
try:
with open(csv_file, 'w') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
writer.writeheader()
for data in dict_data: # One dictionary at a time
writer.writerow(data)
except IOError:
print("I/O error")
This is what I get when I run the above code:
No,Name,Country
1,Alex,India
2,Ben,USA
3,Shri Ram,India
4,Smith,USA
5,Yuva Raj,India
I have this read function where it reads a csv file using csv.DictReader. The file.csv is separated by commas and it fully reads. However, this part of my file has a column that contains multiple commas. My question is, how can I make sure that comma is counted as part of a column? I cannot alter my csv file to meet the criteria.
Text File:
ID,Name,University,Street,ZipCode,Country
12,Jon Snow,U of Winterfell,Winterfell #45,60434,Westeros
13,Steve Rogers,NYU,108, Chelsea St.,23333,United States
20,Peter Parker,Yale,34, Tribeca,32444,United States
34,Tyrion Lannister,U of Casterly Rock,Kings Landing #89, 43543,Westeros
The desired output is this:
{'ID': '12', 'Name': 'Jon Snow', 'University': 'U of Winterfell', 'Street': 'Winterfell #45', 'ZipCode': '60434', 'Country': 'Westeros'}
{'ID': '13', 'Name': 'Steve Rogers', 'University': 'NYU', 'Street': '108, Chelsea St.', 'ZipCode': '23333', 'Country': 'United States'}
{'ID': '20', 'Name': 'Peter Parker', 'University': 'Yale', 'Street': '34, Tribeca', 'ZipCode': '32444', 'Country': 'United States'}
{'ID': '34', 'Name': 'Tyrion Lannister', 'University': 'U of Casterly Rock', 'Street': 'Kings Landing #89', 'ZipCode': '43543', 'Country': 'Westeros'}
As you can tell the 'Street' has at least two commas due to the numbers:
13,Steve Rogers,NYU,108, Chelsea St.,23333,United States
20,Peter Parker,Yale,34, Tribeca,32444,United States
Note: Most of the columns being read splits by a str,str BUT under the 'Street' column it is followed by a str, str (there is an extra space after the comma). I hope this makes sense.
The options I tried looking out is using re.split, but I don't know how to implement it on my read file. I was thinking re.split(r'(?!\s),(?!\s)',x[:-1])? How can I make sure the format from my file will count as part of any column? I can't use pandas.
My current output looks like this right now:
{'ID': '12', 'Name': 'Jon Snow', 'University': 'U of Winterfell', 'Street': 'Winterfell #45', 'ZipCode': '60434', 'Country': 'Westeros'}
{'ID': '13', 'Name': 'Steve Rogers', 'University': 'NYU', 'Street': '108', 'ZipCode': 'Chelsea St.', 'Country': '23333', None: ['United States']}
{'ID': '20', 'Name': 'Peter Parker', 'University': 'Yale', 'Street': '34', 'ZipCode': 'Tribeca', 'Country': '32444', None: ['United States']}
{'ID': '34', 'Name': 'Tyrion Lannister', 'University': 'U of Casterly Rock', 'Street': 'Kings Landing #89', 'ZipCode': '43543', 'Country': 'Westeros'}
This is my read function:
import csv
list = []
with open('file.csv', mode='r') as csv_file:
csv_reader = csv.DictReader(csv_file, delimiter=",", skipinitialspace=True)
for col in csv_reader:
list.append(dict(col))
print(dict(col))
You can't use csv if the file isn't valid CSV format.
You need to call re.split() on ordinary lines, not on dictionaries.
list = []
with open('file.csv', mode='r') as csv_file:
keys = csv_file.readline().strip().split(',') # Read header line
for line in csv_file:
line = line.strip()
row = re.split(r'(?!\s),(?!\s)',line)
list.append(dict(zip(keys, row)))
The actual solution for the problem is modifying the script that generates the csv file.
If you have a chance to modify that output you can do 2 things
Use a delimiter other than a comma such as | symbol or ; whatever you believe it doesn't exist in the string.
Or enclose all columns with " so you'll be able to split them by , which are actual separators.
If you don't have a chance to modify the output.
And if you are sure about that multiple commas are only in the street column; then you should use csv.reader instead of DictReader this way you can get the columns by Indexes that you are already sure. for instance row[0] will be ID row[1] will be Name and row[-1] will be Country row[-2] will be ZipCode so row[2:-2] would give you what you need i guess. Indexes can be arranged but the idea is clear I guess.
Hope that helps.
Edit:
import csv
list = []
with open('file.csv', mode='r') as csv_file:
csv_reader = csv.reader(csv_file, delimiter=",", skipinitialspace=True)
# pass the header row
next(csv_reader)
for row in csv_reader:
list.append({"ID": row[0],
"Name": row[1],
"University": row[2],
"Street": ' '.join(row[3:-2]),
"Zipcode": row[-2],
"Country": row[-1]})
print(list)
--
Here is the output (with pprint)
[{'Country': 'Westeros',
'ID': '12',
'Name': 'Jon Snow',
'Street': 'Winterfell #45',
'University': 'U of Winterfell',
'Zipcode': '60434'},
{'Country': 'United States',
'ID': '13',
'Name': 'Steve Rogers',
'Street': '108 Chelsea St.',
'University': 'NYU',
'Zipcode': '23333'},
{'Country': 'United States',
'ID': '20',
'Name': 'Peter Parker',
'Street': '34 Tribeca',
'University': 'Yale',
'Zipcode': '32444'},
{'Country': 'Westeros',
'ID': '34',
'Name': 'Tyrion Lannister',
'Street': 'Kings Landing #89',
'University': 'U of Casterly Rock',
'Zipcode': '43543'}]
-- second edit
edited the index on the street.
Regards.
I have a dataset (that pull its data from a dict) that I am attempting to clean and republish. Within this data set, there is a field with a sublist that I would like to extract specific data from.
Here's the data:
[{'id': 'oH58h122Jpv47pqXhL9p_Q', 'alias': 'original-pizza-brooklyn-4', 'name': 'Original Pizza', 'image_url': 'https://s3-media1.fl.yelpcdn.com/bphoto/HVT0Vr_Vh52R_niODyPzCQ/o.jpg', 'is_closed': False, 'url': 'https://www.yelp.com/biz/original-pizza-brooklyn-4?adjust_creative=IelPnWlrTpzPtN2YRie19A&utm_campaign=yelp_api_v3&utm_medium=api_v3_business_search&utm_source=IelPnWlrTpzPtN2YRie19A', 'review_count': 102, 'categories': [{'alias': 'pizza', 'title': 'Pizza'}], 'rating': 4.0, 'coordinates': {'latitude': 40.63781, 'longitude': -73.8963799}, 'transactions': [], 'price': '$', 'location': {'address1': '9514 Ave L', 'address2': '', 'address3': '', 'city': 'Brooklyn', 'zip_code': '11236', 'country': 'US', 'state': 'NY', 'display_address': ['9514 Ave L', 'Brooklyn, NY 11236']}, 'phone': '+17185313559', 'display_phone': '(718) 531-3559', 'distance': 319.98144420799355},
Here's how the data is presented within the csv/spreadsheet:
location
{'address1': '9514 Ave L', 'address2': '', 'address3': '', 'city': 'Brooklyn', 'zip_code': '11236', 'country': 'US', 'state': 'NY', 'display_address': ['9514 Ave L', 'Brooklyn, NY 11236']}
Is there a way to pull location.city for example?
The below code simply adds a few fields and exports it to a csv.
def data_set(data):
df = pd.DataFrame(data)
df['zip'] = get_zip()
df['region'] = get_region()
newdf = df.filter(['name', 'phone', 'location', 'zip', 'region', 'coordinates', 'rating', 'review_count',
'categories', 'url'], axis=1)
if not os.path.isfile('yelp_data.csv'):
newdf.to_csv('data.csv', header='column_names')
else: # else it exists so append without writing the header
newdf.to_csv('data.csv', mode='a', header=False)
If that doesn't make sense, please let me know. Thanks in advance!
I am trying to pull 'created' from the Monzo data I'm pulling.
I have made a call to the Monzo api with the following code:
from monzo.monzo import Monzo
client = Monzo(INSERT API KEY)
data = client.get_transactions("INSERT ACCOUNT NUMBER")
print (data)
and I can't quite get the data I need which looks like this:
d': 'merch_000094MPASVBf7xCdrZOz3', 'created': '2016-01-20T21: 26: 33.985Z', 'name': 'DelicedeFrance', 'logo': 'https: //mondo-logo-cache.appspot.com/twitter/deliceuk/?size=large', 'emoji': '🇫🇷', 'category': 'eating_out', 'online': False, 'atm': False, 'address': {'short_formatted': 'LiverpoolStreetStation,
LondonEC2M7PY', 'formatted': 'LiverpoolStreetStation,
LondonEC2M7PY,
UnitedKingdom', 'address': 'LiverpoolStreetStation', 'city': 'London', 'region': 'GreaterLondon', 'country': 'GBR', 'postcode': 'EC2M7PY', 'latitude': 51.518159172221615, 'longitude': -0.08210659649555102, 'zoom_level': 17, 'approximate': False}, 'updated': '2016-02-02T14: 10: 48.664Z', 'metadata': {'foursquare_category': 'Restaurant', 'foursquare_category_icon': 'https: //ss3.4sqi.net/img/categories_v2/food/default_88.png','foursquare_website': '', 'google_places_icon': 'https: //maps.gstatic.com/mapfiles/place_api/icons/restaurant-71.png', 'google_places_name': 'DelicedeFrance', 'suggested_name': 'DelicedeFrance', 'suggested_tags': '#food', 'twitter_id': ''}, 'disable_feedback': False}, 'notes': '', 'metadata': {}, 'account_balance': 3112, 'attachments': [], 'category': 'eating_out', 'is_load': False, 'settled': '2017-04-28T04: 54: 18.167Z', 'local_amount': -199, 'local_currency': 'GBP', 'updated': '2017-04-28T06: 15: 06.095Z', 'counterparty': {}, 'originator': False, 'include_in_spending': True}, {'created': '2017-04-28T08: 54: 10.917Z','amount': -130, 'currency': 'GBP', 'merchant': {'created': '2016-04-21T08: 02: 13.537Z','logo': 'https: //mondo-logo-cache.appspot.com/twitter/MCSaatchiLondon/?size=large', 'emoji': '🍲', 'category': 'eating_out', 'online': False, 'atm': False...
How do I pull the 'created' date?
Try this:
#!/usr/bin/env python
import csv
from pymonzo import MonzoAPI
if __name__ == '__main__':
monzo_api = MonzoAPI()
monzo_transactions = monzo_api.transactions()
with open('monzo_transactions.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
for transaction in monzo_transactions:
writer.writerow([
transaction.amount, transaction.description,
transaction.created,
])
print('All done!')
If this is actually right json code and you just have paste errors, than you can use the python libary json:
import json
data = json.loads(datastring)
If this not json code, you probably have to write a parser on your own.