Just a heads up I'm completely new to the coding scene and I'm having some issues using a json file
I've got the json to open using
json_queue = json.load(open('customer.json'))
but I just cant find the right code that allows me to make use of the info on the json. I think its because the json is an array not an object (probably completely wrong) My json currently looks like this
[
["James", "VW"],
["Katherine", "BMW"],
["Deborah", "renault"],
["Marguerite", "ford"],
["Kenneth", "VW"],
["Ronald", "Mercedes"],
["Donald", "BMW"],
["Al", "vauxhall"],
["Max", "porsche"],
["Carlos", "BMW"],
["Barry", "ford"],
["Donald", "renault"]
]
What I'm trying to do is take the persons name and the car type they are looking for and compare it too another json file that has the stock of cars in a shop but I'm currently stuck as to how I get python to actually use the information in that json.
I think I might of over explained my problem. My issue is that I am just starting a project using .json files and I can get python to open the file, but then I am unsure of how to get python to read that "James" wants a "VW" and then to go check the stock json to check if it is in stock. The stock json looks like this.
{
"VW": 4,
"BMW": 2,
"renault": 0,
"ford": 1,
"mercedes": 2,
"vauxhall": 1,
"porsche": 0,
}
What you have after the json.load() call is a plain python list of lists:
whishlist = [
["James", "VW"],
["Katherine", "BMW"],
["Deborah", "renault"],
["Marguerite", "ford"],
["Kenneth", "VW"],
["Ronald", "Mercedes"],
["Donald", "BMW"],
["Al", "vauxhall"],
["Max", "porsche"],
["Carlos", "BMW"],
["Barry", "ford"],
["Donald", "renault"]
]
where each sublist is a , pair. You can iterate over this list:
for name, car in whishlist:
print("name : {} - car : {}".format(name, car))
Now with your "other json file", what you have is a dict:
stock = {
"VW": 4,
"BMW": 2,
"renault": 0,
"ford": 1,
"mercedes": 2,
"vauxhall": 1,
"porsche": 0,
}
so all you have to do is to iterate over the whishlist list, check whether the car is in stock and print (or do anything else) the result:
for name, car in whishlist:
in_stock = stock.get(car, 0)
print("for {} : car {} in stock : {}".format(name, car, in_stock))
for James : car VW in stock : 4
for Katherine : car BMW in stock : 2
for Deborah : car renault in stock : 0
for Marguerite : car ford in stock : 1
for Kenneth : car VW in stock : 4
for Ronald : car Mercedes in stock : 0
for Donald : car BMW in stock : 2
for Al : car vauxhall in stock : 1
for Max : car porsche in stock : 0
for Carlos : car BMW in stock : 2
for Barry : car ford in stock : 1
for Donald : car renault in stock : 0
Related
So i was trying to find something to code, and i decided to use python to get fortnite stats, i came across the fortnite_python library and it works, but it displays item codes for items in the shop when i want it to display the names. Anyone know how to convert them or just disply the name in the first place? This is my code.
fortnite = Fortnite('c954ed23-756d-4843-8f99-cfe850d2ed0c')
store = fortnite.store()
fortnite.store()
It outputs something like this
[<StoreItem 12511>,
To print out the attributes of a Python object you can use __dict__ e.g.
from fortnite_python import Fortnite
from json import dumps
fortnite = Fortnite('Your API Key')
# ninjas_account_id = fortnite.player('ninja')
# print(f'ninjas_account: {ninjas_account_id}') # ninjas_account: 4735ce91-3292-4caf-8a5b-17789b40f79c
store = fortnite.store()
example_store_item = store[0]
print(dumps(example_store_item.__dict__, indent=2))
Output:
{
"_data": {
"imageUrl": "https://trackercdn.com/legacycdn/fortnite/237112511_large.png",
"manifestId": 12511,
"name": "Dragacorn",
"rarity": "marvel",
"storeCategory": "BRSpecialFeatured",
"vBucks": 0
},
"id": 12511,
"image_url": "https://trackercdn.com/legacycdn/fortnite/237112511_large.png",
"name": "Dragacorn",
"rarity": "marvel",
"store_category": "BRSpecialFeatured",
"v_bucks": 0
}
So it looks like you want to use name attribute of StoreItem:
for store_item in store:
print(store_item.name)
Output:
Dragacorn
Hulk Smashers
Domino
Unstoppable Force
Scootin'
Captain America
Cable
Probability Dagger
Chimichanga!
Daywalker's Kata
Psi-blade
Snap
Psylocke
Psi-Rider
The Devil's Wings
Daredevil
Meaty Mallets
Silver Surfer
Dayflier
Silver Surfer's Surfboard
Ravenpool
Silver Surfer Pickaxe
Grand Salute
Cuddlepool
Blade
Daredevil's Billy Clubs
Mecha Team
Tricera Ops
Combo Cleaver
Mecha Team Leader
Dino
Triassic
Rex
Cap Kick
Skully
Gold Digger
Windmill Floss
Bold Stance
Jungle Scout
It seems that the library doesn't contain a function to get the names. Also this is what the class of a item from the store looks like:
class StoreItem(Domain):
"""Object containing store items attributes"""
and thats it.
I have a code that should write information to excel using selenium. I have 1 list with some information. I need to write all this to excel, and i have solution. But, when i tried to use it i got 'DataFrame' object is not callable. How can i solve it?
All this code into iteration:
for schools in List: #in the List i have data from excel file with Name of schools
data = pd.DataFrame()
data({
"School Name":School_list_result[0::17],
"Principal":School_list_result[1::17],
"Principal's E-mail":School_list_result[2::17],
"Type":School_list_result[8::17],
"Grade Span": School_list_result[3::17],
"Address":School_list_result[4::17],
"Phone":School_list_result[14::17],
"Website":School_list_result[13::17],
"Associations/Communities":School_list_result[5::17],
"GreatSchools Summary Rating":School_list_result[6::17],
"U.S.News Rankings":School_list_result[12::17],
"Total # Students":School_list_result[15::17],
"Full-Time Teachers":School_list_result[16::17],
"Student/Teacher Ratio":School_list_result[17::17],
"Charter":School_list_result[9::17],
"Enrollment by Race/Ethnicity": School_list_result[7::17],
"Enrollment by Gender":School_list_result[10::17],
"Enrollment by Grade":School_list_result[11::17],
})
data.to_excel("D:\Schools.xlsx")
In School_list_result i have this data:
'Cape Elizabeth High School',
'Mr. Jeffrey Shedd',
'No data.',
'9-12',
'345 Ocean House Road, Cape Elizabeth, ME 04107',
'Cape Elizabeth Public Schools',
'8/10',
'White\n91%\nAsian\n3%\nTwo or more races\n3%\nHispanic\n3%\nBlack\n1%',
'Regular school',
'No',
' Male Female\n Students 281 252',
' 9 10 11 12\n Students 139 135 117 142',
'#5,667 in National Rankings',
'https://cehs.cape.k12.me.us/',
'Tel: (207)799-3309',
'516 students',
'47 teachers',
'11:1',
Please follow the syntax about how to create a dataframe
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
So your code should be modified as:
for schools in List: #in the List i have data from excel file with Name of schools
data = pd.DataFrame(data={
"School Name": School_list_result[0::17],
"Principal": School_list_result[1::17],
"Principal's E-mail": School_list_result[2::17],
"Type": School_list_result[8::17],
"Grade Span": School_list_result[3::17],
"Address": School_list_result[4::17],
"Phone": School_list_result[14::17],
"Website": School_list_result[13::17],
"Associations/Communities": School_list_result[5::17],
"GreatSchools Summary Rating": School_list_result[6::17],
"U.S.News Rankings": School_list_result[12::17],
"Total # Students": School_list_result[15::17],
"Full-Time Teachers": School_list_result[16::17],
"Student/Teacher Ratio": School_list_result[17::17],
"Charter": School_list_result[9::17],
"Enrollment by Race/Ethnicity": School_list_result[7::17],
"Enrollment by Gender": School_list_result[10::17],
"Enrollment by Grade": School_list_result[11::17],
})
Do you want to add in an existing xlsx file?
First, create the dictionary and then call the DataFrame method, like this:
r = {"column1":["data"], "column2":["data"]}
data = pd.DataFrame(r)
I'm trying to scrape some text files into a DB - the format is similar to this with a couple of 1000 segments like this :
Posted By
Date
John Keys
31.08.2019, 10:10 AM
Peter Hall 200 150
Ed Parker 14 1
Posted By
Date
John Keys
31.08.2019, 10:15 AM
Rose Stone 200 150
Travis Anderson 14 1
The records that are important are the records that are coming right after "Date" - so the logic is :
inside_match_flag =0
for line in ins:
if inside_match_flag == 1:
inside_match_flag = 2 # add one to it as we will get all lines
if line == "Posted By": # until we see Posted By again (or EOF)
inside_match_flag =0 # we are now outside the segment
if line == "Date" : # lines after Dates are the ones we want
inside_match_flag =1 # the following lines are to be stored
So this is the way I've done it (the above is not the running code) before doing this by keeping track of a flag and depending on the flag_value I know what lines are most likely coming next.
The issue is of course this about 'the lines coming next' - as I'm reading line per line, I can't easy grab out these segments as I don't want to rely on loading the complete file into memory (as it can go huge).
But the code always gets ugly when I implement something like this - and thinking anyone here that would have a lot smarter approach to do this ?
And note - I am also interested if there would be a super-smart compact way to do this if it requires to load all into memory where code doesn't get so ugly, if all is in memory I guess I can just look for DATE field and save all lines between until it sees Posted By again.
Edit 1
Note the number of players can be more than 2 per game, so a record could also look like this :
Posted By
Date
John Keys
31.08.2019, 10:10 AM
Peter Hall 200 150
Ed Parker 54 1
Rose Stone 20 15
Travis Anderson 1 150
Posted By
...
....
My dream format would be to have an object like this - example based on the match above with 4 players :
{
"Game 1:"
{
"posted by" : "john keys"
"date" : "31.08.2019, 10:10 AM"
"players" : {
{ 1, "Peter Hall, "200", "150" }
{ 2, Ed Parker, "54", "1" }
{ 3 , Rose Stone, "20", "15" }
{ 4, Travis Anderson, "1", "150" }
}
}
}
Note : not 100% correct json format there - and doesn't have to be json, just an object as I will throw them into a SQLite database where it's stored per game which should be illustrated above.
Optimized and memory-efficient generator function approach which yields records on demand:
import pprint
def extract_records(fname):
def prepare_record(rec):
return {'posted by': rec[0], 'date': rec[1],
'players': [[i] + p.rsplit(maxsplit=2)
for i, p in enumerate(rec[2:], 1)]}
with open(fname) as f:
record = []
add_item = False
for line in f:
line = line.strip()
if line == 'Date':
add_item = True
continue
elif line == 'Posted By':
add_item = False
if record:
yield prepare_record(record)
record = []
continue
if add_item:
record.append(line)
if record:
yield prepare_record(record)
records_gen = extract_records('datafile.txt') # generator
for rec in records_gen:
pprint.pprint(rec) # further processing, ex. inserting into DB
The output (2 sample records):
{'date': '31.08.2019, 10:10 AM',
'players': [[1, 'Peter Hall', '200', '150'],
[2, 'Ed Parker', '14', '1'],
[3, 'Rose Stone', '20', '15'],
[4, 'Travis Anderson', '1', '150']],
'posted by': 'John Keys'}
{'date': '31.08.2019, 10:15 AM',
'players': [[1, 'Rose Stone', '200', '150'],
[2, 'Travis Anderson', '14', '1']],
'posted by': 'John Keys'}
There is no magic method for this specific case. Here is an example solution:
buf_size = ...
start_marker = "Posted by\n"
date_marker = "Date\n"
def parse_game(filename)
fh = open(filename)
page = ""
buffer = True # just the start value
while buffer:
buffer = fh.read(buf_size)
page += buffer
records = page.split(start_marker)
if buffer:
page = records.pop()
for record in records:
# skip everything before "Date" and split by lines
chunks = record.split(date_marker, 1)[-1].split("\n")
posted_by, date = chunks[:2]
players = [chunk.split() for chunk in chunks[2:]]
yield {
"posted_by": posted_by,
"date": date,
"players": players
}
If you can read the whole file into memory, it will be just:
def read_game(filename):
for record in open(filename).read().split(start_marker):
# skip everything before "Date" and split by lines
chunks = record.split(date_marker, 1)[-1].split("\n")
posted_by, date = chunks[:2]
players = [chunk.split() for chunk in chunks[2:]]
yield {
"posted_by": posted_by,
"date": date,
"players": players
}
This solution is very similar to Roman's. It is slightly less memory efficient (assuming you have buf_size of memory), but will result in less IO
so I'm new to working with JSON and I'm trying to work with the openrecipe database from here. The db dump you get looks like this...
{ "_id" : { "$oid" : "5160756d96cc62079cc2db16" }, "name" : "Hot Roast Beef Sandwiches", "ingredients" : "12 whole Dinner Rolls Or Small Sandwich Buns (I Used Whole Wheat)\n1 pound Thinly Shaved Roast Beef Or Ham (or Both!)\n1 pound Cheese (Provolone, Swiss, Mozzarella, Even Cheez Whiz!)\n1/4 cup Mayonnaise\n3 Tablespoons Grated Onion (or 1 Tbsp Dried Onion Flakes))\n1 Tablespoon Poppy Seeds\n1 Tablespoon Spicy Mustard\n1 Tablespoon Horseradish Mayo Or Straight Prepared Horseradish\n Dash Of Worcestershire\n Optional Dressing Ingredients: Sriracha, Hot Sauce, Dried Onion Flakes Instead Of Fresh, Garlic Powder, Pepper, Etc.)", "url" : "http://thepioneerwoman.com/cooking/2013/03/hot-roast-beef-sandwiches/", "image" : "http://static.thepioneerwoman.com/cooking/files/2013/03/sandwiches.jpg", "ts" : { "$date" : 1365276013902 }, "cookTime" : "PT20M", "source" : "thepioneerwoman", "recipeYield" : "12", "datePublished" : "2013-03-13", "prepTime" : "PT20M", "description" : "When I was growing up, I participated in my Episcopal church's youth group, and I have lots of memories of weekly meetings wh..." }
{ "_id" : { "$oid" : "5160756f96cc6207a37ff777" }, "name" : "Morrocan Carrot and Chickpea Salad", "ingredients" : "Dressing:\n1 tablespoon cumin seeds\n1/3 cup / 80 ml extra virgin olive oil\n2 tablespoons fresh lemon juice\n1 tablespoon honey\n1/2 teaspoon fine sea salt, plus more to taste\n1/8 teaspoon cayenne pepper\n10 ounces carrots, shredded on a box grater or sliced whisper thin on a mandolin\n2 cups cooked chickpeas (or one 15- ounce can, drained and rinsed)\n2/3 cup / 100 g dried pluots, plums, or dates cut into chickpea-sized pieces\n1/3 cup / 30 g fresh mint, torn\nFor serving: lots of toasted almond slices, dried or fresh rose petals - all optional (but great additions!)", "url" : "http://www.101cookbooks.com/archives/moroccan-carrot-and-chickpea-salad-recipe.html", "image" : "http://www.101cookbooks.com/mt-static/images/food/moroccan_carrot_salad_recipe.jpg", "ts" : { "$date" : 1365276015332 }, "datePublished" : "2013-01-07", "source" : "101cookbooks", "prepTime" : "PT15M", "description" : "A beauty of a carrot salad - tricked out with chickpeas, chunks of dried pluots, sliced almonds, and a toasted cumin dressing. Thank you Diane Morgan." }
{ "_id" : { "$oid" : "5160757096cc62079cc2db17" }, "name" : "Mixed Berry Shortcake", "ingredients" : "Biscuits\n3 cups All-purpose Flour\n2 Tablespoons Baking Powder\n3 Tablespoons Sugar\n1/2 teaspoon Salt\n1-1/2 stick (3/4 Cup) Cold Butter, Cut Into Pieces\n1-1/4 cup Buttermilk\n1/2 teaspoon Almond Extract (optional)\n Berries\n2 pints Mixed Berries And/or Sliced Strawberries\n1/3 cup Sugar\n Zest And Juice Of 1 Small Orange\n SWEET YOGURT CREAM\n1 package (7 Ounces) Plain Greek Yogurt\n1 cup Cold Heavy Cream\n1/2 cup Sugar\n2 Tablespoons Brown Sugar", "url" : "http://thepioneerwoman.com/cooking/2013/03/mixed-berry-shortcake/", "image" : "http://static.thepioneerwoman.com/cooking/files/2013/03/shortcake.jpg", "ts" : { "$date" : 1365276016700 }, "cookTime" : "PT15M", "source" : "thepioneerwoman", "recipeYield" : "8", "datePublished" : "2013-03-18", "prepTime" : "PT15M", "description" : "It's Monday! It's a brand new week! The birds are chirping! The coffee's brewing! Everything has such hope and promise! A..." }
I tried the following code to read in the database
import json
f = r'<file_path>\recipeitems-latest.json'
with open(f) as dfile:
data = json.load(dfile)
print(data)
With this I received the following Traceback
Traceback (most recent call last):
File "C:/Users/<redacted>/Documents/<redacted>/project/test_json.py", line 7, in <module>
data = json.load(dfile)
File "C:\Users\<redacted>\AppData\Local\Continuum\Anaconda3\Lib\json\__init__.py", line 265, in load
return loads(fp.read(),
File "C:\Users\<redacted>\AppData\Local\Continuum\Anaconda3\Lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 101915: character maps to <undefined>
The only way I could find around this error was to only have one entry in the json file. Is the db formatted incorrectly or am I reading in the data wrong?
Thanks for any help!
The file is not a json array. Each line of the file is a json document, but the whole file is not in json format.
Read the file by lines, and use json.loads:
with open('some_file') as f:
for line in f:
doc = json.loads(line)
You may also need to pass the encoding parameter to open(). See here.
I have a text file with the details of a set of restaurants given one after the other. The details are name, rating, price and type of cuisines of a particular restaurant. The contents of text file is as given below.
George Porgie
87%
$$$
Canadian, Pub Food
Queen St. Cafe
82%
$
Malaysian, Thai
Dumpling R Us
71%
$
Chinese
Mexican Grill
85%
$$
Mexican
Deep Fried Everything
52%
$
Pub Food
I want to create a set of dictionaries as given below:
Restaurant name to rating:
# dict of {str : int}
name_to_rating = {'George Porgie' : 87,
'Queen St. Cafe' : 82,
'Dumpling R Us' : 71,
'Mexican Grill' : 85,
'Deep Fried Everything' : 52}
Price to list of restaurant names:
# dict of {str : list of str }
price_to_names = {'$' : ['Queen St. Cafe', 'Dumpling R Us', 'Deep Fried Everything'],
'$$' : ['Mexican Grill'],
'$$$' : ['George Porgie'],
'$$$$' : [ ]}
Cuisine to list of restaurant name:
#dic of {str : list of str }
cuisine_to_names = {'Canadian' : ['George Porgie'],
'Pub Food' : ['George Porgie', 'Deep Fried Everything'],
'Malaysian' : ['Queen St. Cafe'],
'Thai' : ['Queen St. Cafe'],
'Chinese' : ['Dumpling R Us'],
'Mexican' : ['Mexican Grill']}
What is the best way in Python to populate the above dictionaries ?
Initialise some containers:
name_to_rating = {}
price_to_names = collections.defaultdict(list)
cuisine_to_names = collections.defaultdict(list)
Read your file into a temporary string:
with open('/path/to/your/file.txt') as f:
spam = f.read().strip()
Assuming the structure is consistent (i.e. chunks of 4 lines separated by double newlines), iterate through the chunks and populate your containers:
restraunts = [chunk.split('\n') for chunk in spam.split('\n\n')]
for name, rating, price, cuisines in restraunts:
name_to_rating[name] = rating
# etc ..
for the main reading loop, you can use enumerate and modulo to know what is the data on a line:
for lineNb, line in enumerate(data.splitlines()):
print lineNb, lineNb%4, line
for the price_to_names and cuisine_to_names dictionnaries, you could use a defaultdict:
from collections import defaultdict
price_to_names = defaultdict(list)