Related
I have a simple Python script that queries an API and parses the JSON data. Specifically, I am trying to find all ids that fall into a rectangle based on given latitude and longitude coordinates. I am having some data type issues since one is being returned as type str and the other as type float. The coordinates are: (37.769754, -122.427050) and, (37.748554, -122.404535).
Below is my code, sample JSON, and the trace.
Code:
import requests
def get_ids():
url = "https://retro.umoiq.com/service/publicJSONFeed?command=vehicleLocations&a=sf-muni&t=0"
response = requests.get(url).json()
id_list = []
for id in response['vehicle']:
lat = id['lat']
lon = id['lon']
if (lat <= 37.769754 and lat >= 37.748554):
if (lon >= -122.427050 and lon <= -122.404535):
id_list.append(id['id'])
return id_list
def main():
print(get_ids())
if __name__ == "__main__":
main()
JSON:
{'lastTime': {'time': '1653259435728'}, 'copyright': 'All data copyright San Francisco Muni 2022.', 'vehicle': [{'routeTag': 'KT', 'predictable': 'true', 'heading': '218', 'speedKmHr': '0', 'lon': '-122.405464', 'id': '1462', 'dirTag': 'KT___O_F20', 'lat': '37.708099', 'secsSinceReport': '33', 'leadingVehicleId': '1487'}, {'routeTag': '33', 'predictable': 'true', 'heading': '165', 'speedKmHr': '35', 'lon': '-122.40744', 'id': '5817', 'dirTag': '33___O_F00', 'lat': '37.763451', 'secsSinceReport': '6'}, {'routeTag': '1', 'predictable': 'true', 'heading': '269', 'speedKmHr': '0', 'lon': '-122.492844', 'id': '5818', 'dirTag': '1____O_F00', 'lat': '37.77985', 'secsSinceReport': '33'}, {'routeTag': '1', 'predictable': 'true', 'heading': '219', 'speedKmHr': '0', 'lon': '-122.493156', 'id': '5819', 'lat': '37.779823', 'secsSinceReport': '6'}, {'routeTag': 'N', 'predictable': 'true', 'heading': '195', 'speedKmHr': '6', 'lon': '-122.457748', 'id': '1453', 'dirTag': 'N____O_F01', 'lat': '37.764671', 'secsSinceReport': '33'}, {'routeTag': '24', 'predictable': 'true', 'heading': '231', 'speedKmHr': '0', 'lon': '-122.412033', 'id': '5813', 'dirTag': '24___I_F00', 'lat': '37.739773', 'secsSinceReport': '20'}, {'routeTag': '1', 'predictable': 'true', 'heading': '80', 'speedKmHr': '0', 'lon': '-122.397522', 'id': '5815', 'dirTag': '1____I_F00', 'lat': '37.795418', 'secsSinceReport': '46'}, {'routeTag': '1', 'predictable': 'true', 'heading': '87', 'speedKmHr': '0', 'lon': '-122.472931', 'id': '5827', 'dirTag': '1____I_F00', 'lat': '37.78437', 'secsSinceReport': '6'}, {'routeTag': 'KT', 'predictable': 'true', 'heading': '330', 'speedKmHr': '32', 'lon': '-122.468117', 'id': '1469', 'dirTag': 'KT___I_F20', 'lat': '37.7290149', 'secsSinceReport': '6'}, {'routeTag': '33', 'predictable': 'true', 'heading': '77', 'speedKmHr': '0', 'lon': '-122.456421', 'id': '5828', 'dirTag': '33___O_F00', 'lat': '37.786957', 'secsSinceReport': '6'}, {'routeTag': '45', 'predictable': 'true', 'heading': '165', 'speedKmHr': '21', 'lon': '-122.406647', 'id': '5829', 'dirTag': '45___I_F00', 'lat': '37.78756', 'secsSinceReport': '6'}
etc...
Trace:
Traceback (most recent call last):
File "/main.py", line 51, in <module>
main()
File "/main.py", line 48, in main
get_ids()
File "/main.py", line 41, in get_ids
if (lat < 37.769754 and lon < -122.427050):
TypeError: '<' not supported between instances of 'str' and 'float'
Try converting the data to floats like so:
lat = float(i['lat'])
lon = float(i['lon'])
This will allow the comparison operators to work correctly when comparing 2 floats.
Keep in mind the operators themselves are wrong (long -179 and lat -179 would fit inside your rectangle).
I took the liberty to improve some of your code and fix the comparison operators:
import requests
VEHICLE_LOCATIONS_URL = "https://retro.umoiq.com/service/publicJSONFeed?command=vehicleLocations&a=sf-muni&t=0"
# (min_lat, max_lat), (min_long, max_long)
BOUNDARIES = ((37.748554, 37.769754), (-122.427050, -122.404535))
def get_ids_in_boundry():
response = requests.get(VEHICLE_LOCATIONS_URL).json()
id_list = []
for vehicle in response['vehicle']:
lat, long = float(vehicle['lat']), float(vehicle['lon'])
if ((BOUNDARIES[0][0] <= lat <= BOUNDARIES[0][1])
and (BOUNDARIES[1][0] <= long <= BOUNDARIES[1][1])):
id_list.append(vehicle['id'])
return id_list
def main():
print(get_ids_in_boundry())
if __name__ == "__main__":
main()
I've changed the URL to be a const, so as the boundaries, and returned the id_list outside of the function. Many other improvements can be added such as asking for boundaries in the function parameter or splitting doing the request and checking the boundaries into 2 different functions.
If I understood everything correctly the aswer is to convert the variables lat and lon to floats.
To do that just modify this in your code
lat = float(i['lat'])
lon = float(i['lon'])
I have a list inside a nested dictionary
body = {'Ready Date': '2020-01-31T12:00:00','Shipment Line List': [{'Description': 'Test', 'Weigth': '5',
'Height': '4.0','Length': '2.0', 'Width': '3.0'}, {'Description': 'Test', 'Weigth': '20', 'Height': '5',
'Length': '30', 'Width': '10']}
I want to iterate over the keys in the nested dictionary and replace "Weigth" with the correct spelling "Weight"
I tried this approach, but I am not getting the expected output
key = {"Weigth":"Weight"}
def find_replace(dict_body, dictionary):
# is the item in the dict?
for item in dict_body:
# iterate by keys
if item in dictionary.keys():
# look up and replace
dict_body = dict_body.replace(item, dictionary[item])
# return updated dict
return dict_body
a = find_replace(body,key)
print(a)
I think a better idea in this particular case is to treat everything as a string, replace and back as a dictionary. Because if you have multiple nested keys, it might be just be easier this way in two lines of code:
from ast import literal_eval
body = literal_eval(str(body).replace("Weigth","Weight"))
This outputs:
{'Ready Date': '2020-01-31T12:00:00',
'Shipment Line List': [{'Description': 'Test',
'Height': '4.0',
'Length': '2.0',
'Weight': '5',
'Width': '3.0'},
{'Description': 'Test',
'Height': '5',
'Length': '30',
'Weight': '20',
'Width': '10'}]}
I want to iterate over the keys in the nested dictionary and replace "Weigth" with the correct spelling "Weight"
something like the below
body = {'Ready Date': '2020-01-31T12:00:00', 'Shipment Line List': [{'Description': 'Test', 'Weigth': '5',
'Height': '4.0', 'Length': '2.0', 'Width': '3.0'},
{'Description': 'Test', 'Weigth': '20',
'Height': '5',
'Length': '30', 'Width': '10'}]}
for entry in body['Shipment Line List']:
entry['Weight'] = entry['Weigth']
del entry['Weigth']
print(body)
output
{'Ready Date': '2020-01-31T12:00:00', 'Shipment Line List': [{'Description': 'Test', 'Height': '4.0', 'Length': '2.0', 'Width': '3.0', 'Weight': '5'}, {'Description': 'Test', 'Height': '5', 'Length': '30', 'Width': '10', 'Weight': '20'}]}
Building off my former post here: How to print data from API call into a CSV file
The API call returns this
[Order({ 'asset_class': 'us_equity',
'asset_id': '8a9-43b6-9b36-662f01e8fadd',
'canceled_at': None,
'client_order_id': 'e38a-b51c-349314bc6e9e',
'created_at': '2020-06-05T16:16:53.307491Z',
'expired_at': None,
'extended_hours': False,
'failed_at': None,
'filled_at': '2020-06-05T16:16:53.329Z',
'filled_avg_price': '7.8701',
'filled_qty': '45',
'id': '8-4888-9c7c-97bf8c2a3a16',
'legs': None,
'limit_price': '7.87',
'order_class': '',
'order_type': 'limit',
'qty': '45',
'replaced_at': None,
'replaced_by': None,
'replaces': None,
'side': 'sell',
'status': 'filled',
'stop_price': None,
'submitted_at': '2020-06-05T16:16:53.293859Z',
'symbol': 'CARS',
'time_in_force': 'day',
'type': 'limit',
'updated_at': '2020-06-08T11:21:51.411547Z'}), Order({ 'asset_class': 'us_equity',
'asset_id': '1aef-42f4-9975-750dbcb3e67d',
'canceled_at': None,
'client_order_id': '2bde-4572-a5d0-bfc32c2bf31a',
'created_at': '2020-06-05T16:16:37.508176Z',
'expired_at': None,
'extended_hours': False,
'failed_at': None,
'filled_at': '2020-06-05T16:16:37.531Z',
'filled_avg_price': '10.8501',
'filled_qty': '26',
'id': '4256-472c-a5de-6ca9d6a21422',
'legs': None,
'limit_price': '10.85',
'order_class': '',
'order_type': 'limit',
'qty': '26',
'replaced_at': None,
'replaced_by': None,
'replaces': None,
'side': 'sell',
'status': 'filled',
'stop_price': None,
'submitted_at': '2020-06-05T16:16:37.494389Z',
'symbol': 'IGT',
'time_in_force': 'day',
'type': 'limit',
'updated_at': '2020-06-08T11:21:51.424963Z'})]
I'd like to repeat the exercise of writing to a CSV as linked in my other post but this time only write a subset of the columns from the API into a CSV. My first attempt here was instead of using the keys from the raw dictionary I would specify the fieldnames as a list, but then I'm having trouble accessing only the keys in the dict entry based on the list of filenames that I'm passing in.
with open('historical_orders.csv', 'w', newline='') as csvfile:
fieldnames = ['id', 'created_at', 'filled_at', 'canceled_at', 'replaced_at', 'symbol', 'asset_class', 'qty',
'filled_qty', 'filled_avg_price', 'order_class', 'order_type', 'type',
'side', 'time_in_force', 'limit_price',
'stop_price', 'status', 'extended_hours', 'legs']
writer = csv.DictWriter(csvfile, fieldnames)
writer.writeheader()
for order in closed_orders:
writer.writerow(order.__dict__['_raw'].fieldnames)
I get AttributeError: 'dict' object has no attribute 'fieldnames'.
Additionally I'd like to add 1 more column that strips out the funky "created_at" value to a date and time. So instead of "created_at" = "'2020-06-05T16:16:53.307491Z'", I'd like to create a column date "'2020-06-05" and time "'16:16:53". I was thinking I could do this by adding a loop in each write row to write one field a time, but wasn't sure if there was a better way.
Can someone help me with these 2 issues?
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
In my python code , I get strings from the text file like :
a = "[{'index': '1', 'selected': 'true', 'length': '0', 'completedLength': '0', 'path': '', 'uris': [{'status': 'used', 'uri': 'http://www.single.com'}]}]"
b ="[{'index': '1', 'selected': 'true', 'length': '0', 'completedLength': '0', 'path': '', 'uris': [{'status': 'used', 'uri': 'http://www.mirrors.com'}, {'status': 'used', 'uri': 'http://www.mirrors2.com'}]}]"
c ="[{'index': '1', 'selected': 'true', 'length': '103674793', 'completedLength': '0', 'path': '/home/dr/Maher_Al-Muaiqly_(MP3_Quran)/002.mp3', 'uris': []}, {'index': '2', 'selected': 'true', 'length': '62043128', 'completedLength': '0', 'path': '/home/dr/Maher_Al-Muaiqly_(MP3_Quran)/004.mp3', 'uris': []}, {'index': '3', 'selected': 'true', 'length': '57914945', 'completedLength': '0', 'path': '/home/dr/Maher_Al-Muaiqly_(MP3_Quran)/003.mp3', 'uris': []}]"
I want to get the text of the value uris , the output should looks like :
a = [{'status': 'used', 'uri': 'http://www.single.com'}]
b = [{'status': 'used', 'uri': 'http://www.mirrors.com'}, {'status': 'used', 'uri': 'http://www.mirrors2.com'}]
c = [[],[],[]]
Many hours I spent in failed trials to get this result by using the string functions ,
uris = str.split('}, {')
for uri in uris :
uri = uri.split(',')
# and so on ...
but , it work so bad especially in the second case , I hope that anyone can do it by regex or any other way.
They are all python literals. You can use ast.literal_eval. No need to use regular expression.
>>> a = "[{'index': '1', 'selected': 'true', 'length': '0', 'completedLength': '0', 'path': '', 'uris': [{'status': 'used', 'uri': 'http://www.single.com'}]}]"
>>> b = "[{'index': '1', 'selected': 'true', 'length': '0', 'completedLength': '0', 'path': '', 'uris': [{'status': 'used', 'uri': 'http://www.mirrors.com'}, {'status': 'used', 'uri': 'http://www.mirrors2.com'}]}]"
>>> c = "[{'index': '1', 'selected': 'true', 'length': '103674793', 'completedLength': '0', 'path': '/home/dr/Maher_Al-Muaiqly_(MP3_Quran)/002.mp3', 'uris': []}, {'index': '2', 'selected': 'true', 'length': '62043128', 'completedLength': '0', 'path': '/home/dr/Maher_Al-Muaiqly_(MP3_Quran)/004.mp3', 'uris': []}, {'index': '3', 'selected': 'true', 'length': '57914945', 'completedLength': '0', 'path': '/home/dr/Maher_Al-Muaiqly_(MP3_Quran)/003.mp3', 'uris': []}]"
>>> import ast
>>> [x['uris'] for x in ast.literal_eval(a)]
[[{'status': 'used', 'uri': 'http://www.single.com'}]]
>>> [x['uris'] for x in ast.literal_eval(b)]
[[{'status': 'used', 'uri': 'http://www.mirrors.com'}, {'status': 'used', 'uri': 'http://www.mirrors2.com'}]]
>>> [x['uris'] for x in ast.literal_eval(c)]
[[], [], []]
in javascript you can do this
a = a.replace(/^.*uris[^[]*(\[[^\]]*\]).*$/, '\1');
if php would be this a way
$a = preg_replace('/^.*uris[^[]*(\[[^\]]*\]).*$/', '\1', $a);
edit: well I see, it wouldn't do your complete task for 'c' -.-
I have the following class setup:
class Card(object):
def __init__(self, name="", attack=0, defense=0, magic=0, shield=0, description=""):
self.name = name
self.attack = int(attack)
self.defense = int(defense)
self.magic = int(magic)
self.shield = int(shield)
self.description = description
I would like to make instances of Card using a list of dictionaries created from csv.dictreader.
Here is what the method for determining my cardList returns:
[
{'Magic': '200', 'Shield': '100', 'NameOfCard': 'Knight', 'Attack': '700', 'Defense': '400', 'Description': ''},
{'Magic': '500', 'Shield': '500', 'NameOfCard': 'Mage', 'Attack': '0', 'Defense': '0', 'Description': ''},
{'Magic': '100', 'Shield': '100', 'NameOfCard': 'Peasant', 'Attack': '100', 'Defense': '100', 'Description': ''},
{'Magic': '0', 'Shield': '0', 'NameOfCard': 'Lancer', 'Attack': '400', 'Defense': '100', 'Description': ''},
{'Magic': '100', 'Shield': '200', 'NameOfCard': 'Guardian', 'Attack': '100', 'Defense': '600', 'Description': ''},
...]
I was hoping to be able to use the 'NameOfCard' values to name the instances, and then map the values to the arguments taken by the __init__ method in the Card class.
My first thought was to do something like this:
Knight = Card(cardList[0]('NameOfCard')...)
But calling print Knight.name returns TypeError: 'dict' object is not callable.
How do I use my list of dicts to create instances of the Card class?
Use argument unpacking:
knight = Card(**dict_of_properties)
This will expand dict_of_properties into named arguments:
knight = Card(name='foo', stuff='bar')
Assuming dict_of_properties looks like:
dict_of_properties = {
'name': 'foo',
'stuff': 'bar'
}
If the argument names were the same as the dict keys then you could use:
Knight = Card(**cardList[0])
As it is you'll need to map the dict keys to the proper argument names first.