Related
I have a variable which stores below dictionary
initial_ltp =
{'s': 'ok',
'd': [{'n': 'MCX:CRUDEOIL23JANFUT',
's': 'ok',
'v': {'ch': 47.0,
'chp': 0.74,
'lp': 6377.0,
'spread': 2.0,
'ask': 6379.0,
'bid': 6377.0,
'open_price': 6330.0,
'high_price': 6393.0,
'low_price': 6305.0,
'prev_close_price': 6330.0,
'volume': 8410,
'short_name': 'CRUDEOIL23JANFUT',
'exchange': 'MCX',
'description': 'MCX:CRUDEOIL23JANFUT',
'original_name': 'MCX:CRUDEOIL23JANFUT',
'symbol': 'MCX:CRUDEOIL23JANFUT',
'fyToken': '1120230119244999',
'tt': 1673481600,
'cmd': {'t': 1673518200,
'o': 6376.0,
'h': 6378.0,
'l': 6375.0,
'c': 6377.0,
'v': 19,
'tf': '15:40'}}},
{'n': 'MCX:SILVERMIC23FEBFUT',
's': 'ok',
'v': {'ch': 485.0,
'chp': 0.71,
'lp': 68543.0,
'spread': 5.0,
'ask': 68545.0,
'bid': 68540.0,
'open_price': 68200.0,
'high_price': 68689.0,
'low_price': 68200.0,
'prev_close_price': 68058.0,
'volume': 49595,
'short_name': 'SILVERMIC23FEBFUT',
'exchange': 'MCX',
'description': 'MCX:SILVERMIC23FEBFUT',
'original_name': 'MCX:SILVERMIC23FEBFUT',
'symbol': 'MCX:SILVERMIC23FEBFUT',
'fyToken': '1120230228242738',
'tt': 1673481600,
'cmd': {'t': 1673518200,
'o': 68525.0,
'h': 68543.0,
'l': 68524.0,
'c': 68543.0,
'v': 140,
'tf': '15:40'}}}]}
I am trying to collect ('n') and ('lp') and save it in a different dictionary using code:
if 'd' in initial_ltp.keys():
ltp[initial_ltp['d'][0]['n']] = initial_ltp['d'][0]['v']['lp']
But it is only taking the first n and lp
ltp
{'MCX:CRUDEOIL23JANFUT': 6377.0}
My expected output:
ltp
{'MCX:CRUDEOIL23JANFUT': 6377.0, 'MCX:SILVERMIC23FEBFUT': 68543.0}
How can I get both the values
You have to loop over the list. Using ltp[initial_ltp['d'][0] will just extract for the first element of the list.
Here is an example:
results = {}
for doc in initial_ltp["d"]:
results[doc["n"]] = doc["v"]["lp"]
print(results)
Use the following approach with dict comprehension:
ltp = {d['n']:d['v']['lp'] for d in initial_ltp['d']} if 'd' in initial_ltp else {}
{'MCX:CRUDEOIL23JANFUT': 6377.0, 'MCX:SILVERMIC23FEBFUT': 68543.0}
when you use the "=" operator, it replaces your value in the dictionary key.
you want to add keys to your dictionary so I suggest using this:
if 'd' in initial_ltp.keys():
for o in initial_ltp['d']:
if n in o:
ltp[initial_ltp['d'][0]['n']].append(initial_ltp['d'][0]['v']
['lp']
It's because you selected only first item in the 'd'.
try a loop like this:
ltp={}
if 'd' in initial_ltp.keys():
for i in range(len(initial_ltp['d'])):
ltp[initial_ltp['d'][i]['n']] = initial_ltp['d'][i]['v']['lp']
print (ltp)
Output:
{'MCX:CRUDEOIL23JANFUT': 6377.0, 'MCX:SILVERMIC23FEBFUT': 68543.0}
to start I am very new to python (and coding period) so I apologize if I am going about this the wrong way.
I am receiving these json messages from a websocket stream:
import websocket, json, dateutil.parser
import dateparser
import csv
import itertools
current_tick = None
dataframe = []
symbols = 'AAPL', 'FB', 'AMZN', 'NFLX'
def on_open(ws):
print("opened")
auth_data = {
"action": "auth",
"params": 'APIKEY'
}
ws.send(json.dumps(auth_data))
for s in symbols:
channel_data = {
"action": "subscribe",
"params": s
}
ws.send(json.dumps(channel_data))
def on_message(ws, message):
global current_tick, dataframe
current_tick = json.loads(message)
print(current_tick)
if current_tick[0]['ev'] == 'T':
dataframe.append(current_tick)
def on_close(ws):
print("closed connection")
socket = "WEBSITE"
ws = websocket.WebSocketApp(socket, on_open=on_open, on_message=on_message, on_close=on_close)
#ws.run_forever()
OUTPUTS:
[{'ev': 'T', 'sym': 'AAPL', 'i': '227161', 'x': 4, 'p': 134.605, 's': 100, 't': 1609275343055, 'z': 3}]
[{'ev': 'T', 'sym': 'AAPL', 'i': '227162', 'x': 4, 'p': 134.605, 's': 3, 'c': [37], 't': 1609275343072, 'z': 3}]
[{'ev': 'T', 'sym': 'AAPL', 'i': '155273', 'x': 12, 'p': 134.6, 's': 25, 'c': [14, 37, 41], 't': 1609275343104, 'z': 3}]
[{'ev': 'T', 'sym': 'FB', 'i': '47501', 'x': 4, 'p': 276.5, 's': 1, 'c': [12, 37], 't': 1609276352067, 'z': 3}]
[{'ev': 'T', 'sym': 'NFLX', 'i': '10420', 'x': 11, 'p': 531.5, 's': 147, 'c': [14, 12, 41], 't': 1609276352376, 'z': 3}]
My goal is to to store these into SQL database as they come in, this is what I have so far
import psycopg2
postgresConnection = psycopg2.connect(
host='HOST',
user='USER',
password='PASSWORD',
database='DATABASE'
cursor= postgresConnection.cursor()
sqlCreateTable = "create table datas(sym varchar(256), price int, size int, exchange int, time int);"
cursor.execute(sqlCreateTable)
postgresConnection.commit()
sqlpoptable = "intert into datas(sym, price, size) VALUES(%(sym)s, %(price)s, %(size)s)", {"sym":current_tick['sym'],"price":current_tick['p'], "size":current_tick['s']}
cursor.execute(sqlpoptable)
postgresConnection.commit()
And my output is:
TypeError: list indices must be integers or slices, not str
Any thoughts? Thanks in advance.
As you may already know, you can access array data using an index. An index is a number that represents the data's position in the array, starting with the number zero.
If you have an array of items like this:
items = ['a', 'z', 'c', 'd']
Then the index of 'a' would be 0, the index of 'z' would be 1, the index of 'c' would be 2, and so on.
If you wrote some Python code that looked like this:
items = ['a', 'z', 'c', 'd']
myChosenItem = items[0]
Then myChosenItem would be equal to 'a'. Make sense?
The error you are getting is telling you that you tried to provide something other than an integer (a whole number) as an index when getting data out of an array.
That would be like writing this:
items = ['a', 'z', 'c', 'd']
myChosenItem = items['chicken']
Python would have no idea what you meant. Look at your code and figure out where you are accessing array data using an index, and then figure out if that index is an integer or not. Then you should be able to get this error to go away.
I have a python script that pulls data out of a database. The problem is it is only pulling one item type and not the complete dataset into the JSON serialized object.
The object I'm trying to get comes from this:
STATS = ['min', 'max', 'mean','percentile1', 'percentile5', 'median', 'percentile95', 'percentile99', 'total']
The only problem is for some reason it will only ever take the first one. It that example it was 'min' if I switch the first to be 'percentile' such as this:
STATS = ['percentile1','min', 'max', 'mean',, 'percentile5', 'median', 'percentile95', 'percentile99', 'total']
Then this will load only the 'percentile' data. It is not including any of the rest. It is querying the correct data with each one but will only pass the first one to the Rickshaw.js to draw the graphs.
I am serializing the data with this:
def get_series(self, stationid, metric, monthly=True):
'''
Format results into json-ready results for Rickshaw.js.
'''
allResults = {}
if monthly:
rs = self.get_monthly_report(stationid, metric)
else:
rs = self.get_daily_report(stationid, metric)
for field in STATS:
series = self.format_series(rs, field)
allResults[field] = series
return json.dumps(allResults, default=json_serial)
def format_series(self, records, field):
'''
JSON formatting helper.
'''
data = []
for record in records:
data.append({'x' : time.mktime(record['date'].timetuple()), 'y' : record[field]})
return data
If you need more of the code. I can gladly provide. Thank you!
I inserted some print commands
def get_series(self, stationid, metric, monthly=True):
'''
Format results into json-ready results for Rickshaw.js.
'''
allResults = {}
if monthly:
rs = self.get_monthly_report(stationid, metric)
else:
rs = self.get_daily_report(stationid, metric)
for field in STATS:
print "The field is"
print (field)
series = self.format_series(rs, field)
print "The Series is"
print (series)
allResults[field] = series
return json.dumps(allResults, default=json_serial)
this is what appears:
The field is
min
The Series is
[{'y': 0, 'x': 1388552400.0}, {'y': 0, 'x': 1391230800.0}, {'y': 0, 'x': 1393650000.0}, {'y': 19, 'x': 1396324800.0}, {'y': 52, 'x': 1398916800.0}, {'y': 13, 'x': 1401595200.0}, {'y': 37, 'x': 1404187200.0}, {'y': 10, 'x': 1406865600.0}, {'y': 4, 'x': 1409544000.0}, {'y': 49, 'x': 1412136000.0}, {'y': 28, 'x': 1414814400.0}, {'y': 0, 'x': 1417410000.0}, {'y': 0, 'x': 1420088400.0}, {'y': 46, 'x': 1422766800.0}, {'y': 60, 'x': 1425186000.0}, {'y': 52, 'x': 1427860800.0}, {'y': 58, 'x': 1430452800.0}, {'y': 69, 'x': 1433131200.0}, {'y': 48, 'x': 1435723200.0}, {'y': 20, 'x': 1438401600.0}, {'y': 22, 'x': 1441080000.0}, {'y': 0, 'x': 1443672000.0}, {'y': 0, 'x': 1446350400.0}, {'y': 0, 'x': 1448946000.0}, {'y': 0, 'x': 1451624400.0}, {'y': 10, 'x': 1454302800.0}, {'y': 48, 'x': 1456808400.0}, {'y': 66, 'x': 1459483200.0}, {'y': 60, 'x': 1462075200.0}, {'y': 58, 'x': 1464753600.0}, {'y': 0, 'x': 1467345600.0}, {'y': 17, 'x': 1470024000.0}, {'y': 27, 'x': 1472702400.0}, {'y': 31, 'x': 1475294400.0}, {'y': 0, 'x': 1477972800.0}, {'y': 10, 'x': 1480568400.0}, {'y': 65, 'x': 1483246800.0}]
The field is
max
The Series is
[]
The field is
mean
The Series is
[]
The field is
percentile1
The Series is
[]
The field is
percentile5
The Series is
[]
The field is
median
The Series is
[]
The field is
percentile95
The Series is
[]
The field is
percentile99
The Series is
[]
The field is
total
The Series is
[]
The return value of get_month_report is of type
<cassandra.cluster.ResultSet object at 0x7fe1a6b6e910>
so when you traverse it once it depletes. You need to turn it to a list before traversing it multiple times, by the "list" operator:
if monthly:
rs = list(self.get_monthly_report(stationid, metric))
else:
rs = list(self.get_daily_report(stationid, metric))
I have a python dictionary representing updates to be made to a mongo document:
{
'1': [{'x': 21, 'y': 37}, {'x': 12, 'y': 41}],
'2': [{'x': 17, 'y': 19}, {'x': 82, 'y': 31}],
...
...
'r2': [{'x': 21, 'y': 37}, {'x': 12, 'y': 41}]
...
...
}
Fields can be many, I have shown just '1', '2' and 'r1' for simplicity.
I want to push all the values in the various arrays to their respective fields in a document already in mongo (using a field 'dated', which I have in a variable).
For ex: If the document in the db was:
{
'1': [{'x': 47, 'y': 33}]
...
}
After update, it should become:
{
'1': [{'x':47, 'y': 33}, {'x': 21, 'y': 37}, {'x': 12, 'y': 41}]
...
}
I do not want to use $pushAll because the mongodb documentation says that its deprecated. So how do I use $push with $each (or some other operator) on every dictionary key whose value is an array (in python 2.7 using pymongo)?
EDIT:
I am finding the document using a field 'dated' which is of the form 'dd-mm-yyyy'.
If in a document a field does not exists it should be created
You can use collection.find_one_and_update in Pymongo 3.x or collection.find_and_modify(DEPRECATED).
d = {
'1': [{'x': 21, 'y': 37}, {'x': 12, 'y': 41}],
'2': [{'x': 17, 'y': 19}, {'x': 82, 'y': 31}]}
for k, v in d.items():
col.find_and_modify({k: {"$exists": True}}, {"$push": {k: {"$each": v}}})
Pymongo 3.x
for k, v in d.items():
col.find_one_and_update({k: {"$exists": True}}, {"$push": {k: {"$each": v}}})
Edit:
Use the update_many with upsert=True or update with upsert = True and multi = True perform an insert if no documents match your filter. However you will need to iterate through each key to update existing document
if col.count() > 0:
col.insert(d)
else:
# your_update
I'm trying to write a simple function that given a letter, you are returned the value of the scrabble tile. Here is what I have:
def letterPoint(letter):
letter = letter.upper()
lettersWorthOne =(['A','E','I','N','O','R','S','T'])
lettersWorthTwo = (['D','G'])
lettersWorthThree = (['B','C','M','P'])
lettersWorthFour = (['F','H','U','V','W','Y'])
lettersWorthFive = (['K'])
lettersWorthEight = (['J','X'])
lettersWorthTen = (['Q','Z'])
if letterWorthOne:
print '1'
if letterWorthTwo:
print '2'
if letterWorthThree:
print '3'
if letterWorthFour:
print '4'
if letterWorthFive:
print '5'
if letterWorthEight:
print '8'
if letterWorthTen:
print '10'
Use a dictionary. Rather than
lettersWorthTwo = (['D','G']), etc.
You would have a data structure along the lines of:
letterValues = {'D':2, 'G':2, ... }
Then a lookup for value is simply:
letterValues['D'] # returns 2 for the value of the tile
To point out why your code doesn't work, because you're not comparing your letter to the list.
#Change from this:
if letterWorthOne:
print '1'
#to this, should work
if letter in letterWorthOne:
print '1'
.....
Use a python dictionary is the way to go.
Further to the solution someone has already posted. You can also construct a more content dictionary like this:
Letters = {
'a': { 'quantity' : 9, 'value': 1},
'b': { 'quantity' : 2, 'value': 3},
'c': { 'quantity' : 2, 'value': 3},
'd': { 'quantity' : 4, 'value': 2},
'e': { 'quantity' : 12, 'value': 1},
'f': { 'quantity' : 2, 'value': 4},
'g': { 'quantity' : 3, 'value': 2},
'h': { 'quantity' : 2, 'value': 4},
'i': { 'quantity' : 9, 'value': 1},
'j': { 'quantity' : 1, 'value': 8},
'k': { 'quantity' : 1, 'value': 5},
'l': { 'quantity' : 4, 'value': 1},
'm': { 'quantity' : 2, 'value': 3},
'n': { 'quantity' : 6, 'value': 1},
'o': { 'quantity' : 8, 'value': 1},
'p': { 'quantity' : 2, 'value': 3},
'q': { 'quantity' : 1, 'value': 10},
'r': { 'quantity' : 6, 'value': 1},
's': { 'quantity' : 4, 'value': 1},
't': { 'quantity' : 6, 'value': 1},
'u': { 'quantity' : 4, 'value': 1},
'v': { 'quantity' : 2, 'value': 4},
'w': { 'quantity' : 2, 'value': 4},
'x': { 'quantity' : 1, 'value': 8},
'y': { 'quantity' : 2, 'value': 4},
'z': { 'quantity' : 1, 'value': 10},
'*': { 'quantity' : 2, 'value': 0}
}
# to get to it's "content", like this:
Letters['a']
{'quantity': 9, 'value': 1}
# you can then get its 'value' or 'quantity' in a tile bag
Letters['a']['value']
1
# if you MUST use a function, do this with above dictionary, although it's quite pointless
def letter_point(letter):
return Letters[letter.upper()]['value']
In letterPoint(), letterWorthOne and lettersWorthOne are separate variables. Each lettersWorth* variable holds a list, and you appear to want letterWorthOne to contain a boolean value (True or False) specifying whether or not letter is in the lettersWorthOne list. To determine whether a value is in a collection, use operator in.
def letterPoint(letter):
letter = letter.upper()
lettersWorthOne =(['A','E','I','N','O','R','S','T'])
lettersWorthTwo = (['D','G'])
letterWorthOne = letter in lettersWorthOne
if letterWorthOne:
print '1'
letterWorthTwo = letter in lettersWorthTwo
if letterWorthTwo:
print '2'
# rest of values omitted for brevity
print 'E worth'
letterPoint('E')
print 'D worth'
letterPoint('D')
This program produces the following output:
E worth
1
D worth
2
This explains why your existing function doesn't work. But in the long run, I'd recommend using a dictionary to hold the value and quantity of each letter, and store the values as numbers instead of strings so that you can add the values of all letters in a word.
To use the code you began with, you could consider changing the lines to:
if letter in lettersWorthOne print 1
...
This is because the data structure you're using is a list (it is surrounded by [] brackets). The way to use these lists in your function is to see if they contain the letter in the list using the code:
if <variable> in <list> print <value>
The () brackets aren't doing anything, as mentioned in another answer so you can get rid of them.
This is just to explain to you why you don't see any result in the function you've written. The other answers which suggest using a dict (dictionary) are a better approach in practice.