Develop Reference

For-Loop Execution in Python - Does the Executable Code Reset? - python

Trying to plot multiple lines on one graph using matplotlib and for loops, but the code doesn't work after the first iteration. Here's the code:
import csv
import matplotlib.pyplot as plt
r = csv.reader(open('CrimeStatebyState.csv', 'rb'))
line1 = r.next()
def crime_rate(*state):
for s in state:
orig_dict = {}
for n in range (1960,2006):
orig_dict[n] = []
for line in r:
if line[0] == s:
orig_dict[int(line[3])].append(int(line[4]))
for y in orig_dict:
orig_dict[y] = sum(orig_dict[y])
plt.plot(orig_dict.keys(), orig_dict.values(),'r')
print orig_dict.values()
print s
crime_rate("Alabama", "California", "New York")
Here's what it returns:
[39920, 38105, 41112, 44636, 53550, 55131, 61838, 65527, 71285, 75090, 85399, 86919, 84047, 91389, 107314, 125497, 139573, 136995, 147389, 159950, 190511, 191834, 182701, 162361, 155691, 158513, 173807, 181751, 188261, 190573, 198604, 219400, 217889, 204274, 206859, 206188, 205962, 211188, 200065, 192819, 202159, 192835, 200331, 201572, 201664, 197071]
Alabama
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
California
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
New York
**[[[Graph of Alabama's values]]]**
Why am I getting zeroes after the loop runs once? Is this why the other two graphs aren't showing up? Is there an issue with the sum function, the "for line in r" loop, or using *state?
Sorry if that's not enough information! Thanks to those kind/knowledgeable enough for helping.

It would appear that your csv reader is exhausted after you have processed the first state and therefore when you next call "for line in r:" on the next state there are no more lines to look at. You can confirm this by putting a print statement straight after it to see what it has to process e.g.
for line in r:
print "test" # Test print
if line[0] == s:
orig_dict[int(line[3])].append(int(line[4]))
If you re-define your csv reader within each state loop you should get your data correctly processed:
import csv
import matplotlib.pyplot as plt
def crime_rate(*state):
for s in state:
r = csv.reader(open('CrimeStatebyState.csv', 'rb'))
line1 = r.next()
orig_dict = {}
for n in range (1960,2006):
orig_dict[n] = []
for line in r:
if line[0] == s:
orig_dict[int(line[3])].append(int(line[4]))
for y in orig_dict:
orig_dict[y] = sum(orig_dict[y])
plt.plot(orig_dict.keys(), orig_dict.values(),'r')
print orig_dict.values()
print s
crime_rate("Alabama", "California", "New York")

Others have already explained the source of your error. May I suggest you use pandas for this task:
import pandas as pd
states = ["Alabama", "California", "New York"]
data = pd.read_csv('CrimeStatebyState.csv') # import data
df = data[(1996 <= data.Year) & (data.Year <= 2005)] # filter by year
pd.pivot_table(df, rows='Year', cols='State', values='Count')[states].plot()

Related

This question already has answers here:
Convert a bytes array into JSON format
(8 answers)
Closed 3 months ago.
I have a variable like this:
result = b'{"Results": {"WebServiceOutput0": [{"Label": 7.0, "f0": 0.0, "f1": 0.0, "f2": 0.0, "f3": 0.0, "f4": 0.0, "f5": 0.0, "f6": 0.0, "f7": 0.0, "f8": 0.0, "f9": 0.0, "f10": 0.0, "f11": 0.0, "f12": 0.0, "f13": 0.0, "f14": 0.0, "f15": 0.0, "f16": 0.0, "f17": 0.0, "f18": 0.0, "f19": 0.0, "f20": 0.0, "f21": 0.0, "f22": 0.0, "f23": 0.0, "f24": 0.0, "f25": 0.0, "f26": 0.0, "f27": 0.0, "f28": 0.0, "f29": 0.0, "f30": 0.0, "f31": 0.0, "f32": 0.0, "f33": 0.0, "f34": 0.0, "f35": 0.0, "f36": 0.0, "f37": 0.0, "f38": 0.0, "f39": 0.0, "f40": 0.0, "f41": 0.0, "f42": 0.0, "f43": 0.0, "f44": 0.0, "f45": 0.0, "f46": 0.0, "f47": 0.0, "f48": 0.0, "f49": 0.0, "f50": 0.0, "f51": 0.0, "f52": 0.0, "f53": 0.0, "f54": 0.0, "f55": 0.0, "f56": 0.0, "f57": 0.0, "f58": 0.0, "f59": 0.0, "f60": 0.0, "f61": 0.0, "f62": 0.0, "f63": 0.0, "f64": 0.0, "f65": 0.0, "f66": 0.0, "f67": 0.0, "f68": 0.0, "f69": 0.0, "f70": 0.0, "f71": 0.0, "f72": 0.0, "f73": 0.0, "f74": 0.0, "f75": 0.0, "f76": 0.0, "f77": 0.0, "f78": 0.0, "f79": 0.0, "f80": 0.0, "f81": 0.0, "f82": 0.0, "f83": 0.0, "f84": 0.0, "f85": 0.0, "f86": 0.0, "f87": 0.0, "f88": 0.0, "f89": 0.0, "f90": 0.0, "f91": 0.0, "f92": 0.0, "f93": 0.0, "f94": 0.0, "f95": 0.0, "f96": 0.0, "f97": 0.0, "f98": 0.0, "f99": 0.0, "f100": 0.0, "f101": 0.0, "f102": 0.0, "f103": 0.0, "f104": 0.0, "f105": 0.0, "f106": 0.0, "f107": 0.0, "f108": 0.0, "f109": 0.0, "f110": 0.0, "f111": 0.0, "f112": 0.0, "f113": 0.0, "f114": 0.0, "f115": 0.0, "f116": 0.0, "f117": 0.0, "f118": 0.0, "f119": 0.0, "f120": 0.0, "f121": 0.0, "f122": 0.0, "f123": 0.0, "f124": 0.0, "f125": 0.0, "f126": 0.0, "f127": 0.0, "f128": 0.0, "f129": 0.0, "f130": 0.0, "f131": 0.0, "f132": 0.0, "f133": 0.0, "f134": 0.0, "f135": 0.0, "f136": 0.0, "f137": 0.0, "f138": 0.0, "f139": 0.0, "f140": 0.0, "f141": 0.0, "f142": 0.0, "f143": 0.0, "f144": 0.0, "f145": 0.0, "f146": 0.0, "f147": 0.0, "f148": 0.0, "f149": 0.0, "f150": 0.0, "f151": 0.0, "f152": 0.0, "f153": 0.0, "f154": 0.0, "f155": 0.0, "f156": 0.0, "f157": 0.0, "f158": 0.0, "f159": 0.0, "f160": 0.0, "f161": 0.0, "f162": 0.0, "f163": 0.0, "f164": 0.0, "f165": 0.0, "f166": 0.0, "f167": 0.0, "f168": 0.0, "f169": 0.0, "f170": 0.0, "f171": 0.0, "f172": 0.0, "f173": 0.0, "f174": 0.0, "f175": 0.0, "f176": 0.0, "f177": 0.0, "f178": 0.0, "f179": 0.0, "f180": 0.0, "f181": 0.0, "f182": 0.0, "f183": 0.0, "f184": 0.0, "f185": 0.0, "f186": 0.0, "f187": 0.0, "f188": 0.0, "f189": 0.0, "f190": 0.0, "f191": 0.0, "f192": 0.0, "f193": 0.0, "f194": 0.0, "f195": 0.0, "f196": 0.0, "f197": 0.0, "f198": 0.0, "f199": 0.0, "f200": 0.0, "f201": 0.0, "f202": 84.0, "f203": 185.0, "f204": 159.0, "f205": 151.0, "f206": 60.0, "f207": 36.0, "f208": 0.0, "f209": 0.0, "f210": 0.0, "f211": 0.0, "f212": 0.0, "f213": 0.0, "f214": 0.0, "f215": 0.0, "f216": 0.0, "f217": 0.0, "f218": 0.0, "f219": 0.0, "f220": 0.0, "f221": 0.0, "f222": 0.0, "f223": 0.0, "f224": 0.0, "f225": 0.0, "f226": 0.0, "f227": 0.0, "f228": 0.0, "f229": 0.0, "f230": 222.0, "f231": 254.0, "f232": 254.0, "f233": 254.0, "f234": 254.0, "f235": 241.0, "f236": 198.0, "f237": 198.0, "f238": 198.0, "f239": 198.0, "f240": 198.0, "f241": 198.0, "f242": 198.0, "f243": 198.0, "f244": 170.0, "f245": 52.0, "f246": 0.0, "f247": 0.0, "f248": 0.0, "f249": 0.0, "f250": 0.0, "f251": 0.0, "f252": 0.0, "f253": 0.0, "f254": 0.0, "f255": 0.0, "f256": 0.0, "f257": 0.0, "f258": 67.0, "f259": 114.0, "f260": 72.0, "f261": 114.0, "f262": 163.0, "f263": 227.0, "f264": 254.0, "f265": 225.0, "f266": 254.0, "f267": 254.0, "f268": 254.0, "f269": 250.0, "f270": 229.0, "f271": 254.0, "f272": 254.0, "f273": 140.0, "f274": 0.0, "f275": 0.0, "f276": 0.0, "f277": 0.0, "f278": 0.0, "f279": 0.0, "f280": 0.0, "f281": 0.0, "f282": 0.0, "f283": 0.0, "f284": 0.0, "f285": 0.0, "f286": 0.0, "f287": 0.0, "f288": 0.0, "f289": 0.0, "f290": 0.0, "f291": 17.0, "f292": 66.0, "f293": 14.0, "f294": 67.0, "f295": 67.0, "f296": 67.0, "f297": 59.0, "f298": 21.0, "f299": 236.0, "f300": 254.0, "f301": 106.0, "f302": 0.0, "f303": 0.0, "f304": 0.0, "f305": 0.0, "f306": 0.0, "f307": 0.0, "f308": 0.0, "f309": 0.0, "f310": 0.0, "f311": 0.0, "f312": 0.0, "f313": 0.0, "f314": 0.0, "f315": 0.0, "f316": 0.0, "f317": 0.0, "f318": 0.0, "f319": 0.0, "f320": 0.0, "f321": 0.0, "f322": 0.0, "f323": 0.0, "f324": 0.0, "f325": 0.0, "f326": 83.0, "f327": 253.0, "f328": 209.0, "f329": 18.0, "f330": 0.0, "f331": 0.0, "f332": 0.0, "f333": 0.0, "f334": 0.0, "f335": 0.0, "f336": 0.0, "f337": 0.0, "f338": 0.0, "f339": 0.0, "f340": 0.0, "f341": 0.0, "f342": 0.0, "f343": 0.0, "f344": 0.0, "f345": 0.0, "f346": 0.0, "f347": 0.0, "f348": 0.0, "f349": 0.0, "f350": 0.0, "f351": 0.0, "f352": 0.0, "f353": 22.0, "f354": 233.0, "f355": 255.0, "f356": 83.0, "f357": 0.0, "f358": 0.0, "f359": 0.0, "f360": 0.0, "f361": 0.0, "f362": 0.0, "f363": 0.0, "f364": 0.0, "f365": 0.0, "f366": 0.0, "f367": 0.0, "f368": 0.0, "f369": 0.0, "f370": 0.0, "f371": 0.0, "f372": 0.0, "f373": 0.0, "f374": 0.0, "f375": 0.0, "f376": 0.0, "f377": 0.0, "f378": 0.0, "f379": 0.0, "f380": 0.0, "f381": 129.0, "f382": 254.0, "f383": 238.0, "f384": 44.0, "f385": 0.0, "f386": 0.0, "f387": 0.0, "f388": 0.0, "f389": 0.0, "f390": 0.0, "f391": 0.0, "f392": 0.0, "f393": 0.0, "f394": 0.0, "f395": 0.0, "f396": 0.0, "f397": 0.0, "f398": 0.0, "f399": 0.0, "f400": 0.0, "f401": 0.0, "f402": 0.0, "f403": 0.0, "f404": 0.0, "f405": 0.0, "f406": 0.0, "f407": 0.0, "f408": 59.0, "f409": 249.0, "f410": 254.0, "f411": 62.0, "f412": 0.0, "f413": 0.0, "f414": 0.0, "f415": 0.0, "f416": 0.0, "f417": 0.0, "f418": 0.0, "f419": 0.0, "f420": 0.0, "f421": 0.0, "f422": 0.0, "f423": 0.0, "f424": 0.0, "f425": 0.0, "f426": 0.0, "f427": 0.0, "f428": 0.0, "f429": 0.0, "f430": 0.0, "f431": 0.0, "f432": 0.0, "f433": 0.0, "f434": 0.0, "f435": 0.0, "f436": 133.0, "f437": 254.0, "f438": 187.0, "f439": 5.0, "f440": 0.0, "f441": 0.0, "f442": 0.0, "f443": 0.0, "f444": 0.0, "f445": 0.0, "f446": 0.0, "f447": 0.0, "f448": 0.0, "f449": 0.0, "f450": 0.0, "f451": 0.0, "f452": 0.0, "f453": 0.0, "f454": 0.0, "f455": 0.0, "f456": 0.0, "f457": 0.0, "f458": 0.0, "f459": 0.0, "f460": 0.0, "f461": 0.0, "f462": 0.0, "f463": 9.0, "f464": 205.0, "f465": 248.0, "f466": 58.0, "f467": 0.0, "f468": 0.0, "f469": 0.0, "f470": 0.0, "f471": 0.0, "f472": 0.0, "f473": 0.0, "f474": 0.0, "f475": 0.0, "f476": 0.0, "f477": 0.0, "f478": 0.0, "f479": 0.0, "f480": 0.0, "f481": 0.0, "f482": 0.0, "f483": 0.0, "f484": 0.0, "f485": 0.0, "f486": 0.0, "f487": 0.0, "f488": 0.0, "f489": 0.0, "f490": 0.0, "f491": 126.0, "f492": 254.0, "f493": 182.0, "f494": 0.0, "f495": 0.0, "f496": 0.0, "f497": 0.0, "f498": 0.0, "f499": 0.0, "f500": 0.0, "f501": 0.0, "f502": 0.0, "f503": 0.0, "f504": 0.0, "f505": 0.0, "f506": 0.0, "f507": 0.0, "f508": 0.0, "f509": 0.0, "f510": 0.0, "f511": 0.0, "f512": 0.0, "f513": 0.0, "f514": 0.0, "f515": 0.0, "f516": 0.0, "f517": 0.0, "f518": 75.0, "f519": 251.0, "f520": 240.0, "f521": 57.0, "f522": 0.0, "f523": 0.0, "f524": 0.0, "f525": 0.0, "f526": 0.0, "f527": 0.0, "f528": 0.0, "f529": 0.0, "f530": 0.0, "f531": 0.0, "f532": 0.0, "f533": 0.0, "f534": 0.0, "f535": 0.0, "f536": 0.0, "f537": 0.0, "f538": 0.0, "f539": 0.0, "f540": 0.0, "f541": 0.0, "f542": 0.0, "f543": 0.0, "f544": 0.0, "f545": 19.0, "f546": 221.0, "f547": 254.0, "f548": 166.0, "f549": 0.0, "f550": 0.0, "f551": 0.0, "f552": 0.0, "f553": 0.0, "f554": 0.0, "f555": 0.0, "f556": 0.0, "f557": 0.0, "f558": 0.0, "f559": 0.0, "f560": 0.0, "f561": 0.0, "f562": 0.0, "f563": 0.0, "f564": 0.0, "f565": 0.0, "f566": 0.0, "f567": 0.0, "f568": 0.0, "f569": 0.0, "f570": 0.0, "f571": 0.0, "f572": 3.0, "f573": 203.0, "f574": 254.0, "f575": 219.0, "f576": 35.0, "f577": 0.0, "f578": 0.0, "f579": 0.0, "f580": 0.0, "f581": 0.0, "f582": 0.0, "f583": 0.0, "f584": 0.0, "f585": 0.0, "f586": 0.0, "f587": 0.0, "f588": 0.0, "f589": 0.0, "f590": 0.0, "f591": 0.0, "f592": 0.0, "f593": 0.0, "f594": 0.0, "f595": 0.0, "f596": 0.0, "f597": 0.0, "f598": 0.0, "f599": 0.0, "f600": 38.0, "f601": 254.0, "f602": 254.0, "f603": 77.0, "f604": 0.0, "f605": 0.0, "f606": 0.0, "f607": 0.0, "f608": 0.0, "f609": 0.0, "f610": 0.0, "f611": 0.0, "f612": 0.0, "f613": 0.0, "f614": 0.0, "f615": 0.0, "f616": 0.0, "f617": 0.0, "f618": 0.0, "f619": 0.0, "f620": 0.0, "f621": 0.0, "f622": 0.0, "f623": 0.0, "f624": 0.0, "f625": 0.0, "f626": 0.0, "f627": 31.0, "f628": 224.0, "f629": 254.0, "f630": 115.0, "f631": 1.0, "f632": 0.0, "f633": 0.0, "f634": 0.0, "f635": 0.0, "f636": 0.0, "f637": 0.0, "f638": 0.0, "f639": 0.0, "f640": 0.0, "f641": 0.0, "f642": 0.0, "f643": 0.0, "f644": 0.0, "f645": 0.0, "f646": 0.0, "f647": 0.0, "f648": 0.0, "f649": 0.0, "f650": 0.0, "f651": 0.0, "f652": 0.0, "f653": 0.0, "f654": 0.0, "f655": 133.0, "f656": 254.0, "f657": 254.0, "f658": 52.0, "f659": 0.0, "f660": 0.0, "f661": 0.0, "f662": 0.0, "f663": 0.0, "f664": 0.0, "f665": 0.0, "f666": 0.0, "f667": 0.0, "f668": 0.0, "f669": 0.0, "f670": 0.0, "f671": 0.0, "f672": 0.0, "f673": 0.0, "f674": 0.0, "f675": 0.0, "f676": 0.0, "f677": 0.0, "f678": 0.0, "f679": 0.0, "f680": 0.0, "f681": 0.0, "f682": 61.0, "f683": 242.0, "f684": 254.0, "f685": 254.0, "f686": 52.0, "f687": 0.0, "f688": 0.0, "f689": 0.0, "f690": 0.0, "f691": 0.0, "f692": 0.0, "f693": 0.0, "f694": 0.0, "f695": 0.0, "f696": 0.0, "f697": 0.0, "f698": 0.0, "f699": 0.0, "f700": 0.0, "f701": 0.0, "f702": 0.0, "f703": 0.0, "f704": 0.0, "f705": 0.0, "f706": 0.0, "f707": 0.0, "f708": 0.0, "f709": 0.0, "f710": 121.0, "f711": 254.0, "f712": 254.0, "f713": 219.0, "f714": 40.0, "f715": 0.0, "f716": 0.0, "f717": 0.0, "f718": 0.0, "f719": 0.0, "f720": 0.0, "f721": 0.0, "f722": 0.0, "f723": 0.0, "f724": 0.0, "f725": 0.0, "f726": 0.0, "f727": 0.0, "f728": 0.0, "f729": 0.0, "f730": 0.0, "f731": 0.0, "f732": 0.0, "f733": 0.0, "f734": 0.0, "f735": 0.0, "f736": 0.0, "f737": 0.0, "f738": 121.0, "f739": 254.0, "f740": 207.0, "f741": 18.0, "f742": 0.0, "f743": 0.0, "f744": 0.0, "f745": 0.0, "f746": 0.0, "f747": 0.0, "f748": 0.0, "f749": 0.0, "f750": 0.0, "f751": 0.0, "f752": 0.0, "f753": 0.0, "f754": 0.0, "f755": 0.0, "f756": 0.0, "f757": 0.0, "f758": 0.0, "f759": 0.0, "f760": 0.0, "f761": 0.0, "f762": 0.0, "f763": 0.0, "f764": 0.0, "f765": 0.0, "f766": 0.0, "f767": 0.0, "f768": 0.0, "f769": 0.0, "f770": 0.0, "f771": 0.0, "f772": 0.0, "f773": 0.0, "f774": 0.0, "f775": 0.0, "f776": 0.0, "f777": 0.0, "f778": 0.0, "f779": 0.0, "f780": 0.0, "f781": 0.0, "f782": 0.0, "f783": 0.0, "Scored Probabilities_0": 1.7306872933250431e-07, "Scored Probabilities_1": 3.3177526751193424e-09, "Scored Probabilities_2": 6.772526557729492e-07, "Scored Probabilities_3": 5.018638683008445e-05, "Scored Probabilities_4": 3.5781842069911e-11, "Scored Probabilities_5": 4.2981825008019914e-08, "Scored Probabilities_6": 6.350046754243676e-14, "Scored Probabilities_7": 0.9999483485221952, "Scored Probabilities_8": 1.3431149602373933e-07, "Scored Probabilities_9": 4.341226706439234e-07, "Scored Labels": 7.0}]}}'
This is the first time I've seen this type of object. How can I get the value of Scored Probabilities_n (n is from 0 to 9) and the Scored Labels? Furthermore, can I get the maximum value (in this case it should be "Scored Probabilities_7": 0.9999483485221952)?
Even when I tried some ways to get the highest value of the variable but it's somehow difficult:
print(max(result))
125
As I can see, there's no 125 value in the variable. So does it mean the normal calculation can not be done on this too? I expected to see the answer on my question above. Thank you!

You can use:
import json
d = json.loads(result.decode('utf-8'))['Results']['WebServiceOutput0'][0]
d['Scored Labels']
# 1.7306872933250431e-07
max_p = max((k for k in d if k.startswith('Scored Probabilities_')), key=d.get)
# 'Scored Probabilities_7'
d[max_p]
# 0.9999483485221952

I have variables as follows:
J = range (1,16)
T = range (1,9)
x= {} # 0,1 decision variable to be determined
These variables turn into combinations of x[j,t].
I am trying to implement a constraint for unacceptable t types in T for x[j,t] combinations that make the x var = 0.
I have a dictionary 'U' with j's as the key and t types and values stored in a list. Zero value means t is unacceptable, 1 is acceptable. The index is range 1-9, not 0-8. So in the example below, j 2, type 3 (bc its at index 3 on range(1,9)) is the only acceptable value.
{1: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
2: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
3: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
4: [1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
5: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
6: [1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
7: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
8: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
9: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
10: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
11: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
12: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
13: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
14: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0],
15: [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0]
}
I am struggling in trying to get the x[j,t] combinations bc of the misaligned index. I set it up like so:
for j,t in x:
if t in U[j]==0:
# do the thing... #addConstr(x[j,t], GRB.EQUAL,0)
So for j 2 the results I need are {(2,1):0, (2,2):0, (2,4):0, (2,5):0, (2,6):0, (2,7):0, (2,8):0} where the index value on range (1,9) becomes the t value in the tupledict.
Any pointers? Thank you!

Assuming your example data is stored in U, what you want to do is:
j_results = []
for j,types in U.items():
results = {}
for t in types:
if t == 0.0:
result[(j,int(t))] = 0
j_results.append(result)
j_results list will contain all results like you described:
for j 2 the results I need are {(2,1):0, (2,2):0, (2,4):0, (2,5):0, (2,6):0, (2,7):0, (2,8):0}
will be in j_result[1] (counter intuitive because your U data start from 1)
Note the int cast, because data you provided has floats, but results you provided are a tuple of ints.

apologies I know this probably a simple question but I'm new to this! I made all these calculations from a csv file and now I'm trying to make the same calculation through every file for the whole month. In the first class,def get_holdings_info(d):, I'm working through my calculations. In the second class, def get_holdings_info_array(): I'm trying to pass the first class through every day of the month. I think my code is just about right, but for some reason I'm getting a return of 0's. Thanks in advance if you can help!
holdings_darray = ['01-03-2020','01-06-2020','01-07-2020','01-08-2020','01-09-2020','01-11-2020','01-14-2020','01-15-2020','01-17-2020','01-21-2020','01-22-2020','01-23-2020',
'01-24-2020','01-27-2020','01-28-2020','01-29-2020','01-30-2020','01-31-2020','02-04-2020']
account_names = ["CATALYST EXCEED DEFINED SHIELD FUND", "SCA/IB FBO CAT EXCEED DEF SHIELD FD"]
bond_name = ["Bond Paying Periodic Income"]
money_market_name = ["Money Market Fund"]
mutual_fund_name = ["Mutual Fund"]
def get_holdings_info(d):
sbhmv = 0
sbhbv = 0
sbhs = 0
setfhmv = 0
setthbv = 0
setfhs = 0
smmhmv = 0
smmhbv = 0
smmhs = 0
holdings_file = 'holdings/Holdings As Of ' + d + '.csv'
df = pd.read_csv(holdings_file, header=1)
account_names = ["Fund_1", "Fund_1"]
bond_name = ["Bond Paying Periodic Income"]
money_market_name = ["Money Market Fund"]
mutual_fund_name = ["Mutual Fund"]
sbh = df[df["Account Name"].isin(account_names) & df["Security Type Name"].isin(bond_name)]
sbhmv = sbh['Market Value'].sum()
sbhbv = sbh['Book Value'].sum()
sbhs = sbh['Shares'].sum()
setfh = df[df["Account Name"].isin(account_names) & df["Security Type Name"].isin(mutual_fund_name)]
setfhmv = setfh['Market Value'].sum()
setthbv = setfh['Book Value'].sum()
setfhs = setfh['Shares'].sum()
smmh = df[df["Account Name"].isin(account_names) & df["Security Type Name"].isin(money_market_name)]
smmhmv = smmh['Market Value'].sum()
smmhbv = smmh['Book Value'].sum()
smmhs = smmh['Shares'].sum()
return sbhmv, sbhbv, sbhs, setfhmv, setthbv, setfhs, smmhmv, smmhbv, smmhs
def get_holdings_info_array():
c = []
for f in holdings_darray:
c.append(get_holdings_info(f))
return(c)
print(get_holdings_info_array())
[(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0)]

What you are in need here, is of tuples. This SO question and it's answers pretty much cover every variable.
Your code, while syntactically correct, semantically is wrong. A python function finishes it's execution after encountering a return statement.
Your function get_holdings_info finishes after reaching this line
return sbhmv
and never executes the rest.
If you want to return all those values you can refer to the question I mention for all possibilities, but the simpler option would be to do:
return sbhmv, sbhbv, sbhs, setfh, setfhmv, setthbv, setfhs, smmh, smmhmv, smmhbv, smmhs
In which you return a tuple with all the values you need.

I am trying to solve a simple quadratic program using CVXOPT and am troubled by the fact that I can guess a feasible solution better than the optimum provided by the solver. The optimisation is of the form:
I will provide the definitions of P,q,G,h,A, and b at the end. When I import and run:
from cvxopt import matrix, spmatrix, solvers
# Code that creates matrices goes here
sol = solvers.qp(P, q, G, h, A, b)
the result is:
pcost dcost gap pres dres
0: 0.0000e+00 -5.5000e+00 6e+00 6e-17 4e+00
1: 0.0000e+00 -5.5000e-02 6e-02 1e-16 4e-02
2: 0.0000e+00 -5.5000e-04 6e-04 3e-16 4e-04
3: 0.0000e+00 -5.5000e-06 6e-06 1e-16 4e-06
4: 0.0000e+00 -5.5000e-08 6e-08 1e-16 4e-08
Optimal solution found.
Objective = 0.0
However I can define a different solution guessed_solution that is feasible and further minimises the objective:
guessed_solution = matrix([0.5,0.5,0.0,0.0,0.0,0.0,0.5,0.5,0.0,0.0,1.0])
# Check Ax = b; want to see zeroes
print(A * guessed_solution - b)
>>>
[ 0.00e+00]
[ 0.00e+00]
[ 2.78e-17]
# Check Gx <= h; want to see non-positive entries
print(G * guessed_solution - h)
>>>
[-5.00e-01]
[-5.00e-01]
[ 0.00e+00]
[ 0.00e+00]
[ 0.00e+00]
[ 0.00e+00]
[-5.00e-01]
[-5.00e-01]
[-1.00e+00]
[-1.00e+00]
[ 0.00e+00]
[ 0.00e+00]
[ 0.00e+00]
[-1.00e+00]
# Check objective
print(guessed_solution.T * P * guessed_solution + q.T * guessed_solution)
>>>[-6.67e-01]
This results in an objective that is -2/3, clearly less than 0. I presume that the 2.78e-17 error in the Ax=b test is not relevant.
Any help on resolving this would be appreciated! And below is the definition of relevant matrices in code (biggest matrix is 11 by 11).
P = matrix([[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0],[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0/3.0, 0.0, 2.0/3.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0/3.0, 2.0/3.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0],[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0/3.0, 0.0, -2.0/3.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0/3.0, -2.0/3.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]).T
q = matrix([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0])
A = matrix([[0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0],[1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],[0.0, 1.0, 1.0/3.0, 2.0/3.0, 0.0, -1.0, -1.0/3.0, -2.0/3.0, 0.0, 0.0, 0.0]]).T
b = matrix([1.0, 1.0, 0.0])
G = spmatrix([-1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, 1.0, 1.0, 1.0, -1.0, -1.0, -1.0], [0,1,2,3,4,5,6,7,8,9,10,11,12,13], [0,1,2,3,4,5,6,7,8,9,10,8,9,10])
h = matrix([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0])

Your quadratic form is not valid in regards to the assumptions.
It needs to be PSD (and symmetric).
Making it symmetric:
P = (P + P.T) / 2
will lead to cvxopt showing an error, which is due to P being indefinite:
import numpy as np
np_matrix = np.array(P)
print(np.linalg.eigvalsh(np_matrix))
#[-8.16496581e-01 -7.45355992e-01 -5.77350269e-01 -2.40008780e-16 -6.33511351e-17 -4.59089160e-17 -3.94415555e-22 5.54077304e-17 5.77350269e-01 7.45355992e-01 8.16496581e-01]
You got a solver designed for convex-optimization problems (if and only if P is PSD) feeded by some non-convex optimization problem. This won't work (in general).

I would like to create a json file viewer in order to iterate through the key/values of the file. For that I decided to go with a qtreewidgetitem where in the first column I would have the key, in the second column the value and in a third column I would like actually to display the values of an numpy array as an image.
For the moment I have written the following script:
# Std
import argparse
import collections
import json
import sys
import numpy as np
# External
from PyQt5 import QtCore
from PyQt5 import QtGui
from PyQt5 import QtWidgets
from PyQt5.QtWidgets import QMainWindow, QApplication, QLabel, QFileDialog, QAction
from PyQt5.QtGui import QIcon, QPixmap
class TextToTreeItem:
def __init__(self):
self.text_list = []
self.titem_list = []
def append(self, text_list, titem):
for text in text_list:
self.text_list.append(text)
self.titem_list.append(titem)
# Return model indices that match string
def find(self, find_str):
titem_list = []
for i, s in enumerate(self.text_list):
if find_str in s:
titem_list.append(self.titem_list[i])
return titem_list
class JsonView(QtWidgets.QWidget):
def __init__(self, fpath):
super(JsonView, self).__init__()
self.find_box = None
self.tree_widget = None
self.text_to_titem = TextToTreeItem()
self.find_str = ""
self.found_titem_list = []
self.found_idx = 0
self.box_name = ""
jfile = open(fpath)
jdata = json.load(jfile, object_pairs_hook=collections.OrderedDict)
box_names = [*self.gen_dict_extract("box_name", jdata)]
result = len(set(box_names)) == 1
if result:
self.box_name = box_names[0]
# Find UI
find_layout = self.make_find_ui()
# Tree
self.tree_widget = QtWidgets.QTreeWidget()
self.tree_widget.setHeaderLabels(["Key", "Value", "Image"])
# self.tree_widget.header().setSectionResizeMode(QtWidgets.QHeaderView.Stretch)
# self.tree_widget.header().setSectionResizeMode(QtWidgets.QHeaderView.ResizeToContents)
root_item = QtWidgets.QTreeWidgetItem([self.box_name])
self.recurse_jdata(jdata, root_item)
self.tree_widget.addTopLevelItem(root_item)
self.tree_widget.expandToDepth(0)
# Add table to layout
layout = QtWidgets.QHBoxLayout()
layout.addWidget(self.tree_widget)
# Group box
gbox = QtWidgets.QGroupBox(fpath)
gbox.setLayout(layout)
layout2 = QtWidgets.QVBoxLayout()
layout2.addLayout(find_layout)
layout2.addWidget(gbox)
self.setLayout(layout2)
def gen_dict_extract(self, key, var):
if isinstance(var,dict) and hasattr(var, 'items'):
for k, v in var.items():
if k == key:
yield v
if isinstance(v, dict):
for result in self.gen_dict_extract(key, v):
yield result
elif isinstance(v, list):
for d in v:
for result in self.gen_dict_extract(key, d):
yield result
elif isinstance(var,list):
for v in var:
if isinstance(v, dict):
for result in self.gen_dict_extract(key, v):
yield result
elif isinstance(v, list):
for d in v:
for result in self.gen_dict_extract(key, d):
yield result
def make_find_ui(self):
# Text box
self.find_box = QtWidgets.QLineEdit()
self.find_box.returnPressed.connect(self.find_button_clicked)
# Find Button
find_button = QtWidgets.QPushButton("Find")
find_button.clicked.connect(self.find_button_clicked)
layout = QtWidgets.QHBoxLayout()
layout.addWidget(self.find_box)
layout.addWidget(find_button)
return layout
def find_button_clicked(self):
find_str = self.find_box.text()
# Very common for use to click Find on empty string
if find_str == "":
return
# New search string
if find_str != self.find_str:
self.find_str = find_str
self.found_titem_list = self.text_to_titem.find(self.find_str)
self.found_idx = 0
else:
item_num = len(self.found_titem_list)
self.found_idx = (self.found_idx + 1) % item_num
self.tree_widget.setCurrentItem(self.found_titem_list[self.found_idx])
def recurse_jdata(self, jdata, tree_widget):
if isinstance(jdata, dict):
for key, val in jdata.items():
self.tree_add_row(key, val, tree_widget)
elif isinstance(jdata, list):
for i, val in enumerate(jdata):
key = str(i)
if isinstance(jdata[i],dict) and 'sens_comp' in jdata[i]:
key = jdata[i]['sens_comp']
self.tree_add_row(key, val, tree_widget)
else:
print("This should never be reached!")
def tree_add_row(self, key, val, tree_widget):
text_list = []
if isinstance(val,list) and key=='sensorData':
text_list.append(key)
val = np.array(val, dtype=object)
# if len(val.shape) != 1:
# val = val.reshape(24,32)
if len(val.shape) == 1:
val = val.reshape(-1,8)
text_list.append(str(val))
row_item = QtWidgets.QTreeWidgetItem([key, str(val)])
elif isinstance(val, dict) or isinstance(val, list):
text_list.append(key)
row_item = QtWidgets.QTreeWidgetItem([key])
self.recurse_jdata(val, row_item)
else:
text_list.append(key)
text_list.append(str(val))
row_item = QtWidgets.QTreeWidgetItem([key, str(val)])
tree_widget.addChild(row_item)
self.text_to_titem.append(text_list, row_item)
class JsonViewer(QtWidgets.QMainWindow):
def __init__(self):
super(JsonViewer, self).__init__()
self.menu_bar()
if len(sys.argv) > 1:
fpath = sys.argv[1]
json_view = JsonView(fpath)
self.setCentralWidget(json_view)
self.resize(640, 480)
self.setWindowTitle("JSON Viewer")
self.show()
def menu_bar (self) :
# filling up a menu bar
bar = self.menuBar()
# File menu
file_menu = bar.addMenu('File')
# adding actions to file menu
open_action = QtWidgets.QAction('Open', self)
close_action = QtWidgets.QAction('Close', self)
file_menu.addAction(open_action)
file_menu.addAction(close_action)
open_action.triggered.connect(self.openPath)
# use `connect` method to bind signals to desired behavior
close_action.triggered.connect(self.close)
def openPath(self):
fpath, _ = QFileDialog.getOpenFileName()
# fpath = sys.argv[1]
json_view = JsonView(fpath)
self.setCentralWidget(json_view)
def keyPressEvent(self, e):
if e.key() == QtCore.Qt.Key_Escape:
self.close()
def main():
qt_app = QtWidgets.QApplication(sys.argv)
json_viewer = JsonViewer()
sys.exit(qt_app.exec_())
if "__main__" == __name__:
main()
and I am trying to visualize the following information (actually all the information is provided as a single line):
[{"aquisTime": "11-18-25-723721", "box_version": "001", "sens_board": "54906430-100", "MACaddr_eth0": "xx:xx:xx:xx:xx:xx", "sensorData": [[0.0, 27.247400283813477, 0.0, 28.69390296936035, 0.0, 28.078027725219727, 0.0, 28.03997230529785, 0.0, 24.897180557250977, 0.0, 24.499963760375977, 0.0, 23.3978214263916, 0.0, 23.5650577545166, 0.0, 23.758142471313477, 0.0, 23.5541934967041, 0.0, 23.370447158813477, 0.0, 24.9819278717041, 0.0, 25.295557022094727, 0.0, 24.3200626373291, 0.0, 24.304040908813477, 0.0, 22.6225528717041], [26.393274307250977, 0.0, 28.582483291625977, 0.0, 27.909997940063477, 0.0, 28.392236709594727, 0.0, 28.00481605529785, 0.0, 25.388696670532227, 0.0, 24.007410049438477, 0.0, 23.434412002563477, 0.0, 23.327539443969727, 0.0, 24.347131729125977, 0.0, 24.070825576782227, 0.0, 24.752283096313477, 0.0, 27.674463272094727, 0.0, 26.057825088500977, 0.0, 24.291162490844727, 0.0, 25.0914249420166, 0.0], [0.0, 26.8185977935791, 0.0, 26.6101016998291, 0.0, 27.82305335998535, 0.0, 28.930749893188477, 0.0, 28.6760196685791, 0.0, 27.718469619750977, 0.0, 24.382287979125977, 0.0, 23.519739151000977, 0.0, 23.821985244750977, 0.0, 23.407678604125977, 0.0, 24.223718643188477, 0.0, 24.299219131469727, 0.0, 26.3620548248291, 0.0, 27.2630558013916, 0.0, 25.204309463500977, 0.0, 24.8760929107666], [24.9536075592041, 0.0, 26.29070472717285, 0.0, 27.222864151000977, 0.0, 29.23284339904785, 0.0, 29.2844181060791, 0.0, 28.030908584594727, 0.0, 26.043237686157227, 0.0, 23.98674964904785, 0.0, 23.5355167388916, 0.0, 23.86419105529785, 0.0, 23.602441787719727, 0.0, 25.0830020904541, 0.0, 25.52703285217285, 0.0, 28.371667861938477, 0.0, 24.92656898498535, 0.0, 26.448205947875977, 0.0], [0.0, 24.1496524810791, 0.0, 25.244165420532227, 0.0, 28.82390785217285, 0.0, 29.858118057250977, 0.0, 28.943506240844727, 0.0, 27.1224308013916, 0.0, 24.10454750061035, 0.0, 23.904870986938477, 0.0, 23.963159561157227, 0.0, 24.309412002563477, 0.0, 24.791040420532227, 0.0, 24.958215713500977, 0.0, 24.861841201782227, 0.0, 25.5935001373291, 0.0, 26.398340225219727, 0.0, 26.2838077545166], [24.494470596313477, 0.0, 24.91155433654785, 0.0, 25.95366859436035, 0.0, 29.70025062561035, 0.0, 29.372522354125977, 0.0, 28.098535537719727, 0.0, 23.87615394592285, 0.0, 23.5194034576416, 0.0, 23.702905654907227, 0.0, 24.23784828186035, 0.0, 24.3256778717041, 0.0, 25.501184463500977, 0.0, 23.957300186157227, 0.0, 25.55767250061035, 0.0, 26.024988174438477, 0.0, 27.061059951782227, 0.0], [0.0, 24.3924503326416, 0.0, 24.31902503967285, 0.0, 23.994043350219727, 0.0, 25.83245277404785, 0.0, 27.86760902404785, 0.0, 25.213464736938477, 0.0, 23.697229385375977, 0.0, 24.060327529907227, 0.0, 23.775232315063477, 0.0, 24.02068519592285, 0.0, 24.35552406311035, 0.0, 24.200159072875977, 0.0, 24.88030433654785, 0.0, 25.765466690063477, 0.0, 28.722192764282227, 0.0, 27.34758949279785], [23.411401748657227, 0.0, 24.815515518188477, 0.0, 23.414087295532227, 0.0, 24.96453285217285, 0.0, 26.157678604125977, 0.0, 26.347681045532227, 0.0, 23.301172256469727, 0.0, 24.099145889282227, 0.0, 23.26299476623535, 0.0, 24.131616592407227, 0.0, 24.30022621154785, 0.0, 24.39019203186035, 0.0, 24.499414443969727, 0.0, 25.25078773498535, 0.0, 26.4195499420166, 0.0, 27.547204971313477, 0.0], [0.0, 24.554773330688477, 0.0, 24.415918350219727, 0.0, 24.222986221313477, 0.0, 24.6810245513916, 0.0, 23.302209854125977, 0.0, 23.7110538482666, 0.0, 23.354944229125977, 0.0, 23.07952308654785, 0.0, 23.18279457092285, 0.0, 23.99627113342285, 0.0, 24.1441593170166, 0.0, 24.74578285217285, 0.0, 24.0407657623291, 0.0, 25.646997451782227, 0.0, 28.602624893188477, 0.0, 28.41668128967285], [24.0495548248291, 0.0, 24.94378089904785, 0.0, 23.739587783813477, 0.0, 24.223840713500977, 0.0, 23.817956924438477, 0.0, 24.2690372467041, 0.0, 23.297449111938477, 0.0, 23.4769229888916, 0.0, 22.755701065063477, 0.0, 23.3546085357666, 0.0, 23.5649356842041, 0.0, 24.7673282623291, 0.0, 24.238183975219727, 0.0, 25.415918350219727, 0.0, 28.852258682250977, 0.0, 31.557764053344727, 0.0], [0.0, 25.1708927154541, 0.0, 25.2442569732666, 0.0, 24.75335121154785, 0.0, 24.432580947875977, 0.0, 24.0013370513916, 0.0, 23.95977210998535, 0.0, 24.051660537719727, 0.0, 23.533716201782227, 0.0, 23.4976749420166, 0.0, 23.9699649810791, 0.0, 23.509180068969727, 0.0, 24.4299259185791, 0.0, 23.87346839904785, 0.0, 26.298608779907227, 0.0, 29.81133460998535, 0.0, 32.801963806152344], [24.9686222076416, 0.0, 25.230737686157227, 0.0, 24.12090492248535, 0.0, 24.8330020904541, 0.0, 24.093530654907227, 0.0, 24.13250160217285, 0.0, 23.650781631469727, 0.0, 23.43535804748535, 0.0, 23.201745986938477, 0.0, 24.51177406311035, 0.0, 24.09893226623535, 0.0, 24.20073890686035, 0.0, 24.4957218170166, 0.0, 25.689172744750977, 0.0, 27.99920082092285, 0.0, 33.069358825683594, 0.0], [0.0, 24.984277725219727, 0.0, 24.74541664123535, 0.0, 24.881860733032227, 0.0, 24.8862247467041, 0.0, 24.106897354125977, 0.0, 23.9283390045166, 0.0, 23.67705726623535, 0.0, 23.661523818969727, 0.0, 23.3705997467041, 0.0, 24.223840713500977, 0.0, 23.91411781311035, 0.0, 24.14556312561035, 0.0, 24.218774795532227, 0.0, 24.572351455688477, 0.0, 26.31304359436035, 0.0, 29.320215225219727], [24.922021865844727, 0.0, 25.5465030670166, 0.0, 25.181848526000977, 0.0, 25.0328311920166, 0.0, 24.540246963500977, 0.0, 24.561731338500977, 0.0, 23.783044815063477, 0.0, 24.41777992248535, 0.0, 23.4245548248291, 0.0, 24.115137100219727, 0.0, 23.5469913482666, 0.0, 24.30474281311035, 0.0, 23.915369033813477, 0.0, 24.704370498657227, 0.0, 24.678918838500977, 0.0, 26.7276554107666, 0.0], [0.0, 25.0797061920166, 0.0, 25.60723304748535, 0.0, 25.208887100219727, 0.0, 24.6856632232666, 0.0, 24.829370498657227, 0.0, 24.5165958404541, 0.0, 24.271814346313477, 0.0, 24.789209365844727, 0.0, 23.733179092407227, 0.0, 23.677026748657227, 0.0, 23.490137100219727, 0.0, 24.016809463500977, 0.0, 24.5273380279541, 0.0, 24.58843421936035, 0.0, 25.003686904907227, 0.0, 24.922754287719727], [25.56633949279785, 0.0, 28.21111488342285, 0.0, 24.992090225219727, 0.0, 24.725061416625977, 0.0, 25.182947158813477, 0.0, 24.303308486938477, 0.0, 24.55571937561035, 0.0, 24.9260196685791, 0.0, 24.511011123657227, 0.0, 23.90618324279785, 0.0, 24.33172035217285, 0.0, 24.201684951782227, 0.0, 25.0439395904541, 0.0, 24.10662269592285, 0.0, 24.121545791625977, 0.0, 25.075098037719727, 0.0], [0.0, 25.8648624420166, 0.0, 25.660181045532227, 0.0, 24.90850257873535, 0.0, 24.78569984436035, 0.0, 24.5445499420166, 0.0, 24.629541397094727, 0.0, 24.470117568969727, 0.0, 25.221155166625977, 0.0, 24.8895206451416, 0.0, 24.377527236938477, 0.0, 23.789026260375977, 0.0, 24.16082191467285, 0.0, 24.84368324279785, 0.0, 24.545007705688477, 0.0, 24.513452529907227, 0.0, 24.727869033813477], [26.7521915435791, 0.0, 25.433618545532227, 0.0, 24.911706924438477, 0.0, 25.127588272094727, 0.0, 24.359338760375977, 0.0, 24.88408851623535, 0.0, 24.20366859436035, 0.0, 24.3344669342041, 0.0, 25.56133460998535, 0.0, 24.5034122467041, 0.0, 24.3403263092041, 0.0, 24.49798011779785, 0.0, 24.347742080688477, 0.0, 25.629236221313477, 0.0, 24.16973304748535, 0.0, 24.77532386779785, 0.0], [0.0, 27.432580947875977, 0.0, 25.407922744750977, 0.0, 25.04790687561035, 0.0, 25.032373428344727, 0.0, 24.75591468811035, 0.0, 24.984766006469727, 0.0, 24.0726261138916, 0.0, 24.694665908813477, 0.0, 24.52007484436035, 0.0, 24.49883460998535, 0.0, 24.015222549438477, 0.0, 24.30108070373535, 0.0, 25.307031631469727, 0.0, 25.149499893188477, 0.0, 24.37603187561035, 0.0, 25.309534072875977], [27.03508949279785, 0.0, 25.10491371154785, 0.0, 24.620569229125977, 0.0, 24.9157657623291, 0.0, 24.773340225219727, 0.0, 24.882715225219727, 0.0, 24.777856826782227, 0.0, 24.66265296936035, 0.0, 24.4468936920166, 0.0, 24.274560928344727, 0.0, 24.427331924438477, 0.0, 23.76751136779785, 0.0, 24.064783096313477, 0.0, 24.4506778717041, 0.0, 24.172388076782227, 0.0, 24.1467227935791, 0.0], [0.0, 24.8056583404541, 0.0, 24.248559951782227, 0.0, 24.90874671936035, 0.0, 25.191797256469727, 0.0, 24.6163272857666, 0.0, 24.895837783813477, 0.0, 24.72405433654785, 0.0, 24.55950355529785, 0.0, 24.4366397857666, 0.0, 24.4924259185791, 0.0, 23.906213760375977, 0.0, 24.440027236938477, 0.0, 24.30083656311035, 0.0, 23.885705947875977, 0.0, 24.707544326782227, 0.0, 23.5994815826416], [24.4078311920166, 0.0, 24.5229434967041, 0.0, 24.397058486938477, 0.0, 24.843835830688477, 0.0, 24.658472061157227, 0.0, 24.904260635375977, 0.0, 24.332971572875977, 0.0, 24.8937931060791, 0.0, 24.77007484436035, 0.0, 24.44011878967285, 0.0, 23.892480850219727, 0.0, 24.33977699279785, 0.0, 24.33587074279785, 0.0, 24.37554359436035, 0.0, 23.89275550842285, 0.0, 24.147912979125977, 0.0], [0.0, 24.730615615844727, 0.0, 24.13128089904785, 0.0, 24.326379776000977, 0.0, 25.565515518188477, 0.0, 24.73760414123535, 0.0, 24.711145401000977, 0.0, 24.516931533813477, 0.0, 24.5921573638916, 0.0, 24.489343643188477, 0.0, 24.31316566467285, 0.0, 24.381433486938477, 0.0, 23.906152725219727, 0.0, 23.602441787719727, 0.0, 24.921167373657227, 0.0, 23.53264808654785, 0.0, 24.28179359436035], [23.799768447875977, 0.0, 24.031763076782227, 0.0, 23.9113712310791, 0.0, 24.697778701782227, 0.0, 25.040552139282227, 0.0, 24.9445743560791, 0.0, 24.569421768188477, 0.0, 24.967676162719727, 0.0, 24.808618545532227, 0.0, 24.8918399810791, 0.0, 24.226831436157227, 0.0, 24.83172035217285, 0.0, 24.483728408813477, 0.0, 23.321191787719727, 0.0, 24.1885929107666, 0.0, 24.445276260375977, 0.0]], "box_model": "DCSMmax", "sens_serNo": [22], "headerVersion": "0021", "sens_i2cAddr": "0x4", "sens_comp": "MLX90640", "aquisDate": "2019-06-14", "hostname": "4d4nx6", "sens_manuf": "Lexis", "box_name": "DCSMmaxNo.22"}]
Now I have two difficulties which I am trying to address. The above works fine and correctly loads the keys and values of the different fields. However, for the sensor_data key while it is a numpy matrix 24x32, instead of showing it in that form what I get is an output where my matrix is formed with multiple line breaks (\n):
which I do not really understand why is happening. The other issue that I would like to address is whether it would be possible to visualize these values as a grayscale image on the third column of my treewidget. Something like below:

I managed to solve my problem by using the pyqtgraph external library and creating the following widget structure:
# Tree
self.tree_widget = QtWidgets.QTreeWidget()
self.tree_widget.setHeaderLabels(["Key", "Value", "Image"])
root_item = QtWidgets.QTreeWidgetItem(["Test Item"])
self.tree_widget.addTopLevelItem(root_item)
# Data
val=numpy.random.normal(size=(32, 24, 3))
# Sub-widgets
tableWidget = pyqtgraph.TableWidget(sortable=False)
tableWidget.setData(val)
im1 = pyqtgraph.ImageView()
im1.setImage(val.T)
self.tree_widget.setItemWidget(root_item, 1, tableWidget)
self.tree_widget.setItemWidget(root_item, 2, im1)