Related
Okay, so I am having this issue with JSON, keys, and strings. I'm using a JSON dump in python to save my game dictionaries and it does work. The issue is when I load the dictionaries the game I'm making uses int values as keys in the world directory but JSON stores keys as strings. Here's a random generation I did.
worldmap = {
'Regions': {
1: 'Zelbridge', 2: 'Forest Path', 3: 'Baronbell', 4: 'Old Path', 5: 'Cariva', 6: 'Prairie Path'},
'Zelbridge': {1: 'Field', 2: 'Prairie Path', 3: 'School', 4: 'Mountain Path', 5: 'Graveyard',
6: 'Old Path', 7: 'Blacksmith', 8: 'Forest Path', 9: 'Doctor', 0: 'Zelbridge'},
'Forest Path': {1: 'Trees', 2: 'Bushes', 3: 'Path', 4: 'Cariva', 5: 'Path',
6: 'Baronbell', 7: 'Path', 8: 'Zelbridge', 9: 'Path', 0: 'Forest Path'},
'Baronbell': {1: 'House', 2: 'Mountain Path', 3: 'Graveyard', 4: 'Old Path', 5: 'Field',
6: 'Forest Path', 7: 'Church', 8: 'Prairie Path', 9: 'Shop', 0: 'Baronbell'},
'Old Path': {1: 'Path', 2: 'Trees', 3: 'Bushes', 4: 'Cariva', 5: 'Path',
6: 'Zelbridge', 7: 'Trees', 8: 'Baronbell', 9: 'Trees', 0: 'Old Path'},
'Cariva': {1: 'Cellar', 2: 'Old Path', 3: 'Graveyard', 4: 'Mountain Path', 5: 'Town Hall',
6: 'Prairie Path', 7: 'School', 8: 'Forest Path', 9: 'Blacksmith', 0: 'Cariva'},
'Prairie Path': {1: 'Bushes', 2: 'Path', 3: 'Path', 4: 'Zelbridge', 5: 'Trees',
6: 'Cariva', 7: 'Trees', 8: 'Baronbell', 9: 'Path', 0: 'Prairie Path'}
}
So when I use the load function I get key errors due to the int's being converted to strings. I attempted a for loop to iterate over the keys and change them back but I get this error about the dictionary changing. Here's an example of me trying to load a different (and also random) world. Its set to print after each loop showing that it works
What was your hero's name? #Input hero name
Loading...
{'2': 'Prairie Path', '3': 'Cariva', '4': 'Old Path', '5': 'Baronbell', '6': 'Mountain Path', 1: 'Zelbridge'} #Region number 1 no longer string
{'3': 'Cariva', '4': 'Old Path', '5': 'Baronbell', '6': 'Mountain Path', 1: 'Zelbridge', 2: 'Prairie Path'} #Region numbers 1 and 2 no longer string
{'4': 'Old Path', '5': 'Baronbell', '6': 'Mountain Path', 1: 'Zelbridge', 2: 'Prairie Path', 3: 'Cariva'} #ect
{'5': 'Baronbell', '6': 'Mountain Path', 1: 'Zelbridge', 2: 'Prairie Path', 3: 'Cariva', 4: 'Old Path'} #ect
{'6': 'Mountain Path', 1: 'Zelbridge', 2: 'Prairie Path', 3: 'Cariva', 4: 'Old Path', 5: 'Baronbell'} # Region numbers 1, 2, 3, 4, and 5 no longer string
{'6': 'Mountain Path', 1: 'Zelbridge', 2: 'Prairie Path', 3: 'Cariva', 5: 'Baronbell'}
Traceback (most recent call last):
File "C:/Users/crazy/PycharmProjects/rpg/main.py", line 581, in load
for key in worldmap["Regions"]:
RuntimeError: dictionary changed size during iteration
Process finished with exit code 1
I'm not exactly sure what's wrong with it but sadly I'll also have to do this for each location within a region. Any and all help is appreciated, as I've looked all over SO and google but to no avail.
with open(world_file) as infile:
worldmap = json.load(infile)
copy = worldmap.copy()
regions = worldmap["Regions"]
locations = copy.pop("Regions")
for key in worldmap["Regions"]:
value = worldmap["Regions"][key]
new_key = int(key)
worldmap["Regions"].update({new_key: value})
worldmap ["Regions"].pop(key)
print(str(worldmap["Regions"]) + "\n")```
Use For loop this way and update the key in loop itself:
mydict = {1: 'a', 2: 'b'}
for index, (key, value) in enumerate(mydict.items()):
print("index: {}, key: {}, value: {}".format(index, key, value))
mydict[index] = mydict.pop(key)
Or use can use List to force a copy of the keys to be made:
mydict = {1: 'a', 2: 'b'}
for index, key in enumerate(list(mydict)):
mydict[index] = mydict.pop(key)
# which will give output like:
# ---------------------------
# {0: 'a', 1: 'b'}
I have this dictionary:
{
0: array([-0.16638531, -0.11749843]),
1: array([-0.2318372 , 0.00917023]),
2: array([-0.42934129, -0.0675385 ]),
3: array([-0.63377579, -0.02102854]),
4: array([-0.26648222, -0.42038916]),
5: array([-0.17250316, -0.73490218]),
6: array([-0.42774336, -0.61259704]),
7: array([-0.55420825, -0.77304496]),
8: array([0.13900166, 0.07800885]),
9: array([0.42223986, 0.16563338]),
10: array([ 0.39895669, -0.09198566]),
12: array([0.24324618, 0.44829616]),
11: array([ 0.55394714, -0.17960723]),
13: array([0.192127 , 0.5988793]),
14: array([0.39554203, 0.7186038 ]),
15: array([0.53721604, 1. ])
}
I want to convert those numpy.ndarray values to tuples, and have something like this:
{
0: (-0.16638531, -0.11749843),
1: (-0.2318372 , 0.00917023),
...
}
From this answer here it looks like for each value in the dictionary you can:
tuple(arr)
So for the whole dictionary you can probably do something like:
new_dict = {key: tuple(arr) for key, arr in old_dict.items()}
Or easier to understand:
new_dict = {}
for key, arr in old_dict.items():
new_dict.update({key: tuple(arr)})
You can use a dictionary comprehension.
Python dictionaries have an .items() method that return a tuple of (key, value) for each key-value pair.
The comprehension recreates a new mapping with the original key and the array cast as a tuple.
from numpy import array
data = {
0: array([-0.16638531, -0.11749843]),
1: array([-0.2318372 , 0.00917023]),
2: array([-0.42934129, -0.0675385 ]),
3: array([-0.63377579, -0.02102854]),
4: array([-0.26648222, -0.42038916]),
5: array([-0.17250316, -0.73490218]),
6: array([-0.42774336, -0.61259704]),
7: array([-0.55420825, -0.77304496]),
8: array([0.13900166, 0.07800885]),
9: array([0.42223986, 0.16563338]),
10: array([ 0.39895669, -0.09198566]),
12: array([0.24324618, 0.44829616]),
11: array([ 0.55394714, -0.17960723]),
13: array([0.192127 , 0.5988793]),
14: array([0.39554203, 0.7186038 ]),
15: array([0.53721604, 1. ])
}
print({key: tuple(value) for key, value in data.items()})
OUTPUT:
{0: (-0.16638531, -0.11749843), 1: (-0.2318372, 0.00917023), 2: (-0.42934129, -0.0675385), 3: (-0.63377579, -0.02102854), 4: (-0.26648222, -0.42038916), 5: (-0.17250316, -0.73490218), 6: (-0.42774336, -0.61259704), 7: (-0.55420825, -0.77304496), 8: (0.13900166, 0.07800885), 9: (0.42223986, 0.16563338), 10: (0.39895669, -0.09198566), 12: (0.24324618, 0.44829616), 11: (0.55394714, -0.17960723), 13: (0.192127, 0.5988793), 14: (0.39554203, 0.7186038), 15: (0.53721604, 1.0)}
mapping = { key: (item[0], item[1]) for key, item in your_dict.items() }
What is the best way to create n groups of n size that have a similar total file size given a DataFrame with a column for files and file sizes? While searching, this problem sounded similar to the knapsack problem, except there are no hard stops. Any solution that is quick and produces groups of any length that are close to average (be it under or over) would be a great improvement.
My first attempt (t1) creates groups by counting in a circular order. Next attempt (t2), sorts the DataFrame by size in hopes of preventing one group from getting a clump of large files, but the approach is basically the same as t1. There are usually ~300 files total. In this context, I was not sure if it is practical to calculate all possible combinations or if there is a better approach.
from itertools import repeat, chain
from math import ceil
import pandas as pd
source_dict = {'file_name_': {0: 'file_0', 1: 'file_1', 2: 'file_2', 3: 'file_3', 4: 'file_4', 5: 'file_5', 6: 'file_6', 7: 'file_7', 8: 'file_8', 9: 'file_9'
, 10: 'file_10', 11: 'file_11', 12: 'file_12', 13: 'file_13', 14: 'file_14', 15: 'file_15', 16: 'file_16', 17: 'file_17', 18: 'file_18'
, 19: 'file_19', 20: 'file_20', 21: 'file_21', 22: 'file_22', 23: 'file_23', 24: 'file_24', 25: 'file_25', 26: 'file_26', 27: 'file_27'
, 28: 'file_28', 29: 'file_29', 30: 'file_30', 31: 'file_31', 32: 'file_32', 33: 'file_33', 34: 'file_34', 35: 'file_35', 36: 'file_36'
, 37: 'file_37', 38: 'file_38', 39: 'file_39', 40: 'file_40', 41: 'file_41', 42: 'file_42', 44: 'file_44', 45: 'file_45', 46: 'file_46'
, 47: 'file_47', 48: 'file_48', 49: 'file_49', 50: 'file_50'}
, 'file_size': {0: 3407245, 1: 3973920, 2: 7408640, 3: 4086426, 4: 12795600, 5: 2155039, 6: 9514856, 7: 13190235, 8: 32043703, 9: 4936240, 10: 9591964
, 11: 70153435, 12: 5106282, 13: 212414, 14: 24998146, 15: 11605646, 16: 2427516, 17: 23634036, 18: 169983, 19: 7011305, 20: 2106828
, 21: 3420304, 22: 11254, 23: 1271220, 24: 1164562, 25: 83613105, 26: 1030701, 27: 366948, 28: 7014895, 29: 8274642, 30: 2731629
, 31: 1596299, 32: 524, 33: 302, 34: 42332100, 35: 5441036, 36: 40633457, 37: 34680208, 38: 123505, 39: 15905009, 40: 52071678
, 41: 10624966, 42: 15425993, 44: 27673986, 45: 144988223, 46: 62619919, 47: 21562386, 48: 10620299, 49: 254661, 50: 232406680}}
sampleSizesDF = pd.DataFrame(source_dict)
desired_groups = 4 # multiprocessing.cpu_count()
group_size = ceil(sampleSizesDF.file_name_.count() / desired_groups)
max_length = sampleSizesDF.file_name_.count() # upper bound for list
# trial 1, count off and group
my_groups = list(chain(*repeat(list(range(0,desired_groups)), group_size)))[:max_length]
sampleSizesDF['pGroup_t1'] = my_groups
# trial 2, sort + trial 1
sampleSizesDF.sort_values('file_size', inplace = True)
sampleSizesDF['pGroup_t2'] = my_groups
pGroupDistDF = pd.concat([
sampleSizesDF.groupby('pGroup_t1').agg({'file_size': 'sum'})
, sampleSizesDF.groupby('pGroup_t2').agg({'file_size': 'sum'})
]
, axis=1)
pGroupDistDF.columns = ['t1', 't2']
pGroupDistDF = pGroupDistDF.merge(pd.DataFrame(pGroupDistDF.values, columns=['t1_dist', 't2_dist']).apply(lambda x: x/x.sum()), left_index=True, right_index=True)
presentation_order = ['t1', 't1_dist', 't2', 't2_dist']
pGroupDistDF[presentation_order]
t1 t1_dist t2 t2_dist
0 304015174 0.281916 291719748 0.270514
1 470551775 0.436347 396619142 0.367788
2 134901157 0.125095 183490246 0.170152
3 168921844 0.156643 206560814 0.191546
I'm working on a program that searches all the files in my database and groups them based on what number is in the file name (from 001 to 100).
The only problem is that python interprets '001' as '1', but '001' is the exact number in the file name, and since I'm using regular expressions to search it doesn't recognize the numbers the way I want it to.
Would really appreciate some help! Here's my code so far:
import sys
import os
import re
import glob
time_data = open("time_data.txt", "w")
space_data = open("space_data.txt", "w")
folder_list = ['/Users/fenyolab/Downloads/root images/pet week img seq - removed 621 and after - ch1 registered', 'C:/Users/fenyolab/Downloads/root images/0329 to 033116 - WERSCR regen - STELLAR - registered', 'C:/Users/fenyolab/Downloads/root images/0406 to 040816 - H2BIAAWOX regen - GOOD - pt II - REGISTERED']
def stack_at_time_point(direc, time_point):
time_list = []
for x in glob.glob('%s/*' % direc):
if re.search("t.*%s_z" % time_point, x) != None and re.search('_c1.*', x) != None:
time_list.append(x)
for i in time_list:
time_data.write("%s\n" % i)
def stack_at_zlocation(direc, location):
location_list = []
for x in glob.glob('%s/*' % direc):
if re.search("_z.*%s_." % location, x) != None and re.search('_c1.*', x) != None:
location_list.append(x)
for i in location_list:
space_data.write("%s\n" % i)
for i in folder_list:
for x in xrange(100):
stack_at_zlocation(i, x)
space_data.write("\n\n\n")
stack_at_time_point(i, x)
time_data.write("\n\n\n")
space_data.close()
time_data.close()
print "Done."
The regex "_z.*%s_." % location would match to _z023_ if the specified location is 23, but if the specified location is 1, the program will return _z001_, _z011_, _z021_, _z031_, _z041_ ... _z091_.
You use printf-style string formatting. Using printf-style formatting, you may specify leading zereos and a field width.
Replace your %s with %03d:
re.search("t.*%03d_z" % time_point, x)
and
re.search("_z.*%03d_." % location, x)
Suggest that you build an index based on a flexible length number portion and use this to lookup the appropriate file. E.g.:
>>> import re
>>>
>>> locations = ['_z{0:03d}_'.format(x) for x in range(1,101)]
>>>
>>> def create_zindex(names):
... reg = re.compile('_z(\d+)_')
... result = {}
... for name in names:
... m = reg.search(name)
... if not m:
... print "Can't find z index in {0!r}".format(name)
... continue
... zindex = int(m.groups()[0])
... if zindex in result:
... print "Duplicate z-index {0} - {1}".format(name,result[zindex])
... continue
... result[zindex] = name
... return result
...
>>> print locations
['_z001_', '_z002_', '_z003_', '_z004_', '_z005_', '_z006_', '_z007_', '_z008_', '_z009_', '_z010_', '_z011_', '_z012_', '_z013_', '_z014_', '_z015_', '_z016_', '_z017_', '_z018_', '_z019_', '_z020_', '_z021_', '_z022_', '_z023_', '_z024_', '_z025_', '_z026_', '_z027_', '_z028_', '_z029_', '_z030_', '_z031_', '_z032_', '_z033_', '_z034_', '_z035_', '_z036_', '_z037_', '_z038_', '_z039_', '_z040_', '_z041_', '_z042_', '_z043_', '_z044_', '_z045_', '_z046_', '_z047_', '_z048_', '_z049_', '_z050_', '_z051_', '_z052_', '_z053_', '_z054_', '_z055_', '_z056_', '_z057_', '_z058_', '_z059_', '_z060_', '_z061_', '_z062_', '_z063_', '_z064_', '_z065_', '_z066_', '_z067_', '_z068_', '_z069_', '_z070_', '_z071_', '_z072_', '_z073_', '_z074_', '_z075_', '_z076_', '_z077_', '_z078_', '_z079_', '_z080_', '_z081_', '_z082_', '_z083_', '_z084_', '_z085_', '_z086_', '_z087_', '_z088_', '_z089_', '_z090_', '_z091_', '_z092_', '_z093_', '_z094_', '_z095_', '_z096_', '_z097_', '_z098_', '_z099_', '_z100_']
>>> print create_zindex(locations)
{1: '_z001_', 2: '_z002_', 3: '_z003_', 4: '_z004_', 5: '_z005_', 6: '_z006_', 7: '_z007_', 8: '_z008_', 9: '_z009_', 10: '_z010_', 11: '_z011_', 12: '_z012_', 13: '_z013_', 14: '_z014_', 15: '_z015_', 16: '_z016_', 17: '_z017_', 18: '_z018_', 19: '_z019_', 20: '_z020_', 21: '_z021_', 22: '_z022_', 23: '_z023_', 24: '_z024_', 25: '_z025_', 26: '_z026_', 27: '_z027_', 28: '_z028_', 29: '_z029_', 30: '_z030_', 31: '_z031_', 32: '_z032_', 33: '_z033_', 34: '_z034_', 35: '_z035_', 36: '_z036_', 37: '_z037_', 38: '_z038_', 39: '_z039_', 40: '_z040_', 41: '_z041_', 42: '_z042_', 43: '_z043_', 44: '_z044_', 45: '_z045_', 46: '_z046_', 47: '_z047_', 48: '_z048_', 49: '_z049_', 50: '_z050_', 51: '_z051_', 52: '_z052_', 53: '_z053_', 54: '_z054_', 55: '_z055_', 56: '_z056_', 57: '_z057_', 58: '_z058_', 59: '_z059_', 60: '_z060_', 61: '_z061_', 62: '_z062_', 63: '_z063_', 64: '_z064_', 65: '_z065_', 66: '_z066_', 67: '_z067_', 68: '_z068_', 69: '_z069_', 70: '_z070_', 71: '_z071_', 72: '_z072_', 73: '_z073_', 74: '_z074_', 75: '_z075_', 76: '_z076_', 77: '_z077_', 78: '_z078_', 79: '_z079_', 80: '_z080_', 81: '_z081_', 82: '_z082_', 83: '_z083_', 84: '_z084_', 85: '_z085_', 86: '_z086_', 87: '_z087_', 88: '_z088_', 89: '_z089_', 90: '_z090_', 91: '_z091_', 92: '_z092_', 93: '_z093_', 94: '_z094_', 95: '_z095_', 96: '_z096_', 97: '_z097_', 98: '_z098_', 99: '_z099_', 100: '_z100_'}
The following code plots an interactive figure where I can toggle specific lines on/off. This works perfectly when I'm working in an Ipython Notebook
import pandas as pd
import numpy as np
from itertools import cycle
import matplotlib.pyplot as plt, mpld3
from matplotlib.widgets import CheckButtons
import matplotlib.patches
import seaborn as sns
%matplotlib nbagg
sns.set(style="whitegrid")
df = pd.DataFrame({'freq': {0: 0.01, 1: 0.02, 2: 0.029999999999999999, 3: 0.040000000000000001, 4: 0.050000000000000003, 5: 0.059999999999999998, 6: 0.070000000000000007, 7: 0.080000000000000002, 8: 0.089999999999999997, 9: 0.10000000000000001, 10: 0.01, 11: 0.02, 12: 0.029999999999999999, 13: 0.040000000000000001, 14: 0.050000000000000003, 15: 0.059999999999999998, 16: 0.070000000000000007, 17: 0.080000000000000002, 18: 0.089999999999999997, 19: 0.10000000000000001, 20: 0.01, 21: 0.02, 22: 0.029999999999999999, 23: 0.040000000000000001, 24: 0.050000000000000003, 25: 0.059999999999999998, 26: 0.070000000000000007, 27: 0.080000000000000002, 28: 0.089999999999999997, 29: 0.10000000000000001}, 'kit': {0: 'B', 1: 'B', 2: 'B', 3: 'B', 4: 'B', 5: 'B', 6: 'B', 7: 'B', 8: 'B', 9: 'B', 10: 'A', 11: 'A', 12: 'A', 13: 'A', 14: 'A', 15: 'A', 16: 'A', 17: 'A', 18: 'A', 19: 'A', 20: 'C', 21: 'C', 22: 'C', 23: 'C', 24: 'C', 25: 'C', 26: 'C', 27: 'C', 28: 'C', 29: 'C'}, 'SNS': {0: 91.198979591799997, 1: 90.263605442199989, 2: 88.818027210899999, 3: 85.671768707499993, 4: 76.23299319729999, 5: 61.0969387755, 6: 45.1530612245, 7: 36.267006802700003, 8: 33.0782312925, 9: 30.739795918400002, 10: 90.646258503400006, 11: 90.306122449, 12: 90.178571428600009, 13: 89.498299319699996, 14: 88.435374149599994, 15: 83.588435374200003, 16: 75.212585034, 17: 60.969387755100001, 18: 47.278911564600001, 19: 37.627551020399999, 20: 90.986394557800011, 21: 90.136054421799997, 22: 89.540816326499993, 23: 88.690476190499993, 24: 86.479591836799997, 25: 82.397959183699996, 26: 73.809523809499993, 27: 63.180272108800004, 28: 50.935374149700003, 29: 41.241496598699996}, 'FPR': {0: 1.0953616823100001, 1: 0.24489252678500001, 2: 0.15106142277199999, 3: 0.104478605177, 4: 0.089172822253300005, 5: 0.079856258734300009, 6: 0.065881413455800009, 7: 0.059892194050699996, 8: 0.059892194050699996, 9: 0.0578957875824, 10: 0.94097291541899997, 11: 0.208291741532, 12: 0.14773407865800001, 13: 0.107805949291, 14: 0.093165635189999998, 15: 0.082518134025399995, 16: 0.074532508152000007, 17: 0.065881413455800009, 18: 0.062554069341799995, 19: 0.061888600519100001, 20: 0.85313103081100006, 21: 0.18899314567100001, 22: 0.14107939043000001, 23: 0.110467824582, 24: 0.099820323417899995, 25: 0.085180009316599997, 26: 0.078525321088700001, 27: 0.073201570506399985, 28: 0.071870632860800004, 29: 0.0705396952153}})
tableau20 = ["#6C6C6C", "#92D050", "#FFC000"]
tableau20 = cycle(tableau20)
kits = ["A","B", "C"]
color = iter(["#6C6C6C", "#92D050", "#FFC000"])
fig = plt.figure(figsize=(12,8))
for kit in kits:
colour = next(color)
for i in df.groupby('kit'):
grouped_df = pd.DataFrame(np.array(i[1]), columns =
['freq', 'SNS', 'FPR', 'kit'])
if grouped_df.kit.tolist()[1] == kit:
x = [float(value) for i, value in enumerate(grouped_df.FPR)]
y = [float(value) for i, value in enumerate(grouped_df.SNS)]
x, y = (list(x) for x in zip(*sorted(zip(x, y))))
label = grouped_df['kit'].tolist()[1]
p = plt.plot(x, y, "-o",label = label, color = colour)
labels = [label.get_text() for label in plt.legend().texts]
plt.legend().set_visible(False)
for i, value in enumerate(labels):
exec('label%s="%s"'%(i, value))
for i in range(len(labels)):
exec('l%s=fig.axes[0].lines[i]'%(i))
rax = plt.axes([0.92, 0.7, 0.2, 0.2], frameon=False)
check = CheckButtons(rax, (labels), ('True ' * len(labels)))
for i, rec in enumerate(check.rectangles):
rec.set_facecolor(tableau20.next())
def func(label):
for i in range(len(labels)):
if label == eval('label%s'%(i)): eval('l%s.set_visible(not l%s.get_visible())'%(i,i))
plt.draw()
check.on_clicked(func)
plt.show()
Problem is, I need to export the notebook as a html to share with colleagues who know nothing about python. How can I export the notebook to html and get it to maintain the interactive (toggle) functionality (which it currently loses)? Thanks!
Maybe you don't need to export jupyter notebook to html, but share the notebook link to the other people and they can visit the url using their browser.
A jupyter notebook plugin would help you do this more efficiently: jupyter/dashboards, it's maintained by official jupyter team, and it helps you share your notebook like a report, and you can control which cell to display and the location of each cell displayed. Worth a try!