Develop Reference

Writing to databases on Python

I have been trying to write to a database and am having trouble setting data using two different classes in the one function.
Firstly, all values are being passed through a GUI and in this case, only the following entries were passed: 'Categories' = C, 'Usage' = Charter, 'DispTHR' = 5000.
Here you can see that I have two classes I am wanting to access (airport and runway) where the function set_opsdatabase_details() will go to the appropriate class and write to our database. This is all well and good when the airport and runway are separate from each other; however, when integrating them in the same function I can't seem to get the corresponding airportvalueDict = {} and runwayvalueDict = {] to display the values I want. Could someone help me understand how to write the correct entry box values into the corresponding valueDict dictionaries?
Thank you in advance! (a screenshot of the output from the print statements is attached)
Function in python
Output of function with the print statements
Example of code in text format:
#edit function for first part of operational window
def edit_details1(self):
airport = self.manager.get_chosen_airport()
runway = airport.get_chosen_runway()
airportlabels = ['Category', 'Usage', 'Curfew',
'Surveyor', 'LatestSurvey',
'FuelAvail', 'FuelPrice', 'TankerPort', 'RFF']
runwaylabels = ['DispTHR', 'VASI', 'TCH']
airportvalueDict = {}
runwayvalueDict = {}
print(self.entryDict['Category']["state"])
if self.entryDict['Category']["state"] == 'disabled':
for entry in self.entryDict.values():
entry.config(state=tk.NORMAL)
self.editButton.config(text="Confirm")
elif self.entryDict['Category']['state'] == 'normal':
airport = self.manager.get_chosen_airport()
runway = airport.get_chosen_runway()
values = [x.get() for x in self.varDict.values()]
for label, value in zip(airportlabels, values):
airportvalueDict[label] = value
airport.set_opsdatabase_details(airportvalueDict)
print(f'airport: {airportvalueDict}')
for label, value in zip(runwaylabels, values):
runwayvalueDict[label] = value
runway.set_opsdatabase_details(runwayvalueDict)
print(f'runway: {runwayvalueDict}')
for entry in self.entryDict.values():
entry.config(state=tk.DISABLED)
self.editButton.config(text="Edit...")

Python: Data-structure and processing of GPS points and properties

I'm trying to read data from a csv and then process it on different way. (For starter just the average)
Data
(OneDrive) https://1drv.ms/u/s!ArLDiUd-U5dtg0teQoKGguBA1qt9?e=6wlpko
The data looks like this:
ID; Property1; Property2; Property3...
1; ....
1; ...
1; ...
2; ...
2; ...
3; ...
...
Every line is a GPS point. All points with same ID together (for example 1) produce one Route. The routes are not of the same length and some IDs are skipped. So it isn't a seamless increase of numbers.
I may need to add, that the points are ALWAYS the same set of meters apart from each other. And I don't need the XY information currently.
Wanted Result
In the end I want something like this:
[ID, AVG_Property1, AVG_Property2...] [1, 1.00595, 2.9595, ...] [2,1.50606, 1.5959, ...]
What I got so far
import os
import numpy
import pandas as pd
data = pd.read_csv(os.path.join('C:\\data' ,'data.csv'), sep=';')
# [id, len, prop1, prop2, ...]
routes = numpy.zeros((data.size, 10)) # 10 properties
sums = numpy.zeros(8)
nr_of_entries = 0;
current_id = 1;
for index, row in data.iterrows():
if(int(row['id']) != current_id): #after the last point of the route
routes[current_id-1][0] = current_id;
routes[current_id-1][1] = nr_of_entries; #how many points are in this route?
routes[current_id-1][2] = sums[0] / nr_of_entries;
routes[current_id-1][3] = sums[1] / nr_of_entries;
routes[current_id-1][4] = sums[2] / nr_of_entries;
routes[current_id-1][5] = sums[3] / nr_of_entries;
routes[current_id-1][6] = sums[4] / nr_of_entries;
routes[current_id-1][7] = sums[5] / nr_of_entries;
routes[current_id-1][8] = sums[6] / nr_of_entries;
routes[current_id-1][9] = sums[7] / nr_of_entries;
current_id = int(row['id']);
sums = numpy.zeros(8)
nr_of_entries = 0;
sums[0] += row[3];
sums[1] += row[4];
sums[2] += row[5];
sums[3] += row[6];
sums[4] += row[7];
sums[5] += row[8];
sums[6] += row[9];
sums[7] += row[10];
nr_of_entries = nr_of_entries + 1;
routes
My problem
1.) The way I did it, I have to copy paste the same code for every other processing approach, since as stated I need to do multiple different way. Average is just an example.
2.) The reading of the data is clumsy and fails when IDs are missing
3.) I'm a C# Developer, so my approach would be to create a Class 'Route' which has all the points and then provide methods for 'calculate average for prop 1'. Or something. This way I could also tweak the data if needed. (extreme values for example). But I have no idea how this would be done in Phyton and if this is a reasonable approach in this language.
4.) Is there a more elegant way to iterate through the original csv and getting like Route ID 1, then Route ID 2 and so on? Maybe something like LINQ Queries in C#?
Thanks for any help.

He is a solution and some ideas you can use. The example features multiple options for the same issue so you have to choose which fits the purpose best. Also it is Python 3.7, you didn't specify a version so i hope this works.
class Route(object):
"""description of class"""
def __init__(self, id, rawdata): # on startup
self.id = id
self.rawdata = rawdata
self.avg_Prop1 = self.calculate_average('Prop1')
self.sum_Prop4 = None
def calculate_average(self, Prop_Name): #selfreference for first argument in class method
return self.rawdata[Prop_Name].mean()
def give_Prop_data(self, Prop_Name): #return the Propdata as list
return self.rawdata[Prop_Name].tolist()
def any_function(self, my_function, Prop_Name): #not sure what dataframes support so turning it into a list first
return my_function(self.rawdata[Prop_Name].tolist())
#end of class definiton
data = pd.read_csv('testdata.csv', sep=';')
# [id, len, prop1, prop2, ...]
route_list = [] #List of all the objects created from the route class
for i in data.id.unique():
print('Current id:', i,' with ',len(data[data['id']==i]),'entries')
route_list.append(Route(i,data[data['id']==i]))
#created the Prop1 average in initialization of route so just accessing attribute
print(route_list[1].avg_Prop1)
for current_route in route_list:
print('Route ',current_route.id , ' Properties :')
for i in current_route.rawdata.columns[1:]: #for all except the first (id)
print(i, ' has average ', current_route.calculate_average(i)) #i is the string of the column not just an id
#or pass any function that you want
route_list[1].sum_Prop4 = (route_list[1].any_function(sum,'Prop4'))
print(route_list[1].sum_Prop4)
#which is equivalent to
print(sum(route_list[1].rawdata['Prop4']))
To adress your individual problems out of order:
For 2. and 4.) Looping only over the existing Ids (data.id.unique()) solves the problem. I have no idea what LINQ Queries are, but i assume they are similar. In general, Python has a great way of looping over objects (like for current_route in route_list), which is worth looking into if you want to use it a little more.
For 1. and 3.) Again looping solves the issue. I created a class in the example, mostly to show the syntax for classes. The benefits and drawbacks for using classes should be the same in Python as in C#.
As it is right now the class probably isn't great, but this depends on how you want to use it. If the class should just be a practical way of storing and accessing data it shouldn't have the methods, because you don't need an individual average method for each route. Then you can just access it's data and use it in a function like in sum(route_list[1].rawdata['Prop4']). If however, depending on the data (amount of rows for example) different calculations are necessary, it might come in handy to use the method calculate_average and differentiate in there.
An other example would be the use of the attributes. If you need the average for Prop1 every time, creating it at the initialization sees a good idea, otherwise i wouldn't bother always calculating it.
I hope this helps!

Maya UI Python change NumberOfCollums dynamically

I want to change (kind of dynamically) the NumberOfColumns of a rowColumnLayout inside a Maya Window. Depending on the value given by 'SLiBThumbSizeComboBox' i want to change the count for the columns.
Here is the problem: when running my 2 functions inside the Script Editor everything is working fine. I execute the first one - the ScrollLayout is created. I execute the second and the rowColumnLayout is created with the right count of columns.
But when I try run it only by the first function, meaning the second function is called at the end of the first one - it's not working anymore?! I get the error message, that NumberOfCollumns has to be greater than 0 ?!
def FunctionA():
if cmds.scrollLayout('SLiBScrollLayout', query = True, exists = True):
cmds.deleteUI('SLiBScrollLayout', layout = True)
cmds.scrollLayout('SLiBScrollLayout', p="SLiB_thumbsframe")
def FunctionB():
iconLayout = cmds.optionMenu('SLiBThumbSizeComboBox', query = True, value = True)
iconSize = iconLayout.split('x')
iconDiv = int(iconSize[0])
n = int(cmds.scrollLayout("SLiBScrollLayout", query=True, saw=1)/iconDiv)
cmds.rowColumnLayout('Icons', numberOfColumns=n, p="SLiBScrollLayout")
Thanks in advance
Daniel

I'm not surprised the rowColumnlayout complains if you give it zero columns: you'll always need 1. Depending on how the are setting up the gui , your query on saw might return 0, which would explain your problem.
Here's a very basic example of what it looks like your trying to do:
w = cmds.window()
c = cmds.columnLayout(adj=True)
v_slider = cmds.intSlider(min = 1, max =10, value=2)
h_slider = cmds.intSlider(min = 1, max =10, value=2)
scroll = cmds.scrollLayout()
form = cmds.formLayout()
def update_row(*_):
# get the row and column counts
rows = cmds.intSlider(v_slider, q=True, v= True)
columns = cmds.intSlider(h_slider, q=True, v= True)
# delete the old layout and rebuild.
# the 'or []` below lets you loop even if there are no children....
for n in cmds.formLayout(form, q=True, ca=True) or []:
cmds.deleteUI(n)
cmds.setParent(form)
# make a new rowColumn
new_row = cmds.rowColumnLayout(nc = columns)
for n in range(rows * columns):
cmds.button(label="button_%i" % n)
cmds.formLayout(form, e=True, af = [(new_row,'top',0), (new_row, 'bottom', 0 ), (new_row, 'left', 0 ), (new_row, 'right', 0)])
# both sliders call the update function
cmds.intSlider(h_slider, e=True, cc =update_row)
cmds.intSlider(v_slider, e=True, cc =update_row)
update_row() # do it once to get started
cmds.showWindow(w)
The key here is the order in which it's declared: the function knows what v_slider, h_slider and form are because it's declared after them and doesn't need to do extra work to find them (you can get a more controlled version of the same effect with a class. You'll notice I don't use the names at all: names are unreliable if there's an old layout lying around, it's simpler to use variables.
I've also set the minimum values for the row and columns to 1 so there's no zero division problem.

Re-writing a python program into VB, how to sort CSV?

About a year back, I wrote a little program in python that basically automates a part of my job (with quite a bit of assistance from you guys!) However, I ran into a problem. As I kept making the program better and better, I realized that Python did not want to play nice with excel, and (without boring you with the details suffice to say xlutils will not copy formulas) I NEED to have more access to excel for my intentions.
So I am starting back at square one with VB (2010 Express if it helps.) The only programming course I ever took in my life was on it, and it was pretty straight forward so I decided I'd go back to it for this. Unfortunately, I've forgotten much of what I had learned, and we never really got this far down the rabbit hole in the first place. So, long story short I am trying to:
1) Read data from a .csv structured as so:
41,332.568825,22.221759,-0.489714,eow
42,347.142926,-2.488763,-0.19358,eow
46,414.9969,19.932693,1.306851,r
47,450.626074,21.878299,1.841957,r
48,468.909171,21.362568,1.741944,r
49,506.227269,15.441723,1.40972,r
50,566.199838,17.656284,1.719818,r
51,359.069935,-11.773073,2.443772,l
52,396.321911,-8.711589,1.83507,l
53,423.766684,-4.238343,1.85591,l
2) Sort that data alphabetically by column 5
3) Then selecting only the ones with an "l" in column 5, sort THOSE numerically by column 2 (ascending order) AND copy them to a new file called coil.csv
4) Then selecting only the ones that have an "r" in column 5, sort those numerically by column 2 (descending order) and copy them to the SAME file coil.csv (appended after the others obviously)
After all of that hoopla I wish to get out:
51,359.069935,-11.773073,2.443772,l
52,396.321911,-8.711589,1.83507,l
53,423.766684,-4.238343,1.85591,l
50,566.199838,17.656284,1.719818,r
49,506.227269,15.441723,1.40972,r
48,468.909171,21.362568,1.741944,r
47,450.626074,21.878299,1.841957,r
46,414.9969,19.932693,1.306851,r
I realize that this may be a pretty involved question, and I certainly understand if no one wants to deal with all this bs, lol. Anyway, some full on code, snippets, ideas or even relevant links would be GREATLY appreciated. I've been, and still am googling, but it's harder than expected to find good reliable information pertaining to this.
P.S. Here is the piece of python code that did what I am talking about (although it created two seperate files for the lefts and rights which I don't really need) - if it helps you at all.
msgbox(msg="Please locate your survey file in the next window.")
mainfile = fileopenbox(title="Open survey file")
toponame = boolbox(msg="What is the name of the shots I should use for topography? Note: TOPO is used automatically",choices=("Left","Right"))
fieldnames = ["A","B","C","D","E"]
surveyfile = open(mainfile, "r")
left_file = open("left.csv",'wb')
right_file = open("right.csv",'wb')
coil_file = open("coil1.csv","wb")
reader = csv.DictReader(surveyfile, fieldnames=fieldnames, delimiter=",")
left_writer = csv.DictWriter(left_file, fieldnames + ["F"], delimiter=",")
sortedlefts = sorted(reader,key=lambda x:float(x["B"]))
surveyfile.seek(0,0)
right_writer = csv.DictWriter(right_file, fieldnames + ["F"], delimiter=",")
sortedrights = sorted(reader,key=lambda x:float(x["B"]), reverse=True)
coil_writer = csv.DictWriter(coil_file, fieldnames, delimiter=",",extrasaction='ignore')
for row in sortedlefts:
if row["E"] == "l" or row["E"] == "cl+l":
row['F'] = '%s,%s' % (row['B'], row['D'])
left_writer.writerow(row)
coil_writer.writerow(row)
for row in sortedrights:
if row["E"] == "r":
row['F'] = '%s,%s' % (row['B'], row['D'])
right_writer.writerow(row)
coil_writer.writerow(row)

One option you have is to start with a class to hold the fields. This allows you to override the ToString method to facilitate the output. Then, it's a fairly simple matter of reading each line and assigning the values to a list of the class. In your case you'll want the extra step of making 2 lists sorting one descending and combining them:
Class Fields
Property A As Double = 0
Property B As Double = 0
Property C As Double = 0
Property D As Double = 0
Property E As String = ""
Public Overrides Function ToString() As String
Return Join({A.ToString, B.ToString, C.ToString, D.ToString, E}, ",")
End Function
End Class
Function SortedFields(filename As String) As List(Of Fields)
SortedFields = New List(Of Fields)
Dim test As New List(Of Fields)
Dim sr As New IO.StreamReader(filename)
Using sr As New IO.StreamReader(filename)
Do Until sr.EndOfStream
Dim fieldarray() As String = sr.ReadLine.Split(","c)
If fieldarray.Length = 5 AndAlso Not fieldarray(4)(0) = "e"c Then
If fieldarray(4) = "r" Then
test.Add(New Fields With {.A = Double.Parse(fieldarray(0)), .B = Double.Parse(fieldarray(1)), .C = Double.Parse(fieldarray(2)), .D = Double.Parse(fieldarray(3)), .E = fieldarray(4)})
Else
SortedFields.Add(New Fields With {.A = Double.Parse(fieldarray(0)), .B = Double.Parse(fieldarray(1)), .C = Double.Parse(fieldarray(2)), .D = Double.Parse(fieldarray(3)), .E = fieldarray(4)})
End If
End If
Loop
End Using
SortedFields = SortedFields.OrderBy(Function(x) x.B).Concat(test.OrderByDescending(Function(x) x.B)).ToList
End Function
One simple way of writing the data to a csv file is to use the IO.File.WriteAllLines methods and the ConvertAll method of the List:
IO.File.WriteAllLines(" coil.csv", SortedFields("textfile1.txt").ConvertAll(New Converter(Of Fields, String)(Function(x As Fields) x.ToString)))
You'll notice how the ToString method facilitates this quite easily.
If the class will only be used for this you do have the option to make all the fields string.

Python Class / Instance /Object Organization Question

The following is a simplified example for something I'm trying to do in python (with pygame, but that's probably irrelevant).
I have a list of 8x8 pixel jpgs, each depicting an English letter:
[a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z]
I want to arrange a 4x4 grid of these, in any pattern I want, as a larger 32x32 picture:
gmmg
gppg
gppg
gmmg
But that pattern is only a single frame of an animation.
For example, I might want 4 animation frames where b's and n's flash side to side alternately while an f moves southwest:
bbnf nnbb bbnn nnbb
bbnn nnfb bbnn nnbb
bbnn nnbb bfnn nnbb
bbnn nnbb bbnn fnbb
I want control over the letter value of each square in each frame to make any animation, so I guess essentially there are 64 separate variables (for a 4-frame animation like the one shown above). Each square also has an [x,y] list position variable and rbg color.
My question is how to organize this all with classes (I'm trying to learn OOP). Logically, it seems that each frame contains squares, and each square contains variables like position, letter and color. But I suppose you could even think of it as each square 'contains' 4 frames...
My guess is make a frame class, and put 4 instances of it in a list (if there's 4 frames) and somehow make each frame instance contain a list of 16 square instances. Maybe usable like frames[2].squares[5].letter = f (Just a fuzzy idea; I'm too new at OOP to know if that's remotely correct or a good idea). But it would be helpful to see how someone who knows what they're doing would organize all this.
Thanks!

Since the size of a frame is fixed, and the number of frames is not, then making a class Frame seems like a good first choice. Each Frame would contain a member grid which could be a list of four lists of four letters. A list of four strings wouldn't work as well since strings are immutable, though having it be a single 16-character string might perform better. You'd have to profile to be sure. For this answer, I'll assume you're going with a list of lists of characters.
Then make a class Animation that has a frames member, which is a list of frames. Then you'll write code that looks like:
myAnimation.frames[10].grid[2][3] = 'f'
I can provide more detail if desired.
EXAMPLE: (Haven't tested this yet, but it should be close. The doc comments should hopefully work with doctest.)
import string
class Frame(object):
"""This is a single frame in an animation."""
def __init__(self, fill=None, color=None):
"""Initializes the frame.
>>> f = Frame()
>>> f.print()
aaaa
aaaa
aaaa
aaaa
>>> g = Frame(fill='c', color=(0, 255, 0))
>>> g.print()
cccc
cccc
cccc
cccc
"""
if fill is None:
fill = 'a' # Or whatever default you want
self.letterGrid = []
for row in range(4):
self.letterGrid.append([fill for col in range(4)])
if color is None:
color = (0, 0, 0)
self.colorGrid = []
for row in range(4):
self.letterGrid.append([fill for col in range(4)])
def set_grid(self, row, col, letter=None, color=None):
"""Sets the letter and/or color at the given grid.
>>> f.set_grid(1, 1, 'b', (255, 0, 0))
>>> f.print()
aaaa
abaa
aaaa
aaaa
>>> f.set_grid(1, 3, letter='x')
>>> f.print()
aaaa
abax
aaaa
aaaa
>>> f.set_grid(3, 3, color=(255, 0, 0))
"""
if letter is not None:
self.letterGrid[row][col] = letter
if color is not None:
self.colorGrid[row][col] = color
def position(row, col):
return (row * 16, col * 16)
def print(self):
"""Simple routine to print a frame as text."""
for row in self.letterGrid:
print(''.join(row))
class Animation(object):
def __init__(self, frames=None):
"""Initializes an animation."""
self.frames = frames or []
Hope this gets you started.

the alternative approach would be to come up with a suitable generic datastructure solely made up from dictionaries, lists, sets and so on, and then write library methods to manipulate that data. that doesn't sound very classical OOP, and it isn't, but i've found that way easier to handle and easier to 'get right'. you can clearly seperate the two concerns of building data containers on the one hand and defining suitable data manipulation code on the other.
as earlier posters suggested, the animation could be modeled as a list of frames; each frame then either contains 32 lists with 32 elements each, or 8 lists with 8 elements each where each element models again the 4x4 grid shown above. of course, whether you actually precompute (or simply define) each frame beforehand, or whether you manipulate the data of a single frame 'live' during the animation depends on further considerations.

#Mike
(replying to you above was limited to 600 characters so I guess I'll show my reply here)
This is my attempt at the Frame class so far. I don't know if I should define one class inside another or whether or how to send a list of instances to the Animation class or something. Each square can have a unique letter, position, and color (position because I intend for the columns or rows to be positionally shiftable). So that's why I put 3 types of grids in there (not sure if that's a good idea, or whether an individual square should have its own class too or something).
class Frame(object):
def __init__(self, letterGrid, positionGrid, colorGrid):
self.letterGrid = letterGrid
self.positionGrid = positionGrid
self.colorGrid = colorGrid
class Animation(object):
def __init__(self, frames):
self.frames = frames
frames = []
frames.append(Frame( [
['b','b','n','f'],
['b','b','n','n'],
['b','b','n','n'],
['b','b','n','n'] ],
[
[[0,0],[16,0],[32,0],[48,0]],
[[0,16],[16,16],[32,16],[48,16]],
[[0,32],[16,32],[32,32],[48,32]],
[[0,48],[16,48],[32,48],[48,48]] ],
[
[[0,0,255],[0,0,0],[0,0,0],[0,0,0]],
[[0,0,255],[0,0,0],[0,0,0],[0,0,0]],
[[0,0,255],[0,0,0],[0,0,0],[0,0,0]],
[[0,0,255],[0,0,0],[0,0,0],[0,0,0]] ]
))
frames.append(Frame( [
['n','n','b','b'],
['n','n','f','b'],
['n','n','b','b'],
['n','n','b','b'] ],
[
[[0,0],[16,0],[32,0],[48,0]],
[[0,16],[16,16],[32,16],[48,16]],
[[0,32],[16,32],[32,32],[48,32]],
[[0,48],[16,48],[32,48],[48,48]] ],
[
[[0,0,0],[0,0,255],[0,0,0],[0,0,0]],
[[0,0,0],[0,0,255],[0,0,0],[0,0,0]],
[[0,0,0],[0,0,255],[0,0,0],[0,0,0]],
[[0,0,0],[0,0,255],[0,0,0],[0,0,0]] ]
))
frames.append(Frame( [
['b','b','n','n'],
['b','b','n','n'],
['b','f','n','n'],
['b','b','n','n'] ],
[
[[0,0],[16,0],[32,0],[48,0]],
[[0,16],[16,16],[32,16],[48,16]],
[[0,32],[16,32],[32,32],[48,32]],
[[0,48],[16,48],[32,48],[48,48]] ],
[
[[0,0,0],[0,0,0],[0,0,255],[0,0,0]],
[[0,0,0],[0,0,0],[0,0,255],[0,0,0]],
[[0,0,0],[0,0,0],[0,0,255],[0,0,0]],
[[0,0,0],[0,0,0],[0,0,255],[0,0,0]] ]
))
frames.append(Frame( [
['n','n','b','b'],
['n','n','b','b'],
['n','n','b','b'],
['n','n','b','b'] ],
[
[[0,0],[16,0],[32,0],[48,0]],
[[0,16],[16,16],[32,16],[48,16]],
[[0,32],[16,32],[32,32],[48,32]],
[[0,48],[16,48],[32,48],[48,48]] ],
[
[[0,0,0],[0,0,0],[0,0,0],[0,0,255]],
[[0,0,0],[0,0,0],[0,0,0],[0,0,255]],
[[0,0,0],[0,0,0],[0,0,0],[0,0,255]],
[[0,0,0],[0,0,0],[0,0,0],[0,0,255]] ]
))
print "3rd frame's colorGrid:\n", frames[2].colorGrid