I'm working on a data science project to make some prediction, and I need to calculate a new column based on other column values.
All is working fine, except that my Jupyter-lab is printing me blank lines in my output and I don't know why.
Here's the code :
# Calcul A :
pas = 1500
TailleTotal = len(df)
limite=TailleTotal-pas
df['A'] = np.empty(TailleTotal, dtype = float)
index=0
while index < limite :
A_temp = 0
A_temp = np.sqrt((df['X'][index]**2)+(df['Y'][index]**2)+(df['Z'][index]**2))
df['A'][index]=A_temp
index = index+1
And when I run it, I have a blank line for every iteration.. My files is making more than 1M lines, I have to scroll all over in my code it's very annoying.
But it's more that I really don't understand why it does this, I have no print function or anything that is supposed to show me something.. So why Python have this need to show me empty lines ? It's because I have no "return" in my loop ?
Edit : It appears to be a "output memory" problem from Jupyter-lab. Right clicking and "clear all output" is resolving my issue
You have an infinite loop, first change the while and try to do maybe 1 or 2 iterations and check if the problem is the same. I'm prety sure that dissapears. The kernel should be consuming resources for your loop.
Related
I'm making an evolution simulator using python 3.8.1 in windows 10.
In the code a matrix called temperatures (storing temperature of every cell in the world matrix) is imported and initialised to contain only 0s, it is also never called in the code except for where the problem arose.
world_data = load_progress()
## other parameters ##
temperatures = world_data["temperatures"]
code to import the matrix ^^
def late_update():
global temperatures
## other updates ##
for row in temperatures:
for tempr in row:
tempr += 1
function where the matrix is modified ^^
while run:
## just some keyboard controls ##
update()
late_update()
updateGFX()
mainloop ^^
(as you can see the temperatures are the last to be updated because updateGFX updates another matrix)
If i exit the mainloop and check the value of temperatures every unit in the matrix is 0 (whenever i kill the code) instead of a positive number. Whereas if i print the output like this :
for row in temperatures:
for tempr in row:
tempr += 1
print(tempr)
in the console only 1s will appear (i waited 20 minutes and checked so i'm quite sure) pointing out that right after the loop the temperatures will reset (also if i print the matrix after the loop it's just 0 so it's confirmed).
I tried to update temperatures directly in the mainloop outside late_update but it was the same.
I also tried making a copy of the matrix with the copy module like this:
temperatures_copy = copy.deepcopy(temperatures)
for row in temperatures_copy:
for tempr in row:
tempr += 1
temperatures = copy.deepcopy(temperatures_copy)
Then literally before posting the question i remembered about list comprehensions and changed the code to:
temperatures = [[tempr + 1 for tempr in row] for row in temperatures]
and it worked.
Since this may be also a problem for future development though, i'll still ask the question about
why a list matrix can't be modified in an annidated loop (inside another loop) but can be if the annidated loop is turned into a list comprehension.
Thanks for your patience.
I have the following code
def numTest():
getNum = "https://sms-activate.ru/stubs/handler_api.php?api_key=" + api + "&action=getNumber&service=go&country=3"
numReq = requests.get(getNum)
smsNum = numReq.text
cleanNum = smsNum.split(":")
print(cleanNum)
reply = cleanNum[:6]
if reply == "ACCESS":
numID = cleanNum[1]
smsNo = cleanNum[2].replace("1", "", 1)
print(numID)
print(smsNo)
else:
numTest()
When the code is ran it doesn't detect the reply properly. So the API can either get back something such as ['ACCESS_NUMBER', '379609689', '12165419985'] or this ['NO_NUMBERS']
If it is the first one I need to split it and just keep array [1] and [2] and if it says No Numbers I need to run the loop again. What happens as well is if I get a number on the first try it stops and works correctly but if I get No numbers it trys again and if it gets a number it keeps going.
cleannum is a list, you're looking to find out if the first elements first 6 characters are ACCESS, not the first 6 elements of the list (which will never equal a string)
reply = cleanNum[0][:6]
Not sure if I understood your problem right. But if the array you mentioned is "cleanNum", then you should be using cleanNum[0][:6] here.
There seems to be something strange going on in this loop. I tried to debug it, and I have no explanation for what is going on.
This code is from another Stack Overflow post, and I made some edits to it.
modelImages = ['/home/lie/Desktop/dylan.jpg','/home/lie/Desktop/melissa.jpg']
for modelImage in modelImages:
**print modelImage**
template=cv2.imread(modelImage)
templateg = cv2.cvtColor(template,cv2.COLOR_BGR2GRAY)
keys = surf.detect(templateg)
keys,desc = surfDescriptorExtractor.compute(templateg, keys)
count = 0
for h,des in enumerate(desc):
des = np.array(des,np.float32).reshape((1,64))
retval, results, neigh_resp, dists = knn.find_nearest(des,1)
res,dist = int(results[0][0]),dists[0][0]
if dist<0.1: # draw matched keypoints in red color
count=count + 1
print "space"**
The important parts have asterisks. This is a portion of the code that was suppose to identify similarities among images of faces. What the code does is not important. The strange part is that this loop is executing 1 time for an array of size two.
The output is:
/home/lie/Desktop/dylan.jpg
/home/lie/Desktop/melissa.jpg
space
Notice that both strings in modelImages are printed before space. By the way this is part of a function that is called from a loop. This seems to be more of a python issue than an opencv issue. It almost seems like there is a hidden continue statment
Thanks
I am writing a program to analyze some of our invoice data. Basically,I need to take an array containing each individual invoice we sent out over the past year & break it down into twelve arrays which contains the invoices for that month using the dateSeperate() function, so that monthly_transactions[0] returns Januaries transactions, monthly_transactions[1] returns Februaries & so forth.
I've managed to get it working so that dateSeperate returns monthly_transactions[0] as the january transactions. However, once all of the January data is entered, I attempt to append the monthly_transactions array using line 44. However, this just causes the program to break & become unrepsonsive. The code still executes & doesnt return an error, but Python becomes unresponsive & I have to force quite out of it.
I've been writing the the global array monthly_transactions. dateSeperate runs fine as long as I don't include the last else statement. If I do that, monthly_transactions[0] returns an array containing all of the january invoices. the issue arises in my last else statement, which when added, causes Python to freeze.
Can anyone help me shed any light on this?
I have written a program that defines all of the arrays I'm going to be using (yes I know global arrays aren't good. I'm a marketer trying to learn programming so any input you could give me on how to improve this would be much appreciated
import csv
line_items = []
monthly_transactions = []
accounts_seperated = []
Then I import all of my data and place it into the line_items array
def csv_dict_reader(file_obj):
global board_info
reader = csv.DictReader(file_obj, delimiter=',')
for line in reader:
item = []
item.append(line["company id"])
item.append(line["user id"])
item.append(line["Amount"])
item.append(line["Transaction Date"])
item.append(line["FIrst Transaction"])
line_items.append(item)
if __name__ == "__main__":
with open("ChurnTest.csv") as f_obj:
csv_dict_reader(f_obj)
#formats the transacation date data to make it more readable
def dateFormat():
for i in range(len(line_items)):
ddmmyyyy =(line_items[i][3])
yyyymmdd = ddmmyyyy[6:] + "-"+ ddmmyyyy[:2] + "-" + ddmmyyyy[3:5]
line_items[i][3] = yyyymmdd
#Takes the line_items array and splits it into new array monthly_tranactions, where each value holds one month of data
def dateSeperate():
for i in range(len(line_items)):
#if there are no values in the monthly transactions, add the first line item
if len(monthly_transactions) == 0:
test = []
test.append(line_items[i])
monthly_transactions.append(test)
# check to see if the line items year & month match a value already in the monthly_transaction array.
else:
for j in range(len(monthly_transactions)):
line_year = line_items[i][3][:2]
line_month = line_items[i][3][3:5]
array_year = monthly_transactions[j][0][3][:2]
array_month = monthly_transactions[j][0][3][3:5]
#print(line_year, array_year, line_month, array_month)
#If it does, add that line item to that month
if line_year == array_year and line_month == array_month:
monthly_transactions[j].append(line_items[i])
#Otherwise, create a new sub array for that month
else:
monthly_transactions.append(line_items[i])
dateFormat()
dateSeperate()
print(monthly_transactions)
I would really, really appreciate any thoughts or feedback you guys could give me on this code.
Based on the comments on the OP, your csv_dict_reader function seems to do exactly what you want it to do, at least inasmuch as it appends data from its argument csv file to the top-level variable line_items. You said yourself that if you print out line_items, it shows the data that you want.
"But appending doesn't work." I take it you mean that appending the line_items to monthly_transactions isn't being done. The reason for that is that you didn't tell the program to do it! The appending that you're talking about is done as part of your dateSeparate function, however you still need to call the function.
I'm not sure exactly how you want to use your dateFormat and dateSeparate functions, but in order to use them, you need to include them in the main function somehow as calls, i.e. dateFormat() and dateSeparate().
EDIT: You've created the potential for an endless loop in the last else: section, which extends monthly_transactions by 1 if the line/array year/month aren't equal. This is problematic because it's within the loop for j in range(len(monthly_transactions)):. This loop will never get to the end if the length of monthly_transactions is increased by 1 every time through.
Solved: A friend of mine helped me add in code that takes the csv files which get outputted and combines them into a single new file. I will add the code in after the weekend in case anyone else with a similar issue wants to see it in the future!
Let me start by sharing my existing, working code. This code takes some raw data from a csv file and generates new csv files from it. The data consists of two columns, one representing voltage and one representing current. If the voltage value is not changing, the current values are sent to a new csv file whose name reflects the constant voltage. Once a new stable voltage is reached, another csv is made for that voltage and so on. Here it is:
for x in range(1,6):
input=open('signalnoise(%d).csv' %x,"r") # Opens raw data as readable
v1 = 0
first = True
for row in csv.reader(input,delimiter='\t'):
v2 = row[0]
if v1==v2:
voltage=float(v1)*1000
if first:
print("Writing spectra file for " +str(voltage) + "mV")
first = False
output=open('noisespectra(%d)' %x +str(voltage)+'mV.csv',"a")
current = [row[1]]
writer=csv.writer(output,delimiter='\t')
writer.writerow(current)
else:
v1 = row[0]
first = True
One note, for some reason the print command doesn't seem to go off until the entire script is done running but it prints the correct thing. This could just be my computer hanging while the script runs.
I would like to change this so that instead of having a bunch of files, I just have one output file with multiple columns. Each column would have its first entry be the voltage value followed by all the currents recorded for that voltage. Here is my idea so far but I'm stuck:
for x in range(1,6):
input=open('signalnoise(%d).csv' %x,"r") # Opens raw data as readable
v1 = 0
first = True
for row in csv.reader(input,delimiter='\t'):
v2 = row[0]
if v1==v2:
voltage=float(v1)*1000
if first:
column = ['voltage']
print("Writing spectra file for " +str(voltage) + "mV")
first = False
column=column+[row[1]] # Adds the current onto the column
saved = True # Means that a column is being saved
elif saved: # This is executed if there is a column waiting to be appended and the voltage has changed
#I get stuck here...
At this point I think I need to somehow use item.append() like the example here but I'm not entirely sure how to implement it. Then I would set saved = False and v1 = row[0] and have the same else statement as the original working code so that on the next iteration things would proceed as desired.
Here is some simple sample data to work with (although mine is actually tab delimited):
.1, 1
.2, 2
.2, 2
.2, 2.1
.2, 2
.3, 3
.4, 4
.5, 5.1
.5, 5.2
.5, 5
.5, 5.1
My working code would take this and give me two files named 'noisespectra(#)200.0mV.csv' and 'noisespectra(#)500.0mV.csv' which are single columns '2,2,2.1,2' and '5.1,5.2,5,5.1' respectively. I would like code which makes a single file named 'noisespectra(#).csv' which is two columns, '200.0mV,2,2,2.1,2' and '500.0mV,5.1,5.2,5,5.1'. In general, a particular voltage will not have the same number of currents and I think this could be a potential problem in using the item.append() technique, particularly if the first voltage has fewer corresponding currents than future voltages.
Feel free to disregard the 'for x in range()'; I am looping through files with similar names but that's not important for my problem.
I greatly appreciate any help anyone can give me! If there are any questions, I will try to address them as quickly as I can.
Keep track of the two sets of values in two lists, then do ...
combined = map(None, list_1, list_2)
And then output the combined list to csv.