Develop Reference

Getting the last two variables in for loop

I am trying to make a program that shows me the data of two specific coins. What it basically does is to takes the data in an infinite "for loop" to display the info until I close the program.
And now I am trying to get the last two elements of this infinite for loop every time it runs again and make calculations with it. I know I can't just hold all the items in a list and I am not sure how to store last two's and use them every time.
for line in lines:
coinsq = line.strip()
url = priceKey + coinsq + "USDT"
data = requests.get(url)
datax = data.json()
print( datax['symbol'] + " " + datax['price'])

Store the data in a deque (from the collections module).
Initialise your deque like this:
from collections import deque
d = deque([], 2)
Now you can append to d as many times as you like and it will only ever have the most recent two entries.
So, for example:
d.append('a')
d.append('b')
d.append('c')
for e in d:
print(e)
Will give the output:
b
c
Adapting your code to use this technique should be trivial.
I recommend this approach in favour of using two variables because it's easier to change if you (for some reason) decided that you want the last N values because all you need to do is change the deque constructor

You can just use two variables that you update for each new elements, at the end you will just have the two last elements seen :
pre_last = None
last = None
for line in lines:
coinsq = line.strip()
url = priceKey + coinsq + "USDT"
data = requests.get(url)
datax = data.json()
print( datax['symbol'] + " " + datax['price'])
pre_last = last
last = datax
#Do the required calculations with last and pre_last
(And just to be exact this isn't an infinite loop otherwise there wouldn't be a 'last' element)

As your script does not have prior information of when the execution is going to halt, I suggest to define a queue-like structure. In each iteration, you update your last item and your previous-to-last. In that way, you just have to keep in memory two elements. I don't know how were you planning on accessing those two elements when the execution has finished, but you should be able to access that queue when the execution is over.
Sorry for not providing code, but this can be done in many ways, I supposed it was better to suggest you a way of proceeding.

You can define a variable for the second-last element of the for loop, and use the datax variable that's already defined in the loop as the last element:
sec_last = None
datax = None
for line in lines:
sec_last = datax
coinsq = line.strip()
url = priceKey + coinsq + "USDT"
data = requests.get(url)
datax = data.json()
print( datax['symbol'] + " " + datax['price'])
print(f"Last element", datax)
print(f"Second Last element", sec_last)

Looping list used for .append function

What I am struggling with now a bit, is getting a set of code to run, where I can automate the cycle of chosen lists, to assign different variables to different lists.
I am using .append, but what I am aiming to do is the following:
x.append(y), I'd want the x list used to be chosen from a set that I have, another list of the list names for example. I have made the following code to capture this:
temp_stat_list = ["10Fouls5","2Corners4","5Crosses6","80Touches92"]
stat_picker = 0
extra_stats = ["Fouls","Corners","Crosses","Touches"]
stat_list = ["foul_list","corner_list","cross_list","touch_list"]
stat_list_len = len(stat_list) - 1
while stat_picker <= stat_list_len:
working_stat = extra_stats[stat_picker]
working_list = stat_list[stat_picker]
stat_len = len(working_stat)
add_stat_home = temp_stat_list[stat_picker]
add_stat_home = add_stat_home[:add_stat_home.find(working_stat)]
stat_list[stat_picker].append(int(add_stat_home))
add_stat_away = temp_stat_list[stat_picker]
add_stat_away = add_stat_away[(add_stat_away.find(working_stat) + stat_len):]
stat_list[stat_picker].append(int(add_stat_away))
stat_picker = stat_picker + 1
Ideally in this code the part:
stat_list[stat_picker].append(int(add_stat_home))
stat_list[stat_picker].append(int(add_stat_away))
the stat list portion would move with the stat picker increase at the end. Effectively capturing the variables in the first run, then moving to the next, and adjusting the list accordingly. Thanks!
I've been successfully implemented this chunk with the different instances of the lists, running individually. This of course adds a fair bit of repetitive code, and I am wanting to run a simple while loop to just automate this.
I can get it to work with just manually running each list instance separately, but as a loop, I can't get it to work.
EDIT: Added temp_stat_list example content.

So, I deleted my other post because I realized I accidentally asked the question as an answer, instead of a comment.
Here's what I think you're looking for, let me know if this is it:
import re
temp_stat_list = ["10Fouls5","2Corners4","5Crosses6","80Touches92"]
fouls = {'home':[],
'away':[]}
corners = {'home':[],
'away':[]}
crosses = {'home':[],
'away':[]}
touches = {'home':[],
'away':[]}
for stat in temp_stat_list:
parsed_stats = re.match(r"([0-9]+)([a-z]+)([0-9]+)", stat, re.I)
stats = parsed_stats.groups()
if stats[1] == "Fouls":
fouls['home'].append(stats[0])
fouls['away'].append(stats[2])
elif stats[1] == "Corners":
corners['home'].append(stats[0])
corners['away'].append(stats[2])
elif stats[1] == "Crosses":
crosses['home'].append(stats[0])
crosses['away'].append(stats[2])
else:
touches['home'].append(stats[0])
touches['away'].append(stats[2])
print("Fouls: ")
print(fouls)
print("Corners: ")
print(corners)
print("Crosses: ")
print(crosses)
print("Touches: ")
print(touches)
This gives an output of:
Fouls:
{'home': ['10'], 'away': ['5']}
Corners:
{'home': ['2'], 'away': ['4']}
Crosses:
{'home': ['5'], 'away': ['6']}
Touches:
{'home': ['80'], 'away': ['92']}

Ok so the answer was a lot simpler than I thought, just needed another set of eyes. Really all we needed to do, was use a list of lists, so then the final code that worked perfectly, is the following:
stat_picker = 0
extra_stats = ["Fouls","Corners","Crosses","Touches"]
stat_list = [foul_list,corner_list,cross_list,touch_list]
stat_list_len = len(stat_list) - 1
while stat_picker <= stat_list_len:
working_stat = extra_stats[stat_picker]
working_list = stat_list[stat_picker]
stat_len = len(working_stat)
add_stat_home = temp_stat_list[stat_picker]
add_stat_home = add_stat_home[:add_stat_home.find(working_stat)]
stat_list[stat_picker].append(int(add_stat_home))
add_stat_away = temp_stat_list[stat_picker]
add_stat_away = add_stat_away[(add_stat_away.find(working_stat) + stat_len):]
stat_list[stat_picker].append(int(add_stat_away))
stat_picker = stat_picker + 1
The change here is that the stat_list needed to be a list of lists instead. A list of strings wouldn't be recognized as lists being applied.

Python - How to read different json files but to keep same list running

This might be a tough question and I will do my best to explain the best I can!
Im trying to create a script where I run different json files through a forloop (Each of these json files has their own "data") - What I want to do is that I want to be able to add those values into a list where I later match the first object in the json and matches if it is found in the script. If it is found with the same Name then we gonna check if its last object is higher or lower than it was previous in that object name. If it is higher then we print that it has been higher and we change the value object to the new one (Which we change inside a list).
Also another thing that I want it to do is that It should only append to the list once and not n times depending on how many json files I use.
I start by showing of the json that I use which we can see it contains same Name but different number.
{
"name": "Albert",
"image": "https://pbs.twimg.com/profile_images/....jpg",
"number": "5"
}
-----------------------------------
{
"name": "Albert",
"image": "https://pbs.twimg.com/profile_images/....jpg",
"number": "6"
}
Ok before I continue to explain. Here is the code I wrote so I can explain it better...
webhook_list = [
'https://discordapp.com/api/webhooks/5133124/slack',
'https://discordapp.com/api/webhooks/5124124/slack',
'https://discordapp.com/api/webhooks/5112412/slack'] #Discord different API_key/Webhook
def get_identifier(thread):
thread_id = thread['name'] #Grab name from Json
try:
thread_image = thread['image'] #Grab Image from Json
except KeyError:
thread_image = None
try:
thread_number = thread['numbers'] #Grab Image from Json
except KeyError:
thread_number = None
identifier = ('{}%{}%{}').format(thread_id, thread_image, thread_number) #Make them all into one "String"
return identifier
def script():
old_list = [] #Old_list where we gonna append if new items arrives.
while True:
for thread in [line.rstrip('\n') for line in open('names.txt')]: #We check all names in txt file (This can be changed to however you want to print, could be a list aswell. Doesn't matter
get_value_identifier = get_identifier(thread) #We send the value thread to get_identifier to grab the values which return of the identifier
if get_identifier(thread) not in old_list: #if this value is not in the old list then we go in here
#Slack/Discord function
directory = os.fsencode('./slack')
for counters, file in enumerate(os.listdir(directory)):
filename = os.fsdecode(file)
if filename.endswith(".json"):
with open('./slack/' + filename) as slackAttachment:
data = json.loads(slackAttachment.read())
data_list = []
# *****************---Picture---*****************
try:
data["attachments"][0]["thumb_url"] = information ['image_url'] #We add everything to data so we can later on be able to print it out to discord/slack
except Exception:
data["attachments"][0][
"thumb_url"] = 'https://cdn.browshot.com/static/images/not-found.png'
# *****************---Footer---*****************
data["attachments"][0]["footer"] = str(
data["attachments"][0]["footer"] + ' | ' + datetime.now().strftime(
'%Y-%m-%d [%H:%M:%S.%f')[:-3] + "]")
# -------------------------------------------------------------------------
a = get_value_identifier.split("%") #We split the identifier meaning it will be name image number
for i, items in zip(range(len(old_list)), old_list): #We split the length of old_list length and the old_list (I didn't think out of anything other way than this. This can be changed)
old_list_value = old_list[i].split("%") #We also split the old_list values the same way as we did with *a =...*
if a[0] in old_list_value[0]: #if the first value of both *a* in inside somewhere in a old_list first value.....
if old_list_value[2] < a[2]: #We check if the number is higher than the old_list. If it is then we do the things below
data["attachments"][0]["title"] = information['name'].upper()
data_list.append((webhook_list[counters], data))
for hook, data in data_list:
threading.Thread(target=sendData, args=(hook, data)).start()
old_list[i] = get_value_identifier
break
elif len(old_list_value[2]) >= len(a[2]): #We check if the number is lower than the old_list. If it is then we do the things below
old_list[i] = get_value_identifier
break
else: #If nothing is found then we just do things below here and add the value to old_list.
data["attachments"][0]["title"] = information['name'].upper()
data_list.append((webhook_list[counters], data))
for hook, data in data_list:
threading.Thread(target=sendData, args=(hook, data)).start()
old_list.append(get_value_identifier)
else:
randomtime = random.randint(3, 7)
logger.warn('No new item found! - retrying in %d secs' % (randomtime))
time.sleep(randomtime)
As you can see, this is my code that I do for opening each json format and we use data = json.loads(slackAttachment.read()) meaning it will add up into "data" which is a json.
directory = os.fsencode('./slack')
for counters, file in enumerate(os.listdir(directory)):
filename = os.fsdecode(file)
if filename.endswith(".json"):
with open('./slack/' + filename) as slackAttachment:
data = json.loads(slackAttachment.read())
Whenever one of the loop is done - It adds up everything into data and later on at the end we can print out the data or send it through a requests to Discord/Slack.
But in the end before I want to print it out to discord/slack - I am checking first of all if the name is already in the list. We do that by splitting each % which will contains "Name image Number"
In the if statement we check if a[0] (Which is current thread found) - if it is somewhere in a old list.
IF it is in the old list then we check the last number on the object
if it is higher or lower
IF it is higher then we print it out since its a increase of value
and then we change the previous old_list value to this new one.
IF it is lower then we just change the previous old_list value to the
new one.
IF there is none in the old_list that matches then we just append it
to the list.
.
a = get_value_identifier.split("%")
for i, items in zip(range(len(old_list)), old_list):
old_list_value = old_list[i].split("%")
if a[0] in old_list_value[0]:
if old_list_value[2] < a[2]:
data["attachments"][0]["title"] = information['name'].upper()
data_list.append((webhook_list[counters], data))
for hook, data in data_list:
threading.Thread(target=sendData, args=(hook, data)).start()
old_list[i] = get_value_identifier
break
elif len(old_list_value[2]) > len(a[2]):
old_list[i] = get_value_identifier
break
else:
data["attachments"][0]["title"] = information['name'].upper()
data_list.append((webhook_list[counters], data))
for hook, data in data_list:
threading.Thread(target=sendData, args=(hook, data)).start()
old_list.append(get_value_identifier)
and here is the mechanic issue.
The issue I am having is that at the beginning when we run
directory = os.fsencode('./slack')
for counters, file in enumerate(os.listdir(directory)):
filename = os.fsdecode(file)
if filename.endswith(".json"):
with open('./slack/' + filename) as slackAttachment:
data = json.loads(slackAttachment.read())
is that it will be looping through the function above
a = get_value_identifier.split("%")
for i, items in zip(range(len(old_list)), old_list):
old_list_value = old_list[i].split("%")......
x times depending on how many files there is in the slack. This will be a issue after second loop becuase if the first loop find a new item. It will add it into the old_list using append but that means when the second loop with slack/discord it means that there will be a value in the old_list that contains it. And here is where it start continuing... Meaning the first loop will always be correct but after that it will give you not the correct answers.
My question in that case is two question
How can I make so whenever the first loop hits etc if statement or elif or else. It should print the same for all discord/slack at once .
If it hits if or elif statement. Inside those I have that it should take the value from old_list[i] and change that value to the "newer" one which is get_value_identifier. How can I make it so it does it only once. Because I believe if I run through 3 slacks in my case. It will be 3 same values in the old_list due to the for loop.
I think that is it for me and I hope I explained so good I could! Please. If there is any more question to be added or anything. I will be pretty active now and I will be able to edit to either upgrade the question due to your questions below!

Add up the value of data[x] to data[x+1]

I have a long list of data which I am working with now,containing a list of 'timestamp' versus 'quantity'. However, the timestamp in the list is not all in order (for example,timestamp[x] can be 140056 while timestamp[x+1] can be 560). I am not going to arrange them, but to add up the value of timestamp[x] to timestamp[x+1] when this happens.
ps:The arrangement of quantity needs to be in the same order as in the list when plotting.
I have been working with this using the following code, which timestamp is the name of the list which contain all the timestamp values:
for t in timestamp:
previous = timestamp[t-1]
increment = 0
if previous > timestamp[t]:
increment = previous
t += increment
delta = datetime.timedelta(0, (t - startTimeStamp) / 1000);
timeAtT = fileStartDate + (delta + startTime)
print("time at t=" + str(t) + " is: " + str(timeAtT));
previous = t
However it comes out with TypeError: list indices must be integers, not tuples. May I know how to solve this, or any other ways of doing this task? Thanks!

The problem is that you're treating t as if it is an index of the list. In your case, t holds the actual values of the list, so constructions like timestamp[t] are not valid. You either want:
for t in range(len(timestamp)):
Or if you want both an index and the value:
for (t, value) in enumerate(timestamp):

When you for the in timestamp you are making t take on the value of each item in timestamp. But then you try to use t as an index to make previous. To do this, try:
for i, t, in enumerate(timestamp):
previous = timestamp[i]
current = t
Also when you get TypeErrors like this make sure you try printing out the intermediate steps, so you can see exactly what is going wrong.

How to match fields from two lists and further filter based upon the values in subsequent fields?

EDIT: My question was answered on reddit. Here is the link if anyone is interested in the answer to this problem https://www.reddit.com/r/learnpython/comments/42ibhg/how_to_match_fields_from_two_lists_and_further/
I am attempting to get the pos and alt strings from file1 to match up with what is in
file2, fairly simple. However, file2 has values in the 17th split element/column to the
last element/column (340th) which contains string such as 1/1:1.2.2:51:12 which
I also want to filter for.
I want to extract the rows from file2 that contain/match the pos and alt from file1.
Thereafter, I want to further filter the matched results that only contain certain
values in the 17th split element/column onwards. But to do so the values would have to
be split by ":" so I can filter for split[0] = "1/1" and split[2] > 50. The problem is
I have no idea how to do this.
I imagine I will have to iterate over these and split but I am not sure how to do this
as the code is presently in a loop and the values I want to filter are in columns not rows.
Any advice would be greatly appreciated, I have sat with this problem since Friday and
have yet to find a solution.
import os,itertools,re
file1 = open("file1.txt","r")
file2 = open("file2.txt","r")
matched = []
for (x),(y) in itertools.product(file2,file1):
if not x.startswith("#"):
cells_y = y.split("\t")
pos_y = cells[0]
alt_y = cells[3]
cells_x = x.split("\t")
pos_x = cells_x[0]+":"+cells_x[1]
alt_x = cells_x[4]
if pos_y in pos_x and alt_y in alt_x:
matched.append(x)
for z in matched:
cells_z = z.split("\t")
if cells_z[16:len(cells_z)]:

Your requirement is not clear, but you might mean this:
for (x),(y) in itertools.product(file2,file1):
if x.startswith("#"):
continue
cells_y = y.split("\t")
pos_y = cells[0]
alt_y = cells[3]
cells_x = x.split("\t")
pos_x = cells_x[0]+":"+cells_x[1]
alt_x = cells_x[4]
if pos_y != pos_x: continue
if alt_y != alt_x: continue
extra_match = False
for f in range(17, 341):
y_extra = y[f].split(':')
if y_extra[0] != '1/1': continue
if y_extra[2] <= 50: continue
extra_match = True
break
if not extra_match: continue
xy = x + y
matched.append(xy)
I chose to concatenate x and y into the matched array, since I wasn't sure whether or not you would want all the data. If not, feel free to go back to just appending x or y.

You may want to look into the csv library, which can use tab as a delimiter. You can also use a generator and/or guards to make the code a bit more pythonic and efficient. I think your approach with indexes works pretty well, but it would be easy to break when trying to modify down the road, or to update if your file lines change shape. You may wish to create objects (I use NamedTuples in the last part) to represent your lines and make it much easier to read/refine down the road.
Lastly, remember that Python has a shortcut feature with the comparative 'if'
for example:
if x_evaluation and y_evaluation:
do some stuff
when x_evaluation returns False, Python will skip y_evaluation entirely. In your code, cells_x[0]+":"+cells_x[1] is evaluated every single time you iterate the loop. Instead of storing this value, I wait until the easier alt comparison evaluates to True before doing this (comparatively) heavier/uglier check.
import csv
def filter_matching_alt_and_pos(first_file, second_file):
for x in csv.reader(open(first_file, 'rb'), delimiter='\t'):
for y in csv.reader(open(second_file, 'rb'), delimiter='\t'):
# continue will skip the rest of this loop and go to the next value for y
# this way, we can abort as soon as one value isn't what we want
# .. todo:: we could make a filter function and even use the filter() built-in depending on needs!
if x[3] == y[4] and x[0] == ":".join(y[:1]):
yield x
def match_datestamp_and_alt_and_pos(first_file, second_file):
for z in filter_matching_alt_and_pos(first_file, second_file):
for element in z[16:]:
# I am not sure I fully understood your filter needs for the 2nd half. Here, I split all elements from the 17th onward and look for the two cases you mentioned. This seems like it might be very heavy, but at least we're using generators!
# same idea as before, we abort as early as possible to avoid needless indexing and checks
for chunk in element.split(":"):
# WARNING: if you aren't 100% sure the 2nd element is an int, this is very dangerous
# here, I use the continue keyword and the negative-check to help eliminate excess overhead. The execution is very similar as above, but might be easier to read/understand and can help speed things along in some cases
# once again, I do the lighter check before the heavier one
if not int(chunk[2])> 50:
# continue automatically skips to the next iteration on element
continue
if not chunk[:1] == "1/1":
continue
yield z
if __name__ == '__main__':
first_file = "first.txt"
second_file = "second.txt"
# match_datestamp_and_alt_and_pos returns a generator; for loop through it for the lines which matched all 4 cases
match_datestamp_and_alt_and_pos(first_file=first_file, second_file=second_file)
namedtuples for the first part
from collections import namedtuple
FirstFileElement = namedtuple("FirstFrameElement", "pos unused1 unused2 alt")
SecondFileElement = namedtuple("SecondFrameElement", "pos1 pos2 unused2 unused3 alt")
def filter_matching_alt_and_pos(first_file, second_file):
for x in csv.reader(open(first_file, 'rb'), delimiter='\t'):
for y in csv.reader(open(second_file, 'rb'), delimiter='\t'):
# continue will skip the rest of this loop and go to the next value for y
# this way, we can abort as soon as one value isn't what we want
# .. todo:: we could make a filter function and even use the filter() built-in depending on needs!
x_element = FirstFileElement(*x)
y_element = SecondFileElement(*y)
if x.alt == y.alt and x.pos == ":".join([y.pos1, y.pos2]):
yield x