Develop Reference

Parsing HTML with BeautifulSoup in Python freeze my script

I'm parsing a web page every 5 seconds to detect changes. This code runs almost all days and nights without problem, except a few times per day it freezes and I have to restart it. I could go around the problem with a timer that restarts automatically my script but I'd like to understand the problem.
Code:
while 1:
[...]
print(">>debug 1")
soup = BeautifulSoup(response, 'html.parser')
print(">>debug 2")
[...]
Output:
[...]
>>debug 1
>>debug 2
>>debug 1
This is where the script freezes. Last output is always ">>debug 1" then it freezes without crashing.
Why would that line freeze the script randomly once every ~3000 executions ? How can I investigate further ?
Thank you

Reading a file every 30 minutes in Python

As you see in the below code, it is possible to open a file in a directory and read it. now i want live_token read the file every 30 minutes and print it. Can anyone help me in this regard?
I found below code as scheduling to do a job but i don't know how to do needful modifications.
schedule.every(30).minutes.do()
Sorry if this question is so basic, I am so new with Python.
def read_key():
live_key_file_loc = r'C:\key.txt'
live_key_file = open(live_key_file_loc , 'r')
global key_token
time.sleep(6)
live_token=live_key_file.read()
print(live_token)

import time
sleep_time = 30 * 60 # Converting 30 minutes to seconds
def read_key():
live_key_file_loc = r'C:\key.txt'
live_key_file = open(live_key_file_loc, 'r')
global key_token
time.sleep(6)
live_token = live_key_file.read()
print(live_token)
while(True): # This loop runs forever! Feel free to add some conditions if you want!
# If you want to read first then wait for 30 minutes then use this-
read_key()
time.sleep(sleep_time)
# If you want to wait first then read use this-
time.sleep(sleep_time)
read_key()

#jonrsharpe is right. Refer to schedule usage. You should have a script which should keep running always to fetch the token every 30 minutes from the file. I have put below a script which should work for you. If you dont want to run this file in python always, look for implementing a scheduled job.
import schedule
import time
def read_key():
with open('C:\\key.txt' , 'r') as live_key_file_loc
live_token = live_key_file_loc.read()
print(live_token)
schedule.every(30).minutes.do(read_key)
while True:
schedule.run_pending()
time.sleep(1)

There are a few steps in this process.
Search for “Task Scheduler” and open Windows Task Scheduler GUI.
Go to Actions > Create Task…
Name your action.
Under the Actions tab, click New
Find your Python Path by entering where python in the command line. Copy the result and put it in the Program/Script input.
In the "Add arguments (optional)" box, put the name of your script. Ex. - in "C:\user\your_python_project_path\yourFile.py", put "yourFile.py".
In the "Start in (optional)" box, put the path to your script. Ex. - in "C:\user\your_python_project_path\yourFile.py", put "C:\user\your_python_project_path".
Click “OK”.
Go to “Triggers” > New and choose the repetition that you want.
For more details check this site -
https://www.jcchouinard.com/python-automation-using-task-scheduler/

Python3: print statement of a variable leads to a different output

I write a Python script to manage my account on a webpage automatically.
Code Description:
The script has a while loop and at the end of the loop, it waits 12 hours before starting again.
Before the while loop starts, it's logging in to my account, and when entering the while loop, it checks if I'm still logged in. If not, it's logging in to my account again.
Problem:
After re-entering the while loop (first time everything goes fine), the script does only work, when print("Name is:") and print(name) is at the very beginning. I tested it several times and maybe it is just a bug/glitch, which was just unlucky to be caused only when the print statements aren't there, but this is very confusing me right now, how those print statements fixed my issue. I would like to know, what is or could causing the issue and how do I have to solve it properly?
Some side info:
The webpage is saving the login credentials through session cookies with a lifetime of ~6 hours. So after re-entering the script loop again, I'm not logged in for sure. If I'm reducing the wait time to 30 minutes instead of 12 hours, the script works also without the print statements.
General notes:
The script is running through nohup on my Raspberry Pi 3
Python version is 3.7.3
Code related notes:
I'm using the post method from requests to log in to my account
For checking, if I'm still logged in, I'm using beautifulSoup4
The following code is abbreviated and in a very basic shape.
"account" is an instance of a self-made class. When instantiating, it is log in itself with arguments, if given
This is the core code:
import time
import requests
from account import Account # costum made class
from bs4 import BeautifulSoup
# login credentials
name = "lol" # I replaced them with placeholders
pw = "lol"
account = Account(name, pw) # instantiating an account class and log in itself with given arguments
while True: # script loop
print("name is:") # Without those both print statements,
print(name) # the code won't work
if not account.stillAlive(): # if not signed in anymore ...
account.login(name, pw) # ... sign in again
account.doStuff() # Do the automating stuff
time.sleep(43200) # Wait 12 hours, before entering the while loop again
This is the doStuff() method from the Account class:
def doStuff(self):
html = requests.get("example.com").text # Note: example.com is only for demonstration purpose only
crawler = BeautifulSoup(html, "lxml")
lol = crawler.find("input", attrs={"name": "Submit2"}).get("value")
# ...
Error message:
So, if I'm executing the program without the print statements, I'm getting this error:
lol = crawler.find("input", attrs={"name": "Submit2"}).get("value")
AttributeError: 'NoneType' object has no attribute 'get'
This does not occur when executing with the print statements. With the print statements, the code runs fine.
My guess
My guess is, what the memory management of Python is deleting the name variable. When entering the script loop in the first time, I'm already logged in and therefore it is skipping the account.login(name, pw) part. Since this is the only part, where name is continued to be, maybe Python is interpreting this as dead code after too many time has passed without the line to be executed, and don't see the reason to keep the name/pw variable and deletes them. Still, I'm just an amateur and I don't have any expertise in this segment.
Side notes:
This is my first question I'm submitting, if I forgot or did something wrong, pls tell me.
I already searched for this problem, but I didn't find anything similar. Maybe I just searched badly, but I searched for a few hours now. If so, I apologize. (I had to wait for every test 12 hours and since I tried it several times, you can tell, I had some time available to search)

Python script with cron only finished the first loop

I have a python script which runs a for loop. I made it executable and put it in a cron job.
It posts a few tweets on twitter. For each loop, it sleeps a few seconds with random times.
However, it appears it only runs the very first loop and then stops. Every time, I only got ONE tweet. I could not figure out why.
Here is the core part of the code.
def post_message(url):
d = parse(url)
entries = d.entries
for entry in entries:
str = entry.title
tweet(str)
t = random.randint(start, stop)
time.sleep(t)
This is how I set it in cron.
0 23 * * * /home/demo/post_message.py
It only post the very first one and then stops. I am wondering if the time.sleep function stops the rest loops in cron?
Thanks.

Have you tried to run this python script from shell?
Is there any exception (Like you posted too frequently led the script to a 403 page which keeps the function from finding any appropriate html element) generated in 2nd tweet() function?

Trying to queuing up multiple files in pygame.music

I'm trying to get a queue of music files in random order for playing on a raspberry pi using pygame. But I'm finding an issue (limitation?) of where it seems to be only able to queue up song.
I've tried to reproduce it on my desktop (so I can debug it) but the queuing doesn't seem to work at all on it. It can't even queue up 1 other file. Hence, I'm wondering why it won't even work on my desktop. I've stripped down my desktop testing/debugging code to as follows:
import pygame
import random
import time
soundType = ".ogg"
# files are in the same directory at the moment
musicLocation = ""
musicFiles = (("takepills01", 0), ("takepistol01", 0), ("takesniper01", 0), ("taunt01", 0), ("teamkillaccident02", 0), ("thanks01", 0))
queuedFilesResut = []
pygame.mixer.init()
# choose a random file to start
selectedMusicName = musicFiles[random.randint(0, len(musicFiles) - 1)][0]
pygame.mixer.music.load(musicLocation + selectedMusicName + soundType)
queuedFilesResut.append(selectedMusicName)
# choose a random file to follow
selectedMusicName = musicFiles[random.randint(0, len(musicFiles) - 1)][0]
pygame.mixer.music.play()
pygame.mixer.music.queue(musicLocation + selectedMusicName + soundType)
queuedFilesResut.append(selectedMusicName)
# not sure if it matters if the play() call is before or after the queue() call
# seen it both ways on a few examples
#pygame.mixer.music.play()
# print to see what was selected
print queuedFilesResut
# now sleep for a bit while the music plays, the testing files are very short, each only 1-2 seconds
time.sleep(5)
What happens right now for the above code is that whatever file is selected first plays, then that's it. Here is how I was attempting to queue up multiple files, before I realised I couldn't even queue up 1 file:
for counter in range(0, 10):
selectedMusicName = musicFiles[random.randint(0, len(musicFiles) - 1)][0]
pygame.mixer.music.queue(musicLocation + selectedMusicName + soundType)
queuedFilesResut.append(selectedMusicName)
I was really expecting pygame to be able to just play a song and queue up 'n' files, then I could just forget about it and leave it run its course.
Although this is just a question more on why the above script doesn't work, I should add that when I go back to the raspberry pi, I'm calling the music playing python script via lighttpd and thus need the method call to return to be responsive -I don't think I can use a loop to check for when the music is done playing/check the event queue. Lighttpd also seems to limit python to just 1 thread.
I do have an idea on how to get around this whole issue/limitation, but it involves running a loop as an entirely separate python process on startup and having a written file to communicate between lighttpd-python and python. I am hoping to find a nicer/cleaner way.

I know this was in 2014 and was with python 2.7 but this could be useful to others with this problem.
I am using python 3.4.
You could try to put your randomly chosen files in a list and play the list like a playlist. The songs will play from left to right in the list.
E.g.
for file in musicList:
#play song here
time.sleep(5)
You can add files to this list mid program by using:
#Chosen random music file
songList.append(musicFile)
Hope this helps.
Cheers