I couldn't find a proper response so I post this question.
The fastest way to understand the question is the goal:
There is a main process and a subprocess (the one I want to create). The main process inspects several websites via webdriver, but sometimes it got stuck at low selenium level and don't want to change the official code. So.. I manually inspect sometimes the monitor to the check whether the process got stuck, and if so, then I change manually the url in the browser and it works again smooth. I don't want to be a human checker.. so i'd like to automate the task with a subprocess that shares the same webdriver and inspects the url by webdriver.current_url and do the work for me.
Here is my try in the minimal representative example form in which the sub-process only detects a change in the url of the webdriver
def test_sub(driver):
str_site0 = driver.current_url # get the site0 url
time.sleep(4) # give some time to the main-process to change to site1
str_site1 = driver.current_url # get the site1 url (changed by main-process)
if str_site0 == str_site1:
print('sub: no change detected')
else:
print('sub: change detected')
#endif
#enddef sub
def test_main():
""" main process changes from site0 (stackoverflow) to site1 (youtube)
sub process detects this change of url of the webdriver object (same pointer) by using
".current_url" method
"""
# init driver
pat_webdriver = r"E:\WPy64-3680\python-3.6.8.amd64\Lib\site-packages\selenium\v83_chromedriver\chromedriver.exe"
driver = webdriver.Chrome(executable_path= pat_webdriver)
time.sleep(2)
# open initial site
str_site0 = 'https://stackoverflow.com'
driver.get(str_site0)
time.sleep(2)
# init sub and try to pass the webdriver object
p = multiprocessing.Process(target=test_sub, args=(driver,)) # PROBLEM HERE! PYTHON UNCAPABLE
p.daemon = False
p.start()
# change site
time.sleep(0.5) # give some time sub query webdriver with site0
str_site1 = 'https://youtube.com' # site 1 (this needs to be detected by sub)
driver.get(str_site1)
# wait the sub to detect the change in url. and kill process (non-daemon insufficient don't know why..)
time.sleep(3)
p.terminate()
#enddef test_main
# init the program (main-process)
test_main()
the corresponding error by executing $python test_multithread.py (it's the name of the test script..) is the following one:
Related
The code I wrote is very basic and it simply works. It takes a value on the Selenium-related site, writes it to a txt, then reads the necessary part of the value and should sound an alarm according to this value. The code terminates before reaching the alarm part or it does not see the alarm part. The problem here may be related to the value I got from the txt, but I could not solve the problem despite my attempts. How can I solve this?
note:There is no problem with the vlc library, it works when used separately and in this example the value in the txt is 12 feb 2022 and it only reads the first character
from selenium import webdriver
import time
import vlc
driver = webdriver.Chrome()
driver.get("https://demoqa.com/automation-practice-form")
driver.maximize_window()
print("Site Title:",driver.title)
#####################################################
nameElement =driver.find_element_by_id("dateOfBirthInput")
nameElement.click()
time.sleep(5)
taleptAttribute = nameElement.get_attribute('value')
print(taleptAttribute)
#print("Talep Sayısı: " + nameElement.get_attribute('value'))
################################################################
talep_satırı = open("talep_satiri.txt", "w")
talep_satırı.write(taleptAttribute)
talep_satırı = open("talep_satiri.txt","r")
talepsayisi=talep_satırı.read(1)
print(talepsayisi)
alarm = vlc.MediaPlayer("path")
if (talepsayisi == 1 ):
alarm.play()
time.sleep(10)
alarm.stop()
else:
alarm.play()
I want to create a program, that can read all the messages from my whatsApp and print them to the screen using python.
In order to do that I tried using the whatsapp-web library https://pypi.org/project/whatsapp-web/.
But when i tried to run their code example I got a timeout error
this is the code
import time
from selenium import webdriver
from simon.accounts.pages import LoginPage
from simon.header.pages import HeaderPage
from simon.pages import BasePage
# Creating the driver (browser)
driver = webdriver.Firefox()
driver.maximize_window()
login_page = LoginPage(driver)
login_page.load()
login_page.remember_me = False
time.sleep(7)
base_page = BasePage(driver)
base_page.is_welcome_page_available()
base_page.is_nav_bar_page_available()
base_page.is_search_page_available()
base_page.is_pane_page_available()
base_page.is_chat_page_available()
# 3. Logout
header_page = HeaderPage(driver)
header_page.logout()
# Close the browser
driver.quit()
and this is the error
base_page.is_welcome_page_available()
File "D:\zoom\venv\lib\site-packages\simon\pages.py", line 18, in wrapper
return func(*args, **kwargs)
File "D:\zoom\venv\lib\site-packages\simon\pages.py", line 51, in is_welcome_page_available
if self._find_element(WelcomeLocators.WELCOME):
File "D:\zoom\venv\lib\site-packages\simon\pages.py", line 77, in _find_element
lambda driver: self.driver.find_element(*locator))
File "D:\zoom\venv\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Your code is not completed to find error or correct.
Because the error is coming from the imported page.
Try by increasing the time limit to load the page,
use time.sleep(15)
You can try to automate the WhatsApp web by yourself without using pip of WhatsApp-web. because I guess that is not updated.
The code is giving error because of WhatsApp web has changed its elements classes names. Because of that the code section is not able to find welcome page in given time limit. So the program execution breaks
I have done the same without using WhatsApp-web pip.
I hope it will work for you.
A complete code reference Example :
from selenium import webdriver
import time
# You can use any web-browser which supported by selenium and which can run WhatsApp web.
# For using GoogleChrome
web_driver = webdriver.Chrome("Chrome_Driver_Path/chromedriver.exe")
web_driver.get("https://web.whatsapp.com/")
# For using Firefox
# web_driver = webdriver.Firefox(executable_path=r"C:/Users/Pascal/Desktop/geckodriver.exe")
# web_driver.get("https://web.whatsapp.com/")
time.sleep(25) # For scan the qr code
# Plese make sure that you have done the qr code scan successful.
confirm = int(input("Press 1 to proceed if sucessfully login or press 0 for retry : "))
if confirm == 1:
print("Continuing...")
elif confirm == 0:
web_driver.close()
exit()
else:
print("Sorry Please Try again")
web_driver.close()
exit()
while True:
unread_chats = web_driver.find_elements_by_xpath("// span[#class='_38M1B']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing the number of unread message showing the contact card inside a green circle before opening the chat room.
# Open each chat using loop and read message.
for chat in unread_chats:
chat.click()
time.sleep(2)
# For getting message to perform action
message = web_driver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in message:
try:
print("Message received : " + str(i.text))
# Here you can use you code to perform action according to your need
except:
pass
Please make sure that the indentation is equal in code blocks if you are copying it.
Can following link for more info about WhatsApp web using python.
https://stackoverflow.com/a/68288416/15284163
I am developing WhatsApp bot using python.
For contribution you can contact at : anurag.cse016#gmail.com
Please give a star on my https://github.com/4NUR46 If this Answer helps you.
I have a selenium set up as:
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600')
driver = webdriver.Chrome(options=options)
driver.implicitly_wait(10)
driver.get('file:///path_to_file')
When I execute the script:
data = driver.execute_script('return document.getElementsByClassName("runner-canvas")[1].getContext("2d").getImageData(0,0,600,150);')['data']`
Data is all zeros: [0,0,0,0, 0,0,0,0 ..., 0,0,0,0].
But when I take a screenshoot, at the same time, with:
driver.save_screenshot(os.path.join(os.path.dirname(os.path.realpath(__file__)), '.', 'screenshot.png'))
I can see that the canvas is populated.
Canvas loads a game that doesn't start till the SPACE is pressed.
Function that is responsible for collecting the canvas data looks similar to this:
# Somewhere before the __get_data is called
self.document.send_keys(Keys.SPACE) # self.document is set to html document
def __get_data(self):
while self.driver.execute_script("return Runner.instance_.started") == False:
print('Waiting to start')
# data is always empty at this stage
data = self.driver.execute_script('return document.getElementsByClassName("runner-canvas")[1].getContext("2d").getImageData(0,0,600,150);')['data']
rgba = np.array(data).reshape((90000, 4))
b = a[:, 2]
return a.reshape((150, 600))
When I run it I can see a lot of 'Waiting to start' in a console, hence I don't think it is a timing issue as by the time while breaks everything should be drawn as the game already started.
Im on Mac running ChromeDriver 2.46.628411
Thought so the image that you are trying to capture or get the data from it it's not loaded properly hence you having the empty data, do one thing find out how much time it's taking to load the image and add that much wait and after that get the data and screenshot
Let us know that if works or not..
I don't really have idea about that so I'd like you to give me some advice if you can.
Generally when I use Selenium I try to search the element that I'm interested in, but now I was thinking to develop some kind of performance test so check how much time take a specific webpage (html, script, etc...) to load.
Do you have some idea how to know the load time of html, script etc without search for a specific element of the page?
PS I use IE or Firefox
You could check the underlying javascript framework for active connections. When there are no active connections you could then assume the page is finished loading.
That, however, requires that you either know what framework the page uses, or that you must systematically check for different frameworks and then check for connections.
def get_js_framework(driver):
frameworks = [
'return jQuery.active',
'return Ajax.activeRequestCount',
'return dojo.io.XMLHTTPTransport.inFlight.length'
]
for f in frameworks:
try:
driver.execute_script(f)
except Exception:
logging.debug("{0} didn't work, trying next js framework".format(f))
continue
else:
return f
else:
return None
def load_page(driver, link):
timeout = 5
begin = time.time()
driver.get(link)
js = _get_js_framework(driver)
if js:
while driver.execute_script(js) and time.time() < begin + timeout:
time.sleep(0.25)
else:
time.sleep(timeout)
I've seen a few instances of this question, but I was not sure how to apply the changes to my particular situation. I have code that monitors a webpage for changes and refreshes every 30 seconds, as follows:
import sys
import ctypes
from time import sleep
from Checker import Checker
USERNAME = sys.argv[1]
PASSWORD = sys.argv[2]
def main():
crawler = Checker()
crawler.login(USERNAME, PASSWORD)
crawler.click_data()
crawler.view_page()
while crawler.check_page():
crawler.wait_for_table()
crawler.refresh()
ctypes.windll.user32.MessageBoxW(0, "A change has been made!", "Attention", 1)
if __name__ == "__main__":
main()
The problem is that Selenium will always show an error stating it is unable to locate the element after the first refresh has been made. The element in question, I suspect, is a table from which I retrieve data using the following function:
def get_data_cells(self):
contents = []
table_id = "table.datadisplaytable:nth-child(4)"
table = self.driver.find_element(By.CSS_SELECTOR, table_id)
cells = table.find_elements_by_tag_name('td')
for cell in cells:
contents.append(cell.text)
return contents
I can't tell if the issue is in the above function or in the main(). What's an easy way to get Selenium to refresh the page without returning such an error?
Update:
I've added a wait function and adjusted the main() function accordinly:
def wait_for_table(self):
table_selector = "table.datadisplaytable:nth-child(4)"
delay = 60
try:
wait = ui.WebDriverWait(self.driver, delay)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, table_selector)))
except TimeoutError:
print("Operation timeout! The requested element never loaded.")
Since the same error is still occurring, either my timing function is not working properly or it is not a timing issue.
I've run into the same issue while doing web scraping before and found that re-sending the GET request (instead of refreshing) seemed to eliminate it.
It's not very elegant, but it worked for me.
I appear to have fixed my own problem.
My refresh() function was written as follows:
def refresh():
self.driver.refresh()
All I did was switch frames right after the refresh() call. That is:
def refresh():
self.driver.refresh()
self.driver.switch_to.frame("content")
This took care of it. I can see that the page is now refreshing without issues.