I'm trying to make an If/Else condition on the title of a specific webpage.
If the title is not 'You have been blocked', my script continue.
I've got an issue with selenium in python with different environment, in the first one (OSX) everything works.
I can grab the title check if it contains specific string and continue or not my script.
In Debian and headless mode activated, I can't access to the title of the web page.
driver.get(url)
print(driver.title)
if driver.title != 'You have been blocked':
print('Ok have fun')
else:
print('blocked')
It seems this function .title is very unstable, is there a better way to achieve it ?
Thanks
I've found the right way to work with .title.
Now my script wait the title of the webpage and and I parse it when it is loaded.
wait = WebDriverWait(driver, 15)
wait.until(EC.presence_of_element_located((By.TAG_NAME,"title")))
if (driver.title != 'You have been blocked'):
print('Ok Have Fun')
else:
print('Blocked')
Thanks
Related
i trying to create a small program usign python and i need some help about python loops.
this small program will automates a fairly boring repetitive task.
I use module : selenium, time and pyautogui
here is the piece of code that i want to repeat until it no longer finds a certain element on the web page :
btnOptions = driver.find_element(By.XPATH, "/html/body/div[1]/div/div[1]/div/div[5]/div/div/div[3]/div/div/div[1]/div[1]/div/div/div[4]/div[2]/div/div[2]/div[3]/div[1]/div/div/div/div/div/div/div/div/div/div/div[8]/div/div[2]/div/div[3]/div/div")
btnOptions.click()
time.sleep(1)
pyautogui.press("down", presses=10)
pyautogui.press("enter")
time.sleep(1)
btn_move = driver.find_element(By.XPATH, "/html/body/div[1]/div/div[1]/div/div[6]/div/div/div[1]/div/div[2]/div/div/div/div/div/div/div[3]/div/div/div/div[1]")
btn_move.click()
as long as the btn_option is found, the program must continue otherwise it must stop.
I can't find a solution, if anyone can help me it would be very appreciated.
thanks a lot
i tried several loops but every time i get an error of course, I'm just new to coding in python :/
I've done a similar task. You may try this:
while True:
try:
btnOptions = driver.find_element(By.XPATH, "/html/body/div[1]/div/div[1]/div/div[5]/div/div/div[3]/div/div/div[1]/div[1]/div/div/div[4]/div[2]/div/div[2]/div[3]/div[1]/div/div/div/div/div/div/div/div/div/div/div[8]/div/div[2]/div/div[3]/div/div")
btnOptions.click()
time.sleep(1)
pyautogui.press("down", presses=10)
pyautogui.press("enter")
time.sleep(1)
btn_move = driver.find_element(By.XPATH, "/html/body/div[1]/div/div[1]/div/div[6]/div/div/div[1]/div/div[2]/div/div/div/div/div/div/div[3]/div/div/div/div[1]")
btn_move.click()
except NoSuchElementException:
break
I'm using Selenium to download several files. The process is simple: I look for a reference, wait until results are ready and download the associated files.
I've a problem with the part "wait until results are ready". The website uses an AJAX table which loads the results. During the update of this table, an attribute appears in the HTML code and when results are ready it dissapears.
The object is always present, it's only the attribute that changes. If I do the next loop just right after clicking the button of search:
for i in range(0,10):
print(self.driver.find_element(By.ID, "gridpoOverview").get_attribute("aria-busy"))
time.sleep(0.05)
It returns (so I know how to detect it):
none
true
true
none
none
none
none
none
none
none
I want to do it using an EC, but the next code doesn't work (Timeout exception):
WebDriverWait(self.driver, 10).until(EC.presence_of_element_located((By.XPATH, '//div[#id="gridpoOverview" and #aria-busy="true"]')))
Seems you are close enough.
As returned by get_attribute("aria-busy") the value of aria-busy attribute turns True.
So you need to replace true with True as follows:
WebDriverWait(self.driver, 10).until(EC.presence_of_element_located((By.XPATH, '//div[#id="gridpoOverview" and #aria-busy="True"]')))
Update
As per your updated question you can use:
WebDriverWait(driver, 10).until(EC.staleness_of(driver.find_element_by_xpath("//div[#id="gridpoOverview" and not(#aria-busy)]")))
Or
WebDriverWait(driver, 10).until(EC.staleness_of(driver.find_element(By.XPATH, "//div[#id="gridpoOverview" and not(#aria-busy)]")))
I've found a solution that works, but I don't know why.
The attribute 'aria-disabled' from the same element is always present. If I include
this line before, it will work.
WebDriverWait(self.driver, 20).until(EC.element_attribute_to_include((By.XPATH, '//div[#id="gridpoOverview"]'),'aria-disabled'))
WebDriverWait(self.driver, 20).until(EC.element_attribute_to_include((By.XPATH, '//div[#id="gridpoOverview"]'),'aria-busy'))
If the first line if not present, it does not work (I've the Timeout exception).
Does anyone knows why ?
Do you see something wrong with this setup?
(selenium, etc. imported earlier on)
It iterates through table_rows until it finds the first row where the “try” is successful, then comes back from the getinfo() function ran from the “try” (which clicks a link, goes to a page, gets info, and then clicks the back button back to the original page), and then keeps iterating through the rest of table_rows.
The correct number of table_rows iterations are performed by the end, and the “try” function is being triggered again (the print() before current_cell works), but the find_element_by_class doesn’t seem to be picking up any more “a0” in the subsequent table_rows iterations, even though there are definitely some there that should be being found (the print() after current_cell never prints after the very first time).
Thank you for your help. I'm new to coding and have learned a ton, but this is stumping me.
def getinfo(current_cell):
link_in_current_cell = current_cell.find_element_by_tag_name("a")
link_in_current_cell.click()
waitfortable2 = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.CLASS_NAME, "top-edit-table"))
)
print("Here is the info about a0.")
driver.back()
return True
for row in table_rows:
print("got the row!")
waitfortable = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.XPATH, "/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr[1]/td[1]"))
)
try:
print("we're trying!")
current_cell = row.find_element_by_class_name("a0")
print("we got an a0!")
getinfo(current_cell)
except:
print("Not an a0 cell.")
pass
continue
Here is more of the code from before "for row in table_rows:" if you need it, but I don't think that part is an issue, as it is still iterating through the rest of table_rows after it finds the first "a0" cell.
try:
WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.XPATH, "/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr[1]/td[1]"))
)
table = driver.find_element_by_xpath("/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table")
table_rows = table.find_elements_by_tag_name("tr")
for row in table_rows:
print("got the row!")
....
....
....(see code box above)
Success! I found my own workaround!
FYI: I still could not see anything wrong with my existing code, as
it correctly found all the a0 cells when I commented out the function
part of that text (#getinfo(current_cell)... thank you #goalie1998 for
the suggestion). And, I didn't change anything in that function for
this new workaround, which works correctly. So, it must have something
to do with Selenium getting messed up when trying to iterate through a
loop that (1) tries to find_element_by something on the page (that
exists multiple times on the page, and that's why you're creating the
loop) and (2) clicks on a link within that loop, goes to a page, goes
back to a page, and then is supposed to keep running through the
iterations with the find_element_by "function" (probably wrong term
usage here) to get the next one that exists on the page. Not sure why
Selenium gets messed up from that, but it does. (More experienced
coders, feel free to elaborate).
Anyway, my workaround thought process, which may help some of you solve this issue for yourselves by doing something similarly, is:
(1) Find all of the links BEFORE clicking on any of them (and create a list of those links)
Instead of trying to find & click the links one-at-a-time as they show up, I decided to find all of them FIRST (BEFORE clicking on them). I changed the above code to this:
# this is where I'm storing all the links
text_link_list = []
for row in table_rows:
print("got the row!")
waitfortable = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.XPATH, "/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr[1]/td[1]"))
)
## Get a0
try:
print("we're trying!")
row.find_element_by_class_name("a0")
print("we got an a0!")
# this next part is just because there are also blank "a0" cells without
# text (aka a link) in them, and I don't care about those ones.
current_row_has_a0 = row.find_element_by_class_name("a0")
if str(current_row_has_a0.text) != "":
text_link_list += [current_row_has_a0.text]
print("text added to text_link_list!")
else:
print("wasn't a text cell!")
except:
pass
continue
(2) Iterate through that list of links, running your Selenium code that includes .click() and .back()
Now that I had my list of links, I could just iterate through that and do my .click() —> perform actions —> .back() function that I created ( getinfo() -- original code in question above).
## brand new for loop, after "for row in table_rows" loop
for text in text_link_list:
# waiting for page to load upon iterations
table = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "/html/body/table[3]/tbody/tr/td[2]/form/table/tbody/tr/td/table/tbody/tr[4]/td/table/tbody/tr[1]/td[1]"))
)
# this is my .click() --> perform actions --> .back() function
getinfo(text)
However, I just needed to make two small changes to my .getinfo() function.
One, I was now clicking on the links via their "link text", not the a0 class I was using before (need to use .find_element_by_link_text).
Two, I could now use my more basic driver.find_element_by instead of my original table.find_element_by ...."table" may have worked as well, but I was worried about the memory of getting to "table" being lost since I was now in my function running the .click() code. I decided to go with "driver" since it was more certain. (I'm still pretty new to coding, this may not have been necessary.)
def getinfo(text):
link_in_current_cell = driver.find_element_by_link_text(text)
link_in_current_cell.click()
waitfortable2 = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.CLASS_NAME, "top-edit-table"))
)
print("Here is the info from this temporary page.")
driver.back()
return True
I hope that this can all be helpful to someone. I was stoked when I did it and it worked! Let me know if it helped you! <3
PS. NOTE IF HAVING STALE ERRORS / StaleElementReferenceException:
If you are iterating through your loops and clicking a link via something like driver.find_element_by (instead of using .back()), you may run into a Stale error, especially if you're just trying to use .click() on a variable that you assigned earlier or outside of the loop. You can fix this (maybe not in the most beautiful way) by redefining your variable right at that point of the loop when you're wanting to click on the link, with code like this:
my_link = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.LINK_TEXT, "linktext"))
)
my_link.click()
continue
"continue" is only necessary if you're wanting to end the current iteration and begin the next one, and is also not likely necessary if this is at the end of your loop
you can change that red "10" number to whatever amount of time you'd like to give the code to find the element (aka the page to reload, most likely) before the script fails
I am using python3, and want to send messages in Instagram using python, package selenium.
the code works great before. Today most of the time the send_keys() does not work.
I increase the sleep() time to be sure that it is not the internet speed problem. still, it does not work. Any idea? or any alternative ways to send text in a box (I am using python)?
this is the code:
webdriver.get('https://www.instagram.com/explore/tags/'+ hashtag_list[tag] + '/')
sleep(5)
check='//*[#id="react-root"]/section/main/article/div[2]/div/div[%d]/div[%d]/a/div' %(i,x)
sleep(10)
#'//*[#id=\"react-root\"]/section/main/article/div[1]/div/div/div[1]/div[%s]/a/div' % (x)
first_thumbnail = webdriver.find_element_by_xpath(check)
first_thumbnail.click()
sleep(10)
webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/div[2]/section[1]/span[2]/button/span').click()
sleep(10)
comment_box = webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/div[2]/section[3]/div/form/textarea')
sleep(20)
comment_box.send_keys("Hi") # this line does not work
sleep(10)
webdriver.find_element_by_xpath('/html/body/div[3]/div[2]/div/article/div[2]/section[3]/div/form/button').click()
sleep(10)
except:
print("Oops!")
sleep(10)
continue
Have you checked to make sure comment_box is actually a valid webelement by printing it? Just to make sure the selector is correct?
Because it should normally work, but perhaps Instagram is somehow doing something to block it.
I'm using Selenium to automate a process and I am using ChroPath to find Elements like XPath for easier use. But sometimes the elements are not available; example:
Element that I'm trying to locate
ChroPath
try:
P_Captcha_ButtonWait = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH("//div[#class='antigate_solver recaptcha in_process']"))))
print("Captcha is being solved. Waiting 40 seconds...")
time.sleep(40)
CaptchaSolved = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//div[#class='antigate_solver recaptcha solved']")))
time.sleep(3)
print("It's rocking!!!!")
driver.find_element_by_xpath(
"/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/form[1]/div[1]/div[1]/p[3]/button[1]").click()
except:
P_Email_Verification = WebDriverWait(driver, 10).until(EC.presence_of_element_located((
By.XPATH, "/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/form[1]/div[1]/div[1]/div[2]/label[1]/div[1]/input[1]")))
driver.find_element_by_xpath(
"/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/form[1]/div[1]/div[1]/div[2]/label[1]/div[1]/input[1]").click()
print("Something went wrong and idk why?")
This is the line of Code I have written for the Captcha part and I am actually using Anti-Captcha.com Extension and I want to locate the text that Anti-Captcha gives when it is solving the captcha. It looks like this: Anti-Captcha
Also, the website I'm trying this on is ProtonMail, the page where you verify your self if you're a human or not.
I have already coded the part where you verify your self using E-Mail, but I want Captcha as a main-use and the E-Mail Verification as a backhand in case things go wrong with Captcha.
I have tried researching for locating the element, but the only thing I could find is with an image searcher?
I have been forced to use PyAutoGui with absolute directions most of the time for some elements like Button clicking or Clicking in certain fields with absolute direction because I couldn't properly find the XPath.
I'd like to know some ways experience users locate elements.
Update:
This is another attempt of me trying to fix the code and trying different methods but still not working.
try:
P_Captcha_ButtonWait = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//a[#class='status'][contains(.,'Solving is in process...')]")))
if P_Captcha_ButtonWait is True:
print("Captcha is being solved. Waiting 40 seconds...")
time.sleep(40)
CaptchaSolved = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "//div[#class='antigate_solver recaptcha solved'][contains(.,'Solved')]")))
if CaptchaSolved is True:
time.sleep(3)
print("It's rocking!!!!")
driver.find_element_by_xpath(
"/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/form[1]/div[1]/div[1]/p[3]/button[1]").click()
except TimeoutException:
# P_Email_Verification = WebDriverWait(driver, 10).until(EC.presence_of_element_located((
# By.XPATH, "/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/form[1]/div[1]/div[1]/div[2]/label[1]/div[1]/input[1]")))
# driver.find_element_by_xpath(
# "/html[1]/body[1]/div[2]/div[1]/div[1]/div[1]/form[1]/div[1]/div[1]/div[2]/label[1]/div[1]/input[1]").click()
print("Something went wrong and idk why?")
driver.close()
I have made the lines in except as comments for now as my main goal is to fix finding the Anti-Captcha Element.
Basically whatever I do, try won't run because the Element isn't founded.
I thought to make it work in another way; if Captcha option is available on selection, driver can wait 30-80 seconds until Captcha is solved. (that's how long anti-captcha takes approx most of the times to solve a captcha). But sometimes they take up to 200 seconds so that will run into errors and 'cause the whole script failing. So that's the reason why I want to locate this element, because it's easier and keeps the script running.