Develop Reference

Selenium Webdriver cannot find element which loads later than DOM

I've been trying to webscrap this page using selenium, and as you may notice on the first load a pop-up appears about agreeing on cookies. This pop-up appears around a second later that the rest of the DOM.
The problem is, that even after adding a sleep or WebDriverWait the driver still cannot click on the Accept button from the pop-up.
Inspecting the page gives that the Xpath of the element is /html/body/div/div[2]/div[4]/button[1] so I tried doing it as:
driver.get("https://www.planespotters.net/airlines")
time.sleep(5)
driver.find_element(By.XPATH, '/html/body/div/div[2]/div[4]/button[1]').click()
with no effect. I get that the element does not exist.
I tried it even with:
driver.execute_script('document.querySelector("#notice > div.message-component.message-row.button-container > button.message-component.message-button.no-children.focusable.primary.sp_choice_type_11").click()')
and
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '/html/body/div/div[2]/div[4]/button[1]'))).click()
Any suggestions on how to bypass this pop-up?

If you scroll up on devtools you'll notice your modal window is in an iframe. I queried the 'planespotters' header image (because it's the top item) and you can see it quite clearly:
For selenium switch to your iframe first - this finds the iframe wher the url contains 'privacy':
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(#src,'privacy')]")))
Then you can complete your action. Be sure to keep that sync in there too.
Then you can switch back to your main window and carry in with your test
driver.switch_to_default_content()

You can try introducing an implicit wait for your driver. This will happen once per script and will affect all objects. This will wait 10 seconds before throwing an error, or continues whenever it is ready:
driver.implicitly_wait(10)

Second iterative of find_elements_by_xpath gives error in selenium python

I'm trying to find all the my subject in my dashboard of my college website.
I'm using selenium to do it.
The site is a little slow so first I wait
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#class='multiline']")))
then I find all the elements with
course = driver.find_elements_by_xpath("//span[#class='multiline']")
after that in a for loop I try to traverse it the 0th place of the "course" works fine and I'm able to click it and go to webpage but when the loop runs for the secon d time that is for the 1st place in "course" it gives me error selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
So I tried adding a lit bit wait time to using 2 method it still gives me error
driver.implicitly_wait(20)
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#class='multiline']")))
the loop
for i in course[1::]:
#driver.implicitly_wait(20)
#WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[#class='multiline']")))
print(i)
i.click()
driver.implicitly_wait(2)
driver.back()
a snippet of the website
Thanks in advance

Answering my own question after extensive research
A common technique used for simulating a tabbed UI in a web app is to prepare DIVs for each tab, but only attach
one at a time, storing the rest in variables. In this case my code have a reference
to an element that is no longer attached to the DOM (that is, that has an ancestor which is "document.documentElement").
If WebDriver throws a stale element exception in this case, even though the element still exists, the reference
is lost. You should discard the current reference you hold and replace it, possibly by locating the element again
once it is attached to the DOM
for i in range(len(course)):
# here you need to find all the elements again because once we
leave the page the reference will be lost and we need to find it again
course = driver.find_elements_by_xpath("//span[#class='multiline']")
print(course[i].text)
course[i].click()
driver.implicitly_wait(2)
driver.back()

Unable to locate element using selenium (Python)

I have been trying to find a button a click on it but no matter what I try it has been unable to locate it. I have tried using all the driver.find_element_by... methods but nothing seems to be working
from selenium import webdriver
import time
driver = webdriver.Chrome(executable_path="/Users/shreygupta/Documents/ComputerScience/PythonLanguage/Automation/corona/chromedriver")
driver.get("https://ourworldindata.org/coronavirus")
driver.maximize_window()
time.sleep(5)
driver.find_element_by_css_selector("a[data-track-note='chart-click-data']").click()
I am trying to click the DATA tab on the screenshot below

You can modify your script to open this graph directly:
driver.get("https://ourworldindata.org/grapher/total-cases-covid-19")
driver.maximize_window()
Then you can add implicitly_wait instead of sleep. An implicit wait tells WebDriver to poll the DOM for a certain amount of time when trying to find any element (or elements) not immediately available (from python documentation). It'll work way faster because it'll interact with an element as soon as it finds it.
driver.implicitly_wait(5)
driver.find_element_by_css_selector("a[data-track-note='chart-click-data']").click()
Hope this helps, good luck.

Here is the logic that you can use, where the script will wait for max 30 for the Data menu item and if the element is present with in 30 seconds it will click on the element.
url = "https://ourworldindata.org/grapher/covid-confirmed-cases-since-100th-case"
driver.get(url)
driver.maximize_window()
wait = WebDriverWait(driver,30)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR,"a[data-track-note='chart-click-data']"))).click()

python selenium cannot locate clickable play button on mobile version of webpage

I am having a hard time locating the play element on a mobile version of a webpage (my python script is passing a mobile user-agent in the header.)
the website url is below (NOTE: must be accessed with a mobile user-agent else it won't show the correct page and reverts to standard browser page instead)
https://m.soundcloud.com/mbmproductions/sets/mark-berrys-playlist
Using inspect in the browser, I have been unable to work out what exactly is needing to be clicked to start it playing. The entire image area around the play button seems to work if I click on it manually, but no element listed by inspect seems to work when using the script to start it playing.
Can anyone explain how to find the correct clickable element in this link so I can add it to a script in order to click it and start it playing. Ideally using XPATH as per my code below, but class or ID or anything would do if it works, even javascript if I have to.
element = WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.XPATH, targetelement))
)
element.click()

I make some amendment to your script. Use action instead of click(). Change clickable explicit to presence.
element = WebDriverWait(driver, 10).until(
EC. presence_of_element_located
((By.CLASS_NAME, ‘g-header-artwork-controls’))
)
el = driver.find_elements_by_css_selector(“.playlist__playButton.g-play-button")
webdriver.ActionChains(driver).move_to_element(el).click(el).perform()

Unable to find links using selenium in python?

I am scraping a webpage using selenium. I first find the link I want and then click on it and download it(Link is pdf). What happens is sometimes I am able to do so, but sometimes selenium says that link not found. I suppose that it is due to the page not loading properly. What can I do about this and am I in the right direction?
This is my previous code:
for b in source_code_2.find_all('a', href=True):
if b.has_attr("title"):
if(b['title']=='Click here to download'):
urllib2.urlretrieve(full_url)
now i want to do it using selenium and element. How can I do this?

I think you should use explicit wait to tell selenium to wait until specific element loads properly , In python you can use explicit wait in following way :
element = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.ID, "yourElement"))
OR
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.ID, "yourElement"))
element.click()
You just need to replace your element ID in above code and you can change 20 seconds to 30 ,40 as per your need. So meaning of above code is your webdriver will wait until 20 seconds to find that specific element.