Develop Reference

How can I scrape an interactive map using python/selenium

I have been trying to scrape a leaflet concert map(see picture below), where the information only shows if you hover over the seat with your mouse. I have searched the internet up and down, I slowly start to understand where the information comes from by going into the Network Tab, but I still cannot fully grasp it.
This is the link to the page
Where is this information saved and how can I access it? I would everything that shows up when you hover over the map and see in the small box or on the side when you select it (everything that is found in class="tickettype-item")
I tried something like this, but I'm sure it's nowhere near enough. I would greatly appreciate the help as I'm really keen on understanding what's happening here and how I can solve this.
list_of = driver.find_element_by_xpath('//*[#id="seatmap-tab"]/div[2]/div/div/section/div[2]/div[2]/div[2]/div/div[2]/div[2]/div[2]/div[2]/canvas')
for item in list_of:
action = ActionChains(driver)
action.move_to_element(item)
try:
description = wait(driver, 3).until(EC.visibility_of_element_located((By.XPATH, '//*[#id="seatmap-tab"]/div[2]/div/div/section/div[2]/div[3]/div[2]/div[2]/div/div'))).text
print(description)
action.perform()
except:
action.perform()

How to find the "See More Posts from..." button? Selenium - Python

Disclaimer: sorry if my explanation is weird. It's my first week learning.
As my first project, I've set out to build a program that counts Instagram posts in a given time frame. Almost everything works, but there is one button standing in my way. (sometimes)
When you open up Instagram with WebDriver, the GDPR notice pops up. This is easy to get rid of since the xpath for this button is always the same. Or less barbaric way:
driver = webdriver.Chrome()
driver.get(ig_link)
driver.find_element_by_xpath('//button[text()="Accept"]').click()
then, I want to scroll down and during it, get URLs of all posts on the page. As I've already mentioned, on some profiles this works just fine. Yet on others, only few recent posts load up and then you are required to press this pesky "Show More Posts from {instagram_tag}"
My idea was to just brute-force it through driver.find_element_by_xpath() but then I found out that on different IG pages, the xpath is different. The button also doesn't have any text directly inside it, so I am not able to use the same approach that I've used with the "Accept" button.
<button class="tCibT qq7_A z4xUb w5S7h">
<div class="Igw0E IwRSH eGOV_ _4EzTm lC6p0 HVWg4">
<div class="_7UhW9 xLCgt qyrsm h_zdq uL8Hv l4b0S ">Show More Posts from choco_afro
</div>
</div>
</button>
My idea is that I check whether the div element under button includes text "Show More Posts" and then click the button, but I don't know how to do that if even possible. Would anybody be so kind and help me with this? :)

I believe, that, this xpath might help you.
//button//div[contains(text(), 'Show More Posts')]/../..
If each IG page has the same HTML design, then this example should help you. Solution: Click Show More Posts Button
driver.find_element(By.XPATH, "//button//div[contains(text(), 'Show More Posts')]/../..").click()

Try this:
btn = driver.find_element_by_xpath('//button[#class = "tCibT qq7_A z4xUb w5S7h"]')
div = btn.find_elements_by_tag_name('div')[1]
if 'Show More Posts' in div.text.strip():
btn.click()

You can use the following to retrieve that button from it's sibling's text by css selector.
//div[text()='Show More Posts from choco_afro']/parent::div/parent::button

Scraping a website that has certain problems

I want to scrape this website and scrape all articles by this author, with Python(response or Selenium libraries) and put them in PDF file.
However, when I click on the button "Show More" that is in the bottom, after 8 times, it doesn't anymore display more articles, hence I can't access them all(idea was to automate selenium, to click on it until all articles are showed, and then scrape them all). Is there a workaround? Alternative ways I can access all articles chronologically and scrape them?
My idea was to somehow analyze if the links come from alternative source, but I'm clueless. However, I scraped successfully those articles that are displayed.
Thanks in advance!

Use findElements and search for <h2 class="css-1j9dxys e1xfvim30">...</h2> which will give you a list of all titles. Each time when you click the Show more the size of the list will get extended by 10 or so. So the idea is to simply click the button untill the size of the list does not change. Use a while loop. Something like:
List<WebElements> oldList = Driver.findElements(by.cssSelector("h2.css-
1j9dxys.e1xfvim30"));
List<WebElements> newList = new ArrayList<>();
WebElement button = Driver.findElement(by.xpath("//button[text()='Show More']"));
while(newList.size!=oldList.size){
button.click();
newList = List<WebElements> newList = Driver.findElements(by.cssSelector("h2.css-
1j9dxys.e1xfvim30));
}
I might have some mistakes in the code but the idea is there. Good luck!

Python Selenium Recaphta I am not a robot - how to bypass

First of all, I am not by any means an expert scraping the web, selenium, or have a good understanding of html...
I am trying to download a CSV file that is hidden behind a 'recaptcha - i am not a robot'. I first must login to the website and then there is link that generates the I am not a robot captcha. It is the kind that you just click a box and must hit 'yes, i am not a robot' button.
I can't figure out how to get this to work. I've seen things about finding the x/y coordinates of where the box pops up and clicking, but I can't seem to figure that out. There seems potentially other methods, but I'm just having a hard time deciphering what to do and then how to do. My code thus far...
import selenium
browser = webdriver.Chrome()
browser.get("mywebsite")
username = browser.find_element_by_id("Email")
password = browser.find_element_by_id("Password")
username.send_keys('<<MY_EMAIL_FOR_LOGIN>>')
password.send_keys('<<MY_PASSWORD_FOR_LOGIN>>')
browser.find_element_by_xpath('//*[#id="container"]/div[3]/div[1]/div[1]/form/div[2]/div[2]/div[1]/input[6]').click() #button to 'login'
# so far so good - I am logged in!
elements = browser.find_element_by_id("patient-export-button")
elements
browser.execute_script("arguments[0].click();", elements)
# the lines above will cause the recaptcha popup to appear.
# now I am STUCK!
I have to now click the checkbox & hit 'yes' in order for my file to download. Sometimes when I hit the checkbox I get all these stupid 'find the car' images which are a pain in the ass for me to solve even as a human...
Any help on solving this recaptcha thing would be appreciated.

Facebook Wall posts - Auto reply - Python Selenium

I've been trying to create a Python Selenium script to reply to facebook wall posts.
I'll be naive in describing the problem since I'm learning both python, selenium and HTML as a beginner.
I'm not able to find a clickable element in my wall and click on the element. When I do manually click, there is a JS element which seems to create an editable element where I can write my comment as reply.
But finding the element to click is really tricky. Any help would be appreciated.
Here is the code snippet I've used.
mydriver.get(baseurl)
mydriver.maximize_window()
mydriver.find_element_by_xpath(xpaths['usernameTxtBox']).clear()
mydriver.find_element_by_xpath(xpaths['usernameTxtBox']).send_keys(username)
mydriver.find_element_by_xpath(xpaths['passwordTxtBox']).clear()
mydriver.find_element_by_xpath(xpaths['passwordTxtBox']).send_keys(password)
mydriver.find_element_by_xpath(xpaths['submitButton']).click()
post_box = mydriver.find_element_by_css_selector(".UFIAddCommentInput")
ActionChains(mydriver).move_to_element(post_box).click().perform()
print("InputContainer Selected")
post_box=mydriver.find_element_by_xpath("//*[#class='_1mf _1mj']")
post_box.send_keys("Testing using Name not ID. Selenium is easy.")
post_box.send_keys(Keys.ENTER)
print ("Posted...")
Here is where I'm trying to register the comment.
http://prntscr.com/bsxmsm
Thanks in advance.