I have been trying to scrape a leaflet concert map(see picture below), where the information only shows if you hover over the seat with your mouse. I have searched the internet up and down, I slowly start to understand where the information comes from by going into the Network Tab, but I still cannot fully grasp it.
This is the link to the page
Where is this information saved and how can I access it? I would everything that shows up when you hover over the map and see in the small box or on the side when you select it (everything that is found in class="tickettype-item")
I tried something like this, but I'm sure it's nowhere near enough. I would greatly appreciate the help as I'm really keen on understanding what's happening here and how I can solve this.
list_of = driver.find_element_by_xpath('//*[#id="seatmap-tab"]/div[2]/div/div/section/div[2]/div[2]/div[2]/div/div[2]/div[2]/div[2]/div[2]/canvas')
for item in list_of:
action = ActionChains(driver)
action.move_to_element(item)
try:
description = wait(driver, 3).until(EC.visibility_of_element_located((By.XPATH, '//*[#id="seatmap-tab"]/div[2]/div/div/section/div[2]/div[3]/div[2]/div[2]/div/div'))).text
print(description)
action.perform()
except:
action.perform()
Related
Right now, my code screenshots a specific section of a webpage. However, since the element it's screenshotting is quite small on the webpage, the picture is also quite small. So, to solve this issue, I thought I could zoom into the page, center onto the element, screenshot the element, and then zoom out – but it's not working.
When I run the following code (no zooming in):
theComment = driver.find_element(By.XPATH, "/html/body/div[1]/div/div[2]/div[3]/div/div/div/div[2]/div[1]/div[2]/div[6]/div/div/div/div[1]")
driver.execute_script('arguments[0].scrollIntoView({block: "center"});', theComment)
theComment.screenshot('comment'+str(verified+1)+'.png')
I get the following image:
But when I run the following code (zooming in):
theComment = driver.find_element(By.XPATH, "/html/body/div[1]/div/div[2]/div[3]/div/div/div/div[2]/div[1]/div[2]/div[6]/div/div/div/div[1]")
driver.execute_script("document.body.style.zoom='150%'")
sleep(1)
driver.execute_script('arguments[0].scrollIntoView({block: "center"});', theComment)
sleep(1)
theComment.screenshot('comment.png')
sleep(1)
driver.execute_script("document.body.style.zoom='100%'")
I get the following image (the screenshot is really far away from where the actual comment is, and smaller than the comment is):
Any help would be greatly appreciated. Thank you.
I want to scrape the content of the top ten posts on my fan page, but the content is hidden and I need to press "seemore", but I can't find the corresponding location with Selenium.
This problem has been bothering me for a long time, hope someone can help
PS: Non-native English speakers, please forgive me if the useful words are inappropriate
driver.find_element(By.XPATH, '//\*\[#id="jsc_c_1m"\]/div/div/span/div\[2\]/div\[3\]/div').click()
button = driver.find_element(By.XPATH, '//\*\[#id="jsc_c_y4"\]/div/div/span/div\[2\]/div\[3\]/div')
driver.excute_script("arguments\[0\].click();",button)
driver.find_element(By.PARTIAL_LINK_TEXT,'seemore').click()
I should start by saying that I'm not a web developer but I have been using Selenium with Python for a few weeks and think I've gotten the basics down.
What I have is a page with a weekly calendar on it. Image as follows:
What happens is that you can click on any of the coloured boxes which will bring up a register for a class. It features items that you can click on which bring up new information. The problem I have is that I can't click on the items - or, rather, I don't know how to automate the click. I have automated clicking with Selenium before in other situations.
That looks as follows:
As you can see, the calendar still appears in the background. That is to say, the item we click on doesn't take us to a new page but, I suppose, runs some kind of method which shows the register and populates it with data.
My problem is this: I want to automate the process so that I can click on each of these items then scrape the information. In order to do that I need to be able to automate the clicking for each of the items.
So what have I done? When I've done this before, I've searched through the web html for the relevant part and then grabbed the xPath to the element I needed. But here I can't do that. Why not? Well, firstly, I just can't find that element!
Take a look at this close up of the first column:
It's divided into columns, but then I'd expect the clickable area to be an element within that. As you can see, it's not. Furthermore, the clickable area is just the coloured box itself, but if you look closely you can see the element goes outside of that area. I have gone very close with my mouse cursor to see exactly what's clickable, and it definitely is just the coloured box.
So I've not been able to get the element at all.
I thought I might be able to just find out where we went after I clicked the button, but when I got the link address, it just said it was the same page with no differences.
I appreciate I'm asking quite a broad question here, but the problem is that I don't really know where to start. If someone could give me at least that much, I would be grateful. Like if I could just click on each of these one at a time... I've found where the populated data is so I could grab that without a problem.
Well, here's to hoping.
Edit: I should add that there are some JavaScript items (tag type script, type='text/javascript'). I presume that the answer is in there somewhere, but there is a lot of Javascript and I'm not adept at reading it. It's hard for me to tell what script does what. If I could at least figure out what script runs when I click the item then I think I'd be onto something, but I have no idea. Even that would help me.
I had encountered similar problem when scraping for Instagram followers in mobile view where it was floating box when showing the accounts followers name. The approach I took was identifying the name of floating dialog box and clicking the element in it. It might different in your case of html.
Trying looking at this link. Selenium Scroll inside of popup div
Hard to say without the HTML. Maybe try Katalon Recorder (chrome extension) and see if that can detect the xpath for you? It might also be you have to use some kind of javascript to invoke the method for the element
First of all, I am not by any means an expert scraping the web, selenium, or have a good understanding of html...
I am trying to download a CSV file that is hidden behind a 'recaptcha - i am not a robot'. I first must login to the website and then there is link that generates the I am not a robot captcha. It is the kind that you just click a box and must hit 'yes, i am not a robot' button.
I can't figure out how to get this to work. I've seen things about finding the x/y coordinates of where the box pops up and clicking, but I can't seem to figure that out. There seems potentially other methods, but I'm just having a hard time deciphering what to do and then how to do. My code thus far...
import selenium
browser = webdriver.Chrome()
browser.get("mywebsite")
username = browser.find_element_by_id("Email")
password = browser.find_element_by_id("Password")
username.send_keys('<<MY_EMAIL_FOR_LOGIN>>')
password.send_keys('<<MY_PASSWORD_FOR_LOGIN>>')
browser.find_element_by_xpath('//*[#id="container"]/div[3]/div[1]/div[1]/form/div[2]/div[2]/div[1]/input[6]').click() #button to 'login'
# so far so good - I am logged in!
elements = browser.find_element_by_id("patient-export-button")
elements
browser.execute_script("arguments[0].click();", elements)
# the lines above will cause the recaptcha popup to appear.
# now I am STUCK!
I have to now click the checkbox & hit 'yes' in order for my file to download. Sometimes when I hit the checkbox I get all these stupid 'find the car' images which are a pain in the ass for me to solve even as a human...
Any help on solving this recaptcha thing would be appreciated.
I am trying to use Selenium with Python to click on a text field, which opens a pop-up panel, select the text entry area of that popup, and enter text to it.
switch_to_window and switch_to_frame don't seem to be working. In a previous question I asked about Selenium, someone told me to pause the program until the element I need is available. The solution worked for that problem, but not this one, so I'm assuming I have a different issue and I'm too new to Selenium to understand what it is.
This is what the original box I'm trying to click on looks like:
And the Inspect Element for this box:
When that description box is clicked, it should open this window:
And select this element to enter text into:
So in my code I have:
descriptionBox = driver.find_element_by_id('kiadvany_fulszoveg_text')
descriptionBox.click()
That does not error the program, but it also doesn't seem to actually be clicking on that element. To make matters more confusing, I got this to work exactly ONCE, where it opened the correct Description text box as pictured above, but it has since not worked at all even when I try the exact same thing.
The panel's ID is:
As I mentioned, switching to this panel ID using switch_to_frame or switch_to_window was the first thing I tried, but I'm getting a No Such Element error.
Because I saw the description box open correctly once, but never again, I'm assuming that's where the problem is. I wish, the one time it did pop up, that I'd tried to put the text into the field to see if that would work too, but I hadn't gotten there yet at that point, so I don't know if that would have worked.
Thank you in advance to anyone who can help with this!
Try this
descriptionBox = driver.find_element_by_id('kiadvany_fulszoveg_text')
driver.execute_script('arguments[0].click();', descriptionBox)
or
actions = ActionChains(driver)
actions.move_to_element(descriptionBox)
actions.click(descriptionBox)
actions.perform()