Webdriver how to get the page has been opened - python

driver = webdriver.chrome()
driver.get(url)
Webdriver are used to open a web page.
But if I turn it.
First open a web page, and then use webdriver access source, feasible? ??
(The first time to come here to ask questions, did not find the forum on python.
I do not know right here!)

For example:
first open manually with firefox stackoverflow.com
Then use python's webdriver to get its source code
Is there a way?
(My English is not good, automatic translation)

Related

how to get Selenium to get other tab information

As it stands and I speak under correction, selenium opens a "private" browser tab.
I wanted to know if there was any way to change that so that the selenium tab can benefit from other tabs that are open, i.e. not need to login because you are already logged in on another non-selenium tab
I think the problem is that Selenium loads up the default browser profile. If you use chrome or a chromium based browser, go to chrome://version and you should see a heading called Profile Path copy that and add it to your program with the options method.
Here is a sample:
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=path_to_chrome_profile") #Path to your chrome profile
driver = webdriver.Chrome(executable_path="path_to_chromedriver", chrome_options=options)
Hope it helps. And yes, make sure you are signed into the account you want selenium to access when you check your path.

How do I input information into a website with python?

I have this python code, which accesses a website using the module webbrowser:
import webbrowser
webbrowser.open('kahoot.it')
How could I input information into a text box on this website?
I suggest you use Selenium for that matter.
Here is an example code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# driver = webdriver.Firefox() # Use this if you prefer Firefox.
driver = webdriver.Chrome()
driver.get('http://www.google.com/')
search_input = driver.find_elements_by_css_selector('input.gLFyf.gsfi')[0]
search_input.send_keys('some search string' + Keys.RETURN)
You can use Selenium better if you know HTML and CSS well. Knowing Javascript/JQuery may help too.
You need the specific webdriver to run it properly:
GeckoDriver (Firefox)
Chrome
There are other webdrivers available, but one of the previous should be enough for you.
On Windows, you should have the executable on the same folder as your code. On Ubuntu, you should copy the webdriver file to /usr/local/bin/
You can use Selenium not only to input information, but also to a lot of other utilities.
I don't think that's doable with the webbrowser module, I suggest you take a look at Selenium
How to use Selenium with Python?
Depending on how complex (interactive, reliant on scripts, ...) your activity is, you can use requests or, as others have suggested, selenium.
Requests allows you to send and get basic data from websites, you would probably use this when automatically submitting an order form, querying an API, checking if a page has ben updated, ...
Selenium gives you programmatic control of a "normal" browser, this seems better for you specific use-case.
The webbrowser module is actually only (more or less) able to open a browser. You can use this if you want to open a link from inside your application.

Selenium and Goodreads' pagination

I'm trying to extract information from Goodreads. The problem is if I go into a url like:
https://www.goodreads.com/shelf/show/programming?page=2
with Selenium chrome webdriver or with BeautifulSoup, it still shows the first page, instead of the second one.
Example with the chrome webdriver:
While on a normal browser, it displays those books instead:
Looks like that happen because you're not logged-in in your selenium session, you will have to login and save the cookies between restarts.
Take a look at this stackoverflow answers to understand how to extract cookies.

Use existing open tab and url in Selenium py

Hi I'm trying to use Selenium python to use a url that is already open on Internet Explorer. I had a look around and not sure if this is possible.
The reason why I wouldn't like to open new brower or tab is because the page changes to different text.
So far my text only opens a new browser
CODE
from selenium import webdriver
driver = webdriver.Ie()
driver.get("https://outlook.live.com/owa/")
This answer helped me with same problem.
By now you can not access previously opened tabs with selenium.
But you can try to recreate your session, passing what is needed using requests library, for example.

How to get html that I see in inspect element?

I'm programming a web-scraper app with python. The website I want to scrape data use JS.
How can I get the source that I see in inspect element?
With javascript pycurl will not work, you need Selenium to get the stuff you need.
import selenium
driver = selenium.webdriver.Firefox()
driver.get("your_url")
Make sure you have Firefox (or another browser selenium supports) installed.

Categories

Resources