This question is a extension from How to get cookies from web-browser with Python?
I would like to extract cookies from the same url for 3 accounts logged into chrome. I've already done the code test from the previous url and it returns me only cookies from a single user. I have already researched the library and apparently does not have this functionality.
That is the code i'm using:
import browser_cookie3
from selenium import webdriver
driver = webdriver.Chrome('C:/Users/pedro/Desktop/chromedriver')
cookies = browser_cookie3.chrome(domain_name='my/url')
print('cookies._cookies')
Look into the selenium module. You are able to run Chrome (in selenium named a webdriver) from Python.
If you start the Chrome browser with selenium (and store it into a variable), you have the selenium webdriver and you are able to log in as you would normally do.
Then cookies = driver.get_cookies() leads to cookies being a dictionary, ready to use.
Reference
Related
I use the following code to open a Chrome window using Selenium:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.youtube.com/")
However, whenever I enter the site, I am logged out. On other sites, I can manually do so, but Google deems it unsafe to log in with Selenium.
I know that Selenium is a test browser, but is it possible to stay logged in with Selenium so that I can access my user data; or is that not the purpose of Selenium, and calling the Chrome subprocess the way to go?
To start with Selenium isn't a browser as such but a bunch of libraries which helps in simulating Browsing Context.
From the question it's not that clear why you are logged out once you login in to the site. But definitely once you are logged in using Selenium driven ChromeDriver initiated google-chrome Browsing Context you must remain logged in and should be able to access your user data.
I am writing my first Python script to navigate a website, and so far I have relied exclusively on a Splinter browser object for things like .find_by_xpath() and .click(). I'm now running into some tasks where the examples I'm finding online and this site all refer to Selenium driver objects (e.g. examples referring to WebDriverWait, or to lines like "driver.execute_script("arguments[0].click();", element").
My question is this: Because I have relied on Splinter, I haven't actually explicitly defined a driver object anywhere in my code (i.e. all I wrote was browser = Browser(options here), and Splinter did the rest). Has a driver object nonetheless been created in the background when I created my browser object? If so, how can I access it? If not, do I need to rewrite my code using only Selenium, or is there a workaround to link my existing browser object with a Selenium driver object?
More generally, can you help me understand the relationship between a Splinter browser object, and a Selenium driver object?
Splinter provides an an abstraction layer for selenium.
As you can see in the splinter repo in github
When splinter creates a Browser Object is creating a Selenium Webdriver Object.
This browser object is a top layer for the selenium webdriver object.
I think that you can access to the driver like this:
from splinter import Browser
browser = Browser()
driver = browser.driver
I have this python code, which accesses a website using the module webbrowser:
import webbrowser
webbrowser.open('kahoot.it')
How could I input information into a text box on this website?
I suggest you use Selenium for that matter.
Here is an example code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
# driver = webdriver.Firefox() # Use this if you prefer Firefox.
driver = webdriver.Chrome()
driver.get('http://www.google.com/')
search_input = driver.find_elements_by_css_selector('input.gLFyf.gsfi')[0]
search_input.send_keys('some search string' + Keys.RETURN)
You can use Selenium better if you know HTML and CSS well. Knowing Javascript/JQuery may help too.
You need the specific webdriver to run it properly:
GeckoDriver (Firefox)
Chrome
There are other webdrivers available, but one of the previous should be enough for you.
On Windows, you should have the executable on the same folder as your code. On Ubuntu, you should copy the webdriver file to /usr/local/bin/
You can use Selenium not only to input information, but also to a lot of other utilities.
I don't think that's doable with the webbrowser module, I suggest you take a look at Selenium
How to use Selenium with Python?
Depending on how complex (interactive, reliant on scripts, ...) your activity is, you can use requests or, as others have suggested, selenium.
Requests allows you to send and get basic data from websites, you would probably use this when automatically submitting an order form, querying an API, checking if a page has ben updated, ...
Selenium gives you programmatic control of a "normal" browser, this seems better for you specific use-case.
The webbrowser module is actually only (more or less) able to open a browser. You can use this if you want to open a link from inside your application.
I'm trying to extract information from Goodreads. The problem is if I go into a url like:
https://www.goodreads.com/shelf/show/programming?page=2
with Selenium chrome webdriver or with BeautifulSoup, it still shows the first page, instead of the second one.
Example with the chrome webdriver:
While on a normal browser, it displays those books instead:
Looks like that happen because you're not logged-in in your selenium session, you will have to login and save the cookies between restarts.
Take a look at this stackoverflow answers to understand how to extract cookies.
I have a webdriver using selenium that opens a browser for me, points it to an ip Address, does a bunch of stuff and closes.
I want to know all of the urls accessed during this time. That is, any ads that are loaded, any css calls that were made out to any url and so on.
Here is the code im using
from selenium import webdriver
browser = webdriver.Firefox(profile) # Get local session of firefox
browser.get(url) # Open a url and wait for it to finish
I did it by loading the firefox plugins Firebug and Netexport. The first is a tool that allows you to see all the exchange of information, the second allows to write all of it in a file (.har extension). So basically selenium has to load the plugins, the website and wait the time you want, and when it closes, you get a file with the result.
Its not a python solution.. But you can add fiddler plug in to Firefox. We needed to do exact same thing about a year ago. We used selenium to open browser and all UI stuff and in background Fiddler captured all traffic (http and https) .. This also list all JS CSS src and you can debug later with inspector see what request is sent and what response is received