So, I'm trying to login in this website with Selenium:
https://carrinho.pontofrio.com.br/Checkout?ReturnUrl=%2fSite%2fMeusPedidos.aspx#login
And I'm using anti-captcha, here's my login code:
my_driver = webdriver.Chrome(executable_path=chrome_path)
wait = WebDriverWait(my_driver, 20)
#Realizar o Login
def login():
my_driver.get(url)
time.sleep(4)
my_driver.find_element_by_id('Email').send_keys(usuario)
my_driver.find_element_by_id('Senha').send_keys(senha)
my_driver.find_element_by_id('Senha').send_keys(Keys.ENTER)
time.sleep(1)
solver = recaptchaV2Proxyless()
solver.set_verbose(1)
solver.set_key("")
solver.set_website_url('https://carrinho.pontofrio.com.br/Checkout?ReturnUrl=%2fSite%2fMeusPedidos.aspx#login')
solver.set_website_key("6LfeX6kZAAAAAIhuSyQ1XRwZdOS26O-r4UJbW3y1")
# solver.set_data_s('"data-s" token from Google Search results "protection"')
g_response = solver.solve_and_return_solution()
if g_response != 0:
print("g-response: " + g_response)
else:
print("task finished with error " + solver.error_code)
time.sleep(1)
my_driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "%s"' % g_response)
time.sleep(1)
my_driver.execute_script(f"callbackCaptcha('{g_response}');")
login()
Website Key is correct, but the website is not accepting my Captcha responses.
So I've tried to check how the Login Process happens with the developer tools, and it goes like that:
The callback function happens after a function that I don't know what it that calls the website:
https://www.google.com/recaptcha/api2/userverify?k=6LfeX6kZAAAAAIhuSyQ1XRwZdOS26O-r4UJbW3y1
Post Method before callback method
And I'm not being able to find a way to simulate this post method, since Selenium doesn't do post methods.
Is there anyway that I can listen to all the Javascript events (the codes called) while running the page?
Any help would be much appreciated, thanks!
I was able to solve the validation thing, with the following code:
options.add_argument('--disable-blink-features=AutomationControlled')
But the Anti-Captcha is still giving me a wrong answer :(
I solved the problem
I've finally managed to resolve this myself. In case anyone else is struggling with a similar issue, here was my solution:
Open the console and execute the following cmd: ___grecaptcha_cfg.clients
Find the path which has the callback function, in my case it's ___grecaptcha_cfg.clients[0].O.O
Use the following code: driver.execute_script(f"___grecaptcha_cfg.clients[0].O.O.callback('{new_token}')") (Remember to change the path accordingly)
This Article will help you to find the ___grecaptcha_cfg.clients of your recaptcha site
driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "{}";'.format(g_response))
driver.execute_script(f"___grecaptcha_cfg.clients[0].O.O.callback('{g_response}')")
So yeah, I've discovered that there were 2 problems, first one being the validation, which I solved with this option:
options.add_argument('--disable-blink-features=AutomationControlled')
And the other one was that the website was generating a new token when I clicked the login button. So I decided to solve the captcha before asking to login, and using the callbackCaptcha to ask for login.
Related
I would like to bypass the funcaptchas on the Outlook login page. At the moment it is solved and I get the code, but I don't know how to change the HTML with Python to get further.
Hope you know what I mean and can help me. Thanks :)
Thats the Output:
result: {'captchaId': '72351812278', 'code': '81263a01d67793f12.2257185204|r=ap-southeast-1|metabgclr=%23ffffff|maintxtclr=%231B1B1B|mainbgclr=%23ffffff|guitextcolor=%23747474|metaiconclr=%23757575|meta_height=325|meta=7|pk=B7D8911C-5CC8-A9A3-35B0-554ACEE604DA|at=40|ag=101|cdn_url=https%3A%2F%2Fclient-api.arkoselabs.com%2Fcdn%2Ffc|lurl=https%3A%2F%2Faudio-ap-southeast-1.arkoselabs.com|surl=https%3A%2F%2Fclient-api.arkoselabs.com|smurl=https%3A%2F%2Fclient-api.arkoselabs.com%2Fcdn%2Ffc%2Fassets%2Fstyle-manager'}
def solveCaptcha():
time.sleep(10)
sys.path.append(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
api_key = f'{API_KEY}'
solver = TwoCaptcha(api_key)
try:
result = solver.funcaptcha(sitekey=f'{SITE_KEY}',
url=f'{outlookurl}',
surl='https://client-api.arkoselabs.com')
except Exception as e:
sys.exit(e)
else:
sys.exit('result: ' + str(result)
Like i said i want to bypass the funcatpchas for the outlook signup page.
I'm building a bot that logs into zoom at specified times and the links are being obtained from whatsapp. So i was wondering if it is was possible to retrieve those links from whatsapp directly instead of having to copy paste it into python. Google is filled with guides to send messages but is there any way to READ and RETRIEVE those messages and then manipulate it?
You can, at most, try to read WhatsApp messages with Python using Selenium WebDriver since I strongly doubt that you can access WhatsApp APIs.
Selenium is basically an automation tool that lets you automate tasks in your browser so, perhaps, you could write a Python script using Selenium that automatically opens WhatsApp and parses HTML information regarding your WhatsApp web client.
First of all, we mentioned Selenium, but we will use it only to automate the opening and closing of WhatsApp, now we have to find a way to read what's inside the WhatsApp client, and that's where the magic of Web Scraping comes is hand.
Web scraping is a process of extracting data from a website, in this case, the data is represented by the Zoom link you need to automatically obtain, while the web site is your WhatsApp client. To perform this process you need a way to extract (parse) information from the website, to do so I suggest you use Beautiful Soup, but I advise you that a minimum knowledge of how HTML works is required.
Sorry if this may not completely answer your question but this is all the knowledge I have on this specific topic.
You can open WhatsApp on browser using https://selenium-python.readthedocs.io/ in Python.
Selenium is basically an automation tool that lets you automate tasks in your browser so, perhaps, you could write a Python script using Selenium that automatically opens WhatsApp and parses HTML information regarding your WhatsApp web client.
I learn and use code from "https://towardsdatascience.com/complete-beginners-guide-to-processing-whatsapp-data-with-python-781c156b5f0b" this site. Go through the details written on mentioned link.
You have to install external python library "whatsapp-web" from this link --- "https://pypi.org/project/whatsapp-web/". Just type in command prompt / windows terminal by "python -m pip install whatsapp-web".
It will show result ---
python -m pip install whatsapp-web
Collecting whatsapp-web
Downloading whatsapp_web-0.0.1-py3-none-any.whl (21 kB)
Installing collected packages: whatsapp-web
Successfully installed whatsapp-web-0.0.1
You can read all the cookies from whatsapp web and add them to headers and use the requests module or you can also use selenium with that.
Update :
Please change the xpath's class name of each section from the current time class name of WhatsApp web by using inspect element section in WhatsApp web to use the following code. Because WhatsApp have changed its element's class names.
I have tried that in creating a WhatsApp bot using python.
But there are still many bugs because of I am also beginner.
steps based on my research :
Open browser using selenium webdriver
Login on WhatsApp using qr code
If you know from which number you are going to received the meeting link then use this step otherwise check the following process mention after this process.
Find and open the chat room where you are going to received zoom meeting link.
For getting message from known chat room to perform action
#user_name = "Name of meeting link Sender as in your contact list"
Example :
user_name = "Anurag Kushwaha"
#In above variable at place of `Anurag Kushwaha` pass Name or number of Your Teacher
# who going to sent you zoom meeting link same as you have in your contact list.
user = webdriver.find_element_by_xpath('//span[#title="{}"]'.format(user_name))
user.click()
# For getting message to perform action
message = webdriver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in message:
try:
if "zoom.us" in str(i.text):
# Here you can use you code to preform action according to your need
print("Perform Your Action")
except:
pass
If you do not know by which number you are going to received the link.
Then you can get div class of any unread contact block and get open all the chat room list which are containing that unread div class.
Then check all the unread messages of open chat and get the message from the div class.
When you don't know from whom you gonna received zoom meeting link.
# For getting unread chats you can use
unread_chats = webdriver.find_elements_by_xpath("// span[#class='_38M1B']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing the number of unread message showing the contact card inside a green circle before opening the chat room.
# Open each chat using loop and read message.
for chat in unread_chats:
chat.click()
# For getting message to perform action
message = webdriver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in messge:
try:
if "zoom.us" in str(i.text):
# Here you can use you code to preform action according to your need
print("Perform Your Action")
except:
pass
Note : In the above code 'webdriver' is the driver by which you open web.whatsapp.com
Example :
from selenium import webdriver
webdriver = webdriver.Chrome("ChromePath/chromedriver.exe")
webdriver.get("https://web.whatsapp.com")
# This wendriver variable is used in above code.
# If you have used any other name then please rename in my code or you can assign your variable in that code variable name as following line.
webdriver = your_webdriver_variable
A complete code reference Example :
from selenium import webdriver
import time
webdriver = webdriver.Chrome("ChromePath/chromedriver.exe")
webdriver.get("https://web.whatsapp.com")
time.sleep(25) # For scan the qr code
# Plese make sure that you have done the qr code scan successful.
confirm = int(input("Press 1 to proceed if sucessfully login or press 0 for retry : "))
if confirm == 1:
print("Continuing...")
elif confirm == 0:
webdriver.close()
exit()
else:
print("Sorry Please Try again")
webdriver.close()
exit()
while True:
unread_chats = webdriver.find_elements_by_xpath("// span[#class='_38M1B']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing the number of unread message showing the contact card inside a green circle before opening the chat room.
# Open each chat using loop and read message.
for chat in unread_chats:
chat.click()
time.sleep(2)
# For getting message to perform action
message = webdriver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in messge:
try:
if "zoom.us" in str(i.text):
# Here you can use you code to preform action according to your need
print("Perform Your Action")
except:
pass
Please make sure that the indentation is equal in code blocks if you are copying it.
Can read my another answer in following link for more info about WhatsApp web using python.
Line breaks in WhatsApp messages sent with Python
I am developing WhatsApp bot using python.
For contribution you can contact at : anurag.cse016#gmail.com
Please give a star on my https://github.com/4NUR46 If this Answer helps you.
Try This Its A bit of a hassle but it might work
import pyautogui
import pyperclip
import webbrowser
grouporcontact = pyautogui.locateOnScreen("#group/contact", confidence=.6) # Take a snip of the group or contact name/profile photo
link = pyperclip.paste()
def searchforgroup():
global link
time.sleep(5)
webbrowser.open("https://web.whatsapp.com")
time.sleep(30)#for you to scan the qr code if u have done it then u can edit it to like 10 or anything
grouporcontact = pyautogui.locateOnScreen("#group/contact", confidence=.6)
x = grouporcontact[0]
y = grouporcontact[1]
if grouporcontact == None:
#Do any other option in my case i just gave it my usual link as
link = "mymeetlink"
else:
pyautogui.moveTo(x,y, duration=1)
pyautogui.click()
# end of searching group
def findlink():
global link
meetlink = pyautogui.locateOnScreen("#", confidence=.6)#just take another snap of a meet link without the code after the "/"
f = meetlink[0]
v = meetlink[1]
if meetlink == None:
#Do any other option in my case i just gave it my usual link as
link = "mymeetlink"
else:
pyautogui.moveTo(f,v, duration=.6)
pyautogui.rightClick()
pyautogui.moveRel(0,0, duration=2) # You Have to play with this it basically is considered by your screen size so just edit that and edit it till it reaches the "Copy Link Address"
pyautogui.click()
link = pyperclip.paste()
webbrowser.open(link) # to test it out
So Now You Have It Have To Install pyautogui, pyperclip
and just follow the comments in the snippet and everything should work :)
Good morning, everyone,
I want to create a script which automatically update an issue on RedMine when someone make a pull-request on our GitHub based on the pull-request comment.
I wrote a script in Python using selenium and redmine REST API that retrieves the comment of a pull-request on GitHub made by its requester, but I have to execute it manually.
Do you know if it is possible to execute a python script automatically just after a pull request?
(Currently the script is stored on my computer, but ideally it will be stored on an external server so that I and my partners can use it more easily)
I have searched some solutions based on WebHooks or CRON, but nothing seems to answer my problem.
I am using Python 2.7
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import test
# Xpath to retrieve number of the fix
DISCONNECTED_XPATH = "//div[4]/div/main/div[2]/div[1]/div/div[2]/div[3]/div[2]/div[1]/div[1]/div[2]/div/div[2]/task-lists/table/tbody/tr/td/p"
CONNECTED_XPATH = "//div[4]/div/main/div[2]/div[1]/div/div[1]/div[3]/div[2]/div[1]/div[1]/div[2]/div/div[2]/task-lists/table/tbody/tr/td/p"
PULL_URL = "https://github.com/MaxTeiger/TestCoopengo/pull/1"
# Init
print("Opening the browser...")
driver = webdriver.Firefox()
# Go to the specified pull
print("Reaching " + PULL_URL)
driver.get(PULL_URL)
assert "GitHub" in driver.title
print("Finding the pull comment...")
# retrieve the fix id
elem = driver.find_element_by_xpath(DISCONNECTED_XPATH)
issueID = elem.text
print("Closing driver")
driver.close()
issueID = int(issueID.split('#')[1])
print("Issue ID : " +str(issueID))
print("Updating ticket on RedMine...")
test.updateIssueOnRedMineFromGit(issueID, PULL_URL)
Thank you if you can help me or if you have a better solution to my problem
I finally found an answer to my problem and it turns out that the webhooks proposed by GitHub answer my problem (Repo > Settings > Webhooks).
Now, I just need to set up a server that calls my script when I make an HTML Post request, but I don't know how to retrieve the URL of the wanted pull-request.
I've seen a few instances of this question, but I was not sure how to apply the changes to my particular situation. I have code that monitors a webpage for changes and refreshes every 30 seconds, as follows:
import sys
import ctypes
from time import sleep
from Checker import Checker
USERNAME = sys.argv[1]
PASSWORD = sys.argv[2]
def main():
crawler = Checker()
crawler.login(USERNAME, PASSWORD)
crawler.click_data()
crawler.view_page()
while crawler.check_page():
crawler.wait_for_table()
crawler.refresh()
ctypes.windll.user32.MessageBoxW(0, "A change has been made!", "Attention", 1)
if __name__ == "__main__":
main()
The problem is that Selenium will always show an error stating it is unable to locate the element after the first refresh has been made. The element in question, I suspect, is a table from which I retrieve data using the following function:
def get_data_cells(self):
contents = []
table_id = "table.datadisplaytable:nth-child(4)"
table = self.driver.find_element(By.CSS_SELECTOR, table_id)
cells = table.find_elements_by_tag_name('td')
for cell in cells:
contents.append(cell.text)
return contents
I can't tell if the issue is in the above function or in the main(). What's an easy way to get Selenium to refresh the page without returning such an error?
Update:
I've added a wait function and adjusted the main() function accordinly:
def wait_for_table(self):
table_selector = "table.datadisplaytable:nth-child(4)"
delay = 60
try:
wait = ui.WebDriverWait(self.driver, delay)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, table_selector)))
except TimeoutError:
print("Operation timeout! The requested element never loaded.")
Since the same error is still occurring, either my timing function is not working properly or it is not a timing issue.
I've run into the same issue while doing web scraping before and found that re-sending the GET request (instead of refreshing) seemed to eliminate it.
It's not very elegant, but it worked for me.
I appear to have fixed my own problem.
My refresh() function was written as follows:
def refresh():
self.driver.refresh()
All I did was switch frames right after the refresh() call. That is:
def refresh():
self.driver.refresh()
self.driver.switch_to.frame("content")
This took care of it. I can see that the page is now refreshing without issues.
I've been writing a program to log in to Facebook and update the status as a side project. I managed to get the program to login. However, I'm having trouble selecting the textarea that ends up being the "Enter your status here" box. Using "Inspect Element" in Chrome, I'm able to see the form under which it's located, but listing the forms in the program doesn't seem to list said form...
import mechanize
import re
br = mechanize.Browser()
usernamecorrect = 0
while usernamecorrect == 0:
username = raw_input("What is the username for your Facebook Account? ")
matchmail = re.search(r'[\w.-]+#[\w.-]+', username)
if matchmail:
print matchmail.group()
usernamecorrect = 1
else:
print "That is not a valid username; please enter the e-mail address registered with your account.\n"
password = raw_input("What is the password for your account?")
print "Logging in..."
br.set_handle_robots(False)
br.open("https://www.facebook.com/")
br.select_form(nr = 0)
br['email'] = username
br['pass'] = password
br.submit()
raw_input("Login successful!")
print "Forms: \n"
for f in br.forms():
print f.name
The full output is as follows:
What is the username for your Facebook Account? myemail#website.com
What is the password for your account? thisisapassword
Logging in...
Login successful!
Forms:
navSearch
None
I took a look through the source of Facebook via Inspect Elements again, and "navSearch" is the "Find People, things, etc." search bar, and the unnamed form appears to have to do with the logout button. However, while Inspect Elements gives at least 2 more forms, one of which holds the status update box. I haven't been able to determine if it's because of JavaScript or not (while the status update box code block is encapsulated in , so are the navSearch and logout forms.) The most relevant thing I've been able to find is that navSearch and the logout forms are in a separate div, but I somehow feel as though that shouldn't be much of a problem for mechanize. Is there just something wrong with my code, or is it something else entirely?
Is there just something wrong with my code, or is it something else entirely?
Your whole approach is wrong:
I've been writing a program to log in to Facebook and update the status
That’s what the Graph API is for.
Scraping FB pages and trying to act as a “browser” is not the way to go. Apart from the fact, that FB policies do not allow that, you see how difficult it gets on a page that uses JavaScript/AJAX so much.
Go with the API, it’s the easy way.