Develop Reference

Anti-Captcha not working, validation happening before callback - Selenium

So, I'm trying to login in this website with Selenium:
https://carrinho.pontofrio.com.br/Checkout?ReturnUrl=%2fSite%2fMeusPedidos.aspx#login
And I'm using anti-captcha, here's my login code:
my_driver = webdriver.Chrome(executable_path=chrome_path)
wait = WebDriverWait(my_driver, 20)
#Realizar o Login
def login():
my_driver.get(url)
time.sleep(4)
my_driver.find_element_by_id('Email').send_keys(usuario)
my_driver.find_element_by_id('Senha').send_keys(senha)
my_driver.find_element_by_id('Senha').send_keys(Keys.ENTER)
time.sleep(1)
solver = recaptchaV2Proxyless()
solver.set_verbose(1)
solver.set_key("")
solver.set_website_url('https://carrinho.pontofrio.com.br/Checkout?ReturnUrl=%2fSite%2fMeusPedidos.aspx#login')
solver.set_website_key("6LfeX6kZAAAAAIhuSyQ1XRwZdOS26O-r4UJbW3y1")
# solver.set_data_s('"data-s" token from Google Search results "protection"')
g_response = solver.solve_and_return_solution()
if g_response != 0:
print("g-response: " + g_response)
else:
print("task finished with error " + solver.error_code)
time.sleep(1)
my_driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "%s"' % g_response)
time.sleep(1)
my_driver.execute_script(f"callbackCaptcha('{g_response}');")
login()
Website Key is correct, but the website is not accepting my Captcha responses.
So I've tried to check how the Login Process happens with the developer tools, and it goes like that:
The callback function happens after a function that I don't know what it that calls the website:
https://www.google.com/recaptcha/api2/userverify?k=6LfeX6kZAAAAAIhuSyQ1XRwZdOS26O-r4UJbW3y1
Post Method before callback method
And I'm not being able to find a way to simulate this post method, since Selenium doesn't do post methods.
Is there anyway that I can listen to all the Javascript events (the codes called) while running the page?
Any help would be much appreciated, thanks!
I was able to solve the validation thing, with the following code:
options.add_argument('--disable-blink-features=AutomationControlled')
But the Anti-Captcha is still giving me a wrong answer :(

I solved the problem
I've finally managed to resolve this myself. In case anyone else is struggling with a similar issue, here was my solution:
Open the console and execute the following cmd: ___grecaptcha_cfg.clients
Find the path which has the callback function, in my case it's ___grecaptcha_cfg.clients[0].O.O
Use the following code: driver.execute_script(f"___grecaptcha_cfg.clients[0].O.O.callback('{new_token}')") (Remember to change the path accordingly)
This Article will help you to find the ___grecaptcha_cfg.clients of your recaptcha site
driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "{}";'.format(g_response))
driver.execute_script(f"___grecaptcha_cfg.clients[0].O.O.callback('{g_response}')")

So yeah, I've discovered that there were 2 problems, first one being the validation, which I solved with this option:
options.add_argument('--disable-blink-features=AutomationControlled')
And the other one was that the website was generating a new token when I clicked the login button. So I decided to solve the captcha before asking to login, and using the callbackCaptcha to ask for login.

How do I read whatsapp messages from a contact using python?

I'm building a bot that logs into zoom at specified times and the links are being obtained from whatsapp. So i was wondering if it is was possible to retrieve those links from whatsapp directly instead of having to copy paste it into python. Google is filled with guides to send messages but is there any way to READ and RETRIEVE those messages and then manipulate it?

You can, at most, try to read WhatsApp messages with Python using Selenium WebDriver since I strongly doubt that you can access WhatsApp APIs.
Selenium is basically an automation tool that lets you automate tasks in your browser so, perhaps, you could write a Python script using Selenium that automatically opens WhatsApp and parses HTML information regarding your WhatsApp web client.
First of all, we mentioned Selenium, but we will use it only to automate the opening and closing of WhatsApp, now we have to find a way to read what's inside the WhatsApp client, and that's where the magic of Web Scraping comes is hand.
Web scraping is a process of extracting data from a website, in this case, the data is represented by the Zoom link you need to automatically obtain, while the web site is your WhatsApp client. To perform this process you need a way to extract (parse) information from the website, to do so I suggest you use Beautiful Soup, but I advise you that a minimum knowledge of how HTML works is required.
Sorry if this may not completely answer your question but this is all the knowledge I have on this specific topic.

You can open WhatsApp on browser using https://selenium-python.readthedocs.io/ in Python.
Selenium is basically an automation tool that lets you automate tasks in your browser so, perhaps, you could write a Python script using Selenium that automatically opens WhatsApp and parses HTML information regarding your WhatsApp web client.
I learn and use code from "https://towardsdatascience.com/complete-beginners-guide-to-processing-whatsapp-data-with-python-781c156b5f0b" this site. Go through the details written on mentioned link.
You have to install external python library "whatsapp-web" from this link --- "https://pypi.org/project/whatsapp-web/". Just type in command prompt / windows terminal by "python -m pip install whatsapp-web".
It will show result ---
python -m pip install whatsapp-web
Collecting whatsapp-web
Downloading whatsapp_web-0.0.1-py3-none-any.whl (21 kB)
Installing collected packages: whatsapp-web
Successfully installed whatsapp-web-0.0.1

You can read all the cookies from whatsapp web and add them to headers and use the requests module or you can also use selenium with that.

Update :
Please change the xpath's class name of each section from the current time class name of WhatsApp web by using inspect element section in WhatsApp web to use the following code. Because WhatsApp have changed its element's class names.
I have tried that in creating a WhatsApp bot using python.
But there are still many bugs because of I am also beginner.
steps based on my research :
Open browser using selenium webdriver
Login on WhatsApp using qr code
If you know from which number you are going to received the meeting link then use this step otherwise check the following process mention after this process.
Find and open the chat room where you are going to received zoom meeting link.
For getting message from known chat room to perform action
#user_name = "Name of meeting link Sender as in your contact list"
Example :
user_name = "Anurag Kushwaha"
#In above variable at place of `Anurag Kushwaha` pass Name or number of Your Teacher
# who going to sent you zoom meeting link same as you have in your contact list.
user = webdriver.find_element_by_xpath('//span[#title="{}"]'.format(user_name))
user.click()
# For getting message to perform action
message = webdriver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in message:
try:
if "zoom.us" in str(i.text):
# Here you can use you code to preform action according to your need
print("Perform Your Action")
except:
pass
If you do not know by which number you are going to received the link.
Then you can get div class of any unread contact block and get open all the chat room list which are containing that unread div class.
Then check all the unread messages of open chat and get the message from the div class.
When you don't know from whom you gonna received zoom meeting link.
# For getting unread chats you can use
unread_chats = webdriver.find_elements_by_xpath("// span[#class='_38M1B']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing the number of unread message showing the contact card inside a green circle before opening the chat room.
# Open each chat using loop and read message.
for chat in unread_chats:
chat.click()
# For getting message to perform action
message = webdriver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in messge:
try:
if "zoom.us" in str(i.text):
# Here you can use you code to preform action according to your need
print("Perform Your Action")
except:
pass
Note : In the above code 'webdriver' is the driver by which you open web.whatsapp.com
Example :
from selenium import webdriver
webdriver = webdriver.Chrome("ChromePath/chromedriver.exe")
webdriver.get("https://web.whatsapp.com")
# This wendriver variable is used in above code.
# If you have used any other name then please rename in my code or you can assign your variable in that code variable name as following line.
webdriver = your_webdriver_variable
A complete code reference Example :
from selenium import webdriver
import time
webdriver = webdriver.Chrome("ChromePath/chromedriver.exe")
webdriver.get("https://web.whatsapp.com")
time.sleep(25) # For scan the qr code
# Plese make sure that you have done the qr code scan successful.
confirm = int(input("Press 1 to proceed if sucessfully login or press 0 for retry : "))
if confirm == 1:
print("Continuing...")
elif confirm == 0:
webdriver.close()
exit()
else:
print("Sorry Please Try again")
webdriver.close()
exit()
while True:
unread_chats = webdriver.find_elements_by_xpath("// span[#class='_38M1B']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing the number of unread message showing the contact card inside a green circle before opening the chat room.
# Open each chat using loop and read message.
for chat in unread_chats:
chat.click()
time.sleep(2)
# For getting message to perform action
message = webdriver.find_elements_by_xpath("//span[#class='_3-8er selectable-text copyable-text']")
# In the above line Change the xpath's class name from the current time class name by inspecting span element
# which containing received text message of any chat room.
for i in messge:
try:
if "zoom.us" in str(i.text):
# Here you can use you code to preform action according to your need
print("Perform Your Action")
except:
pass
Please make sure that the indentation is equal in code blocks if you are copying it.
Can read my another answer in following link for more info about WhatsApp web using python.
Line breaks in WhatsApp messages sent with Python
I am developing WhatsApp bot using python.
For contribution you can contact at : anurag.cse016#gmail.com
Please give a star on my https://github.com/4NUR46 If this Answer helps you.

Try This Its A bit of a hassle but it might work
import pyautogui
import pyperclip
import webbrowser
grouporcontact = pyautogui.locateOnScreen("#group/contact", confidence=.6) # Take a snip of the group or contact name/profile photo
link = pyperclip.paste()
def searchforgroup():
global link
time.sleep(5)
webbrowser.open("https://web.whatsapp.com")
time.sleep(30)#for you to scan the qr code if u have done it then u can edit it to like 10 or anything
grouporcontact = pyautogui.locateOnScreen("#group/contact", confidence=.6)
x = grouporcontact[0]
y = grouporcontact[1]
if grouporcontact == None:
#Do any other option in my case i just gave it my usual link as
link = "mymeetlink"
else:
pyautogui.moveTo(x,y, duration=1)
pyautogui.click()
# end of searching group
def findlink():
global link
meetlink = pyautogui.locateOnScreen("#", confidence=.6)#just take another snap of a meet link without the code after the "/"
f = meetlink[0]
v = meetlink[1]
if meetlink == None:
#Do any other option in my case i just gave it my usual link as
link = "mymeetlink"
else:
pyautogui.moveTo(f,v, duration=.6)
pyautogui.rightClick()
pyautogui.moveRel(0,0, duration=2) # You Have to play with this it basically is considered by your screen size so just edit that and edit it till it reaches the "Copy Link Address"
pyautogui.click()
link = pyperclip.paste()
webbrowser.open(link) # to test it out
So Now You Have It Have To Install pyautogui, pyperclip
and just follow the comments in the snippet and everything should work :)

How can I create a clickable windows app link in spotify?

In a piece of code I am working on, I have created 2 ways to print the link for a song on spotify, should it exist. The first of which gives the link to the web based player, the second gives the format for a link that will try to open an installed app. However, only the printed link for the web player is clickable, as it is a http://www. type of link. Is there a way to format the print function to turn it into a link, so that way I could click on it in the output and try to open the spotify app?
try:
print("Spotify Web Player: " + "https://open.spotify.com/track/" + data['metadata']['music'][0]['external_metadata']['spotify']['track']['id'])
print("Spotify App: " + "spotify:track:" + data['metadata']['music'][0]['external_metadata']['spotify']['track']['id'])
Note, the direct app link looks like this, and if you paste it into the URL bar, it asks to open the app: spotify:track:60jpDubMmVyR5molJp2TCm

You can achieve this using the os module built into Python:
import os
os.startfile("spotify:track:60jpDubMmVyR5molJp2TCm")
This will successfully open the song inside of Spotify.

Python Proxy Settings

I was using the wikipedia module in which you can get the information that is present about that topic on wikipedia. When I run the code it is unable to connect because of proxy. When I connected PC to a proxy free network it's working. It also happened while using the Beautiful soup module for scraping. I have tried to set environment variable like http://username:password#proxy_url:port but when the run the code in IDLE it's not working. Please help.

It worked:
import os
os.environ["HTTPS_PROXY"] = "http://user_id:pass#proxy:port"

If you don't want to store your password in code file:
import os
pxuser = "your.corporate.domain\\your_username"
pxpass = input(f"Password for {pxuser}: ")
env_px = f"http://{pxuser}:{pxpass}#your_proxy:port"
os.environ["HTTPS_PROXY"] = env_px

How do I submit data to a web form in python?

I'm trying to automate the process of creating an account for something, lets call it X, but I cant figure out what to do.
I saw this code somewhere,
import urllib
import urllib2
import webbrowser
data = urllib.urlencode({'q': 'Python'})
url = 'http://duckduckgo.com/html/'
full_url = url + '?' + data
response = urllib2.urlopen(full_url)
with open("results.html", "w") as f:
f.write(response.read())
webbrowser.open("results.html")
But I cant figure out how to modify it for my use.

I would highly recommend utilizing Selenium+Webdriver for this, since your question appears UI and browser-based. You can install Selenium via 'pip install selenium' in most cases. Here are a couple of good references to get started.
- http://selenium-python.readthedocs.io/
- https://pypi.python.org/pypi/selenium
Also, if this process needs to drive the browser headlessly, look into including PhantomJS (via GhostDriver), which can be downloaded from the phantomjs.org website.