Develop Reference

Login and Clicking Buttons with Python "requests" Module

I have been playing with requests module on Python for a while as part of studying HTTP requests/responses; and I think I grasped most of the fundamental things on the topic that are supposed to be understood. With a naive analogy it basically works on ping-pong principle. You send a request in a packet to server and then it send back to you another packet. For instance, logging in to a site is simply sending a post request to server, I managed to do that. However, what I have trouble is to fail clicking on buttons through HTTP post request. I searched for it here and there, but I could not find a valid answer to my inquiry other than utilizing selenium module, which is what I do not want to if there is another way with requests module too. I am also aware of the fact that they created such a module called selenium for a thing.
QUESTIONS:
1) What kind of parameters do I have to take into account for being able to click on buttons or links from the account I accessed through HTTP requests? For instance, when I watch network activity for request header and response header with my browser's built-in inspect tool, I get so many parameters sent back by server, e.g. sec-fetch-dest, sec-fetch-mode, etc.
2) Is it too complicated for a beginner or is there too much advanced stuff going on behind the scene to do that so selenium was created for that reason?

Theoretically, you could write a program to do this with requests, but you would be duplicating much of the functionality that is already built and optimized in other tools and APIs. The general process would be:
Load the HTML that is normally rendered in your browser using a get request.
Process the HTML to find the button in question.
Then, if it's a simple form:
Determine the request method the button will carry out (e.g. using the formmethod argument, see here).
Perform the specified request with the required information in your request packet.
If it's a complex page (i.e. it uses JavaScript):
Find the button's unique identifier.
Process the JavaScript code to determine what action is performed when the button is clicked.
If possible, perform the JavaScript action using requests (e.g. following a link or something like that). I say if possible because JavaScript can do many things that, to my knowledge, simple HTTP request cannot, like changing rendered CSS in order to change the background color of a <div> when a button is clicked.
You are much better off using a tool like selenium or beautiful soup, as they have created APIs that do a lot of the above for you. If you've used the built-in requests library to learn about the basic HTTP request types and how they work, awesome--now move on to the plethora of excellent tools that wrap requests up into a more functional and robust API.

How do i receive JSON payloads from the slack API to retrieve messages using python?

I have been trying to make a slack bot for quite some time but there seems to be very little documentation on the slack URL, the most in-depth instructions I have found is to use ngrok and then you should receive JSON payloads from the slack API but as I am a beginner programmer I don't have much knowledge on this. I used an ngrok https URL and appended /slack/actions as documented from slack and just tried a little piece of code to see if the URL was ok to receive payloads from
import requests
r = requests.get('https://f39b39ebcdbf.ngrok.io/slack/actions')
print(r)
and I got a 405 error
So I am not sure what code I need to receive these payloads
as you can probably tell I am very new at this
so any help would be much appreciated

Use POST Method Instead of GET.
USE URL LIKE : https://xxxxxx.ngrok.io/api/YourSlackReceviver/Action

Logging into instagram using Python requests

I'm trying to log into Instagram using Python Requests. I figured it would be as simple as creating a requests.Session object and then sending a post request i.e.
session.post(login_url, data={'username':****, 'password':****})
This didn't work. I didn't know why so I tried manually entering the browsers headers (I used Chrome dev tools to see the headers of the post request) and passing them along with the request (headers={...}) even though I figured the session would deal with that. I tried sending a get request to the login URL first in order to get a cookie (and CSRF token I think) then doing the steps mentioned before. None of this worked.
I dont have much experience at all with this type of thing and I just dont understand what differentiates my post requests from google chromes (I must be doing something wrong). Thanks

Should I use GET or POST to send commands to server?

I am developing a simple application in python that provides a browser based interface using a local server. The structure is like this:
[Interface] <===> { [Server] <---> [Application] }
My application has a class function called, say, compute(). On my browser interface, I have a button Compute. When the user clicks it, I want to run compute(). My current approach is to send a GET request like /path/compute. The server identifies the request path and runs the function. Is this the correct way to go about it?
From this accepted answer:
GET is used for viewing something, without changing it, while POST is used for changing something
I am not using GET here to view something immediately, but to send a command that changes the state of the application. Should I use POST instead? Is there another method?

Yes indeed, you must use a POST request here. You may want to read about REST APIs for more about the correct use of http verbs. The HTTP rfc is also a must-read...

urlfetch change in April and simpleoauth

I am using this crhym3/simpleauth for oauth authentication with Google, Linkedin and Twitter in my project. It uses GAE's urlfetch.
Google is planning to change the behaviour of urlfetch in late April. I reproduce their notice here:
Currently, the URL Fetch service preserves your original HTTP method
(e.g., GET, POST) when it receives and responds to a 302 Moved
Temporarily response. Modern user agents typically issue a GET request
in response to a 302. After the update, URL Fetch will only issue a
GET request after receiving a 302 response, rather than preserving the
original method. This may cause requests to be routed differently
and/or return 404s or other errors, and will drop the message body
from POST requests.
I have posted a question on the project's forum but I haven't got a reply yet.
My question is:
What is the best way to test this piece of software is safe from the change? I am thinking of adding follow_redirects=False to the urlfetch calls to see what redirections I get from google, linkedin and twitter.

They are just following the specifications. I'm pretty sure that all of them (google, linkedin and twitter) are accepts GET request after redirect as its defined in the specifications.
So I think that you don't need to do anything.