Develop Reference

How do I send and receive a request using the requests library?

there is a task-from page get the text of all posts with more than 0 likes. As I understand it, you must first get all the tokens and IDs of possible posts from the page (which was not difficult) and make a request to the server using the requests library and these ids and get a response, since the post itself is in the code only in the form of a form without information about likes. But I don't understand much about the requests themselves and I can't figure out how to make such a request and get the html code of the post? Do I need a token? They are usually used for security and are generated by each user.
Directly finding the assumed token and request
Number of likes

To do this, you simply import the requests library and use requests.get(). More detailed response can be found here: https://realpython.com/python-requests/.

Understanding the Python requests module

So I'm currently learning the python requests module but I'm a bit confused and was wondering if someone could steer me in the right direction. I've seen some people post headers when they want to log into the website, but where do they get these headers from and when do you need them? I've also seen some people say you need an authentication token, but I've seen some other solutions not even use headers or an authentication token at all. This is supposedly the authentication token but I'm not sure where to go from here after I post my username and password.
<input type="hidden" name="lt" value="LT-970332-9KawhPFuLomjRV3UQOBWs7NMUQAQX7" />

Although your question is a bit vague, I'll try to help you.
Authentication
A web browser (client) can authenticate on the target server by providing data, usually the pair login/password, which is usually encoded for security reasons.
This data can be passed from client to server using the following parts of HTTP request:
URL parameters (http://httpbin.org/get?foo=bar)
headers
body (this is where POST parameters from HTML forms usually go)
Tokens
After successful authentication server generates a unique token and sends it to client. If server wants client to store token as a cookie, it includes Set-Cookie header in its response.
A token usually represents a unique identifier of a user session. In most cases token has an expiration date for security reasons.
Web browsers usually store token as a cookie in internal cookie storage and use them in all subsequent requests to corresponding website. A single website can use multiple tokens and other cookies for a single user.
Research
Every web site has its own authentication format, rules and restrictions, so first thing you need to do is a little research on target website. You need to get information about the client sends auth information to server, what server replies and where session data is being stored (usually you can find it in client request headers).
In order to do that, you may use a proxy (Burp for example) to intercept browser traffic. It can help you to get the data passed from client to server and back.
Try to authenticate and then browse some pages on target site using your web browser with a proxy. After that, using your proxy, examine what parts of HTTP request/response do client and browser use to store information about sessions and authentication.
After that you can finally use python and requests to do what you want.

Logging into instagram using Python requests

I'm trying to log into Instagram using Python Requests. I figured it would be as simple as creating a requests.Session object and then sending a post request i.e.
session.post(login_url, data={'username':****, 'password':****})
This didn't work. I didn't know why so I tried manually entering the browsers headers (I used Chrome dev tools to see the headers of the post request) and passing them along with the request (headers={...}) even though I figured the session would deal with that. I tried sending a get request to the login URL first in order to get a cookie (and CSRF token I think) then doing the steps mentioned before. None of this worked.
I dont have much experience at all with this type of thing and I just dont understand what differentiates my post requests from google chromes (I must be doing something wrong). Thanks

Python urllib2 accesses page without sending authentication details

I was reading urllib2 tutorial wherein it mentions that in order to access a page that requires authentication (e.g. valid username and password), the server first sends an HTTP header with error code 401 and (python) client then sends a request with authentication details.
Now, the problem in my case is that there exist two different versions of a webpage, one that can be accessed without supplying any authentication details and one that is quite different when authentication details are supplied (i.e. when the user is logged in the system). As an example think about url www.gmail.com, when you are not logged in you get a log-in page, but if your browser remembers you from your last login then the result is your email account homepage with your inbox displayed.
I follow all the details to set up an handler for authentication and install an opener. However everytime I request the page get back the version of the webpage that does not have the user logged-in.
How can I access the other version of webpage that has the user logged-in?

Requests makes this easy. As its creators say:
Python’s standard urllib2 module provides most of the HTTP capabilities you need, but the API is thoroughly broken.

Try using Mechanize. It has cookie handling features that would allow your program to be "logged in" even though it's not a real person.

How to submit a request by POST in Pyramid?

In Pyramid, send a request by GET can be done by create URL like this:
#view_config(route_name="test")
def test(request):
return HTTPFound("http://test.com/redirect_to_another_test?a=1")
But it seems that the HTTPFound can't do that by POST, then how can I do that?
Does anyone have ideas about this? Thanks!

You can't do this in Pyramid or any other Server-Side Framework or Language.
Your example code isn't showing a form submission, the code is showing a HTTP redirect. It is instructing the Browser to visit another URL, or in other words, telling the browser to resubmit the request.
This Stack Overflow question discusses this same concept, although in ASP not Python - Response.Redirect with POST instead of Get?
If you were to "submit a request" in Pyramid via GET or POST, you would have to use a library like urllib2, requests, or similar. In those instances, the libraries would have the Pyramid server act as the "submitter" of the request.
If you want to have the User/web-broswer submit the request by POST, you would have to do some fancy footwork/trickery to make the browser do that.
Possible ways to accomplish that would include:
use JavaScript AJAX form submission, and return an error code or instruction via JSON telling your JavaScript library to resubmit the same form via POST.
redirect the user to a GET page which has the form-data filled out (via the GET arguments) , and use javascript to resubmit the form via POST onload
You can't, in any server side language, tell the browser to resubmit a request by POST. Browsers don't work like that.
You should also know that "Most Browsers" will generally interpret any redirect request to be fetched via GET -- even if the original was a POST. That's not to spec - certain redirect codes are supposed to keep a POST a POST - however browsers don't follow all the specs. There is also no HTTP status code or command to purposefully switch from GET to POST .