I have a program I'd like to exploit. I have seen (through burp suite) that when it opens it does a request to a server and depending on the answer of the server it does multiple things. The thing is that I'd like to edit this request. I'd like to create a proxy (I have seen mitmproxy can fit my needs but if you have other suggestions feel free to post them), that "passtrough" all the http request except for the one I'd like. On this "special" request all it needs to do is give a custom response. I'm making you an example. The program does 10 request to google.com, I'd like to let this request pass back and forth, than it does one special request to example.org and example.org answer this request with "OK". I'd like to change this answer to "Wrong". Is there a way to do this? I have seen something similar but nothing like this. Can you help me?
P.S. I know how to program in python so if you link me an article is more than fine!
Have a nice day!
Edit:
I wrote this simple code i copied online but it doesn't seem to work...
from mitmproxy import http
def response(self, flow: http.HTTPFlow) -> None:
if flow.response and flow.response.content:
flow.response.content = flow.response.content.replace(
b"</head>",
b"<style>body {transform: scaleX(-1);}</style></head>"
)
I inject this with mitmproxy -s main.py
Am i doing something wrong?
Related
First of all, I apologize for my bad English. Also, I would appreciate it if you could explain your answers simply, since I am only 15. My maths and programming skills are not so good yet.
I am currently trying to program an HTTP server from Scratch. I am in the process of implementing PHP, but I need to read many values from the header.
The code works with Edge, OperaGX and even Internet Explorer. But Google Chrome doesn't send the values from the POST form, so I can't read them either.
Because I can't read the values, I can't pass them to PHP. And because of this, the $_POST variable doesn't work when the user is using Chrome.
If someone wants to reproduce the whole thing (its currently windows only), here is the py code:
https://pastebin.com/8qmx7G5G. And to start the Server:
def main() -> None:
server: Server = Server(docs_dir='docs', port=3033)
server.activate_php(php_dir='C:\\php',
php_ini='c:\\php\\php.ini')
server.start()
Anyway, an POST request usually looks like this:
POST /index.php?submit=1 HTTP/1.0
Accept: ...
Refferer: ...
username=Fido&password=example
But chrome sends only:
POST /index.php?submit=1 HTTP/1.0
Accept: ...
Refferer: ...
It's not loading the .php file, after submit the form. Its only raising Error ERR_CONNECTION_RESET.
Update:
When i access my localhost over http://localhost:3033, then it's not working (Only in Chrome, in Edge, IE and Opera its working). But if i access it with http://192.168.178.45:3033 then it's working. Is this perhaps a security feature of Chrome?
I'm trying to make one simple request:
ua=UserAgent()
req = requests.get('https://www.casasbahia.com.br/' , headers={'User-Agent':ua.random})
I would understand if I received <Response [403] or something like that, but instead, a recive nothing, the code keep runing with no response.
using logging I see:
I know I could use a timeout to avoid keeping the code running, but I just want to understand why I don't get an response
thanks in advance
I never used this API before, but from what I researched on here just now, there are sites that can block requests from fake users.
So, for reproducing this example on my PC, I installed fake_useragent and requests modules on my Python 3.10, and tried to execute your script. It turns out that with my Authentic UserAgent string, the request can be done. When printed on the console, req.text shows the entire HTML file received from the request.
But if I try again with a fake user agent, using ua.random, it fails. The site was probably developed to detect and reject requests from fake agents (or bots).
Though again, this is just theory. I have no ways to access this site's server files to comprove it.
So I need to build an HTTP server that will contact a client and send him data like pictures or calculations and create a page with those things. I guess you understood that I do not really know what I'm doing... :(
I know python and the basic(+) of the client-server project but I don't understand that HTTP protocol and didn't understand anything from what I read on the internet...
Can anyone explain to me how to work with this protocol? What is the form of HTTP packets?
Here an example of 1 problem that I don't understand: I have been asked to get a packet (which I did) and understand what is the request there, then send back the name of the file the client wants and after it the file itself. I printed the packet and didn't understand where is the request or what the client wants...
Thank you very very much!
Can anyone explain to me how to work with this protocol? What is the form of HTTP packets?
The specification might be helpful.
Concerning the webz, you find a lot of specification on the RFCs.
More to HTTP below.
(Since you seem to be new to programming, I figured I might want to tell you the following:)
Usually one doesn't directly interact with HTTP(S) packets. Instead you use a framework, such as flask, django, aiohttp and many more. The choice of framework depends on the use-case. E.g.:
You need a database, authentication and any imaginable feature? Go with Django.
You just want to create a WebApplication without a bloated framework? Go with Flask.
You need the bare minimum or want to act as a client? Go with aiohttp.
More frameworks are listed here.
The advantage of using such frameworks is that they usually include useful things, that are battletested (i.e. usually no bugs), while you don't have to figure out pecularities of certain protocols.
You just import the framework and write awesomeness! :)
(Anyways, here is a little very oversimplified overview for completeness)
So, HTTP is an text protocol over TCP, which basically means that you send text over a simple tcp socket. When you receive your request you have to "parse" (i.e. comprehend its contents). Luckily for us the requests are standarized and follow the same scheme.
The smallest request would look like this:
GET / HTTP/1.0
Host: www.server.com
The first line starts with a verb (also called request method), in our example the verb is GET. The / denotes the path. Think of file paths on your HDD. The last part of the first line, namely HTTP/1.0, tells the receiver with which version of HTTP we are operating on. Currently the there is HTTP 1.0 and HTTP 1.1; however, I wouldn't bother with HTTP 1.1 yet and stick with HTTP 1.0, if you're implementing the requests your self.
Lastly the Host: www.server.com line tells us which server we want to talk to, since multiple instances of an HTTP server could be running under the same ip. This is used to revole the subdomain.
If you send this request to an HTTP Server, you're likely to receive an response like this:
HTTP/1.0 200 OK
Server: Apache/1.3.29 (Unix) PHP/4.3.4
Content-Length: 1337
Connection: close
Content-Type: text/html
<DATA>
This response contains the status in the first line HTTP/1.0 200 OK. The number and the 'OK' represent a status code, telling us that everything is fine. There are many status codes with their own meaning and usages.
The lines following the first are so-called Response-Headers. They provide additional useful information about the response. For instance, when we open a site like 'stackoverflow.com', the server transmits an HTML file to us for the browser to interpret. Before we can do that, we need to know the size of the HTML file.
Luckily the server tells us beforehand with Content-Length: 1337 line, that the file is 1337 bytes big. The file itself would be present where the <DATA> placeholder stands.
There are, yet again, many of these headers.
As you can see, there are many things to account for when working with HTTP, showing that it is not feasible, without a very good reason, to implement a HTTP client/server from scratch.
Instead it's preferred to use one of the frameworks (for python) listed above.
As a last note:
In the process of trying to explain the concepts as simple as possible I probably left-out or oversimplified some things. If you find any mistake, please let me know.
I am trying to make little automated-testing script. It shoud be able to make HTTP request based on string provided, something like follows:
import coollib # non-existent library
r = coollib.make_raw_request(
# Lets assume, python's tripple quoted string spacing is not a problem.
"""
GET / HTTP/1.1\n\r
Host: example.com\n\r
My-Faulty-Header: status\n\r
"""
)
print(r.response_body)
Intention behind this is to insert a little mistakes into request, to test, how web server copes with faulty requests.
Any idea how to do this? Any insights are welcome.
As Ronald Aaronson mentioned in the comment, your weapon of choice here should probably be the socket library. There's an example in this related question: Creating a raw HTTP request with sockets
If I point Firefox at http://bitbucket.org/tortoisehg/stable/wiki/Home/ReleaseNotes, I get a page of HTML. But if I try this in Python:
import urllib
site = 'http://bitbucket.org/tortoisehg/stable/wiki/Home/ReleaseNotes'
req = urllib.urlopen(site)
text = req.read()
I get the following:
500 Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
What am I doing wrong?
You are not doing anything wrong, bitbucket does some user agent detection (to detect mercurial clients for example). Just changing the user agent fixes it (if it doesn't have urllib as a substring).
You should fill an issue regarding this: http://bitbucket.org/jespern/bitbucket/issues/new/
You're doing nothing wrong, on the surface, and as the error page says you should contact the site's administrators because they're the ones with the server logs which may explain what's happening. Fortunately, bitbucket's site admins are a friendly bunch!
No doubt there is some header or combination of headers that browsers set one way, urllib sets another way, and a bug on the server gets tickled in the latter case. You may want to see exactly what headers are being sent e.g. with firebug in firefox, and reproduce those until you isolate exactly the server bug; most likely it's going to be the user agent or some "accept"-ish header that's tickling that bug.
I don't think you're doing anything wrong -- it looks like this server was just down? Your script worked fine for me ('text' contained the same data as that displayed in the browser).