I am trying to make little automated-testing script. It shoud be able to make HTTP request based on string provided, something like follows:
import coollib # non-existent library
r = coollib.make_raw_request(
# Lets assume, python's tripple quoted string spacing is not a problem.
"""
GET / HTTP/1.1\n\r
Host: example.com\n\r
My-Faulty-Header: status\n\r
"""
)
print(r.response_body)
Intention behind this is to insert a little mistakes into request, to test, how web server copes with faulty requests.
Any idea how to do this? Any insights are welcome.
As Ronald Aaronson mentioned in the comment, your weapon of choice here should probably be the socket library. There's an example in this related question: Creating a raw HTTP request with sockets
Related
I have a program I'd like to exploit. I have seen (through burp suite) that when it opens it does a request to a server and depending on the answer of the server it does multiple things. The thing is that I'd like to edit this request. I'd like to create a proxy (I have seen mitmproxy can fit my needs but if you have other suggestions feel free to post them), that "passtrough" all the http request except for the one I'd like. On this "special" request all it needs to do is give a custom response. I'm making you an example. The program does 10 request to google.com, I'd like to let this request pass back and forth, than it does one special request to example.org and example.org answer this request with "OK". I'd like to change this answer to "Wrong". Is there a way to do this? I have seen something similar but nothing like this. Can you help me?
P.S. I know how to program in python so if you link me an article is more than fine!
Have a nice day!
Edit:
I wrote this simple code i copied online but it doesn't seem to work...
from mitmproxy import http
def response(self, flow: http.HTTPFlow) -> None:
if flow.response and flow.response.content:
flow.response.content = flow.response.content.replace(
b"</head>",
b"<style>body {transform: scaleX(-1);}</style></head>"
)
I inject this with mitmproxy -s main.py
Am i doing something wrong?
So I need to build an HTTP server that will contact a client and send him data like pictures or calculations and create a page with those things. I guess you understood that I do not really know what I'm doing... :(
I know python and the basic(+) of the client-server project but I don't understand that HTTP protocol and didn't understand anything from what I read on the internet...
Can anyone explain to me how to work with this protocol? What is the form of HTTP packets?
Here an example of 1 problem that I don't understand: I have been asked to get a packet (which I did) and understand what is the request there, then send back the name of the file the client wants and after it the file itself. I printed the packet and didn't understand where is the request or what the client wants...
Thank you very very much!
Can anyone explain to me how to work with this protocol? What is the form of HTTP packets?
The specification might be helpful.
Concerning the webz, you find a lot of specification on the RFCs.
More to HTTP below.
(Since you seem to be new to programming, I figured I might want to tell you the following:)
Usually one doesn't directly interact with HTTP(S) packets. Instead you use a framework, such as flask, django, aiohttp and many more. The choice of framework depends on the use-case. E.g.:
You need a database, authentication and any imaginable feature? Go with Django.
You just want to create a WebApplication without a bloated framework? Go with Flask.
You need the bare minimum or want to act as a client? Go with aiohttp.
More frameworks are listed here.
The advantage of using such frameworks is that they usually include useful things, that are battletested (i.e. usually no bugs), while you don't have to figure out pecularities of certain protocols.
You just import the framework and write awesomeness! :)
(Anyways, here is a little very oversimplified overview for completeness)
So, HTTP is an text protocol over TCP, which basically means that you send text over a simple tcp socket. When you receive your request you have to "parse" (i.e. comprehend its contents). Luckily for us the requests are standarized and follow the same scheme.
The smallest request would look like this:
GET / HTTP/1.0
Host: www.server.com
The first line starts with a verb (also called request method), in our example the verb is GET. The / denotes the path. Think of file paths on your HDD. The last part of the first line, namely HTTP/1.0, tells the receiver with which version of HTTP we are operating on. Currently the there is HTTP 1.0 and HTTP 1.1; however, I wouldn't bother with HTTP 1.1 yet and stick with HTTP 1.0, if you're implementing the requests your self.
Lastly the Host: www.server.com line tells us which server we want to talk to, since multiple instances of an HTTP server could be running under the same ip. This is used to revole the subdomain.
If you send this request to an HTTP Server, you're likely to receive an response like this:
HTTP/1.0 200 OK
Server: Apache/1.3.29 (Unix) PHP/4.3.4
Content-Length: 1337
Connection: close
Content-Type: text/html
<DATA>
This response contains the status in the first line HTTP/1.0 200 OK. The number and the 'OK' represent a status code, telling us that everything is fine. There are many status codes with their own meaning and usages.
The lines following the first are so-called Response-Headers. They provide additional useful information about the response. For instance, when we open a site like 'stackoverflow.com', the server transmits an HTML file to us for the browser to interpret. Before we can do that, we need to know the size of the HTML file.
Luckily the server tells us beforehand with Content-Length: 1337 line, that the file is 1337 bytes big. The file itself would be present where the <DATA> placeholder stands.
There are, yet again, many of these headers.
As you can see, there are many things to account for when working with HTTP, showing that it is not feasible, without a very good reason, to implement a HTTP client/server from scratch.
Instead it's preferred to use one of the frameworks (for python) listed above.
As a last note:
In the process of trying to explain the concepts as simple as possible I probably left-out or oversimplified some things. If you find any mistake, please let me know.
I am writing a simple SOAP client application in Python.
WSDL file can be found here: https://clients.nationalmailing.com.au/ServiceTest/OrderService.svc?wsdl
Unfortunately the server declared usage of wsHttpBinding in its WSDL file and I had to learn how many troubles it brings to not-.NET developers.
I have working C# code (and it is pretty simple there) and used Fiddler to capture the traffic and analyze messages. Now I know the structure to follow. Client sends 2 subsequental messages.
I managed to create and send first request and receive a response from the server. BUT second request is a way more complex. I have found a library signxml which helped me to create <Signature> structure with all the fields that should present (as per captured traffic).
But the server continues to answer with "Error 500: An error occurred when verifying security for the message."
I realized that in the first message I put just random values for the following structure:
<s:Body>
<trust:RequestSecurityToken xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
<trust:TokenType>http://docs.oasis-open.org/ws-sx/ws-secureconversation/200512/sct</trust:TokenType>
<trust:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType>
<trust:Entropy>
<trust:BinarySecret
u:Id="uuid-0649fd7a-9ae2-4f9f-964c-e3aa5d68e8cd-1"
Type="http://docs.oasis-open.org/ws-sx/ws-trust/200512/Nonce">h/MaeQVSL5Br30Hnt/SAl274flYfZVZyx2Fri9zNuEY=</trust:BinarySecret>
</trust:Entropy>
<trust:KeySize>256</trust:KeySize>
</trust:RequestSecurityToken>
</s:Body>
The value of BinarySecret is just a random string encoded with Base64. I think this should be an issue on this stage. I also do not use the same parameters from server's response.
Could anyone explain how should I use Entropy.BinarySecret - should it take part in the calculations of Signature and how it is used?
Answering my own question. Yes, the issue was in improper usage of Entropy parameter.
To sign the message you need to generate a key, it consists of two parts (client entropy and server's entropy). They get combined with P_SHA1 algorithm into a key.
To anyone who find this post in the future: for Python have a look on signxml library and section 4 of ws-trust spec.
can anyone guide me in the right direction when it comes to http requests with python? What I'm after is a Excel-VBA add-in which will track which workbooks the user opens and when etc. While that's already done, I would now like to get the information to a database.
For that purpose I can imagine running a very simple Python server which would be used to store the information. The question thus is, how do I set up a simple http server so that VBA can post a simple string which then gets stored?
Thanks!
EDIT:
Thanks chf! I went ahead and followed your advice - I replaced flask with django though as I had some brief experience with that. I've got my first API ever created now but can't post using the VBA code you posted. I can do httpie like so: "http POST http:/127.0.0.1 name="somename" workbookname="someworkbook".
Sub TestFramework()
Dim newClient As New WebClient
Dim newRequest As New WebRequest
Dim Response As WebResponse
newClient.BaseUrl = "http://127.0.0.1:8000/api/create/"
newRequest.Method = HttpPost
newRequest.Format = WebFormat.plaintext
newRequest.AddBodyParameter "name", "somename"
newRequest.AddBodyParameter "workbook_name", "Sheet1"
Set Response = newClient.Execute(newRequest)
End Sub
Any chance you might point me to the right way?
RuntimeError: You called this URL via POST, but the URL doesn't end in a slash and you have APPEND_SLASH set. Django can't redirect to the slash URL while maintaining POST data. Change your form to point to 127.0.0.1:8000/api/create/ (note the trailing slash), or set APPEND_SLASH=False in your Django settings.
[03/Aug/2016 20:13:18] "POST /api/create HTTP/1.1" 500 60534
Edit2: nevermind, got it working :)
Nice project!
You can use Flask for the Python part to build a small REST Api and for the VBA part you can use VBA-WEB to consume that API.
Both Flask than VBA-WEB are very well documented with a lot of examples.
I use VBA-WEB in a lot of small "SAK" (swiss army knife) utilities in Excel and it's very useful and powerful.
For some rest apis with json output Flask is a nice tool to use.
I am using Python 2.7 with the requests module to send http post with parameters. I encountered a strange problem.
To do http post, it is just one line;
x = requests.post(URL, params)
I have no problem with the params. It is the URL that puzzled me.
Sometimes, this URL http://hostname/path/post works. Sometimes, I use http://hostname/path without the /post to get the HTTP post to work. I am puzzled why is this so. What is the difference between the two? Under what conditions do I use which one?
'http://hostname/path/post' is a path. You could in principle issue and HTTP GET request to that same path (although probably you wouldn't get anything meaningful back).
In general, you should look at the site's API documentation and post to the url that they say you should post to without adding anything extra to the url.
There are two different concepts, url and HTTP method. You are confused by trying to mix them.
url - an address you talk to
The url is addressing something on some server. If you get valid url, you can take it as a string, do not read in, and use it. Consider it to be a string.
If I would link it to a visiting your friend, url is address of a doors to come to.
HTTP method (POST, GET, DELETE...)
There are multiple HTTP methods which differ in the way, how you talk to given url.
Linking it to visiting a friend, it would be the way, you try to make the doors open (use the bell, knock or use a hammer)