Working with a project, where using cookie for user identification.
When user arrives, it calls the service (which is running in localhost) and the service sending cookie with the response header looks like below:
curl 'http://127.0.0.1:8000/api/v1.0/tracking' -X OPTIONS -H 'Access-Control-Request-Method: POST' -H 'Origin: http://local.com:8080' -H 'Access-Control-Request-Headers: content-type,x-forwarded-for' --compressed
The response header looks like below:
HTTP/1.1 200 OK
Connection: keep-alive
Keep-Alive: 60
Access-Control-Allow-Headers: Content-Type, Access-Control-Allow-Headers, x-forwarded-for
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, PATCH, GET
Content-Length: 0
Content-Type: text/plain; charset=utf-8
Set-Cookie: id=random_id_123_123; expires=Wed, 06-Dec-2017 10:57:36 GMT; Domain=.local.com; Path=/
And then after a specific user action, the app is sending following API request:
curl 'http://127.0.0.1:8000/api/v1.0/tracking?event=video_added&user_id=123123123' -H 'Origin: http://local.com:8080' -H 'Accept: */*' -H 'Referer: http://local.com:8080/' -H 'Connection: keep-alive' --compressed
The request header for the above request looks like below:
GET api/v1.0/tracking?event=video_added&user_id=123123123 HTTP/1.1
Host: 127.0.0.1:8000
Connection: keep-alive
Accept: */*
Origin: http://local.com:8080
User-Agent: My user agent
Referer: http://local.com:8080/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
I was expecting the cookie (random_id_123_123) to be received with the first request as response header would be the request header for the second request.
The website is running on: http://local.com:8080 (which actually running on local machine and my vhost config pointing 127.0.0.1 local.com) and its being served by python SimpleHTTPServer.
The backend service which is setting the cookie is running on port 8000 in localhost also.
Seems I have missed something during the implementation. Whats that?
Edit: Here is the code.
Your issue is that cookies are only sent based on the domain. Your code has
var settings = {
"crossDomain": true,
"url": "http://127.0.0.1:8000/api/v1.0/tracking?event=video_added&tracking_id=123123123",
"method": "GET",
}
The url is 127.0.0.1:8000 and it should be local.com:8000 if you want the cookies to be passed.
Last time I checked, curl doesn't have enabled the cookies by default.
To do so you will need to:
Use the parameter -b /path/to/cookiejar to read cookies.
Use the parameter -c /path/to/cookiejar to write cookies.
So your requests should become:
curl -c cookiejar 'http://127.0.0.1:8000/api/v1.0/tracking' \
-X OPTIONS -H 'Access-Control-Request-Method: POST' \
-H 'Origin: http://local.com:8080' \
-H 'Access-Control-Request-Headers: content-type,x-forwarded-for' \
--compressed
And:
curl -b cookiejar 'http://127.0.0.1:8000/api/v1.0/tracking?event=video_added&user_id=123123123' \
-H 'Origin: http://local.com:8080' \
-H 'Accept: */*' \
-H 'Referer: http://local.com:8080/' \
-H 'Connection: keep-alive' --compressed
Related
Question
How do I use Basic Auth and Bearer Token Auth with python requests module?
Background
I write curl script as same as I want to do with python requests module.
curl --location --request POST 'https://username#password:example.com/path/to/endpoint' \
--header 'Authorization: Bearer SOME_TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
"example": "example"
}'
Below code is python edition for the above script.
import requests
import json
url = "https://username#password:example.com/path/to/endpoint"
payload = json.dumps({
"example": "example"
})
headers = {
'Authorization': 'Bearer SOME_TOKEN',
'Content-Type': 'application/json'
}
response = requests.request("POST", url, headers=headers, data=payload)
But this Python code does not work as intended.
I captured http request that this python code sends.
The result is below.
POST /path/to/endpoint HTTP/1.1
Host: example
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Basic xxxxx
Connection: keep-alive
Content-Length: 31
Content-Type: application/json
User-Agent: python-requests/2.28.1
{"example": "example"}
I don't know why Auhorization: Basic xxxx appears.
I intend that Authorization: Bearer SOME_TOKEN.
Please teach me how to use Basic auth and Bearer token auth with python requests module.
appendix
I think that I can't use both Basic auth and Bearer token auth with python requests module.
Because this module depends on python http.client module, and I think this module does not support basic auth with including username and password in url(https://username:password#example.com/ format).
I examined the http request that the above codes sends to http server.
Is there a library/tool out there to convert raw application/json requests to curl command line (like show code in postman, or the curl in openapi).
Simple example under with just json:
---------------- request ----------------
POST https://xxxx.yyyy.zzzz/idpro-test-api/mid/business
User-Agent: python-requests/2.27.1
Accept-Encoding: gzip, deflate
accept: application/json
Connection: keep-alive
Content-Type: application/json
Content-Length: 86
{"user_id": "987000205", "ssn": "21010008287", "mid_issuer_id": 32, "activate": false}
---------------- response ----------------
<deleted example of a 500 I like to reproduce>
------------------------------------------
curl.exe in powershell (needs to escape the double-quotes):
curl.exe -vv -X POST -H "accept: application/json" -H "Content-Type: application/json" \
"https://xxxx.yyyy.zzzz/idpro-test-api/mid/business" \
-d '{\"user_id\": \"987000205\", \"ssn\": \"21010008287\", \"mid_issuer_id\": 32, \"activate\": false}'
curl in linux bash:
curl -vv -X POST -H "accept: application/json" -H "Content-Type: application/json" \
"https://xxxx.yyyy.zzzz/idpro-test-api/mid/business" \
-d '{"user_id": "987000205", "ssn": "21010008287", "mid_issuer_id": 32, "activate": false}'
I found curlify which is exactly what I want: https://github.com/ofw/curlify, (why cant pyCurl mention this on their site).
I have the code below in Python which is making a POST request for an OAuth2 token. It uses basic authentication.
The code works fine, but I would like to "translate it" to curl.
The code:
#Authorization: Basic c29tZV91c2VyOnBhc3M=
#some_user:pass = base64decode('c29tZV91c2VyOnBhc3M=')
def get_access_token():
burp0_url = "https://myurl:443/api/oauth/token"
burp0_headers = {"Accept": "application/json", "Authorization": "Basic c29tZV91c2VyOnBhc3M=", "Content-Type": "application/x-www-form-urlencoded", "Connection": "close", "Accept-Encoding": "gzip, deflate", "User-Agent": "okhttp/3.0.1"}
burp0_data={"grant_type": "client_credentials"}
return json.loads(requests.post(burp0_url, headers=burp0_headers,
data=burp0_data).text)['access_token']
My guess was be that it would look something like this:
curl -v -XPOST -H 'Authorization: Basic c29tZV91c2VyOnBhc3M=' --header 'Accept: application/json' --header 'Connection: close' --header 'Accept-Encoding: gzip, deflate' --header 'User-Agent: okhttp/3.0.1' --data '{"grant_type": "client_credentials"}' https://myurl:443/api/oauth/token
However I keep getting a HTTP/1.1 400 and the following
* Failed writing body (0 != 10)
* Failed writing data
* stopped the pause stream!
* Closing connection 0
Can you help me?
Looks like you forgot to copy the "Content-Type": "application/x-www-form-urlencoded" header into your curl command.
This would also suggest the data is not submitted as a JSON string as you're currently doing, but as regular form data. You can probably use -F 'grant_type=client_credentials' for that and drop the --data argument.
I have a Python script using the Requests library that is of this form:
uhash = '1234567abcdefg'
cookies = {
'uhash':uhash
}
payload = {
'action':'trade.bump',
'hash':uhash,
'tradeid':'12345678'
}
r = requests.post(
'http://www.target_url.com/api/core',
cookies=cookies,
params=payload
)
Above is my Python attempt at creating the following cURL request (written with bash):
HASH="1234567abcdefg"
TRADEID="12345678"
curl 'http://www.target_url.com/api/core' -H "Cookie: uhash=$HASH" --data "action=trade.bump&hash=$HASH&tradeid=$TRADEID"
In summary, both scripts contain:
The cookie - uhash
Three data parameters called action, hash, and tradeid
My issue currently is, the bash script works - the server response for when I use the bash script is this:
{"meta":{"code":200},"data":{"bumped":true,"count":15}}
However, if I use the Python script, with the SAME cookie and parameter values as the bash script, I get:
{"meta":{"code":301},"data":{"message":"You can't bump a trade that doesn't exist ;_;"}}
The above error tells me the trade doesn't exist, despite that tradeid existing and the exact same one as my bash script's parameters.
I tried to debug using Firefox' convenient copy-as-curl tool to copy that curl command, which was how I made the bash script. However, once I tried to translate it to the Python script, it will tell me the aforementioned error. Maybe I am using the Requests library incorrectly, and I am missing something.
Attached is the full cURL request taken from Firefox (don't worry, the parameters were sanitized, meaning, they're not the real values):
curl 'http://www.tf2outpost.com/api/core' -H 'Host: www.tf2outpost.com' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:35.0) Gecko/20100101 Firefox/35.0' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'X-Requested-With: XMLHttpRequest' -H 'Referer: http://www.tf2outpost.com/trades' -H 'Cookie: __qca=P0-6517545-1420724809746; __utma=5135382.11011755.14224810.14331180.14180489.7; __utmz=51353782.1420724810.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); uhash=abcdefg12345678; mb_uid2=3211475230616776; CTag61=14338638870; __utmb=513532.9.10.14180489; __utmc=513782; __utmt=1; __utmt_b=1; __utmt_c=1; OX_plg=sl|qt|pm; HIRO_COOKIE=data=&newSession=true&id=2237524293×tamp=1433506185; HIRO_CLIENT_ID=67751187' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' --data 'action=trade.bump&hash=abcdefg12345678&tradeid=12345678'
Not quite sure why that is happening.
Try using data or json key instead of params, use json.dumps(payload) if data is your preferred method.
There are a couple of default headers that HTTPie sets. I'm wondering if there is a way to remove some header, like Accept-Encoding?
The reason I like to unset Accept-Encoding is to check our server's behavior about HTTP Compression.
Per https://github.com/jakubroztocil/httpie#http-headers , you can override those headers. For example, set Accept-Encoding to empty to achieve the same effect as if you had removed it -- per the rules at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.3 .
Add the Headers follow by a colon.
No headers:
http -v https://jsonplaceholder.typicode.com/todos/1 \
Accept: \
Accept-Encoding: \
Connection: \
Host: \
User-Agent:
Request:
GET /todos/1 HTTP/1.1
Host: jsonplaceholder.typicode.com
Response:
HTTP/1.1 200 OK
...
Standard:
http -v https://jsonplaceholder.typicode.com/todos/1
Request:
GET /todos/1 HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: jsonplaceholder.typicode.com
User-Agent: HTTPie/0.9.8
Response:
HTTP/1.1 200 OK
...
The -v option display the request. Also, remember no spaces after \ in multiline bash commands.