Develop Reference

Python Request POST - Upload file via JOOMLA Community Builder Import

I have a JOOMLA 3 system with Community Builder plugin. I need to update Users from a CSV file.
Here is the code I have tried:
if __name__ == "__main__":
s = fn_login()
r = s.get(cb_users)
print(r)
with open(IMPORT_FILE, 'rb') as f:
files = {'files': f}
r = s.post(post_url, files=files)
The session is ok. I have tried to figure out the POST message, but something may be missing. Nothing happens when I send the post. I have got response 200. What else do I need to consider?
Here is the request header:
curl 'https://3joomla.domain/administrator/index.php'
-X 'POST'
-H 'authority: 3joomla.domain'
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9'
-H 'accept-language: en-US,en;q=0.9,hu;q=0.8'
-H 'cache-control: max-age=0'
-H 'content-length: 6710'
-H 'content-type: multipart/form-data; boundary=----WebKitFormBoundaryALBLy00y6zolw7hC'
-H 'cookie: ....
-H 'origin: https://3joomla.domain'
-H 'referer: https://3joomla.domain/administrator/index.php'
-H 'sec-ch-ua: "Not?A_Brand";v="8", "Chromium";v="108", "Google Chrome";v="108"'
-H 'sec-ch-ua-mobile: ?0'
-H 'sec-ch-ua-platform: "Linux"'
-H 'sec-fetch-dest: document'
-H 'sec-fetch-mode: navigate'
-H 'sec-fetch-site: same-origin'
-H 'sec-fetch-user: ?1'
-H 'upgrade-insecure-requests: 1'
-H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36'
--compressed

application/json request to curl command line for reproduction

Is there a library/tool out there to convert raw application/json requests to curl command line (like show code in postman, or the curl in openapi).
Simple example under with just json:
---------------- request ----------------
POST https://xxxx.yyyy.zzzz/idpro-test-api/mid/business
User-Agent: python-requests/2.27.1
Accept-Encoding: gzip, deflate
accept: application/json
Connection: keep-alive
Content-Type: application/json
Content-Length: 86
{"user_id": "987000205", "ssn": "21010008287", "mid_issuer_id": 32, "activate": false}
---------------- response ----------------
<deleted example of a 500 I like to reproduce>
------------------------------------------
curl.exe in powershell (needs to escape the double-quotes):
curl.exe -vv -X POST -H "accept: application/json" -H "Content-Type: application/json" \
"https://xxxx.yyyy.zzzz/idpro-test-api/mid/business" \
-d '{\"user_id\": \"987000205\", \"ssn\": \"21010008287\", \"mid_issuer_id\": 32, \"activate\": false}'
curl in linux bash:
curl -vv -X POST -H "accept: application/json" -H "Content-Type: application/json" \
"https://xxxx.yyyy.zzzz/idpro-test-api/mid/business" \
-d '{"user_id": "987000205", "ssn": "21010008287", "mid_issuer_id": 32, "activate": false}'

I found curlify which is exactly what I want: https://github.com/ofw/curlify, (why cant pyCurl mention this on their site).

Getting 302 with Curl, but 200 with python requests

The site that I am trying to scrape must follow 302 observed by the browser network tools. I've copied the network request as Curl and its working fine, but when I convert it to python requests its just returning 200.
Curl:
curl -v 'https://my-site.com/CardAuthentication.aspx' \
-H 'Connection: keep-alive' \
-H 'Pragma: no-cache' \
-H 'Cache-Control: no-cache' \
-H 'Upgrade-Insecure-Requests: 1' \
-H 'Origin: https://my-site.com' \
-H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryBN2XwBbcAwvRWZzk' \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
-H 'Sec-GPC: 1' \
-H 'Sec-Fetch-Site: same-origin' \
-H 'Sec-Fetch-Mode: navigate' \
-H 'Sec-Fetch-User: ?1' \
-H 'Sec-Fetch-Dest: document' \
-H 'Referer: https://my-site.com/CardAuthentication.aspx' \
-H 'Accept-Language: en-US,en;q=0.9' \
-H 'Cookie: ASP.NET_SessionId=...; TS0192fa71=...; ServiceProvider=UID=...; .ASPXAUTH=...' \
--data-raw $'------WebKitFormBoundaryBN2XwBbcAwvRWZzk\r\ DATA \r\n' \
--compressed
this is returning:
* Trying IP:443...
........
........
< HTTP/1.1 302 Found
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
< Expires: -1
< Location: my-site.com/MemberDetails.aspx
< Date: Mon, 07 Feb 2022 09:54:49 GMT
< Content-Length: 61211
< Set-Cookie: TS0192fa71=...; Path=/; Domain=.my-site.com; Secure
Python:
import requests
import logging
logging.basicConfig(level=logging.DEBUG)
cookies = {
"ASP.NET_SessionId": ".....",
"TS0192fa71": ".....",
"ServiceProvider": ".....",
".ASPXAUTH": ".....",
}
headers = {
......
"Content-Type": "multipart/form-data; boundary=----WebKitFormBoundaryBN2XwBbcAwvRWZzk",
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36",
"Referer": "https://my-site.com/CardAuthentication.aspx",
"Accept-Language": "en-US,en;q=0.9",
}
data = {
"------WebKitFormBoundaryBN2XwBbcAwvRWZzk\r\nContent-Disposition: form-data; name": '"__EVENTTARGET"\r\n\r\n\r\n-- DATA
}
response = requests.post(
"https://my-site.com/CardAuthentication.aspx",
headers=headers,
cookies=cookies,
data=data,
)
And the return code is 200, with response history empy.
Is there something wrong with requests library or in the way that request library is processing the data? How can I solve this?

Is there something wrong with requests library or in the way that
request library is processing the data? How can I solve this?
This is default requests behavior, you need to set allow_redirects to False if you want to not follow in case of 3xx response code, example
import requests
r = requests.get("http://github.com", allow_redirects=False)
print(r.status_code) # 301
If you want to know more read request.request docs.

How to consume this API in Python? I just can't

So i'm trying to consume this API, I got this URL http://www.ventamovil.com.mx:9092/service.asmx?op=Check_Balance
There you can write this {"User":"6144135400","Password":"Prueba$$"} on the input field and you get a response.
https://i.stack.imgur.com/RTEii.png
Response
But when i try to consume this api on python i just can't, i don't exactly know how to consume correctly:
My Code
As you can see i got a different response with my code, i should be getting the same response as the "Response" image.

To save yourself some time, you can use their request to build python code automatically, all you have to do is:
Just as you did at first, enter the json in the input field and invoke.
open the network tab, copy the post request they made as curl
curl 'http://www.ventamovil.com.mx:9092/service.asmx/Check_Balance' -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36' -H 'Origin: http://www.ventamovil.com.mx:9092' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'Referer: http://www.ventamovil.com.mx:9092/service.asmx?op=Check_Balance' -H 'Accept-Language: en-US,en;q=0.9,ar;q=0.8,pt;q=0.7' --data 'jrquest=%7B%22User%22%3A6144135400%2C+%22Password%22%3A+%22Prueba%24%24%22%7D' --compressed --insecure
Go to postman and import the curl, then click code and select python, and here you go you have all the right headers needed
import requests
url = "http://www.ventamovil.com.mx:9092/service.asmx/Check_Balance"
payload = 'jrquest=%7B%22User%22%3A6144135400%2C+%22Password%22%3A+%22Prueba%24%24%22%7D'
headers = {
'Upgrade-Insecure-Requests': '1',
'Content-Type': 'application/x-www-form-urlencoded',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
}
response = requests.request("POST", url, headers=headers, data = payload)
print(response.text.encode('utf8'))
As you can see, they accept their input as form encoded payload.
You need to modify this request to be parameterized with user/password you want each time you use.
Btw, the output of this python code is:
b'<?xml version="1.0" encoding="utf-8"?>\r\n<string xmlns="http://www.ventamovil.com.mx/ws/">{"Confirmation":"00","Saldo_Inicial":"10000","Compras":"9360","Ventas":"8416","Comision":"469","Balance":"10345.92"}</string>'

Incorrect input parameters when using the Python Requests library

I have a Python script using the Requests library that is of this form:
uhash = '1234567abcdefg'
cookies = {
'uhash':uhash
}
payload = {
'action':'trade.bump',
'hash':uhash,
'tradeid':'12345678'
}
r = requests.post(
'http://www.target_url.com/api/core',
cookies=cookies,
params=payload
)
Above is my Python attempt at creating the following cURL request (written with bash):
HASH="1234567abcdefg"
TRADEID="12345678"
curl 'http://www.target_url.com/api/core' -H "Cookie: uhash=$HASH" --data "action=trade.bump&hash=$HASH&tradeid=$TRADEID"
In summary, both scripts contain:
The cookie - uhash
Three data parameters called action, hash, and tradeid
My issue currently is, the bash script works - the server response for when I use the bash script is this:
{"meta":{"code":200},"data":{"bumped":true,"count":15}}
However, if I use the Python script, with the SAME cookie and parameter values as the bash script, I get:
{"meta":{"code":301},"data":{"message":"You can't bump a trade that doesn't exist ;_;"}}
The above error tells me the trade doesn't exist, despite that tradeid existing and the exact same one as my bash script's parameters.
I tried to debug using Firefox' convenient copy-as-curl tool to copy that curl command, which was how I made the bash script. However, once I tried to translate it to the Python script, it will tell me the aforementioned error. Maybe I am using the Requests library incorrectly, and I am missing something.
Attached is the full cURL request taken from Firefox (don't worry, the parameters were sanitized, meaning, they're not the real values):
curl 'http://www.tf2outpost.com/api/core' -H 'Host: www.tf2outpost.com' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:35.0) Gecko/20100101 Firefox/35.0' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'X-Requested-With: XMLHttpRequest' -H 'Referer: http://www.tf2outpost.com/trades' -H 'Cookie: __qca=P0-6517545-1420724809746; __utma=5135382.11011755.14224810.14331180.14180489.7; __utmz=51353782.1420724810.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); uhash=abcdefg12345678; mb_uid2=3211475230616776; CTag61=14338638870; __utmb=513532.9.10.14180489; __utmc=513782; __utmt=1; __utmt_b=1; __utmt_c=1; OX_plg=sl|qt|pm; HIRO_COOKIE=data=&newSession=true&id=2237524293&timestamp=1433506185; HIRO_CLIENT_ID=67751187' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' --data 'action=trade.bump&hash=abcdefg12345678&tradeid=12345678'
Not quite sure why that is happening.

Try using data or json key instead of params, use json.dumps(payload) if data is your preferred method.