Develop Reference

Python Requests Response 404

I would get access to this website, I get it from another website, https://www.betexplorer.com/soccer/algeria/ligue-1/bordj-bou-arreridj-tlemcen/tfvAHu7U/, in the Network section of Developer tools.
Code
import requests
url = 'https://www.betexplorer.com/archive-odds/4urejxv464x0xd4645/18/'
response = requests.get(url, headers={'User-Agent': 'my user agent'})
print(response)
Output <Response [404]>

Set Referer HTTP header to obtain correct response:
import json
import requests
url = "https://www.betexplorer.com/archive-odds/4urejxv464x0xd4645/18/"
headers = {
"Referer": "https://www.betexplorer.com",
}
data = requests.get(url, headers=headers).json()
print(json.dumps(data, indent=4))
Prints:
[
{
"date": "23.07. 17:36",
"odd": "1.88",
"change": "+0.08"
},
{
"date": "23.07. 17:34",
"odd": "1.80",
"change": "-0.01"
},
...

Python - Get Request with Cookies

I've got a puzzling requests that I do not seem able to dig myself out of, despite several attempts.
Context / Workflow:
1. Logging to a webservice
2. Make a Get request
3. Obtain JSoN
Issue:
- JSON obtained by going via browser is different (read: has more data) than the one obtained via my code
My code:
with requests.Session() as s:
s.headers = {'Accept': 'application/json; indent=4', 'Content-Type': 'application/json'}
secret = {
"client_id": "xxx",
"client_secret": "yyy"
}
datas = json.dumps(secret)
url = 'https://XXXX.yyy/sign_in'
response = s.post(url, data=json.dumps(secret), headers=s.headers, verify=False)
print (s.headers)
print(response)
This part works OK, returns me a cookie, which I save on
ck = s.cookies.get_dict()
I then need to make a Get request with the cookies saved after successful login
url = 'https://give.me/tests'
print ("Connecting to: " + url)
# here we do our get, passing the cookies
response = s.get(url, cookies=ck, verify=False)
data = response.text
# print (s.headers) # is OK, returns 200
# print(data)
with open('data.txt', 'w') as f:
f.write(data)
return 0
Expected Result:
"phase": {
"type": "phase",
"activity_level": null,
"logical_name": "phase.test_manual.obsolete",
"name": "Obsolete",
"index": 4,
"id": "phase.test_manual.obsolete"
},
Actual Result:
"phase": {
"type": "phase",
"id": "phase.test_manual.obsolete"
},
Documentation, in case I'm doing something wrong functionally:

How to Redirect to another url after POST request to PARSE / GRAB data ? Python 3.x

I am sending a JSON post request to a URL with headers and para:
headers = {
'channel':'mobiApp',
'Content-Type': 'application/json; charset=UTF-8',
'Content-Length': '939',
'Cookie': 'xyzxcasd',
'User-Agent': 'okhttp/3.8.0'
}
url = 'https://api.xyz.com/account.jsp'
data = {'act':'login','class':'profile','prftcf':code1,'p':password1,'u':username1,'ver':'0.9.1'}
content = json.dumps(data)
print ("")
print (content)
print ("")
r = requests.post(url,headers=headers,data=content,allow_redirects=True)
z = r.text
(above is the request I am sending)
I am getting this as response:
{
"s": 0,
"err": "",
"errCode": "",
"status": "",
"ccnt": 1,
"em": "blueyes81382#yahoo.com",
"pid": "2436999645",
"bvUserToken": "ee936b7065353389878696fc7cc4d71a646174653d3230313730393137267573657269643d32343336393939363435"
}
What I want to do is redirect after this post request (if bvUSertoken exists) to some URL, i.e.: https://api.xyz.com/acount/summary, and print the response.

okay so i found a solution to this
p['Set-cookie'] parse cookie from previous response header
cookies = (p['Set-Cookie'])
headers1 = {
'channel':'mobiApp',
'Content-Type': 'application/json; charset=UTF-8',
'Cookie' : cookies,
'User-Agent': 'okhttp/3.8.0'
}
url = 'your redirect url'
urlx = url
parsing = requests.get(urlx,headers=headers1)

Using Python to Post json contact list to Qualtrics API, error with Content-Type

I'm trying to import contacts into a contact list in Qualtrics. I am using python to do this.
Token = 'MyToken' #when running the code I put in my actual token and id
ContactsID = 'MyContactsID'
data = open('contacts.json', 'rb')
headers = {'X-API-TOKEN': Token, 'Content-Type':'application/json',}
r = requests.post('https://az1.qualtrics.com/API/v3/mailinglists/' + ContactsID +'/contactimports', headers=headers, data=data)
r.text
This code gives me the following error: '{"meta":{"httpStatus":"400 - Bad Request","error":{"errorMessage":"Invalid Content-Type. expected=multipart/form-data found=application/json","errorCode":"RP_0.1"},"requestId":null}}'
I changed the content type to multipart/form-data that it says it is expecting and received the response "413", which qualtrics explains means "The request body was too large. This can also happen in cases where a multipart/form-data request is malformed."
I have tested my json and verified that it is valid. Also, I don't know why the request body would be too large because it's only 13 contacts that I'm trying to import. Any ideas?

With the help of Qualtrics Support, I was eventually able to get the following code to work:
Token = 'MyToken' #when running the code I put in my actual token and id
ContactsID = 'MyContactsID'
url = "https://az1.qualtrics.com/API/v3/mailinglists/" + ContactsID + "/contactimports/"
headers = {
'content-type': "multipart/form-data; boundary=---BOUNDRY",
'x-api-token': "Token"
}
files = {'contacts': ('contacts', open('contacts.json', 'rb'), 'application/json')}
request = requests.post(url, headers=headers, files=files)
print(request.text)
Please note that if you want to use this code, you will need to change "az1" in the URL to your own Qualtrics datacenter ID.

You need to use files = .. for a multipart request:
Token = 'MyToken' #when running the code I put in my actual token and id
ContactsID = 'MyContactsID'
data = open('contacts.json', 'rb')
headers = {'X-API-TOKEN': Token}
r = requests.post('https://az1.qualtrics.com/API/v3/mailinglists/' + ContactsID +'/contactimports',files={"file":data}, headers=headers)
r.text
Once you do requests will take care of the rest:
In [36]: url = 'http://httpbin.org/post'
In [37]: headers = {'X-API-TOKEN': "123456789"}
In [38]: files = {'file': open('a.csv', 'rb')}
In [39]: r = requests.post(url, files=files, headers=headers)
In [40]: print r.text
{
"args": {},
"data": "",
"files": {
"file": "a,b,c\n1,2,3"
},
"form": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Content-Length": "152",
"Content-Type": "multipart/form-data; boundary=3830dbe5fa6141f69d3d85dee4ba6e78",
"Host": "httpbin.org",
"User-Agent": "python-requests/2.10.0",
"X-Api-Token": "123456789"
},
"json": null,
"origin": "51.171.98.185",
"url": "http://httpbin.org/post"
}
In [41]: print(r.request.body)
--3830dbe5fa6141f69d3d85dee4ba6e78
Content-Disposition: form-data; name="file"; filename="a.csv"
a,b,c
1,2,3
--3830dbe5fa6141f69d3d85dee4ba6e78--
looking at the docs, you actually want something closer to:
Token = 'MyToken' #when running the code I put in my actual token and id
ContactsID = 'MyContactsID'
data = open('contacts.json', 'rb')
files = {'file': ('contact', data ,'application/json', {'X-API-TOKEN': Token})}
r = requests.post('https://az1.qualtrics.com/API/v3/mailinglists/' + ContactsID +'/contactimports',files=files)

Python3: JSON POST Request WITHOUT requests library

I want to send JSON encoded data to a server using only native Python libraries. I love requests but I simply can't use it because I can't use it on the machine which runs the script. I need to do it without.
newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"}
params = urllib.parse.urlencode(newConditions)
params = params.encode('utf-8')
req = urllib.request.Request(conditionsSetURL, data=params)
urllib.request.urlopen(req)
My server is a local WAMP server. I always get an
urllib.error.HTTPError: HTTP Error 500: Internal Server Error
I am 100% sure that this is NOT a server issue, because the same data, with the same url, on the same machine, with the same server works with the requests library and Postman.

You are not posting JSON, you are posting a application/x-www-form-urlencoded request.
Encode to JSON and set the right headers:
import json
newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"}
params = json.dumps(newConditions).encode('utf8')
req = urllib.request.Request(conditionsSetURL, data=params,
headers={'content-type': 'application/json'})
response = urllib.request.urlopen(req)
Demo:
>>> import json
>>> import urllib.request
>>> conditionsSetURL = 'http://httpbin.org/post'
>>> newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"}
>>> params = json.dumps(newConditions).encode('utf8')
>>> req = urllib.request.Request(conditionsSetURL, data=params,
... headers={'content-type': 'application/json'})
>>> response = urllib.request.urlopen(req)
>>> print(response.read().decode('utf8'))
{
"args": {},
"data": "{\"con4\": 40, \"con2\": 20, \"con1\": 40, \"password\": \"1234\", \"con3\": 99}",
"files": {},
"form": {},
"headers": {
"Accept-Encoding": "identity",
"Connection": "close",
"Content-Length": "68",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "Python-urllib/3.4",
"X-Request-Id": "411fbb7c-1aa0-457e-95f9-1af15b77c2d8"
},
"json": {
"con1": 40,
"con2": 20,
"con3": 99,
"con4": 40,
"password": "1234"
},
"origin": "84.92.98.170",
"url": "http://httpbin.org/post"
}