I'm doing a GET request on POSTMAN. The request is just this URL https://www.google.com/search?q=pip+google-images&tbm=isch and I get a 853639 characters long response(Which is the response I want).
So I want to do the same thing with Python. I used the Postman's GENERATE CODE SNIPPETS and copied the code for Python Requests, I pasted it into my own python script and ran it.
But the response I got was only 22490 characters long(The response I don't want).
Why is it happening?
Python's code:
import requests
url = "https://www.google.com/search"
querystring = {"q":"pip google-images","tbm":"isch"}
headers = {
'cache-control': "no-cache",
'postman-token': "6b5e997f-6651-2178-1371-5d6a555984a7"
}
response = requests.request("GET", url, headers=headers, params=querystring)
print(response.text)
You need to set the User-Agent, e.g.
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'
}
Include this in your headers.
Related
I'm using python and trying to read the metadata from a token on solscan.
I am looking for the name, image, etc from metadata.
I am currently using JSON request which seems to work (ie not fail), but it only returns me:
{"holder":0}
Process finished with exit code 0
I am doing several other requests to website, so I think my request is correct.
I tried looking at the documentation on https://public-api.solscan.io/docs and I believe I am requesting the correct info, but I dont get it.
Here is my current code:
import requests
headers = {
'accept': 'application/jsonParsed',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'
}
params = (
('tokenAddress', 'EArf8AxBi44QxFVnSab9gZpXTxVGiAX2YCLokccr1UsW'),
)
response = requests.get('https://public-api.solscan.io/token/meta', headers=headers, params=params)
#response = requests.get('https://arweave.net/viPcoBnO9OjXvnzGMXGvqJ2BEgl25BMtqGaj-I1tkCM', headers=headers)
print(response.content.decode())
Any help appreciated!
This code sample works:
import requests
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'
}
params = {
'address': 'EArf8AxBi44QxFVnSab9gZpXTxVGiAX2YCLokccr1UsW',
}
response = requests.get('https://api.solscan.io/account', headers=headers, params=params)
print(response.content.decode())
I use another URL and parameters in my sample: https://api.solscan.io/account used instead of https://public-api.solscan.io/token/meta and address param instead of tokenAddress.
I had a Python script that would automatically add tickets to your basket on https://tickets.liverpoolfc.com/PagesPublic/Home/home.aspx
Now the script won't work, yet the same request works fine in Postman.
The request section works like this:
addheaders = {'origin': "https://tickets.liverpoolfc.com",
'upgrade-insecure-requests': "1",
'content-type': "application/x-www-form-urlencoded",
'user-agent': "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36",
'accept': "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"}
body={'__VIEWSTATE':'/wEPDwUJMzkxOTk4MjAyD2QWAmYPZBYCZg9kFgICAw9kFgJmD2QWCAIDDw8WAh4HVmlzaWJsZWhkZAIHDw8WAh8AaGRkAhEPDxYCHwBoZGQCIQ9kFgICAQ9kFgICCQ9kFgQCHQ9kFgICAQ9kFgRmD2QWHAIBD2QWAgIBDxYCHgRUZXh0BRhTZWxlY3QgQnkgU3RhbmQgYW5kIEFyZWFkAgMPFgIfAWVkAgUPFgIfAQUNVGlja2V0IEZpbHRlcmQCBw8WAh8AZxYKAgEPFgIfAQUUV2hhdCBpcyB5b3VyIGJ1ZGdldD9kAgMPDxYCHwEFDU1pbmltdW0gUHJpY2VkZAIEEBAPFgYeDkRhdGFWYWx1ZUZpZWxkBQVQcmljZR4NRGF0YVRleHRGaWVsZAUFUHJpY2UeC18hRGF0YUJvdW5kZxYCHghvbmNoYW5nZQXJAWRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdTaXRlTWFzdGVyQm9keV9Db250ZW50UGxhY2VIb2xkZXIxX3VzY1NlYXRTZWxlY3Rpb25fdXNjU3RhbmRBbmRBcmVhU2VsZWN0aW9uX2hkZlNlbGVjdGVkTWluaW11bVByaWNlJykudmFsdWU9dGhpcy52YWx1ZTsgUmVEcmF3U3RhZGl1bShmYWxzZSwgdHJ1ZSwgZmFsc2UsIGZhbHNlLCBmYWxzZSwgZmFsc2UpOxAVAQfCozIyLjAwFQEFMjIuMDAUKwMBZ2RkZAIGDw8WAh8BBQ1NYXhpbXVtIFByaWNlZGQCBxAQDxYGHwIFBVByaWNlHwMFBVByaWNlHwRnFgIfBQXJAWRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdTaXRlTWFzdGVyQm9keV9Db250ZW50UGxhY2VIb2xkZXIxX3VzY1NlYXRTZWxlY3Rpb25fdXNjU3RhbmRBbmRBcmVhU2VsZWN0aW9uX2hkZlNlbGVjdGVkTWF4aW11bVByaWNlJykudmFsdWU9dGhpcy52YWx1ZTsgUmVEcmF3U3RhZGl1bShmYWxzZSwgZmFsc2UsIHRydWUsIGZhbHNlLCBmYWxzZSwgZmFsc2UpOxAVAQfCozIyLjAwFQEFMjIuMDAUKwMBZ2RkZAIJDxYCHwBnFgYCAQ8WAh8BBRVQaWNrIGEgcHJpY2UgY2F0ZWdvcnlkAgMPDxYCHwEFFVBpY2sgYSBwcmljZSBjYXRlZ29yeWRkAgQQEA8WBh8CBQxQcmljZUJyZWFrSUQfAwUVUHJpY2VCcmVha0Rlc2NyaXB0aW9uHwRnFgIfBQXJAWRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdTaXRlTWFzdGVyQm9keV9Db250ZW50UGxhY2VIb2xkZXIxX3VzY1NlYXRTZWxlY3Rpb25fdXNjU3RhbmRBbmRBcmVhU2VsZWN0aW9uX2hkZlNlbGVjdGVkUHJpY2VCcmVha0lkJykudmFsdWU9dGhpcy52YWx1ZTsgUmVEcmF3U3RhZGl1bSh0cnVlLCBmYWxzZSwgZmFsc2UsIGZhbHNlLCBmYWxzZSwgZmFsc2UpOxAVAwALQUQvSlUgQVJFQVMKQURVTFQgwqMyMhUDATAFMTMyNjMFMTMyNTIUKwMDZ2dnZGRkAgsPFgIfAQUZU2VsZWN0IHRoZSBzdGFuZCBhbmQgYXJlYWQCDQ8WAh8BBRlXaGVyZSBkbyB5b3Ugd2FudCB0byBzaXQ/ZAIPDxYCHwBoFgICAhAQZGQWAGRkAhEPZBYIAgEPDxYCHwEFBVN0YW5kZGQCAhAQDxYIHgdUb29sVGlwZB8DBRBTdGFuZERlc2NyaXB0aW9uHwIFCVN0YW5kQ29kZR8EZxYCHghPbkNoYW5nZQXrA0phdmFzY3JpcHQ6IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdTaXRlTWFzdGVyQm9keV9Db250ZW50UGxhY2VIb2xkZXIxX3VzY1NlYXRTZWxlY3Rpb25fdXNjU3RhbmRBbmRBcmVhU2VsZWN0aW9uX2hkZlN0YW5kU2VsZWN0ZWQnKS52YWx1ZT10aGlzLnZhbHVlOyBMb2FkQXJlYXNGb3IxOExHQUModGhpcy5mb3JtKTtKYXZhc2NyaXB0OiBkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnU2l0ZU1hc3RlckJvZHlfQ29udGVudFBsYWNlSG9sZGVyMV91c2NTZWF0U2VsZWN0aW9uX3VzY1N0YW5kQW5kQXJlYVNlbGVjdGlvbl9oZGZBcmVhU2VsZWN0ZWQnKS52YWx1ZT1kb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnYXJlYURyb3BEb3duTGlzdCcpLnZhbHVlO3ZhbGlkYXRlQXJlYURyb3Bkb3duKCk7SmF2YXNjcmlwdDogb25RdWFudGl0eUNoYW5nZWQoKTtKYXZhc2NyaXB0OiBSZURyYXdTdGFkaXVtKGZhbHNlLCBmYWxzZSwgZmFsc2UsIHRydWUsIGZhbHNlLCBmYWxzZSk7EBUFCUFueSBTdGFuZBJBTkZJRUxEIFJPQUQgTE9XRVIOS09QIEdSQU5EU1RBTkQQTUFJTiBTVEFORCBMT1dFUhBNQUlOIFNUQU5EIFVQUEVSFQUAAkFMA0tPUANNU0wDTVNVFCsDBWdnZ2dnZGRkAgQPDxYCHwEFBEFyZWFkZAIFEBAPZBYCHwUF+AFKYXZhc2NyaXB0OiBkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnU2l0ZU1hc3RlckJvZHlfQ29udGVudFBsYWNlSG9sZGVyMV91c2NTZWF0U2VsZWN0aW9uX3VzY1N0YW5kQW5kQXJlYVNlbGVjdGlvbl9oZGZBcmVhU2VsZWN0ZWQnKS52YWx1ZT10aGlzLnZhbHVlO0phdmFzY3JpcHQ6IG9uUXVhbnRpdHlDaGFuZ2VkKCk7SmF2YXNjcmlwdDogUmVEcmF3U3RhZGl1bShmYWxzZSwgZmFsc2UsIGZhbHNlLCBmYWxzZSwgdHJ1ZSwgZmFsc2UpOxAVAQhBbnkgQXJlYRUBABQrAwFnZGRkAhMPFgIfAGhkAhUPZBYEAgEPFgIfAGhkAgIPZBYIAgEPDxYCHwEFCFF1YW50aXR5ZGQCAw8PFgIeCU1heExlbmd0aAICFggeB29ua2V5dXAFFG9uUXVhbnRpdHlDaGFuZ2VkKCk7HwUFFG9uUXVhbnRpdHlDaGFuZ2VkKCk7HgNtaW4FATAeA21heAUCOTlkAgQPDxYIHgxFcnJvck1lc3NhZ2UFOlF1YW50aXR5IG11c3QgYmUgYmV0d2VlbiAwIGFuZCA5OSBmb3IgTEZDIEZPVU5EQVRJT04gRVZFTlQeDE1heGltdW1WYWx1ZQUCOTkeBFR5cGULKixTeXN0ZW0uV2ViLlVJLldlYkNvbnRyb2xzLlZhbGlkYXRpb25EYXRhVHlwZQEeDE1pbmltdW1WYWx1ZQUBMGRkAgYPDxYCHwEFA0JVWWRkAhcPFgIfAGhkAhkPFgIfAGdkAkEPFgIfAQWLFQ0KPHNjcmlwdCBsYW5ndWFnZT0iamF2YXNjcmlwdCIgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij4NCmZ1bmN0aW9uIExvYWRBcmVhc0ZvcjE4TEdBQyh0aGlzZm9ybSl7DQp2YXIgZGRsID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ2FyZWFEcm9wRG93bkxpc3QnKTsNCnZhciBuZXdTdGFuZENvZGUgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnc3RhbmREcm9wRG93bicpLnZhbHVlOw0KaWYgKHRyaW0obmV3U3RhbmRDb2RlKSA9PSAiIikgew0KcmVtb3ZlQWxsT3B0aW9ucyhkZGwpOw0KYWRkT3B0aW9uKGRkbCwgIiIsICJBbnkgQXJlYSIpOw0KfQ0KaWYgKHRyaW0obmV3U3RhbmRDb2RlKSA9PSAiQUwiKSB7DQpyZW1vdmVBbGxPcHRpb25zKGRkbCk7DQphZGRPcHRpb24oZGRsLCAiIiwgIkFueSBBcmVhIik7DQphZGRPcHRpb24oZGRsLCAiMTIzIiwgIkJMT0NLIDEyMyIpOw0KYWRkT3B0aW9uKGRkbCwgIjEyNiIsICJCTE9DSyAxMjYiKTsNCmFkZE9wdGlvbihkZGwsICIxMjciLCAiQkxPQ0sgMTI3Iik7DQphZGRPcHRpb24oZGRsLCAiMTI4IiwgIkJMT0NLIDEyOCIpOw0KYWRkT3B0aW9uKGRkbCwgIjEyOSIsICJCTE9DSyAxMjkiKTsNCn0NCmlmICh0cmltKG5ld1N0YW5kQ29kZSkgPT0gIktPUCIpIHsNCnJlbW92ZUFsbE9wdGlvbnMoZGRsKTsNCmFkZE9wdGlvbihkZGwsICIiLCAiQW55IEFyZWEiKTsNCmFkZE9wdGlvbihkZGwsICIxMDgiLCAiQkxPQ0sgMTA4Iik7DQphZGRPcHRpb24oZGRsLCAiMTA5IiwgIkJMT0NLIDEwOSIpOw0KYWRkT3B0aW9uKGRkbCwgIjIwMiIsICJCTE9DSyAyMDIiKTsNCmFkZE9wdGlvbihkZGwsICIyMDciLCAiQkxPQ0sgMjA3Iik7DQphZGRPcHRpb24oZGRsLCAiMjA4IiwgIkJMT0NLIDIwOCIpOw0KYWRkT3B0aW9uKGRkbCwgIjMwNCIsICJCTE9DSyAzMDQiKTsNCmFkZE9wdGlvbihkZGwsICIzMDUiLCAiQkxPQ0sgMzA1Iik7DQphZGRPcHRpb24oZGRsLCAiMzA2IiwgIkJMT0NLIDMwNiIpOw0KfQ0KaWYgKHRyaW0obmV3U3RhbmRDb2RlKSA9PSAiTVNMIikgew0KcmVtb3ZlQWxsT3B0aW9ucyhkZGwpOw0KYWRkT3B0aW9uKGRkbCwgIiIsICJBbnkgQXJlYSIpOw0KYWRkT3B0aW9uKGRkbCwgIkwxIiwgIkJMT0NLIEwxIik7DQphZGRPcHRpb24oZGRsLCAiTDEwIiwgIkJMT0NLIEwxMCIpOw0KfQ0KaWYgKHRyaW0obmV3U3RhbmRDb2RlKSA9PSAiTVNVIikgew0KcmVtb3ZlQWxsT3B0aW9ucyhkZGwpOw0KYWRkT3B0aW9uKGRkbCwgIiIsICJBbnkgQXJlYSIpOw0KYWRkT3B0aW9uKGRkbCwgIlUxIiwgIkJMT0NLIFUxIik7DQphZGRPcHRpb24oZGRsLCAiVTIiLCAiQkxPQ0sgVTIiKTsNCmFkZE9wdGlvbihkZGwsICJVMyIsICJCTE9DSyBVMyIpOw0KYWRkT3B0aW9uKGRkbCwgIlU0IiwgIkJMT0NLIFU0Iik7DQphZGRPcHRpb24oZGRsLCAiVTUiLCAiQkxPQ0sgVTUiKTsNCmFkZE9wdGlvbihkZGwsICJVNiIsICJCTE9DSyBVNiIpOw0KYWRkT3B0aW9uKGRkbCwgIlU3IiwgIkJMT0NLIFU3Iik7DQphZGRPcHRpb24oZGRsLCAiVTgiLCAiQkxPQ0sgVTgiKTsNCmFkZE9wdGlvbihkZGwsICJVOSIsICJCTE9DSyBVOSIpOw0KfQ0KfQ0KZnVuY3Rpb24gTG9hZERlZmF1bHRzRm9yMThMR0FDKHN0YW5kRERMLCBhcmVhRERMKSB7DQp2YXIgdGhlU3RhbmRDb2RlID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ3N0YW5kRHJvcERvd24nKS52YWx1ZTsNCnZhciB0aGVBcmVhQ29kZSA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdhcmVhRHJvcERvd25MaXN0JykudmFsdWU7DQpkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnU2l0ZU1hc3RlckJvZHlfQ29udGVudFBsYWNlSG9sZGVyMV91c2NTZWF0U2VsZWN0aW9uX3VzY1N0YW5kQW5kQXJlYVNlbGVjdGlvbl9oZGZTdGFuZFNlbGVjdGVkJykudmFsdWU9dGhlU3RhbmRDb2RlOw0KZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ1NpdGVNYXN0ZXJCb2R5X0NvbnRlbnRQbGFjZUhvbGRlcjFfdXNjU2VhdFNlbGVjdGlvbl91c2NTdGFuZEFuZEFyZWFTZWxlY3Rpb25faGRmQXJlYVNlbGVjdGVkJykudmFsdWU9dGhlQXJlYUNvZGU7DQpyZW1vdmVBbGxPcHRpb25zKGFyZWFEREwpOw0KYWRkT3B0aW9uKGFyZWFEREwsICIiLCAiIik7DQpzdGFuZERETC5zZWxlY3RlZEluZGV4PTA7DQp9DQpmdW5jdGlvbiB0cmltKHMpIHsgDQp2YXIgcj0vXGIoLiopXGIvLmV4ZWMocyk7IA0KcmV0dXJuIChyPT1udWxsKT8iIjpyWzFdOyANCn0NCmZ1bmN0aW9uIHZhbGlkYXRlQXJlYURyb3Bkb3duKCkgew0KLy8gaWYgdGhlcmUgYXJlIDIgb3B0aW9ucyBvbmx5IC0gaW4gYXJlYSBkcm9wZG93biB0aGVuIGJ5IGRlZmF1bHQgc2VsZWN0IHRoZSBzZWNvbmQgdmFsdWUgaW5zdGVhZCBvZiAiQW55IGFyZWEiDQppZihkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnYXJlYURyb3BEb3duTGlzdCcpLm9wdGlvbnMubGVuZ3RoID09IDIpIHsgDQpkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnYXJlYURyb3BEb3duTGlzdCcpLnNlbGVjdGVkSW5kZXggPSAxOw0KZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ1NpdGVNYXN0ZXJCb2R5X0NvbnRlbnRQbGFjZUhvbGRlcjFfdXNjU2VhdFNlbGVjdGlvbl91c2NTdGFuZEFuZEFyZWFTZWxlY3Rpb25faGRmQXJlYVNlbGVjdGVkJykudmFsdWUgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZCgnYXJlYURyb3BEb3duTGlzdCcpLm9wdGlvbnNbMV0udmFsdWUgOw0KfQ0KfQ0KPC9zY3JpcHQ+ZAIBD2QWAgIBDxYCHwFlZAIgD2QWAgIEEBAPFgYfAgULUHJvZHVjdENvZGUfAwUSUHJvZHVjdERlc2NyaXB0aW9uHwRnZBAVAhNMSVZFUlBPT0wgViBXQVRGT1JEFExGQyBGT1VOREFUSU9OIEVWRU5UFQIGMThMSFdUBjE4TEdBQxQrAwJnZxYBAgFkZBgBBS5jdGwwMCRjdGwwMCR1c2NQZXJzb25hbGlzYXRpb25CYXJUb3AkTG9naW5WaWV3Dw9kAgFkHnzO56IhX/YZIlxUnOQWGCL0xwM=',
'ctl00$ctl00$SiteMasterBody$ContentPlaceHolder1$uscSeatSelection$btnBuy':'Buy',
'ctl00$ctl00$SiteMasterBody$ContentPlaceHolder1$uscSeatSelection$ddlGameSelection':'18LGAC',
'ctl00$ctl00$SiteMasterBody$ContentPlaceHolder1$uscSeatSelection$hdfSelectedSeats':'MSU-U9/82/0251#A',
'ctl00$ctl00$SiteMasterBody$ContentPlaceHolder1$uscSeatSelection$uscStandAndAreaSelection$hdfProductCode':'18LGAC',
'ctl00$ctl00$SiteMasterBody$ContentPlaceHolder1$uscSeatSelection$uscStandAndAreaSelection$hdfProductPriceBand':'A'}
body = urllib.urlencode(body)
addreq = requests.post('https://tickets.liverpoolfc.com/PagesPublic/ProductBrowse/VisualSeatSelection.aspx?campaign=&product=18LGAC&productIsHomeAsAway=N&productsubtype=HOME&stadium=AS&type=H', headers=addheaders, data=body, verify=True)
I recieve a 200 status code, but the ticket is not added to the basket, however if I send this request through Postman then the ticket is added to the basket.
I have a simple Get request I'd like to make using Python's Request library.
import requests
HEADERS = {'user-agent': ('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)'
'AppleWebKit/537.36 (KHTML, like Gecko)'
'Chrome/45.0.2454.101 Safari/537.36'),
'referer': 'http://stats.nba.com/scores/'}
url = 'http://stats.nba.com/stats/playbyplayv2?EndPeriod=10&EndRange=55800&GameID=0021500281&RangeType=2&Season=2016-17&SeasonType=Regular+Season&StartPeriod=1&StartRange=0'
response = requests.get(url, timeout=5, headers=HEADERS)
However, when I make the requests.get call, I get the error requests.exceptions.ReadTimeout: HTTPConnectionPool(host='stats.nba.com', port=80): Read timed out. (read timeout=5). But I am able to copy/paste that url into my browser and view the resulting JSON. Why is requests not able to get the result?
Your HEADERS format is wrong. I tried with this code and it worked without any issues:
import requests
HEADERS = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36',
}
url = 'http://stats.nba.com/stats/playbyplayv2?EndPeriod=10&EndRange=55800&GameID=0021500281&RangeType=2&Season=2016-17&SeasonType=Regular+Season&StartPeriod=1&StartRange=0'
response = requests.get(url, timeout=5, headers=HEADERS)
print(response.text)
I have a REST url like:
url='http://xx.xx.xx.xx/server/rest/line/125'
I can get correct return json by command curl, like this:
curl -i http://xx.xx.xx.xx/server/rest/line/125
But when i use Python3 requests, it allways return 404, Python code:
import requests
resp = requests.get(r'http://xx.xx.xx.xx/server/rest/line/125')
print(resp.status_code)
Anyone can tell me the problem? Is the server block the request?
A 404 is displayed when:
The URL is incorrect and the response is actually accurate.
Trailing spaces in the URL
The website may not like HTTP(S) requests coming from Python code. Change your headers by adding "www." to your Referer url.
resp = requests.get(r'http://www.xx.xx.xx.xx/server/rest/line/125')
or
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'
}
result = requests.get('https://www.transfermarkt.co.uk', headers=headers)
In your case, try option #3.
i would like to take the response data about a specific website.
I have this site:
https://enjoy.eni.com/it/milano/map/
and if i open the browser debuger console i can see a posr request that give a json response:
how in python i can take this response by scraping the website?
Thanks
Apparently the webservice has a PHPSESSID validation so we need to get it first using proper user agent:
import requests
import json
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'
}
r = requests.get('https://enjoy.eni.com/it/milano/map/', headers=headers)
session_id = r.cookies['PHPSESSID']
headers['Cookie'] = 'PHPSESSID={};'.format(session_id)
res = requests.post('https://enjoy.eni.com/ajax/retrieve_vehicles', headers=headers, allow_redirects=False)
json_obj = json.loads(res.content)