Develop Reference

Get in python what web server is used by a website

How can I know if a website is using apache, nginx or other and get this information in python? Thanks in advance

This information if available is given in the header of the response to a HTTP Request. With Python you can perform HTTP requests using the module requests.
Make a simple GET request to the interested site and then print the headers parameter of the returned object.
import requests
r = requests.get(YOUR_SITE)
print(r.headers)
The output is made of a dictionary of keys and value, you have to look for the Server parameter
server = r.headers['Server']
You must be aware that not all websites return this information for several reasons, so you could not find this key in the response header.

python requests.post is interpreted as GET

I am interfacing with an external API using python's request module. When I submit the request using requests.post() the API returns with
{"Message":"The requested resource does not support http method 'GET'."}
When I try the exact same request with my browser's RESTED client (also using POST method), it works just fine.
Not really much in the way of sample code to give, it's just a simple
r = requests.post(url, data=data, headers=headers)
It would seem the fault is either on the API's side and/or python for not submitting an entirely standard POST request (since it works with RESTED). Has anyone ever run into this scenario before? Is there a python alternative to requests.post()?
I'm on Python 3.9.1 on Windows.

Freedom of information act API. API key error

I am having some trouble running the Freedom of information act API in python. I am sure it is related to how I am implementing my API key but I am uncertain as to where I am dropping the ball. Any help is greatly appreciated.
import requests
apikey= ''
api_base_url = f"https://api.foia.gov/api/webform/submit"
endpoint = f"{api_base_url}{apikey}"
r = requests.get(endpoint)
print(r.status_code)
print(r.text)
there error I receive is requests.exceptions.InvalidSchema: No connection adapters were found for this website. Thanks again

According to the documentation, the API requires the API key to be passed as a request header parameter ("X-API-Key"). Your python code appears to be simply concatenating the API key and the URL.
The following Q&A explains how to set a request header using requests.
Using headers with the Python requests library's get method
It would be something like this:
import requests
apikey= ...
api_base_url = ...
r = requests.get(api_base_url,
headers={"X-API-Key": apikey})
print(r.status_code)
print(r.text)
Note that the documentation for the FOIA site explains what you need to do to submit a FIOA request form. It is significantly different to what your Python code is apparently trying to do. I would advise you to read the documentation. Also read the manual entry for the "curl" command so that you understand the requests that the examples show.

Rest API programming: Requests vs urllib2 in Python 2.7 -- the API needs authentication (Error 401)

I am a beginner trying to learn REST API programming through Python 2.7 to get data from Socialcast API. From my research it looks like requests or urllib2 would work. I need to authenticate with username and id for the API. I tried using urllib2 and it gave me error 401.
Which one should I use? My goal is to produce .csv files from the data so I can visualize it. Thank you in advance.

The question will yield a bit of an opinion based response, but I would suggest using Requests. I find that when making request that require parameters using Requests is easier to manage. An example for the Socialcast using Requests would be
parameters={"email" : emailAddress, "passoword" : password}
r = requests.post(postUrl, parameters)
The post url would be the url to make the post request and emailAddress and password would be the vales you use to login in.
For the csv, take a look here which includes a tutorial on going from json to csv.

Python requests call fails with HTTPS

I am running a Flask restful API behind an NGINX web server on AWS. I am hitting that with a python module from my Pi.
Everything worked fine when I was using HTTP to make calls to the api. But I just locked down my api so only HTTPS is possible. I changed the UIRL used by my python module but it now fails. The code is quite simple...here is an extract:
jsonpkg = {'subscriberID': self.api_login, 'token': self.api_token,
'content': speech_content}
headers = {'Content-Type': 'application/json'}
r = requests.post(self.api_apiurl, data=json.dumps(jsonpkg), headers=headers)
The values are being correct set by the class init section. And I am importing the requests module at the top. Error messages indicate it is using python 2.7. However when I monitor the API I can see its not even hitting the server. I can point a browser to the api and its working fine.
Am I to understand the requests module in python 2.7 does not support https?
Are there additional parameters I need to send for https?

Aha! With a little more digging into the request module docs I found the answer. If I use the following
r = requests.post(self.api_apiurl, data=json.dumps(jsonpkg), headers=headers, verify=False)
then it works. So the issue is with verifying the cert. I am not quite sure why the browser gets by without this...but perhaps it does the extra stuff automatically. So I either need to NOT verify the cert or have a local copy(?) that can be verified.
Final Update:
I finally worked out how to concatenate my site certificate with the chain certificate (and understand why). This site here was a great help. Also, once they are concatenated you will probably get a second error, which if you google it you will find is caused by the need for a carriage return after the first certificate and before the second (edit the resulting concatenated file with notepad). I then was able to return the post to using "verify=True" which made the warnings about no verification go away.