Develop Reference

Connect MongoDB to Python

I try to connect to MongoDB database from https://cloud.mongodb.com/
My code:
import pymongo
from pymongo.server_api import ServerApi
client = pymongo.MongoClient("mongodb+srv://<My-Username:<My-Password#test.umh3xqu.mongodb.net/?retryWrites=true&w=majority", server_api=ServerApi('1'))
print(client.list_database_names())`
Errors:
pymongo.errors.ServerSelectionTimeoutError: ac-gyw4foz-shard-00-01.umh3xqu.mongodb.net:27017: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992),ac-gyw4foz-shard-00-00.umh3xqu.mongodb.net:27017: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992),ac-gyw4foz-shard-00-02.umh3xqu.mongodb.net:27017: [SSL: CERTIFICATE_VERIFY_FAILED]
How to fix???
List all databases

Microsoft-Graph: Failing to get token from python code: Error SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]

I need to call a web API. For that I need a bearer token.
I am using databricks(python) code to first get authenticated over Microsoft AAD. Then get bearer token for my service_user. I Followed the microsoft docs docs
But facing problem where it hits our Company server and asking for SSL certificate.
I can't install any certificate. What could be a better way to avoid it. Below is my short code taken from above microsoft and Git repos. but its not working.
Can i get help!
clientId = "42xx-xx-xx5f"
authority = "https://login.microsoftonline.com/tenant_id/"
app = msal.PublicClientApplication(client_id=clientId, authority=authority)
user = "serviceuser#company.com"
pwd = "password"
scope = "Directory.Read.All"
result = app.acquire_token_by_username_password(scopes=[scope], username=user, password=pwd)
print(result)
Got below error
HTTPSConnectionPool(host='mycompany.com', port=443): Max retries exceeded with url: /adfs/services/trust/mex (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)')))

The problem is that the code uses the requests library that relies on the certifi package instead of using Linux certificate chain (so existing instructions doesn't work). To solve that problem it's better to use cluster init script that will install SSL certificate when cluster starts. Something like this (requests and certifi are installed by default), just replace CERT_FILE with actual path to the .pem file with CA certificate:
CERT_FILE="/dbfs/....."
CERTIFI_HOME="$(python -m certifi 2>/dev/null)"
cat $CERT_FILE >> $CERTIFI_HOME

SSL: CERTIFICATE_VERIFY_FAILED certificate verify failed - Google Colab

I am trying to scrape data from a url using beautifulsoup. Below is my code
import requests
URL = "https://bigdataldn.com/speakers/"
page = requests.get(URL)
print(page.text)
However I am getting the following error when I run the code in google colab.
SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)
During handling of the above exception, another exception occurred:
MaxRetryError Traceback (most recent call last)
MaxRetryError: HTTPSConnectionPool(host='bigdataldn.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))
The above code works fine for other urls.
Can someone help me figure out how to solve this issue.

It's not your fault - their certificate chain is not properly configured. What you can do is disabling the certificate verification (you should not do this when you're handling sensitive information!) but it might be fine for a webscraper.
page = requests.get(URL, verify=False)

enter image description here
Your SSL certificate is not installed properly , you can follow godaddy ssl install instruction maybe its helpfull .
https://in.godaddy.com/help/install-my-ssl-certificate-16623?sp_hp=B&xpmst=A&xpcarveout=B

Validating an external certificate not in certifi package

I have a Python script that requests an https URL using the requests package. In so doing, I get a certificate error:
import requests
resp = requests.get('https://comicskingdom.com/', verify=True)
The error I see is:
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
My system has the certifi package installed, but apparently the target server's certificate cannot be validated using that package's bundle. How can I verify this certificate properly? Where do I look to download the appropriate certificate chain? In the future, how do I know where to find the right certificate chain for any given certificate?

Solution:
requests documentation: https://requests.readthedocs.io/en/master/user/advanced/
(check args and kwargs possibilities (cert=...) in chapter SSL Cert Verification)
but to quickly resolve your issue:
(Firefox) go to your site. Click on the https icon left to the browser url (usually the icon looks like a lock),click on an arrow next to 'connection secure', click more info, click View certificates and scroll down to download Chain certificate. (You can even try here on stackoverflow site)
Then, in your requests.get, add path to the chain file
>>> requests.get('https://comicskingdom.com', verify='{path}/comicskingdom-com-chain.pem')
<Response [200]>
The certificate has some issue, so I will post here what I was able to find
What is problem?
What exactly is a problem can be found through this link or through finding the error
Source: https://security.stackexchange.com/questions/16085/how-to-get-public-key-of-a-secure-webpage
for you to examine the problem, run this command
This command will show you the certificate is ok, but there is issue
openssl s_client -connect comicskingdom.com:443 | openssl x509 -pubkey -noout
which outputs
openssl s_client -connect comicskingdom.com:443 | openssl x509 -pubkey -noout
depth=0 OU = Domain Control Validated, CN = *.comicskingdom.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 OU = Domain Control Validated, CN = *.comicskingdom.com
verify error:num=21:unable to verify the first certificate
verify return:1
Note this part
verify error:num=20:unable to get local issuer certificate
which matches requests error that I received with requests
requests.exceptions.SSLError: HTTPSConnectionPool(host='comicskingdom.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))

SSL: CERTIFICATE_VERIFY_FAILED certificate verify failed: Hostname mismatch, certificate is not valid for 'url''

I am trying to access Solr using urllib2 as instaructed here: https://lucene.apache.org/solr/guide/7_3/using-python.html using urllib
but running into error raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'the_dns-for-lb'
The cert exist in AWS ACM and there is custom auth, can someone guide me how to establish connection?
from urllib.request import urlopen
connection = urlopen('https://dns-for-lb/solr/design').read()
response = eval(connection.read())
tried:
Using urllib gives SSL error
Using urllib gives SSL error
pip install fails with "connection error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)"