I have cert information for using HTTPS in my repos stored in /etc/yum.conf at the bottom:
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
...
sslclientcert=/path/to/cert.pem
sslclientkey=/path/to/privatekey.pem
sslcacert=/path/to/ca.pem
When I use dnf via the terminal, it can communicate with the repos and retrieve repodata/repomd.xml (and package information and all) just fine. However, when I do it via python:
import dnf
with dnf.Base() as base:
base.read_all_repos()
base.fill_sack()
I get:
Errors during downloading metadata for repository '<reponame>':
- Curl error (60): Peer certificate cannot be authenticated with given CA certificates for <repo-path>/repodata/repomd.xml [SSL certificate problem: self signed certificate in certificate chain]
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/dnf/repo.py", line 574, in load
ret = self._repo.load()
File "/usr/lib64/python3.6/site-packages/libdnf/repo.py", line 397, in load
return _repo.Repo_load(self)
libdnf._error.Error: Failed to download metadata for repo '<reponame>': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/dnf/base.py", line 399, in fill_sack
self._add_repo_to_sack(r)
File "/usr/lib/python3.6/site-packages/dnf/base.py", line 139, in _add_repo_to_sack
repo.load()
File "/usr/lib/python3.6/site-packages/dnf/repo.py", line 581, in load
raise dnf.exceptions.RepoError(str(e))
dnf.exceptions.RepoError: Failed to download metadata for repo '<reponame>': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
How I know it's an SSL/HTTPS problem:
Implemented HTTPS on my personal repo a few days ago, and that's when the issues started (it worked just fine prior to that)
If I change the URL to http:// instead of https://, it works fine (but this isn't a viable solution).
If I run the dnf via commandline, it also works just fine.
I know that dnf has the ability to pull those certs in from the yum.conf file, but does anyone know (or can figure out) how it's actually done? I've spent a good while digging through the code now and can't figure it out (I'm not particularly familiar with how swig works, which is what I'm getting caught up on witin the dnf code itself).
Any help is appreciated.
Figured it out a little while ago, so maybe this'll help someone else out there:)
import dnf
from libdnf.conf import Option
with dnf.Base() as base:
# Read in the config from yum.conf
with open('/etc/yum.conf') as inf:
yum_config = inf.read()
# Split the config into a dictionary
yum_settings = yum_config.split('\n')
yum_settings = {k.split('=')[0]:k.split('=')[1] for k in yum_settings in '=' in k}
# The settings below will require a priority. This is the third highest one out of about 10, so I went with this one
priority = Option.Priority_DROPINCONFIG
base.conf._config.sslclientcert().set(priority, yum_settings['sslclientcert'])
base.conf._config.sslclientkey().set(priority, yum_settings['sslclientkey'])
base.conf._config.sslcacert().set(priority, yum_settings['sslcacert'])
base.read_all_repos()
base.fill_sack()
And now it works! It uses the SSL certs in the files in /etc/yum.conf to connect to the repos and all is well!
Related
Python Version = 3.9,
Foreman Version = 2.5.4
I'm using the Python Foreman (python-foreman) module to connect to Foreman to do a bunch of automation tasks, however as soon as i'm connecting to Foreman as below:
from foreman.client import Foreman
f = Foreman('http://foreman-url.com/api/hosts', ('admin', 'password'), api_version = 2)
I'm getting a bunch of conflicting statements as below and eventually erroring out.
There is a conflict trying to redefine a method for a foreign resource (foreman_tasks_index):
resource:
apipie_resource: foreman_tasks
new_api: <resource:, name:foreman_tasks_index>
new_url: /foreman_tasks/api/tasks/:parent_task_id/sub_tasks
old_api: <resource:, name:foreman_tasks_index>
old_url: /foreman_tasks/api/tasks
There is a conflict trying to redefine a method for a foreign resource (sync_plans_rganizations):
resource:
apipie_resource: sync_plans
new_api: <resource:organizations, name:sync_plans_rganizations>
new_url: /katello/api/organizations/:organization_id/sync_plans/:id/add_products
old_api: <resource:organizations, name:sync_plans_rganizations>
old_url: /katello/api/organizations/:organization_id/sync_plans/:id
Eventually the script is erroring out at this:
Traceback (most recent call last):
File "", line 1, in
File "/data/sfreport/python3.9/lib/python3.9/site-packages/foreman/client.py", line 619, in init
self._generate_api_defs(use_cache, strict_cache)
File "/data/sfreport/python3.9/lib/python3.9/site-packages/foreman/client.py", line 820, in generate_api_defs
new_resource, extra_foreign_methods = parse_resource_definition(
File "/data/sfreport/python3.9/lib/python3.9/site-packages/foreman/client.py", line 375, in parse_resource_definition
functions[api.name] = api.generate_func()
File "/data/sfreport/python3.9/lib/python3.9/site-packages/foreman/client.py", line 282, in generate_func
six.exec(code)
File "", line 1
def lifecycle_environments_update(self, id, organization_id=None, new_name=None, description=None, registry_name_pattern=None, registry_unauthenticated_pull=None, async=None):
^
SyntaxError: invalid syntax
Can you pls help with this or let me know the best way to connect to Foreman API via Python to carry out the automations?
The error message you are getting states that (sine Python 3.5) async is a reserved keyword, and cannot be a variable name. This means that the version of Foreman you are using is incompatible with Python 3.9. Try updating Foreman, downgrading Python, or (if you are comfortable doing it) try to fix the issue yourself (changing the variable name from async to something else, and hoping it works).
I am not sure whether the offending code comes directly from Foreman. Try to search your code (and libraries) for async=None pattern, as it may also not be Foreman's fault.
I'm trying to run a test in a docker container, which runs locally with no issues:
I want to upload a correct.csv file from 'correct' directory
*** Keyword ***
Upload file
[Arguments] ${directory} ${file}
Choose File ${choose_file_input} ${EXECDIR}/Files/${directory}/${file}
** Test case ***
Upload
Upload file correct correct.csv
But when running test in docker I get a FAIL with the AttributeError: module 'base64' has no attribute 'encodestring'. Is it because there is no GUI in docker? or the encoding needs to be fixed? Or eventually maybe there is another solution I can use for uploading files?
15:20:01.250 INFO Sending /App/Files/correct/correct.csv to browser.
15:20:01.251 DEBUG POST http://192.168.1.29:4444/wd/hub/session/4b6d453b394adaaa51bb4149e9ba8678/elements {"using": "xpath", "value": "//div[#id=\"upload\"]//input"}
15:20:01.252 DEBUG Starting new HTTP connection (1): 192.168.1.29:4444
15:20:01.305 DEBUG http://192.168.1.29:4444 "POST /wd/hub/session/4b6d453b394adaaa51bb4149e9ba8678/elements HTTP/1.1" 200 90
15:20:01.305 DEBUG Finished Request
15:20:01.618 FAIL AttributeError: module 'base64' has no attribute 'encodestring'
15:20:01.619 DEBUG Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/SeleniumLibrary/__init__.py", line 490, in run_keyword
return DynamicCore.run_keyword(self, name, args, kwargs)
File "/usr/local/lib/python3.9/site-packages/robotlibcore.py", line 103, in run_keyword
return self.keywords[name](*args, **(kwargs or {}))
File "/usr/local/lib/python3.9/site-packages/SeleniumLibrary/keywords/formelement.py", line 224, in choose_file
self.find_element(locator).send_keys(file_path)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 475, in send_keys
value = self._upload(local_file)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 695, in _upload
content = base64.encodestring(fp.getvalue())
Based on the traceback you have found this issue:
Selenium 3 is incompatible with Python 3.9
This is the fix for the issue: DeprecationWarning of base64.encodestring().
They won't back port this fix:
Thanks for the issue. We won't be releasing another version 3 as we're
heading to finishing off Selenium 4. It is a drop in replacement to
use Selenium 4.0.0.a5 so should work the same. There should not be any
breaking changes.
So you could upgrade selenium to Selenium 4.0.0.a5 or
Downgrade Python to 3.7 for example. I suppose locally you do not run 3.9.
We were running into this issue as well, but going back to an older version of Python was not an option due to incompatibilities with other libraries. If you find yourself in the same spot, you can re-create the alias like so:
import base64
base64.encodestring = base64.encodebytes
In whatever your entry-point is.
The whois.whois function always gets a timed out error.
At first, I thought it was because my project is written in Python 2.7 but I also checked in 3.7 and got the same error.
I checked the address on the online website that uses whois and the link worked and didn't get this error.
Anyone knows why this is happening?
import whois
w = whois.whois("https://stackoverflow.com")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files\Python37\lib\site-packages\whois\__init__.py", line 43, in whois
text = nic_client.whois_lookup(None, domain.encode('idna'), flags)
File "C:\Program Files\Python37\lib\site-packages\whois\whois.py", line 264, in whois_lookup
result = self.whois(query_arg, nichost, flags)
File "C:\Program Files\Python37\lib\site-packages\whois\whois.py", line 142, in whois
s.connect((hostname, 43))
socket.timeout: timed out
Your code has at least two problems, and you may have a network problem also.
However, there is no reason for it not to work on Python2.
About the code
This works perfectly fine:
In [7]: import whois
In [8]: print whois.query('stackoverflow.com').expiration_date
2020-02-02 11:59:59
Note two things:
whois is about domain names, not URLs; so you should pass a domain name; note more generally that for new endeavors you should have a look at RDAP instead of whois since you will get a far better experience
you need to use whois.query not whois.whois (you are not saying which version of the library you use, but at its documentation page on https://pypi.org/project/whois/ you can clearly see it is whois.query so I do not know where your whois.whois` comes from).
About the network
You show a network error. It is not 100% clear but you may or may not have access to the whois servers you want to query.
Easy way to test: just use the command line whois from the same box as your code (but again use a domain name, not a URL as parameter) and you will see what is happening.
You can even do directly a telnet on port 43 as whois does nothing else.
$ echo 'stackoverflow.com' | nc whois.verisign-grs.com 43 | grep 'Expiry'
Registry Expiry Date: 2020-02-02T11:59:59Z
I'm trying use del.icio.us API and following the examples from the book Programming Collective Intelligence
When I use these commands in python 3.6.2:
>>> from deliciousrec import *
>>> delusers=initializeUserDict('programming')
I get this error:
<urlopen error [Errno 11001] getaddrinfo failed>, 4 tries left.
<urlopen error [Errno 11001] getaddrinfo failed>, 3 tries left.
<urlopen error [Errno 11001] getaddrinfo failed>, 2 tries left.
<urlopen error [Errno 11001] getaddrinfo failed>, 1 tries left.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File"C:\Users\user\AppData\Local\Programs\Python\Python36\deliciousrec.py", line 10, in initializeUserDict
for p1 in get_popular(tag=tag)[0:count]:
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\pydelicious-0.6-py3.6.egg\pydelicious\__init__.py", line 1042, in get_popular
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\pydelicious-0.6-py3.6.egg\pydelicious\__init__.py", line 1026, in getrss
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\pydelicious-0.6-py3.6.egg\pydelicious\__init__.py", line 455, in dlcs_rss_request
File "C:\Users\user\AppData\Local\Programs\Python\Python36\lib\site-packages\pydelicious-0.6-py3.6.egg\pydelicious\__init__.py", line 239, in http_request
UnboundLocalError: local variable 'e' referenced before assignment
I can not open pydelicious-0.6-py3.6.egg and access the init file that is asked to be modified here.
Has anyone seen this type of error before. How do I solve it?
This library's PyPI page is written in the past tense:
Was a complete Python interface to del.icio.us Bookmarks’ HTTP API’s.
Its home page is a Github repo that says:
Old pydelicious work (--2010)
The only update since 2010 is:
[2016] del.icio.us was gone since about 2009, and redirected to delicious.com. At the moment there is another login page back at del.icio.us. Not sure what they're up to now."
Even the old 2010 page said:
IMPORTANT: pydelicious has not been updated to use the OAuth protocol. New users with a Yahoo account/email will not be able to use this library.
So, it seems very likely that this library was never updated for Python 3.x, or for up-to-date versions of whichever libraries it depends on, or for whatever the new del.icio.us is.
But meanwhile, if you want to debug it yourself, you can.
To look at source inside a .egg archive, you can either do it from within Python:
>>> import deliciousrec
>>> import inspect
>>> inspect.getsource(deliciousrec)
… or you can extract the .egg archive using your favorite ZIP-file tool, because that's all .egg files are.
… or you can just clone the GitHub repo, or view it online, which is probably the best solution.
As Hamms pointed out in a comment, if you look at the function that's raising, it's doing this:
except urllib2.HTTPError, e:
# reraise unexpected protocol errors as PyDeliciousException
raise PyDeliciousException, "%s" % e
In Python 2, that meant to handle exceptions of type urllib2.HTTPError, and bind e to the exception for the rest of the function.
In Python 2.6, this was soft-deprecated in favor of this new syntax:
except urllib2.HTTPError as e:
# reraise unexpected protocol errors as PyDeliciousException
raise PyDeliciousException, "%s" % e
In Python 3, the old syntax went away entirely, so it's now illegal.
The 2to3 tool will automatically fix this particular issue for you, but it may not fix everything; you'll need to manually port the code to 3.x. And of course that may not do any good, since you're trying to call an API that went away in 2010.
During my pandas/Google Analytics API setup, I basically did everything as described in this link:
http://blog.yhathq.com/posts/pandas-google-analytics.html
The client_secrets.json is in the pandas/io folder. When i now try to execute a statement of the form
>>>from pandas.io import ga
>>>df = ga.read_ga(metrics, dimensions, start_date)
the following error occurs:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "\Anaconda\lib\site-packages\pandas\io\ga.py", line 110, in read_ga
reader = GAnalytics(**reader_kwds)
File "\Anaconda\lib\site-packages\pandas\io\ga.py", line 179, in __init__
self._service = self._init_service(secrets)
File "\Anaconda\lib\site-packages\pandas\io\ga.py", line 191, in _init_service
http = self.authenticate(secrets)
File "\Anaconda\lib\site-packages\pandas\io\ga.py", line 151, in authenticate
return auth.authenticate(flow, self.token_store)
File "\Anaconda\lib\site-packages\pandas\io\auth.py", line 108, in authenticate
credentials = tools.run(flow, storage)
AttributeError: 'module' object has no attribute 'run'
According to the yhat link, my browser should open for authentication.
Note: I did not not create the Client ID for "installed application", since I did not have this choice in the menu when creating the ID. Instead, i chose "other". This shouldn't be the cause of the error, though.
Second Note: I recently updated my pandas to 0.17.1. When importing pandas.io.ga, i got the message that the .ga module is deprecated. Furthermore, i manually installed the gflags module, because it was needed when I tried to import .io.ga the first time.
Either file a ticket with the owners of Pandas to change (currently) line 108 of pandas/io/auth.py from run() to run_flow(), or make the fix yourself and file a PR. (Yes, it would've been nice if Google had just made run_flow() and alias of run(), but as you can imagine, this is not how this change evolved, so we have to live with it.)
For other developers running into this error: If you have the latest version (as of Feb 2016) of the Google APIs Client Library for Python, just rename your call from tools.run() to tools.run_flow(), and you should be good-to-go. More about this change in a PSA (public service announcement) blogpost I wrote back in mid-2015 but update periodically to stay current.
The fastest way to upgrade your Client Library is with:
pip install -U google-api-python-client # or pip3 for 3.x