I had to use Kerberos authentication for the first time, it kinda works, but I feel like missing a lot of understanding what is going on and how to cook it properly.
Basically, what I need is my Python script to run every couple of hours and send a request to a remote web server using domain account in MS AD environment.
The following code provides me with the ready to go session instance:
from requests import Session
import gssapi
from requests_gssapi import HTTPSPNEGOAuth
session = Session()
session.auth = HTTPSPNEGOAuth(mech=gssapi.mechs.Mechanism.from_sasl_name("SPNEGO"))
The script was added to the crontab of a user in a linux box and kinit was used to obtain a ticket-granting ticket:
kinit -kt ~/ad_user.keytab ad_user#DOMAIN.COM
But after a while it all stopped because of the expired ticket. The solution was simple: adding the kinit to the crontab to run every 8 hours solved the issue.
I'm wondering if there is a better and more proper way to achieve the same? If I don't want/need to create a principal for the server in the AD, but simply want some code to always have a valid ticket - can I avoid having a dedicated task in users' crontab?
Why don't you initiate the ticket cache directly in your code? This might help be more transparent that your job relies on a kerberos login and where it's located. (In 5 years when you come back to this code it will be hard to remember and this might save you some grief later.)
It will also help to ensure that someone can't accidentally disable your job by removing a cron job.
kinit from python script using keytab
I would do this in every script you require authentication to reduce external dependencies.
I'm doing some research and I need to download a very large number of git repositories, something like 17k+.
I wrote a very simple Python script to automate the cloning routine from a dataset containing the GitHub URLs.
first_10 = data.url
name = data.project_name
for x, i in zip(first_10, name):
os.system('git clone {} D:\gitres\{}'.format(x,i))
It just iterates over those two pandas dataframe columns for the URL to be cloned and the folder name.
Here comes the problem: every time the script finds a URL that no longer exists on GitHub, the script halts its routine, asks for credentials and won't resume until my input. Doesn't matter if I input correct credentials or gibberish, it will do this every time it finds an invalid GitHub URL. How do I stop git from asking those credentials?
The reason GitHub sends you a 401 to prompt for credentials if the repository is missing is because they don't want to leak whether a private repository exists. If they didn't prompt, you could easily determine that the repository does exist by getting a 401 and that it doesn't by getting a 404. Instead, GitHub always prompts for credentials, and only then returns a 404 if the repository doesn't exist or isn't accessible to you.
If your desire is not to be prompted at all, as torek mentioned, you can simply set the environment variable GIT_ASKPASS to false and this will work. You could also set GIT_TERMINAL_PROMPT to 0 and that would prevent any prompting for credentials.
However, I strongly recommend that you do indeed set some credentials because GitHub will much more aggressively rate-limit you if you don't set any credentials, and if you do end up using an excessive amount of resources, it's much easier for GitHub to contact you about the problem and ask you to fix it, rather than just block you or make an abuse report to your network provider.
On that note, your Python script is not likely to handle the case where you have a large number of failures for that reason, so you should strongly consider handling that case most robustly. In general, anyone making a large number of HTTP requests to any server needs to learn to gracefully back off.
If you decide you do want to pass credentials, you can do so from the environment using a custom credential helper, or you can use an SSH key and SSH URLs to do this.
I'd suggest you've overspecified the problem (turning this into an XY problem): you don't specifically need to make Git not ask for credentials since you could instead merely clone those repositories that do exist.
That said, there are two ways to prevent Git from asking for credentials:
Use a URL that cannot take credentials. (Any given server may or may not accept such URLs. With GitHub, you could try to log in as git#github.com via ssh, and present a valid public key. After ssh has authenticated you, Git will give you access to any accessible URL, and deny you access to any inaccessible or invalid URL, without asking for further credentials.)
Supply a credential helper that never actually provides any credentials, without asking for any. For instance, you could run with GIT_ASKPASS=false in the environment. See the credentials documentation for details.
(There's one more option as well, which is to allow Git to ask for credentials but redirect the input to a program. This is trickier than just overriding GIT_ASKPASS so there is no reason to cover it here.)
To solve the problem better, find a way to list out the repositories that do exist, and do not attempt to clone the ones that don't. This is likely to go significantly faster.
My guess is that you are using https:// data urls. If you use a personal access token, then GitHub shouldn't be asking you for a username/password. Take a look here on how to set it up.
I think that if you use ssh:// data urls instead, then you wouldn't encounter that problem because git defaults to using your ssh-key for authentication instead of password.
You probably want to check that the repo exists before attempting the clone. There are answers on Stack Overflow for this here.
Alternatively, if you switch to using subprocess instead of os.system, you can simply "trick" it by reading input from /dev/null which will prevent the prompt - that way, no intervention will be needed and the invalid URLs will simply be skipped.
for x, i in zip(first_10, name):
subprocess.call(['git', 'clone', x, i], shell=False, stdin=subprocess.DEVNULL, start_new_session=True)
I have come across another useful trick that might come in handy working in desktop environments.
in conjunction with using GIT_TERMINAL_PROMPT=0, git -c credential.helper= <rest of commands> helps to also suppress credential manager windows to pop up.
I want to write a program which allows me to pull my git remote without having to type my username and password all the time.
When I use git pull in terminal i get this output:
Username for 'https://github.com':
Password for 'https://github.com':
After googling on the issue, I found this
import os
os.system("git pull")
Now I need to know if it's possible to set my username and password in the python program so I don't need to put it in myself.
Putting a name and especially password directly inside the code of any program is dirty and you definitely want to avoid this for several reasons.
In this case, I suggest using SSH keys to authorize without the need of password. Refer to GitHub Help for info on how to achieve this. Once your SSH keys are setup, make sure to use ssh:// repository links instead of https://.
If you're really stuck on using https instead of the ssh protocol as your remote URL for whatever reason (GitHub actually recommends it), might I suggest making use of git's credential caching to make life easier. Just use:
git config credential.helper 'cache [options]'
For example, issuing the command
git config credential.helper 'cache --timeout=3600'
will add
[credential]
helper = cache --timeout=3600
to your gitconfig file, forcing you to only enter your password once every hour for the given remote.
I'm writing a python program which allow user to login to it. I don't want to implement my own authentication but would rather take advantage of the OS(linux) mechanism. That is, when the user is trying to sign in my app by inputing username/password pair(which should be a valid OS user), I need to authenticate the pair by the OS. How to do that ? It may need the subprocess module, yet I've tried with no luck.
Try using PAM via Python PAM or similar
That should be possible by having your script read the /etc/passwd and /etc/shadow files, which contain details about usernames and passwords on a Linux system. Do note that the script will have to have read access to the files, which depending on the situation may or may not be possible.
Here are two good articles explaining the format of those files, which should tell you everything you need to know in order to have your script read and understand them:
Understanding /etc/passwd File Format
Understanding /etc/shadow File Format
By the way, when it talks about encrypted password, it means that it has been encrypted using the DES algorithm. You'll probably need to use pyDes or another python implementation of the DES algorithm in order for your script to create an encrypted password that it can compare to the one in /etc/shadow.
I have a question related to user permissions in python. I have a project that uses the file system to store information about specific tasks. Of course this can be edited/deleted if user wants to. Now say a user starts the program using sudo sh start.sh. He then does some operations and data is stored. Now if anyone else or even the same user will start the program without sudo rights when trying to delete/modify those file a permission denied OSError is raised. Now this seems logical but my question is:
Does python offer any way to choose the permissions when writing/creating a file/folder? More precisely can you create files that will be accessible to all users from sudo ?
Does python offer any way to request a users permission when starting the program/ during program execution?
Regards,
Bogdan
Looks like you need os.fchown().
I don't know what you mean by your second question - do you want to ask for a root password and use it to gain access to a file that the normal user wouldn't have access to?