I'm studying how to deploy a django application with kubernetes, I found this tutorial, which explains about the Devspace tool, so that following the tutorial just like it returns me a message that I can't solve.
the message is this:
returns only this [fatal] pullSecrets[0].registry: cannot be empty
Can someone help me ?
DevSpace maintainer here. Check your devspace.yaml and you'll see that there is a pullSecrets section defined and the first entry in this section has registry either not defined or empty string.
To fix this, you can either remove the entire pullSecrets or provide a valid registry hostname + username and password. For the username and especially for the password, you should use variables such as ${REGISTRY_PASSWORD} and then set them via environment variables in your terminal or let DevSpace set them via devspace set var REGISTRY_PASSWORD=xxx etc.
First thing in your setup at very first you missed two steps
in which it's asking about the docker registry and username & authentication.
If you will pass the detail you won't face any issue and secret will get created automatically.
You have to set the value for pull secret to true
createPullSecret: true
pull seceret is used by Kubernetes config when you have to fetch the container (docker) image from the private container repository for example private docker repository.
If your config pull already exist but you have not created it you can create it manually by providing a few details
kubectl create secret docker-registry my-pull-secret --docker-server=[REGISTRY_URL] --docker-username=[REGISTRY_USERNAME] --docker-password=[REGISTRY_PASSWORD] --docker-email=[YOUR_EMAIL]
Read more at : https://devspace.cloud/docs/v3.5.18/image-building/registries/pull-secrets
There is same issue reported previously : https://github.com/loft-sh/devspace/issues/1317
Related
I am just trying to create a python package on testPyPI for some testing (using twine), following this official guide. I am using a token, which succeeded exactly one time (username __token__, password is the token itself). Then I made a change and wanted to repeat that process, it doesn't work anymore.
related post
This seems to be a common issue, however I couldn't fix it so far. I'm on Windows 10, this is what I tried...
different ways of pasting into the console and different consoles (so that's not the issue)
using a .pypirc file for authentication details
a new token
a new account
a new email
also directly inserting username and password into the twine command (which should be avoided, I guess)
And I'm running out of ideas. Any clue?
I was struggling with the same issue. The tutorial OP referred to is, in my opinion, not explicit enough, because there tend to be two ways, which can easily be mixed (as I was doing, however, I am a newbie following the tutorial for a reason ;) ).
Solution:
Assuming a /home/.pypirc file is created you can either
use credentials (i.e, use username and password for logging into the website):
[testpypi]
username = <your username>
password = <your password>
or use the API token created on the website:
[testpypi]
username = __token__
password = pypi-<Rest of token>
Hope that helps others following the tutorial.
I'm doing some research and I need to download a very large number of git repositories, something like 17k+.
I wrote a very simple Python script to automate the cloning routine from a dataset containing the GitHub URLs.
first_10 = data.url
name = data.project_name
for x, i in zip(first_10, name):
os.system('git clone {} D:\gitres\{}'.format(x,i))
It just iterates over those two pandas dataframe columns for the URL to be cloned and the folder name.
Here comes the problem: every time the script finds a URL that no longer exists on GitHub, the script halts its routine, asks for credentials and won't resume until my input. Doesn't matter if I input correct credentials or gibberish, it will do this every time it finds an invalid GitHub URL. How do I stop git from asking those credentials?
The reason GitHub sends you a 401 to prompt for credentials if the repository is missing is because they don't want to leak whether a private repository exists. If they didn't prompt, you could easily determine that the repository does exist by getting a 401 and that it doesn't by getting a 404. Instead, GitHub always prompts for credentials, and only then returns a 404 if the repository doesn't exist or isn't accessible to you.
If your desire is not to be prompted at all, as torek mentioned, you can simply set the environment variable GIT_ASKPASS to false and this will work. You could also set GIT_TERMINAL_PROMPT to 0 and that would prevent any prompting for credentials.
However, I strongly recommend that you do indeed set some credentials because GitHub will much more aggressively rate-limit you if you don't set any credentials, and if you do end up using an excessive amount of resources, it's much easier for GitHub to contact you about the problem and ask you to fix it, rather than just block you or make an abuse report to your network provider.
On that note, your Python script is not likely to handle the case where you have a large number of failures for that reason, so you should strongly consider handling that case most robustly. In general, anyone making a large number of HTTP requests to any server needs to learn to gracefully back off.
If you decide you do want to pass credentials, you can do so from the environment using a custom credential helper, or you can use an SSH key and SSH URLs to do this.
I'd suggest you've overspecified the problem (turning this into an XY problem): you don't specifically need to make Git not ask for credentials since you could instead merely clone those repositories that do exist.
That said, there are two ways to prevent Git from asking for credentials:
Use a URL that cannot take credentials. (Any given server may or may not accept such URLs. With GitHub, you could try to log in as git#github.com via ssh, and present a valid public key. After ssh has authenticated you, Git will give you access to any accessible URL, and deny you access to any inaccessible or invalid URL, without asking for further credentials.)
Supply a credential helper that never actually provides any credentials, without asking for any. For instance, you could run with GIT_ASKPASS=false in the environment. See the credentials documentation for details.
(There's one more option as well, which is to allow Git to ask for credentials but redirect the input to a program. This is trickier than just overriding GIT_ASKPASS so there is no reason to cover it here.)
To solve the problem better, find a way to list out the repositories that do exist, and do not attempt to clone the ones that don't. This is likely to go significantly faster.
My guess is that you are using https:// data urls. If you use a personal access token, then GitHub shouldn't be asking you for a username/password. Take a look here on how to set it up.
I think that if you use ssh:// data urls instead, then you wouldn't encounter that problem because git defaults to using your ssh-key for authentication instead of password.
You probably want to check that the repo exists before attempting the clone. There are answers on Stack Overflow for this here.
Alternatively, if you switch to using subprocess instead of os.system, you can simply "trick" it by reading input from /dev/null which will prevent the prompt - that way, no intervention will be needed and the invalid URLs will simply be skipped.
for x, i in zip(first_10, name):
subprocess.call(['git', 'clone', x, i], shell=False, stdin=subprocess.DEVNULL, start_new_session=True)
I have come across another useful trick that might come in handy working in desktop environments.
in conjunction with using GIT_TERMINAL_PROMPT=0, git -c credential.helper= <rest of commands> helps to also suppress credential manager windows to pop up.
I found out there are lot of ways to store it as variables, hooks and other ways using encryption. I would like to know what's the best way to do it.
Currently there 2 ways of storing secrests:
1) Airflow Variables: Value of a variable will be hidden if the key contains any words in (‘password’, ‘secret’, ‘passwd’, ‘authorization’, ‘api_key’, ‘apikey’, ‘access_token’) by default, but can be configured to show in clear-text as shown in the image below.
However, there is a known-bug where anyone with an access to UI can export all the variables which will expose the secrets.
2) Airflow Connections:
You can use the Passwords field in Airflow connections which will encrypt that field if you had installed the crypto package (pip install apache-airflow[crypto]). The password field would just appear as blank in the UI as shown in the screenshot.
More on Securing connections: https://airflow.apache.org/howto/secure-connections.html
I recommend the 2nd approach as even if someone gets access to the UI, he/she won't be able to get your secrets. Keep in mind though that you need to install the crypto package for this.
You can then access the secrets as below:
from airflow.hooks.base_hook import BaseHook
connection = BaseHook.get_connection(CONN_ID)
slack_token = connection.password
You can set the CONN_ID as the name of your connection.
I get an error when trying to deallocate a virtual machine with the Python SDK for Azure.
Basically I try something like:
credentials = ServicePrincipalCredentials(client_id, secret, tenant)
compute_client = ComputeManagementClient(credentials, subscription_id, '2015-05-01-preview')
compute_client.virtual_machines.deallocate(resource_group_name, vm_name)
pprint (result.result())
-> exception:
msrestazure.azure_exceptions.CloudError: Azure Error: AuthorizationFailed
Message: The client '<some client UUID>' with object id '<same client UUID>' does not have authorization to perform action 'Microsoft.Compute/virtualMachines/deallocate/action' over scope '/subscriptions/<our subscription UUID>/resourceGroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<our-machine>'.
What I don't understand is that the error message contains an unknown client UUID that I have not used in the credentials.
Python is version 2.7.13 and the SDK version was from yesterday.
What I guess I need is a registration for an Application, which I did to get the information for the credentials. I am not quite sure which exact permission(s) I need to register for the application with IAM. For adding an access entry I can only pick existing users, but not an application.
So is there any programmatic way to find out which permissions are required for an action and which permissions our client application has?
Thanks!
As #GauravMantri & #LaurentMazuel said, the issue was caused by not assign role/permission to a service principal. I had answered another SO thread Cannot list image publishers from Azure java SDK, which is similar with yours.
There are two ways to resolve the issue, which include using Azure CLI & doing these operations on Azure portal, please see the details of my answer for the first, and I update below for the second way which is old.
And for you want to find out these permissions programmatically, you can refer to the REST API Role Definition List to get all role definitions that are applicable at scope and above, or refer to Azure Python SDK Authentication Management to do it via the code authorization_client.role_definitions.list(scope).
Hope it helps.
Thank you all for your answers! The best recipe for creating an application and to register it with the right role - Virtual Machine Contributor - is presented indeed on https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal
The main issue I had was that there is a bug in the adding a role within IAM. I use add. I select "Virtual Machine Contributor". With "Select" I get presented a list of users, but not the application that I have created for this purpose. Entering the first few letters of the name of my application will give a filtered output that includes my application this time though. Registration is then finished and things can proceed.
I want to write a program which allows me to pull my git remote without having to type my username and password all the time.
When I use git pull in terminal i get this output:
Username for 'https://github.com':
Password for 'https://github.com':
After googling on the issue, I found this
import os
os.system("git pull")
Now I need to know if it's possible to set my username and password in the python program so I don't need to put it in myself.
Putting a name and especially password directly inside the code of any program is dirty and you definitely want to avoid this for several reasons.
In this case, I suggest using SSH keys to authorize without the need of password. Refer to GitHub Help for info on how to achieve this. Once your SSH keys are setup, make sure to use ssh:// repository links instead of https://.
If you're really stuck on using https instead of the ssh protocol as your remote URL for whatever reason (GitHub actually recommends it), might I suggest making use of git's credential caching to make life easier. Just use:
git config credential.helper 'cache [options]'
For example, issuing the command
git config credential.helper 'cache --timeout=3600'
will add
[credential]
helper = cache --timeout=3600
to your gitconfig file, forcing you to only enter your password once every hour for the given remote.