Develop Reference

getpass behaves different in pychram IDE and terminal

paaword.py is a script where getpass() asked the user about the password and validates it. but i want to automate the whole process and used subprocess for it (main.py). And i am using python3.10
Problem:
problem is when i run the main.py in pycharm IDE it works normally (it automates the process). but when I run the script python3 main.py in ubuntu terminal it asked for the input.
I dont know why it behaves deifferent in in IDE and terminal?
password.py
import warnings
import getpass
import time
# Suppress warnings
warnings.filterwarnings("ignore", category=getpass.GetPassWarning)
for x in range(10):
print(f"curnt index {x}")
time.sleep(5)
password = getpass.getpass("Enter your password: ")
if password != "test":
print("wrong password")
else:
print("correct password")
main.py
import subprocess
# subprocess
proc = subprocess.Popen(["python", "password.py"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
password = "test"
input_data = f"{password}\n"
# read output from the subprocess in real-time
while True:
if proc.poll() is not None:
break
proc.stdin.write(input_data.encode())
proc.stdin.flush()
output = proc.stdout.readline().decode().strip()
if output:
print(output)
output in pycharm:
output in ubuntu terminal (20.04)

Judging by the screenshots, your OS is Linux.
In Linux, getpass() first tries to read directly from the process' controlling terminal (/dev/tty), or, if that fails, stdin using direct terminal I/O; and only if that fails, it falls back to regular I/O, displaying a warning.
Judging by the warnings in the IDE, the latter is exactly what happens in your first case.
Lib/getpass.py:
def unix_getpass(prompt='Password: ', stream=None):
<...>
try:
# Always try reading and writing directly on the tty first.
fd = os.open('/dev/tty', os.O_RDWR|os.O_NOCTTY)
tty = io.FileIO(fd, 'w+')
<...>
input = io.TextIOWrapper(tty)
<...>
except OSError:
# If that fails, see if stdin can be controlled.
<...>
try:
fd = sys.stdin.fileno()
except (AttributeError, ValueError):
fd = None
passwd = fallback_getpass(prompt, stream) # fallback_getpass is what displays the warnings
input = sys.stdin
<...>
if fd is not None:
try:
old = termios.tcgetattr(fd)
<...>
except termios.error:
<...>
passwd = fallback_getpass(prompt, stream)
<...>
return passwd
As you can see, getpass() is specifically designed to be interactive and resist intercepting its input. So if you need to provide a password automatically, use another way:
store it in a file readable only by you (e.g. SSH does that; you can provide that file as an argument and store other arguments there as well), or
use the system's keyring
and only fall back to getpass if the password was not provided that way and/or if you detect that the program is being run interactively (sys.stdin.isatty())
while it's also possible to provide the password on the command line -- in that case, you have to overwrite it in your process' stored command line to hide it from snooping. I couldn't find a way to do that in Python.
You can check Secure Password Handling in Python | Martin Heinz | Personal Website & Blog for a more detailed rundown of the above. (note: it suggests using envvars and load them from .env which would probably not apply to you. That's designed for .NET projects which due to the rigid structure of MS Visual Studio's build system, have had to rely on envvars for any variable values.)

using subprocess.run to automate a command line application (windows 10)

trying to use python to automate a usage of a command line application called slsk-cli
manually, the procedure is straight-forward - i open a command prompt window and type 'soulseek login', then a prompt requests username, after i type in and press enter i'm requested a password.
so far, i manage to get the prompt of the username but not getting passed that.
subprocess.run('soulseek login',shell=True)
this results in the ?Login output in the python console but also the process is stuck, when i run in debug or also in run
is there a better way to go about this?

Interacting continuously with a system via subprocess can be tricky. However, it seems that your interface prompts are one after the other, which can therefore be chained together, via newline characters which act as Return key strokes.
For example, the program shown below simply prompts a user for their username and a password, to which the 'user' (your script) provides the input via the proc.communicate() method. Once these are provided, the user is asked if they'd like to continue (and do the same thing again). The following subprocess call feeds the following input into the prompter.py script:
username
password
continue reply (y or n)
Example code:
import subprocess
uid = 'Bob'
pwd = 'MyPa$$w0rd'
reply = 'n'
with subprocess.Popen('./prompter.py',
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
text=True) as proc:
stdout, stderr = proc.communicate(input='\n'.join([uid, pwd, reply]))
Output:
# Check output.
>>> print(stdout)
Please enter a username: Password: uid='Bob'
pwd='MyPa$$w0rd'
Have another go? [y|n]:
# Sanity check for errors.
>>> print(stderr)
''
Script:
For completeness, I've included the contents of the prompter.py script below.
#!/usr/bin/env python
from time import sleep
def prompter():
while True:
uid = input('\nPlease enter a username: ')
pwd = input('Password: ')
print(f'{uid=}\n{pwd=}')
sleep(2)
x = input('Have another go? [y|n]: ')
if x.lower() == 'n':
break
if __name__ == '__main__':
prompter()

raw_input() prompt disappears when redirecting output to file

I am writing a CLI that accepts an email and password for auth.
The email prompt uses raw_input() and the password prompt uses getpass() for obfuscation.
This setup works fine when outputting directly to console, but falters when redirecting the output to a log file.
Sample code:
user_email = raw_input('Email: ')
user_password = getpass('Password: ')
Sample output without redirection:
$ python script_that_does_stuff.py
Email: me#email.com
Password:
Doing stuff...
Sample output with redirection:
$ python script_that_does_stuff.py > stuff.log
Because I know that it's expecting a user input here, I can type the email, hit enter, and then it will show:
$ python script_that_does_stuff.py > stuff.log
me#email.com
Password:
After inputting a password, it continues as usual, however the log shows the following:
$ cat stuff.log
Email:Doing stuff...
Question:
How can I force the raw_input() prompt to show up in console like the getpass() prompt does when redirecting output to a file?
Environment
This script lives in a legacy Python 2.7 codebase, and is run primarily on Mac OS systems, occasionally Linux.

You can override sys.stdout temporarily to write to the terminal. For example,
import contextlib
import sys
#contextlib.contextmanager
def output_to_terminal():
try:
with open("/dev/tty") as f:
sys.stdout = f
yield
finally:
# Ensure sys.stdout is restored in the event of an error
sys.stdout = sys.__stdout__
with output_to_terminal():
x = raw_input("> ")
print(x)
(This was derived independently; you may want to check source for Python 3's redirect_stdout, also found in the contextlib module, and back port it for your use.)

This answer on another question seems to work for me.
In short, create a custom input function:
def email_input(prompt=None):
if prompt:
sys.stderr.write(str(prompt))
return raw_input()
The calling code then becomes:
user_email = email_input('Email: ')
user_password = getpass('Password: ')
This results in both the Email and Password prompts being sent to stderr (printing to console), and not messing with the redirected log output.

According to official documentation getpass([prompt[, stream]]) has the second optional parameter which indicates output stream to print the prompt to (stderr by default).
When you redirect the output (stdout) the prompt is still printed to stderr for getpass but raw_input does not support setting an output stream so its prompt is redirecting to to the target file.
So to solve your issue, you have to print your prompt to stderr for email as well.

Automated database restore from *.sql files

I am trying to write a python script with which to restore our database. We store all our tables (individually) in the repository. Since typing "source table1.sql, source table2.sql,.." Will be cumbersome I've written a script to do this automatically.
I've found a solution using Popen.
process = Popen('mysql %s -u%s -p%s' % (db, "root", ""), stdout=PIPE, stdin=PIPE, shell=True)
output = process.communicate('source' + file)[0]
The method appears to work very well, however, for each table, it prompts me for a password. How do I bypass this to either get it prompt for a password only once or have the subprocess read the password from a config file?
Is there a better way to do this? I've tried to do this using a windows batch script, but as you'll expect, this is a lot less flexible than using python (for e.g)

Since apparently you have an empty password, remove the -p option, -p without a password makes mysql prompt
from subprocess import Popen, PIPE
process = Popen('mysql %s -u%s' % (db, "root"), stdout=PIPE, stdin=PIPE, shell=True)
output = process.communicate('source' + file)[0]

In order to prevent exposing the password to anyone with permission to see running processes, it's best to put the password in a config file, and call that from the command-line:
The config file:
[client]
host=host_name
user=user_name
password=your_pass
Then the command-line:
mysql --defaults-extra-file=your_configfilename

Well, you could pass it on in the command line after reading it from a file
with open('secret_password.txt', 'r') as f:
password = f.read()
process = Popen('mysql %s -u%s -p%s' % (db, "root", password), stdout=PIPE, stdin=PIPE,
Otherwise you could investigate pexpect, which lets you interact with processes. Other alternatives for reading from a config file (like ConfigParser) or simply making it a python module and importing the password as a variable.

How to hide the password in fabric when the command is printed out?

Say I have a fabfile.py that looks like this:
def setup():
pwd = getpass('mysql password: ')
run('mysql -umoo -p%s something' % pwd)
The output of this is:
[host] run: mysql -umoo -pTheActualPassword
Is there a way to make the output look like this?
[host] run: mysql -umoo -p*******
Note: This is not a mysql question!

Rather than modifying / overriding Fabric, you could replace stdout (or any iostream) with a filter.
Here's an example of overriding stdout to censor a specific password. It gets the password from Fabric's env.password variable, set by the -I argument. Note that you could do the same thing with a regular expression, so that you wouldn't have to specify the password in the filter.
I should also mention, this isn't the most efficient code in the world, but if you're using fabric you're likely gluing a couple things together and care more about manageability than speed.
#!/usr/bin/python
import sys
import string
from fabric.api import *
from fabric.tasks import *
from fabric.contrib import *
class StreamFilter(object):
def __init__(self, filter, stream):
self.stream = stream
self.filter = filter
def write(self,data):
data = data.replace(self.filter, '[[TOP SECRET]]')
self.stream.write(data)
self.stream.flush()
def flush(self):
self.stream.flush()
#task
def can_you_see_the_password():
sys.stdout = StreamFilter(env.password, sys.stdout)
print 'Hello there'
print 'My password is %s' % env.password
When run:
fab -I can_you_see_the_password
Initial value for env.password:
this will produce:
Hello there
My password is [[TOP SECRET]]

It may be better to put the password in the user's ~/.my.cnf under the [client] section. This way you don't have to put the password in the python file.
[client]
password=TheActualPassword

When you use the Fabric command run, Fabric isn't aware of whether or not the command you are running contains a plain-text password or not. Without modifying/overriding the Fabric source code, I don't think you can get the output that you want where the command being run is shown but the password is replaced with asterisks.
You could, however, change the Fabric output level, either for the entire Fabric script or a portion, so that the command being run is not displayed. While this will hide the password, the downside is that you wouldn't see the command at all.
Take a look at the Fabric documentation on Managing Output.

Write a shell script that invokes the command in question with the appropriate password, but without echoing that password. You can have the shell script lookup the password from a more secure location than from your .py files.
Then have fabric call the shell script instead.
This solves both the problem of having fabric not display the password and making sure you don't have credentials in your source code.

from fabric.api import run, settings
with settings(prompts={'Enter password: ': mysql_password}):
run("mysql -u {} -p -e {}".format(mysql_user,mysql_query))
or if no prompt available:
from fabric.api import run, hide
with hide('output','running','warnings'):
run("mycommand --password {}".format(my_password))