PostgreSQL on MacOSX - python

I just wanted to install a PostgreSQL Database. After 3 hours of trying I do not know what else to do. My last try included installing PostgreSQL via Homebrew -> Works perfectly fine.
But typing this:
which psql
I got this: /usr/local/bin/psql
From my view this sort of Path is wrong, a I saw a different one in most tutorials. But I have no idea what to do.
But I went on trying:
createuser -U postgres yrkIO -P
And the terminal asked me for a password only to give me this:
createuser: could not connect to database postgres: FATAL: role "postgres" does not exist
What can I do, I just want to run a PostgreSQL on my Python Flask App?

Have you tried it without forcing a password?
createuser -s -r postgres
This worked for me.
Also, remember to start a server
Start:
pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
Stop server:
pg_ctl -D /usr/local/var/postgres stop -s -m fast

Related

Read data tables from an SQL file containing an entire database [duplicate]

How can I import a database with mysql from terminal?
I cannot find the exact syntax.
Assuming you're on a Linux or Windows console:
Prompt for password:
mysql -u <username> -p <databasename> < <filename.sql>
Enter password directly (not secure):
mysql -u <username> -p<PlainPassword> <databasename> < <filename.sql>
Example:
mysql -u root -p wp_users < wp_users.sql
mysql -u root -pPassword123 wp_users < wp_users.sql
See also:
4.5.1.5. Executing SQL Statements from a Text File
Note: If you are on windows then you will have to cd (change directory) to your MySQL/bin directory inside the CMD before executing the command.
Preferable way for windows:
Open the console and start the interactive MySQL mode
use <name_of_your_database>;
source <path_of_your_.sql>
mysql -u <USERNAME> -p <DB NAME> < <dump file path>
-u - for Username
-p - to prompt the Password
Eg. mysql -u root -p mydb < /home/db_backup.sql
You can also provide password preceded by -p but for the security reasons it is not suggestible. The password will appear on the command itself rather masked.
Directly from var/www/html
mysql -u username -p database_name < /path/to/file.sql
From within mysql:
mysql> use db_name;
mysql> source backup-file.sql
Open Terminal Then
mysql -u root -p
eg:- mysql -u shabeer -p
After That Create a Database
mysql> create database "Name";
eg:- create database INVESTOR;
Then Select That New Database "INVESTOR"
mysql> USE INVESTOR;
Select the path of sql file from machine
mysql> source /home/shabeer/Desktop/new_file.sql;
Then press enter and wait for some times if it's all executed then
mysql> exit
From Terminal:
mysql -uroot -p --default-character-set=utf8 database_name </database_path/database.sql
in the terminal type
mysql -uroot -p1234; use databasename; source /path/filename.sql
Below command is working on ubuntu 16.04, I am not sure it is working or not other Linux platforms.
Export SQL file:
$ mysqldump -u [user_name] -p [database_name] > [database_name.sql]
Example : mysqldump -u root -p max_development > max_development.sql
Import SQL file:
$ mysqldump -u [user_name] -p [database_name] < [file_name.sql]
Example: mysqldump -u root -p max_production < max_development.sql
Note SQL file should exist same directory
I usually use this command to load my SQL data when divided in files with names : 000-tableA.sql, 001-tableB.sql, 002-tableC.sql.
for anyvar in *.sql; do <path to your bin>/mysql -u<username> -p<password> <database name> < $anyvar; done
Works well on OSX shell.
Explanation:
First create a database or use an existing database. In my case, I am using an existing database
Load the database by giving <name of database> = ClassicModels in my case and using the operator < give the path to the database = sakila-data.sql
By running show tables, I get the list of tables as you can see.
Note : In my case I got an error 1062, because I am trying to load the same thing again.
mysql -u username -ppassword dbname < /path/file-name.sql
example
mysql -u root -proot product < /home/myPC/Downloads/tbl_product.sql
Use this from terminal
After struggling for sometime I found the information in https://tommcfarlin.com/importing-a-large-database/
Connect to Mysql (let's use root for both username and password):
mysql -uroot -proot
Connect to the database (let's say it is called emptyDatabase (your should get a confirmation message):
connect emptyDatabase
3 Import the source code, lets say the file is called mySource.sql and it is in a folder called mySoureDb under the profile of a user called myUser:
source /Users/myUser/mySourceDB/mySource.sql
Open the MySQL Command Line Client and type in your password
Change to the database you want to use for importing the .sql file data into. Do this by typing:
USE your_database_name
Now locate the .sql file you want to execute.
If the file is located in the main local C: drive directory and the .sql script file name is currentSqlTable.sql, you would type the following:
\. C:\currentSqlTable.sql
and press Enter to execute the SQL script file.
If you are using sakila-db from mysql website,
It's very easy on the Linux platform just follow the below-mentioned steps, After downloading the zip file of sakila-db, extract it. Now you will have two files, one is sakila-schema.sql and the other one is sakila-data.sql.
Open terminal
Enter command mysql -u root -p < sakila-schema.sql
Enter command mysql -u root -p < sakila-data.sql
Now enter command mysql -u root -p and enter your password, now you have entered into mysql system with default database.
To use sakila database, use this command use sakila;
To see tables in sakila-db, use show tables command
Please take care that extracted files are present in home directory.
First connect to mysql via command line
mysql -u root -p
Enter MySQL PW
Select target DB name
use <db_name>
Select your db file for import
SET autocommit=0; source /root/<db_file>;
commit;
This should do it. (thanks for clearing)
This will work even 10GB DB can be imported successfully this way. :)
In Ubuntu, from MySQL monitor, you have already used this syntax:
mysql> use <dbname>
-> The USE statement tells MySQL to use dbname as the default database for subsequent statements
mysql> source <file-path>
for example:
mysql> use phonebook;
mysql> source /tmp/phonebook.sql;
Important: make sure the sql file is in a directory that mysql can access to like /tmp
If you want to import a database from a SQL dump which might have "use" statements in it, I recommend to use the "-o" option as a safeguard to not accidentially import to a wrong database.
• --one-database, -o
Ignore statements except those those that occur while the default
database is the one named on the command line. This filtering is
limited, and based only on USE statements. This is useful for
skipping updates to other databases in the binary log.
Full command:
mysql -u <username> -p -o <databasename> < <filename.sql>
For Ubuntu/Linux users,
Extract the SQL file and paste it somewhere
e.g you pasted on desktop
open the terminal
go to your database and create a database name
Create database db_name;
Exit Mysql from your terminal
cd DESKTOP
mysql -u root -p db_name < /cd/to/mysql.sql
Enter the password:....
Before running the commands on the terminal you have to make sure that you have MySQL installed on your terminal.
You can use the following command to install it:
sudo apt-get update
sudo apt-get install mysql-server
Refrence here.
After that you can use the following commands to import a database:
mysql -u <username> -p <databasename> < <filename.sql>
The simplest way to import a database in your MYSQL from the terminal is done by the below-mentioned process -
mysql -u root -p root database_name < path to your .sql file
What I'm doing above is:
Entering to mysql with my username and password (here it is root & root)
After entering the password I'm giving the name of database where I want to import my .sql file. Please make sure the database already exists in your MYSQL
The database name is followed by < and then path to your .sql file. For example, if my file is stored in Desktop, the path will be /home/Desktop/db.sql
That's it. Once you've done all this, press enter and wait for your .sql file to get uploaded to the respective database
There has to be no space between -p and password
mysql -u [dbusername] -p[dbpassword] [databasename] < /home/serverusername/public_html/restore_db/database_file.sql
I always use it, it works perfectly. Thanks to ask this question. Have a great day. Njoy :)

mysqldump command working in cmd but not in python

I'm trying to do some tests dumping data from one database to another with mysqldump.
The mysqldump is set in PATH, and the command runs perfectly in the CMD interface, or via a .cmd.
It seems to run ok in python wrapped in a simple try/except block, but I don't get any result in the target database.
Working with:
MariaDB 10.1 & 10.5 /
Python 3.9
The command looks similar to this:
mysqldump --no-create-info --no-create-db --user root -p****** --port=some_port -h 127.0.0.1 some_database some_table |
mysql --user root -p***** --port=some_port -h 127.0.0.1 target_table
What I've tried is different variations on Popen (also trying to handle the pipe via subprocess, and splitting the args), but starting with the simplest soltuion given elsewhere on stackoverflow:
subprocess.Popen("mysqldump --no-create-info --no-create-db --user root -p****** --port=some_port -h 127.0.0.1 some_database some_table |
mysql --user root -p***** --port=some_port -h 127.0.0.1 target_table", shell=True)
I also made a .cmd file and ran this in windows and it works fine. Passing it to either Popen() or os.system() in python gives the same problem, seems to check out in a try/except, but no result seen in the target database.
What could be the problem, and how do I get this right?
Solved for anyone who wants to know.
Put the absolute paths in the command with the short names generated for non-8dot3 file names like so:
'C:\\PROGRA~1\\MARIAD~1.1\\bin\\mysqldump.exe --no-create-info --no-create-db --user root -p****** --port=some_port -h 127.0.0.1 some_database some_table |
C:\\PROGRA~1\\MARIAD~1.1\\bin\\mysql.exe --user root -p****** --port=some_port -h 127.0.0.1 some_database'
It's my work laptop where they only give permissions to add to PATH on my user account, and not the entire system. Got my Python configured to look at system PATH.
Was a bit confusing to see it work in the command line and a .cmd script, but in no way in Python. And when running, not telling me it isn't "recognized as an internal command..." when in other cases it does.

how to connect oracle database from python from unix server

How to connect oracle database server from python inside unix server ?
I cant install any packages like cx_Orcale, pyodbc etc.
Please consider even PIP is not available to install.
It my UNIX PROD server, so I have lot of restriction.
I tried to run the sql script from sqlplus command and its working.
Ok, so there is sqlplus and it works, this means that oracle drivers are there.
Try to proceed as follows:
1) create a python virtualenv in your $HOME. In python3
python -m venv $HOME/my_venv
2) activate it
source $HOME/my_venv/bin/activate[.csh] # .csh is for cshell, for bash otherwise
3) install pip using python binary from you new virtualenv, it is well described here: https://pip.pypa.io/en/stable/installing/
TL;DR:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get_pip.py (this should install pip into your virtualenv as $HOME/my_env/bin/pip[3]
4) install cx_Oracle:
pip install cx_Oracle
Now you should be able to import it in your python code and connect to an oracle DB.
I tried to connect Oracle database via SQLPLUS and I am calling the script with below way :
os.environ['ORACLE_HOME'] = '<ORACEL PATH>'
os.chdir('<DIR NAME>')
VARIBALE=os.popen('./script_to_Call_sql_script.sh select.sql').read()
My shell script: script_to_Call_sql_script.sh
#!/bin/bash
envFile=ENV_FILE_NAME
envFilePath=<LOACTION_OF_ENV>${envFile}
ORACLE_HOME=<ORACLE PATH>
if [[ $# -eq 0 ]]
then
echo "USAGES: Please provide the positional parameter"
echo "`$basename $0` <SQL SCRIPT NAME>"
fi
ECR=`$ORACLE_HOME/bin/sqlplus -s /#<server_name><<EOF
set pages 0
set head off
set feed off
#$1;
exit
EOF`
echo $ECR
Above things help me to do my work done on Production server.

Connect to gitlab production postgresql database with psycopg2?

I am trying to connect to GitLab production (installed with omnibus package) postgresql database with psycopg2.
My configuration is like below:
onn = psycopg2.connect(database="gitlabhq_production", user="gitlab-psql", host="/var/opt/gitlab/postgresql", port="5432")
It gives the following error:
FATAL: Peer authentication failed for user "gitlab-psql"
I can connect to the postgresql server on command line with:
sudo -u gitlab-psql -i bash /opt/gitlab/embedded/bin/psql --port 5432 -h /var/opt/gitlab/postgresql -d gitlabhq_production
Does anyone know what will be the correct parameters to pass into?
Peer authentication works by checking the user the process is running as. In your command line example you switch to gitlab-psql using sudo.
There are two ways to fix this:
Assign a password to the gitlab-psql postgres user (not the system user!) and use that to connect via python. Setting the password is just another query you need to run as a superuser like so:
sudo -u postgres psql -c "ALTER USER gitlab-psql WITH PASSWORD 'ReplaceThisWithYourLongAndSecurePassword';"
Run your python script as gitlab-psql like so:
sudo -u gitlab-psql python /path/to/your/script.py

Force delete of any previous test database (autoclobber) when running Django unit tests, eg, in PyCharm

I am running Django unit tests against a multithreaded app. Often a thread hasn't terminated by the time the unit test finishes, so the test database cannot be deleted. When I next run the tests, I get the message:
Type 'yes' if you would like to try deleting the test database 'test_appname', or 'no' to cancel`
The create_test_db autoclobber option is the functionality I want, but how can I use that? I can't find any examples or clues. I'm working in the PyCharm IDE, which is pretty configurable. I just want to delete the test database silently every time.
I'm putting tests in Transaction TestCase classes, running setup_test_environment() then Client().post(reverse(etc..))..
In Pycharm django tests, you can enable the option input and enter --noinput
see screenshot below
If you're using PyCharm and want to run a single test using the green arrow but you keep getting this error, you can modify the default django tests configuration template, so that you don't have to keep setting the --noinput option on each.
If you are using flush in some other way, (eg, on the pre-existing development database like here), the --noinput option supresses the user prompt, eg:
from django.core.management import call_command
call_command('flush', '--noinput')
My answer was to create a script like this:
export PGPASSWORD=the_password
if [[ `psql -h 127.0.0.1 -d postgres -U username -p 5432 -tAc "SELECT 1 FROM pg_database WHERE datname='test_djangoprojectname'"` == "1" ]]
then
psql -h 127.0.0.1 -d postgres -U username -p 5432 -a -w -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'test_djangoprojectname' AND pid <> pg_backend_pid();"
psql -h 127.0.0.1 -d postgres -U username -p 5432 -a -w -c "DROP DATABASE test_djangoprojectname;"
fi
The -d setting is database name - it can be any database your user has access to, except the one you are deleting.
The default username is postgres.
The -p setting is the port your database is on - 5432 is the default.
Save as (for example) del_test_db.sh (Windows users see below), then
chmod +x del_test_db.sh
Then in PyCharm:
Run, Edit Configurations...
Unfold Defaults, click Django tests
In the Before launch window click +, External tools, click +
Under Program, select your file del_test_db.sh, give the command a name (eg, 'del test db') and click OK.
Select your tool in the list and click OK
You may need to unfold Django tests in the left and delete existing test configurations
Then the script force deletes the test database before every run.
This works on Mac OS X and Ubuntu etc. For Windows the process is the same, except instead of export use SET, save the commands as a .bat file instead of .sh, and you don't need to chmod +x, and use the following syntax for the IF statement in the batch file:
if 'command' == '1' (
...
)
Apologies I'm unable to check this as I don't have a Windows machine.
Thanks to this answer for the code checking whether the database exists.

Categories

Resources