Using Python/Psycopg2/PopstgreSQL and Cron.
I'd like to take remote server information(see below) and add it into a PostGreSQL database on the host computer.
Using #!/usr/bin/python
import socket
import commands
import string
import os
hostname = socket.gethostname()
print hostname
ip = commands.getoutput("ifconfig").split("\n")[1].split()[1][5:]
print ip
os = commands.getoutput("lsb_release -d")
print os[13:34]
kernel = commands.getoutput("uname -p")
print kernel
reboot = commands.getoutput("who -b")
print reboot[22:38]
This is the 'connect to database' script:
#!/usr/bin/python
import psycopg2
import sys
try:
conn = psycopg2.connect('host=*** dbname=*** user=*** password=***')
print "Connected to Database"
except:
print "No Connection"
cur = conn.cursor()#cursor_factory=psycopg2.extras.DictCursor)
try:
cur.execute('SELECT * FROM new')
rows = cur.fetchall()
print "\n Show: \n"
for row in rows:
print " ", row
except:
print "Not Working"
I'm able to connect, I'm able to pull the data. I need to combine the two scripts and insert the returned information into the database.
Your local python script would have these lines:
import psycopg2 as db
remote_connection = db.connect('host=that_host dbname=that_db user=user password=pwd')
local_connection = db.connect('host=localhost dbname=local_db user=user password=pwd')
Related
os.eviron['username'] is sent from OpenVPN, but for debugging I changed it to "test" as user in the Database.
os.eviron['password'] is sent from OpenVPN, but for debugging I changed it to the working password "password", and everything worked as designed.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import mysql.connector as database
import sys
import hashlib
from config import DB_NAME, DB_ADMIN, DB_PASSWORD, DB_HOST, HASH_ALGORITHM, HASH_SALTY
connection = database.connect(
user=DB_ADMIN,
password=DB_PASSWORD,
host=DB_HOST,
database=DB_NAME
)
cursor = connection.cursor()
hash_func = getattr(hashlib, HASH_ALGORITHM)
salty = hash_func(HASH_SALTY.encode("utf-8")).hexdigest()
ovpnuser = (os.environ['username'])
ovpnuser = "SELECT * FROM users WHERE usernames = '" + ovpnuser + "';"
cursor.execute(ovpnuser)
result = cursor.fetchone()
if result is None:
sys.exit(1)
id, usernames, passwords = result
passw = (os.environ['password'] + salty) #Salt this mofo
if hash_func(passw.encode("utf-8")).hexdigest() != passwords:
sys.exit(1)
sys.exit(0)
Now that I use OpenVPN to call this plugin as you see it now, it returns sys.exit(1) Failed to authenticate. I've tried to debug it, but I can't get ovpn to print and errors from connections or calls to this plugin. I even added echo to write to a log file in the python code, and it didn't work.
Any thoughts?
Relatively new to python scripts, so bare with.
I have used speedtest-cli before. I have edited the script so it will insert the values into a sql table as below, however having an issue with one of the inserts. It will insert ping, and download ok, however, the upload is always 2.74 or 2.75 for example, but ONLY when run from a crontab.. very weird.
If I run the python script from cli it will insert values fine.
This is my query, and the values ping, download and upload are coming from the speedtest-cli script.
Here is the full script
import re
import subprocess
import time
import mysql.connector
from mysql.connector import Error
from mysql.connector import errorcode
print "----------------------------------"
print 'Started: {} {}'.format(time.strftime('%d/%m/%y %H:%M:%S'), "")
response = subprocess.Popen('speedtest-cli --simple', shell=True, stdout=subprocess.PIPE).stdout.read()
ping = re.findall('Ping:\s(.*?)\s', response, re.MULTILINE)
download = re.findall('Download:\s(.*?)\s', response, re.MULTILINE)
upload = re.findall('Upload:\s(.*?)\s', response, re.MULTILINE)
ping[0] = ping[0].replace(',', '.')
download[0] = download[0].replace(',', '.')
upload[0] = upload[0].replace(',', '.')
try:
if os.stat('/var/www/html/speed/log.txt').st_size == 0:
print 'Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)'
except:
pass
print 'PING: {}, DOWN: {}, UP: {}'.format(ping[0], download[0], upload[0])
try:
connection = mysql.connector.connect(host='localhost',
database='dev',
user='dev',
password='dev1')
sql_insert_query = ("""INSERT INTO speedtest(ping, download, upload) VALUES (%s,%s,%s)""", (ping[0], download[0], upload[0]))
cursor = connection.cursor()
result = cursor.execute(*sql_insert_query)
connection.commit()
print ("Insert success into speedtest tbl")
except mysql.connector.Error as error :
connection.rollback() #rollback if any exception occured
print("Failed inserting record into speedtest table {}".format(error))
finally:
#closing database connection.
if(connection.is_connected()):
cursor.close()
connection.close()
print("MySQL conn closed")
print 'Finished: {} {}'.format(time.strftime('%d/%m/%y %H:%M:%S'), "")
Manual script runs ok, just from crontab I get unexpected values. Not sure how to solve.
I am trying to connect to my database via Python 2.7 with this code:
import csv
import psycopg2
try:
conn = psycopg2.connect("dbname='student', user='postgres',password='password', host='localhost'")
cursor = conn_cursor()
reader = csv.reader(open('last_file.csv', 'rb'))
print "connected"
except:
print "not Connected"
It did work last week and we don't think we've changed anything, but now it won't connect.
We've tried using it with the database open and closed, nothing worked.
The database does exist in Postgres.
import psycopg2
try:
conn = psycopg2.connect("dbname='database_name' user='postgres_user_name' host='localhost' password='user_passwd'")
except:
print "I am unable to connect to the database"
cur = conn.cursor()
cur.execute("""SELECT * from table_name""")
rows = cur.fetchall()
print "\nShow me the data:\n"
for row in rows:
print " ", row[0]
print " ", row[1]
Exception part add like this to see what is error
except Exception as ex:
print "not Connected"
print "Error: "+ str(ex)
Try this:
import csv
import psycopg2
try:
conn = psycopg2.connect("dbname='student', user='postgres',password='password', host='localhost'")
except:
print "I am unable to connect to the database."
cursor = conn.cursor()
try:
reader = csv.reader(open('last_file.csv', 'rb'))
print "connected"
except:
print "not Connected"
Seems like there are something wrong with your postgres.
Try and see postgres log.
Location of postgres log by default :
tail -f /var/log/postgresql/<>/main/postgresql.log
something like this.
Also don't forget to check firewall. Maybe someone disable it by accident.
Also try for pip install PyGreSQL package. Since psycopg2 (some of versions) is under GPL license. It could be tricky for open source license. Just for your information.
I have the following code in python for sending data to a mysql database
import time
import datetime
import MySQLdb
from time import strftime
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
PIR_PIN = 21
GPIO.setup(PIR_PIN, GPIO.IN)
# Variables for MySQL
db = MySQLdb.connect(host="*******", user="root",passwd="*****", db="sensor1")
cur = db.cursor()
while True:
i = GPIO.input(PIR_PIN)
print i
datetimeWrite = (time.strftime("%Y-%m-%d ") + time.strftime("%H:%M:%S"))
print datetimeWrite
sql = ("""INSERT INTO templog (datetime,temperature) VALUES (%s,%s)""",(datetimeWrite,i))
try:
print "Writing to database..."
# Execute the SQL command
cur.execute(*sql)
# Commit your changes in the database
db.commit()
print "Write Complete"
except:
# Rollback in case there is any error
db.rollback()
print "Failed writing to database"
cur.close()
db.close()
break
My problem is that my XAMPP server is installed in pc where I want to view the the data from raspberry pi in mysql database.
So for getting the connection established what should I write in host= "?"
The connection-string should be like this:
db = MySQLdb.connect(host="192.168.0.xxx", user="root",passwd="*****", db="sensor1")
See this Question
Host will be IP address of your system on which XAMPP is installed e.g. 192.168.x.x
I have a database available on a remote host. When I use putty and SSH, I am able to access it. The database itself has no password. Only, the SSH connection has a password. But, when I try to connect to it using Python, it asks for password. I'm new to postgres and paramiko.
Here's what I've tried:
import psycopg2
import paramiko
import time
t = paramiko.Transport(('xxx.com', 22))
t.connect(username="xxx", password='xxx')
c = paramiko.Channel(t)
conn = psycopg2.connect("dbname='xxx'")
curs = conn.cursor()
sql = "select * from xxx"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
Other method that I tried was:
import os, psycopg2
os.system("ssh xxx#xxx.com -fNL 5432:localhost:5432 -p 22")
while True:
try:
conn = psycopg2.connect("dbname='xxx'")
curs = conn.cursor()
sql = "select * from xxx"
curs.execute(sql)
rows = curs.fetchall()
print(rows)
except:
print "I am unable to connect to the database"
This gives me a 'Could not request local forwarding' error.
Is there some other way to go about this?
I have a Windows 7 (x64) machine with Python 2.7. Please help me. Thanks.
You should connect to the remote server and make port forwarding of remote PostgreSQL to a local port.
Without paramiko, it's something like this:
# start port forwarding
$ ssh -L PGSQL_LOCAL_PORT:localhost:PGSQL_REMOTE_PORT user#xxx.com
# in python
psycopg.connect("dbname='xxx' host='localhost' port='PGSQL_LOCAL_PORT'")
Here is an example of doing this with paramiko
https://code.ros.org/trac/wg-ros-pkg/browser/pkg/trunk/paramiko/demos/forward.py?rev=30
Note: Port forwarding is blocking operation. So, you have to start port forwarding in separate thread/process.