I'm attempting to develop a web app using tornado/torndb and am running into some issues with my database interactions. I've written a "Database" class which wraps torndb in order to provide some common database functionality for my web app. When invoking any of the methods from the Database class I've written there seems to be a problem with the connection to the database:
"ERROR:root:Error connecting to MySQL on localhost"
My constructor opens the connection so I'm a bit confused as to why I see this message after the connection has been opened. I expect this is a scoping and/or GC issue that I am not understanding. The goal is to to create the Database object once and thus just have that
single connection persist throughout the life of the server, the db is stored
The following code snippet does work as expected which led me to the scoping or GC issue possibly:
#!/usr/bin/python
import torndb
class Database:
def __init__(self):
try:
self.__dbh = torndb.Connection(
'localhost',
'mydb',
user = 'myuser',
password = 'mypass')
except Exception as e:
print e
def user_add(self, user, email, passwd):
insert = "INSERT INTO users (username, email, passwd) VALUES " + \
"(%s, %s, %s)" % (user, email, passwd)
rowid = 0
try:
rowid = self.__dbh.execute(insert)
except Exception as e:
print e
if rowid is 0:
return (False, 'User exists');
return (True, None)
if __name__ == "__main__":
print 'Testing'
raw_input('Hit enter to connect to the DB')
d = Database();
users = []
raw_input('Hit enter to create some users')
for i in range(5):
users.append(str(i))
d.user_add(users[i], users[i], users[i])
<- snip ->
The issue is when I try to create a Database object from anywhere other than the main of the module that defines the Database class, for example:
import tornado.ioloop
import tornado.httpserver
import tornado.web
from register import Register
from logon import Logon
from db import Database
class Application(tornado.web.Application):
def __init__(self):
resources = [
(r"/logon", Logon),
(r"/register", Register)
]
self.db = Database()
tornado.web.Application.__init__(self, resources)
try:
self.db.user_add('testuser', 'testemail', 'password')
except Exception as e:
print e
if __name__ == "__main__":
app = Application()
# Start the server.
server = tornado.httpserver.HTTPServer(app)
server.listen(8080)
tornado.ioloop.IOLoop.instance().start()
The above when executed prints (due to the call to self.__dbh.execute()):
ERROR:root:Error connecting to MySQL on localhost
Some other bits of information:
I can connect to the db from the console without any issues.
I'm following this example https://github.com/facebook/tornado/blob/master/demos/blog/blog.py#L60
torndb version is 2.4.1, torndb version is LATEST (pulled using pip).
Questions:
Why is there a difference when I create my Database object in the main of the module that defines the class compared to creating the Database object anywhere else?
The problem was due to the string arguments passed to the query not being escaped, with the following change it works:
def user_add(self, user, email, passwd):
insert = "INSERT INTO users (username, email, passwd) VALUES " + \
"(\'%s\', \'%s\', \'%s\')" % (user, email, passwd)
rowid = 0
try:
rowid = self.__dbh.execute(insert)
except Exception as e:
print e
if rowid is 0:
return (False, 'User exists');
return (True, None)
Related
I'm having difficulties connecting to my Neo4j Database in VS Code.
I started with the boiler plate code that is given on the Neo4J documentation:
from neo4j import GraphDatabase
import logging
from neo4j.exceptions import ServiceUnavailable
class Neo4jConnection:
def __init__(self, uri, user, pwd):
self.__uri = uri
self.__user = user
self.__pwd = pwd
self.__driver = None
try:
self.__driver = GraphDatabase.driver(self.__uri, auth=(self.__user, self.__pwd))
except Exception as e:
print("Failed to create the driver:", e)
def close(self):
if self.__driver is not None:
self.__driver.close()
def query(self, query, db=None):
assert self.__driver is not None, "Driver not initialized!"
session = None
response = None
try:
session = self.__driver.session(database=db) if db is not None else self.__driver.session()
response = list(session.run(query))
except Exception as e:
print("Query failed:", e)
finally:
if session is not None:
session.close()
return response
Then I connected to my database:
conn = Neo4jConnection(uri="neo4j+s://7022d007.databases.neo4j.io", user="neo4j", pwd="****")
Then I attempted to call for neo4j to run a task in the database:
query_string = '''
CALL db.schema.visualization()
'''
conn.query(query_string, db='MARA')
Which then failed and gave me the error:
Unable to retrieve routing information
Query failed: Unable to retrieve routing information
This could be due to the certificate error. It worked for me after changing the certificate to self signed SSL. You could try using neo4j+ssc://{IP_address}:{Port} as the link to DB.
I am developing a web app using Angular 9 for frontend and Flask in python 3.8 for backend, everything was going well till i tried to connect the backend with the data base, where the code apparently its ok, flask start running succesfully but when i tried to use my endpoint for authentication, flask throws the error:
**local variable 'conn' referenced before assignment**
i have been checking some forums and so on but i dont understand what is going on. Thanks in advance for your help.
import pymysql
database_name = "dbx80"
database_user = "user_auth"
database_password = "Jan2019#"
def conection_database(db):
try:
conn = (pymysql.connect(
unix_socket="/cloudsql/servicisEX:us-central1:dbx43",
port = 3306,
user = database_user,
password = database_password,
database = database_name,
charset="utf8"
))
print(f"Connection {database_name} Succesful")
except Exception as e:
print(f"Error connecting to {database_name}")
return conn
Converting comment to an answer.
return conn is called no matter the result of the try except block.
If you put the return statement inside the try block, the error will not occur.
Example:
import pymysql
database_name = "dbx80"
database_user = "user_auth"
database_password = "Jan2019#"
def conection_database(db):
try:
conn = (pymysql.connect(
unix_socket="/cloudsql/servicisEX:us-central1:dbx43",
port = 3306,
user = database_user,
password = database_password,
database = database_name,
charset="utf8"
))
print(f"Connection {database_name} Succesful")
return conn
except Exception as e:
print(f"Error connecting to {database_name}")
I have created a test database called test inside it has a table called testTable with an autoincrement id value and a name field that takes a varchar(30).
The PREPARE statement queries (4 of them) execute fine when copied into phpmyadmin but I get the error 👍 2021-01-08 18:26:53,022 (MainThread) [ERROR] (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET\n #name = 'fred';\nEXECUTE\n statement USING #name;\nDEALLOCATE\nPREPARE\n ' at line 5")
The test code:
import pymysql
import logging
class TestClass():
def __init__(self):
# mysqlconnections
self.mySQLHostName = "localhost"
self.mySQLHostPort = 3306
self.mySQLuserName = "userName"
self.mySQLpassword = "pass"
self.MySQLauthchandb = "mysql"
def QueryMYSQL (self, query):
try:
#logging.info("QueryMYSQL : " + str( query)) # Uncomment to print all mysql queries sent
conn = pymysql.connect(host=self.mySQLHostName, port=self.mySQLHostPort, user=self.mySQLuserName, passwd=self.mySQLpassword, db=self.MySQLauthchandb, charset='utf8')
conn.autocommit(True)
cursor = conn.cursor()
if cursor:
returnSuccess = cursor.execute(query)
if cursor:
returnValue = cursor.fetchall()
#logging.info ("return value : " + str(returnValue)) # Uncomment to print all returned mysql queries
if cursor:
cursor.close()
if conn:
conn.close()
return returnValue
except Exception as e:
logging.error("Problem in ConnectTomySQL")
logging.error(query)
logging.error(e)
return False
# Default error logging log file location:
logging.basicConfig(format='%(asctime)s (%(threadName)-10s) [%(levelname)s] %(message)s', filename= 'ERROR.log',filemode = "w", level=logging.DEBUG)
logging.info("Logging Started")
test = TestClass()
result = test.QueryMYSQL("Describe test.testTable")
print(result)
query = """
PREPARE
statement
FROM
'INSERT INTO test.testTable (id, name) VALUES (NULL , ?)';
SET
#name = 'fred';
EXECUTE
statement USING #name;
DEALLOCATE
PREPARE
statement;
"""
result = test.QueryMYSQL(query)
print(result)
I'm assuming this is a library issue rather than a mysql issue? I am trying to use prepared statements to prevent code injection from user input as I understand this prepared statements are the best way to do this rather than trying to pre filter user input and missing something.
I asked this question on the github but one of the authors (methane Inada Naoki) replied with this:
========
Multistatement can be used by attacker when there is a query injection vulnerability. So it is disabled by default.
as I understand this prepared statements are the best way
You are totally wrong. Your use of prepared statement doesn't protect you from SQL injection at all. If you enable multistatement, your "prepared statement" can be attacked by SQL injection.
But I am not free tech support nor free teacher for you. OSS maintainers are not. Please don't ask here.
and he closed the issue.
Is he correct?
The author book I am reading Robin Nixon,"Learning PHP, MySQL and JavaScript" O'Reilly 5th edition. He appears to be under the misconception and I quote "Let me introduce the best and recommended way to interact with MySQL, which is pretty much bulletproof in terms of Security" Its in the Using Placeholders section pg 260. Is he wrong?
Because I bought this book to improve my security practices and now I'm not sure what is correct.
I found out from the developer of pymysql that the library does not support the PREPARE mysql statement. Also the pymysql library by default does not execute multi-statements.
I understand that my first attempt at substituting values into the INSERT statement is inherently unsafe if multi-statements are enabled. This can be done by using the client_flag=pymysql.constants.CLIENT.MULTI_STATEMENTS in the connect constructor.
The pymysql library does however allow for placeholders to be used in MySQL queries using the cursor.execute(query, (tuple)) method.
To demonstrate this I wrote the following test code example.
import pymysql
import logging
class TestClass():
def __init__(self):
# mysqlconnections
self.mySQLHostName = "localhost"
self.mySQLHostPort = 3306
self.mySQLuserName = "name"
self.mySQLpassword = "pw"
self.MySQLauthchandb = "mysql"
def QueryMYSQL (self, query, data = ()):
try:
logging.info("QueryMYSQL : " + str( query)) # Uncomment to print all mysql queries sent
conn = pymysql.connect(host=self.mySQLHostName, port=self.mySQLHostPort, user=self.mySQLuserName, passwd=self.mySQLpassword, db=self.MySQLauthchandb, charset='utf8', client_flag=pymysql.constants.CLIENT.MULTI_STATEMENTS) #code injection requires multistatements to be allowed this is off in pymysql by default and has to be set on manually.
conn.autocommit(True)
cursor = conn.cursor()
if cursor:
if data:
returnSuccess = cursor.execute(query, data)
else:
returnSuccess = cursor.execute(query)
if cursor:
returnValue = cursor.fetchall()
logging.info ("return value : " + str(returnValue)) # Uncomment to print all returned mysql queries
if cursor:
cursor.close()
if conn:
conn.close()
return returnValue
except Exception as e:
logging.error("Problem in ConnectTomySQL")
logging.error(e)
logging.error(query)
if data:
logging.error("Data {}".format(str(data)))
return False
# Default error logging log file location:
logging.basicConfig(format='%(asctime)s (%(threadName)-10s) [%(levelname)s] %(message)s', filename= 'ERROR.log',filemode = "w", level=logging.DEBUG)
logging.info("Logging Started")
def usePlaceholder(userInput):
query = "INSERT INTO test.testTable (id, name) VALUES (NULL , %s)"
data = (userInput,)
result = test.QueryMYSQL(query,data)
print(result)
def useSubstitution(userInput):
query = "INSERT INTO test.testTable (id, name) VALUES (NULL , '{}')".format(userInput) # this is unsafe.
result = test.QueryMYSQL(query)
print(result)
test = TestClass()
#Create the test database and testTable.
query = "CREATE DATABASE test"
test.QueryMYSQL(query)
query = "CREATE TABLE `test`.`testTable` ( `id` INT NOT NULL AUTO_INCREMENT , `name` VARCHAR(256) NULL DEFAULT NULL , PRIMARY KEY (`id`)) ENGINE = InnoDB;"
test.QueryMYSQL(query)
#Simulated user input.
legitUserEntry = "Ringo"
injectionAttempt = "333' ); INSERT INTO test.testTable (id, name) VALUES (NULL , 666);#" #A simulated user sql injection attempt.
useSubstitution(legitUserEntry) # this will also insert Ringo - but could be unsafe.
usePlaceholder(legitUserEntry) # this will insert Ringo - but is safer.
useSubstitution(injectionAttempt) # this will inject the input code and execute it.
usePlaceholder(injectionAttempt) # this will insert the input into the database without executing the injected code.
So from this exercise, I shall henceforth improve my security by keeping multi-statements set to off (the default) AND using the placeholders and data tuple rather than substitution.
I've created a new lambda function in Python with a handler called handler. In the Configuration section, AWS requires me to put the name of the function in the form file-name.function-name (as described here).
However I have no idea what the filename is supposed to be. I've created a blank function and didn't specify a filename at any point. My function name is "MySQLTest" so I've tried various things like "MySQLTest.handler", "my-sql-test.handler", "mysqltest.handler" but none of it seems to work.
Any idea what I should put as a filename?
For info, this is the test code I'm using:
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host = "*******"
# "rds-instance-endpoint"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
port = 3306
logger = logging.getLogger()
logger.setLevel(logging.INFO)
server_address = (rds_host, port)
try:
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
sys.exit()
logger.info("SUCCESS: Connection to RDS mysql instance succeeded")
def handler(event, context):
"""
This function fetches content from mysql RDS instance
"""
item_count = 0
try:
with conn.cursor() as cur:
cur.execute("create table Employee3 ( EmpID int NOT NULL, Name varchar(255) NOT NULL, PRIMARY KEY (EmpID))")
cur.execute('insert into Employee3 (EmpID, Name) values(1, "Joe")')
cur.execute('insert into Employee3 (EmpID, Name) values(2, "Bob")')
cur.execute('insert into Employee3 (EmpID, Name) values(3, "Mary")')
cur.execute("select * from Employee3")
for row in cur:
item_count += 1
logger.info(row)
#print(row)
finally:
conn.close()
return "Added %d items from RDS MySQL table" %(item_count)
If you are adding the function via the AWS Console the name you supply should be <name of lambda function>.handler (assuming you called your Python function handler.
So, if the name of my Lambda function is fooBar and my Python function is called handler I would use fooBar.handler.
I have run into another problem now, with below given code i am unable to login, it still says Access is Denied, even though the username and password is correct .No errors on the console. Looks like i am missing something after the connection.
All i am trying to do here is modify the TODO section so that it runs the computed query against my Oracle database after connecting and a successful login, it should show the results from permissions table.
import cgi
import cx_Oracle
print("Content-type: text/html\n")
print("<title>Test</title>")
print("<body><center>")
try:
# get post data
form = cgi.FieldStorage()
name = form['name'].value if 'name' in form else ''
pwd = form['pwd'].value if 'pwd' in form else ''
permissions = []
# query to check password and get permissions
query = "SELECT PERMISSIONS FROM USERS WHERE NAME='{}' and PWD='{}'".format(name, pwd)
# TODO: connect to database and run query
host = '123.abc.com'
port = 1521
SID = 'orcl'
dsn_tns = cx_Oracle.makedsn(host, port, SID)
connection = cx_Oracle.connect('abuser', 'userpass', dsn_tns)
curs = connection.cursor()
result = curs.execute(query)
# TODO section ends
if len(permissions) > 0:
print("<H1>Access granted. You have the following permissions: {}.</H1>".format(permissions[0][0]))
else:
print("<H1>Access denied.</H1>")
connection.close()
except cx_Oracle.DatabaseError as e:
# for ease of debugging
print("Database Error: {}".format(e))
print("<br>Query: {}".format(query))
print("""
<form action="../login.html" method="GET">
<input type="submit" value="Back to Login">
</form>
""")
print('</center></body>')
Your indent is too great starting on line 36 (I guess you uploaded partial source code), where it starts host =
There is an extra indent in the line starting with host. Indents in Python usually follow the :. The below code should fix the indentation error that you are getting. More about this here - http://www.diveintopython.net/getting_to_know_python/indenting_code.html
import cgi
import cx_Oracle
print("Content-type: text/html\n")
print("<title>Test</title>")
print("<body><center>")
try:
# get post data
form = cgi.FieldStorage()
name = form['name'].value if 'name' in form else ''
pwd = form['pwd'].value if 'pwd' in form else ''
permissions = []
# query to check password and get permissions
query = "select permissions from users where name='{}' and pwd='{}'".format(name, pwd)
# Connect to database and run query
host = '123.abc.com'
port = 1521
SID = 'orcl'
dsn_tns = cx_Oracle.makedsn(host, port, SID)
connection = cx_Oracle.connect('abcuser', 'abcuserpassword', dsn_tns)
results = connection.execute(query)
# TODO section ends
if len(permissions) > 0:
print("<H1>Access granted. You have the following permissions: {}.</H1>".format(permissions[0][0]))
else:
print("<H1>Access denied.</H1>")
except cx_Oracle.DatabaseError as e:
# for ease of debugging
print("Database Error: {}".format(e))
print("<br>Query: {}".format(query))