how to populate database using python faker - python

I am trying to populate a table using python faker and I am getting this error . Here is my code
import psycopg2
from faker import Faker
fake = Faker()
conn = psycopg2.connect(database="testdb", user="****", password="****", host="127.0.0.1", port="5432")
print "Opened database successfully"
cur = conn.cursor()
for i in range (10):
Id =fake.random_digit_not_null()
name = fake.name()
age=fake.random_number(digits=None)
adress =fake.address()
salary = fake.random_int(min=0, max=9999)
cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) \
VALUES (Id,name,age,adress,salary)");
conn.commit()
print "Records created successfully";
conn.close()
here is the error
Traceback (most recent call last):
File "fakegenerator.py", line 16, in <module>
VALUES (Id,name,age,adress,salary)");
psycopg2.ProgrammingError: column "id" does not exist
LINE 1: ...OMPANY (ID,NAME,AGE,ADDRESS,SALARY) VALUES (Id,name,ag...
^
HINT: There is a column named "id" in table "company", but it cannot be referenced from this part of the query.

You're not filling in the values into your query, instead you're sending the string as-is to the database. This would actually fill your query with values:
cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) VALUES (%s, %s, %s, %s, %s)", (Id, name, age, adress, salary));
This wraps the variables filled with the values you want to insert into a tuple and let's psycopg2 handle quoting your strings correctly which is less work for you and keeps you safe from SQL injection, should you use your code as a base for productive code. This is documented in the module's documentation.

the sql request in cur.execute seems to be a problem try this
cur.execute("INSERT INTO COMPANY (ID,NAME,AGE,ADDRESS,SALARY) \
VALUES ({},{},{},{},{})".format(Id,name,age,adress,salary));

Related

Trying to insert data into SQL database. Error 1046 No Database Selected

I have some data in a tab-delimited file that i'm trying to insert into an SQL database. Here is my code
import csv
import MySQLdb
new_db = MySQLdb.connect(host='localhost',user='me')
cursor = new_db.cursor()
cursor.execute('create database if not exists my_database')
maketablestr = """CREATE TABLE info (Disease VARCHAR(10), NCBI Acc. Number VARCHAR(10) ,Length VARCHAR(10), Sequence VARCHAR(10));"""
cursor.execute(maketablestr)
new_db.commit()
tsv_read = csv.reader(file('marfan.tsv'), delimiter = '\t')
for row in tsv_read:
cursor.execute('INSERT INTO info(Disease, NCBI Acc. Number, Length, Sequence ) VALUES(%s, %s, %s, %s)', row)
new_db.commit()
cursor.close()
new_db.close()
print("Database updated")
When I run the code, it gives me the error 1046 'No Database Selected'. I'm a little confused as to why I'm getting this error since the code that I wrote is mostly taken from others who are trying to do the same thing as me.
There are two approaches to solve this:
You run cursor.execute('use my_database;')
Or you adapt your SQL statements to specify the DB like:
cursor.execute('INSERT INTO my_database.info(Disease, NCBI Acc. Number, Length, Sequence ) VALUES(%s, %s, %s, %s)', row)
Note, with the later approach you need to adapt all sql statements

SQL Insert query from flask not working

db.execute('CREATE TABLE IF NOT EXISTS test (name TEXT)')
print("Table created successfully")
db.execute('INSERT INTO %s VALUES (%s)' % ('test', 'sample'))
db.close()
I am using Python. The table is created successfully with a column of "name", but I am unable to insert anything.
I get the error:
sqlite3.OperationalError: no such column: sample
Why?
I think you need something like:
INSERT INTO test (name)
VALUES
('sample');
Translated into python:
db.execute('INSERT INTO %s (name) VALUES (\'%s\')' % ('test', 'sample'))
You forgot the column name?
db.execute('CREATE TABLE IF NOT EXISTS test (name TEXT)')
print("Table created successfully")
db.execute('INSERT INTO %s (name) VALUES (%s)' % ('test', 'sample'))
db.close()
You need to commit your changes. If you don't call db.commit(), your changes (including creating the table) will be rolled back when you close the database.
This should work:
db.execute('INSERT INTO test VALUES (%s)' % ('sample'))
You definitely don't want to use string substitutions. That opens you to https://www.owasp.org/index.php/SQL_Injection. Instead, use database binding which incidentally fixes your actual error (which is that you are treating sample as a column name, rather than data).
Ok, might as well write it up:
import sqlite3
db = sqlite3.connect(':memory')
#you control this stuff, as the db schema isn't typically coming from user data
#so less likely to be a mess...
#i.e. build your query templates with string substitutions, but exec with binds.
tablename = 'test'
db.execute('CREATE TABLE IF NOT EXISTS %s (name TEXT)' % (tablename))
print("Table created successfully")
qry = 'INSERT INTO %s VALUES (?)' % (tablename)
#the data is where you want to be careful
db.execute(qry, ('sample',))
print ("insert done")
db.close()
which gives:
Table created successfully
insert done
Check out docs # https://docs.python.org/2/library/sqlite3.html Starting at # Never do this -- insecure!.

INSERT WHERE NOT EXISTS clause using python variables

I am attempting to write an SQL query to insert rows into a table when the script is run. I have been able to get it working whilst duplicating data, but am unable to set up an insert that can determine whether or not the record is unique.
I have tried following these threads:
Most efficient way to do a SQL 'INSERT IF NOT EXISTS'
Python MySQLdb / MySQL INSERT IGNORE & Checking if Ignored
Currently, I can get the system to add all records, but due to the frequency the script is ran at the database is being filled with the same records. This is my current code:
for post in r.get_subreddit("listentothis").get_new(limit=5):
if 'youtube.com' in post.url or 'youtu.be' in post.url:
print(str(post.title))
print(str(post.url))
sql = """INSERT INTO listentothis2(title,url) VALUES ('%s', '%s') WHERE NOT EXISTS (SELECT 1 FROM listentothis2 WHERE url = '%s') """ % (post.title, post.url, post.url)
try:
cursor.execute(sql)
db.commit()
except:
db.rollback()
I have also tried:
sql = """INSERT IGNORE INTO listentothis2(title,url) VALUES ('%s', '%s') """ % (post.title, post.url)
However, this adds the 3 most recent records whether or not they are commited to the database already. Any help is greatly appreciated!

How do I insert data into table?

I have created table using this create command as:
CREATE TABLE test_table(id INT PRIMARY KEY,name
VARCHAR(50),price INT)
i want to insert into this table wherein values are stored already in variable
bookdb=# name = 'algorithms'
bookdb-# price = 500
bookdb-# INSERT INTO test_table VALUES(1,'name',price);
I get the following error:
ERROR: syntax error at or near "name"
LINE 1: name = 'algorithms'
Can anyone point out the mistake and propose solution for the above?
Thanks in advance
Edit:
import psycopg2
import file_content
try:
conn = psycopg2.connect(database='bookdb',user='v22')
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS book_details")
cur.execute("CREATE TABLE book_details(id INT PRIMARY KEY,name VARCHAR(50),price INT)")
cur.execute("INSERT INTO book_details VALUES(1,'name',price)")
conn.commit()
except:
print "unable to connect to db"
I have used the above code to insert values into table,variables name and price containing the values to be inserted into table are available in file_content python file and i have imported that file.The normal INSERT statement takes values manually but i want my code to take values which are stored in variables.
SQL does not support the concept of variables.
To use variables, you must use a programming language, such as Java, C, Xojo. One such language is PL/pgSQL, which you can think of as a superset of SQL. PL/PgSQL is often bundled as a part of Postgres installers, but not always.
I suggest you read some basic tutorials on SQL.
See this similar question: How do you use script variables in PostgreSQL?
don't have postgres installed here, but you can try this
import psycopg2
import file_content
try:
conn = psycopg2.connect(database='bookdb',user='v22')
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS book_details")
cur.execute("CREATE TABLE book_details(id INT PRIMARY KEY,name VARCHAR(50),price INT)")
cur.execute("INSERT INTO book_details VALUES(1, '%s', %s)" % (name, price))
conn.commit()
except:
print "unable to connect to db"
If you are using PSQL console:
\set name 'algo'
\set price 10
insert into test_table values (1,':name',:price)
\g

What's the best way to insert data from dbfpy into MySQL?

I'm using the dbfpy module to read data from DBF files with the intention of writing that same data to equivalent MySQL tables. Here's a rough version of my code:
###################
# BEGIN CONFIG
###################
import_root = '/Users/shawn/Dropbox/ITistic Inc/Digital Aspire/Clients/MVP/Automated Sales Report Project/pdq dbf export 1.30.2013'
concept_id = 1
###################
# END CONFIG
###################
import os
import datetime
from dbfpy import dbf
import MySQLdb
# Connect to MySQL
db = MySQLdb.connect('localhost', 'dbposireporting', 'posi', 'dbposireporting')
cur = db.cursor()
discount = dbf.Dbf(os.path.join(path, 'DISCOUNT.DBF'))
for rec in discount:
print rec['date']
print
# LINE BELOW NOT WORKING:
cur.execute("INSERT INTO discount VALUES (%s, %s, %s, %s, %s, %s)", rec)
discount.close()
db.close()
The MySQL table I'm trying to insert into contains one additional field which I need to populate with the concept_id value set at the top of the script. That value is not part of the DBF records (rec). What's the best way for me to insert this data?
I would think you could replace rec in the cur.execute() line with
tuple(rec) + (concept_id,)
Oh, and don't forget to add one more %s for it.
If any of the fields are strings, you'll need to wrap them in quotes for the SQL to be valid.

Categories

Resources