I want column values as list output to be
['Repair and Maintenance - General', 'Advance salary', 'EXIM Deposit', 'Office Cleaning Expenses']
but I'm able to obtain only this type of output
[('Repair and Maintenance - General',), ('Advance salary',), ('EXIM Deposit',), ('Office Cleaning Expenses',)]
with the following code, Can anyone please help me
import mysql
import mysql.connector
import pandas as pd
conn = mysql.connector.connect(host="localhost", user="root",
passwd="Abcd11",database="entry",auth_plugin="mysql_native_password")
query= "select ledger from l_db_name"
cur=conn.cursor()
cur.execute(query)
rows=cur.fetchall()
conn.commit()
print(rows);
extract the first element of each tuple with a list comprehension
rows = cur.fetchall()
rows = [row[0] for row in rows]
Related
I am new to python and I don't really understand the sql thing that well. Currently on the 6th week of team treehouse so please bare with me here if these are noob questions.
Goal
Import CSV with stock_tickers and 5 other columns of data
Convert CSV into pandas dataframe
Import dataframe into database. If there is already the unique stock_ticker for it to not add a new row, but next to check if the data in the other 5 columns is different. If it is than update it.
Right now I can do steps #1 and #2 and half of #3. With the help on here was able to get the looping thing to work. If there is a new stock_ticker row in the csv it will add it to database. If the data changes for an existing stock_ticker it won't do any updates.
for i in range(len(df)):
try:
df[df.index == i].to_sql(name='stocks', con=conn, if_exists='append', index=False)
conn.commit()
except sqlite3.IntegrityError:
pass
Current Code looks like this
import pandas as pd
from pandas import DataFrame
from pandas import ExcelWriter
import csv
import sqlite3
### IMPORT CSV ###
stock_csv_file = pd.read_csv (r'C:\Users\home\Desktop\code_projects\FIRE_Dashboard\db\alpha_vantage_active_stocks.csv')
### CHANGING INDEX NAMES FROM CSV TO TABLE NAMES ###
df = pd.DataFrame(stock_csv_file)
df = df.rename(columns = {"symbol":"stock_ticker", "name":"stock_name", "exchange":"stock_exchange", "ipoDate":"stock_ipoDate", "delistingDate":"stock_delistingDate", "status":"stock_status"})
### UPDATING DATABSE WITH SQLITE3###
conn = sqlite3.connect('stockmarket.db')
c = conn.cursor()
insert_statement = """
INSERT INTO stocks (stock_ticker,
stock_name,
stock_exchange,
stock_ipoDate,
stock_delistingDate,
stock_status
)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT (stock_ticker) DO UPDATE
SET (stock_status)"""
for i in range(len(df)):
values = tuple(df.iloc[i])
c.execute(insert_statement, values)
The error I am getting
Traceback (most recent call last):
File "update_stock_tickers.py", line 71, in <module>
c.execute(insert_statement, values)
sqlite3.OperationalError: incomplete input
Found these posts that talk about it, but still getting lost >.<
How to use variables in SQL statement in Python?
python Datetime and SQLite
Loop through individual rows and update those rows SQLite3 Python
Any help is much appreciated.
Code after solution
import pandas as pd
from pandas import DataFrame
from pandas import ExcelWriter
import csv
import sqlite3
### IMPORT CSV ###
stock_csv_file = pd.read_csv (r'C:\Users\home\Desktop\code_projects\FIRE_Dashboard\db\alpha_vantage_active_stocks.csv')
### CHANGING INDEX NAMES FROM CSV TO TABLE NAMES ###
df = pd.DataFrame(stock_csv_file)
df = df.rename(columns = {"symbol":"stock_ticker", "name":"stock_name", "exchange":"stock_exchange", "ipoDate":"stock_ipoDate", "delistingDate":"stock_delistingDate", "status":"stock_status"})
### UPDATING DATABSE WITH SQLITE3###
conn = sqlite3.connect('stockmarket.db')
c = conn.cursor()
insert_statement = """
INSERT INTO stocks (stock_ticker,
stock_name,
stock_exchange,
stock_ipoDate,
stock_delistingDate,
stock_status
)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT (stock_ticker) DO UPDATE
SET stock_status = EXCLUDED.stock_status"""
for i in range(len(df)):
values = tuple(df.iloc[i])
c.execute(insert_statement, values)
conn.commit()
This is the ON CONFLICT clause of your query:
ON CONFLICT (stock_ticker) DO UPDATE
SET (stock_status)
This is not valid SQLite syntax. If you want to update stock_status when another row already exists with the same stock_ticker, you can use pseudo-table EXCLUDED like so:
INSERT INTO stocks (stock_ticker,
stock_name,
stock_exchange,
stock_ipoDate,
stock_delistingDate,
stock_status
)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT (stock_ticker) DO UPDATE
SET stock_status = EXCLUDED.status
I have done the following code and I would like to ask the user to input how many new records want and after to fill column by column those records.
import MySQLdb
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="Adam!977",
database="testdb1"
)
cur = mydb.cursor()
get_tables_statement = """SHOW TABLES"""
cur.execute(get_tables_statement)
tables = cur.fetchall()
table = tables(gene)
x=input("How many records you desire: ")
x
print "Please enter the data you would like to insert into table %s" %(table)
columns = []
values = []
for j in xrange(0, len(gene)):
column = gene[j][0]
value = raw_input("Value to insert for column '%s'?"%(gene[j][0]))
columns.append(str(column))
values.append('"' + str(value) + '"')
columns = ','.join(columns)
values = ','.join(values)
print columns
print values
The error that i get is about table gene( The table exist in db of SQL)
Traceback (most recent call last):
File "C:\Users\Admin\Desktop\π.py", line 25, in
table = tables(gene)
NameError: name 'gene' is not defined
Also, even I don't know if working properly the code. Please, I need help. Thank you
The error being returned by python is down to the lack of definition of a variable gene. In the following line you reference gene, without it existing:
table = tables(gene)
In the documentation for the python mysql connector, under cursor.fetchall() you'll notice that this method returns either a list of tuples or an empty list. It is therefore somewhat puzzling why you call tables as a function and attempt to pass a parameter to it - this is not correct syntax for accessing a list, or a tuple.
At the beginning of your code example you fetch a list of all of the tables in your database, despite knowing that you only want to update a specific table. It would make more sense to simply reference the name of the table in your SQL query, rather than querying all of the tables that exist and then in python selecting one. For example, the following query would give you 10 records from the table 'gene':
SELECT * FROM gene LIMIT 10
Below is an attempt to correct your code:
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
passwd="Adam!977",
database="testdb1"
)
x=input("How many records you desire: ")
cur = mydb.cursor()
get_rows_statement = """SELECT * FROM gene"""
cur.execute(get_rows_statement)
results = cur.fetchall()
This should give you all of the rows within the table.
I am trying to take a dataframe and convert it into sql. I am creating the table first to set the unique indexing to allow for a rolling update with out having duplicates if there happens to be two A. Rods over time. Though I can't seem to shake this table column error and i don't know why.
import pandas as pd
import sqlite3 as sq
conn = sq.connect('test.db')
c = conn.cursor()
def set_table():
c.execute("""CREATE TABLE IF NOT EXISTS players(
"#" INTEGER,
" " REAL,
"Named" TEXT,
"B/T" TEXT,
"Ht" TEXT,
"Wt" TEXT,
"DOB" TEXT);""")
conn.commit()
def set_index_table():
c.execute(""" CREATE UNIQUE INDEX index_unique
ON players (Named, DOB)""")
conn.commit()
set_table()
set_index_table()
roster_active = pd.read_html('http://m.yankees.mlb.com/roster',index_col=0)
df = roster_active[0]
df = df.rename(columns={'Name': 'Named'})
df.to_sql('players', conn, if_exists='append')
conn.commit()
conn.close()
sqlite3.OperationalError: table players has no column named
Thank you for your time.
So I am not completely sure why this doesn't work but I found how I could get it to work. I believe it had something to do with the dataframe index. So I defined what columns I wanted to select for the dataframe and that worked.
df = df[['Named','B/T', 'Ht','Wt','DOB']]
I have the data in pandas dataframe which I am storing in SQLITE database using Python. When I am trying to query the tables inside it, I am able to get the results but without the column names. Can someone please guide me.
sql_query = """Select date(report_date), insertion_order_id, sum(impressions), sum(clicks), (sum(clicks)+0.0)/sum(impressions)*100 as CTR
from RawDailySummaries
Group By report_date, insertion_order_id
Having report_date like '2014-08-12%' """
cursor.execute(sql_query)
query1 = cursor.fetchall()
for i in query1:
print i
Below is the output that I get
(u'2014-08-12', 10187, 2024, 8, 0.3952569169960474)
(u'2014-08-12', 12419, 15054, 176, 1.1691244851866613)
What do I need to do to display the results in a tabular form with column names
In DB-API 2.0 compliant clients, cursor.description is a sequence of 7-item sequences of the form (<name>, <type_code>, <display_size>, <internal_size>, <precision>, <scale>, <null_ok>), one for each column, as described here. Note description will be None if the result of the execute statement is empty.
If you want to create a list of the column names, you can use list comprehension like this: column_names = [i[0] for i in cursor.description] then do with them whatever you'd like.
Alternatively, you can set the row_factory parameter of the connection object to something that provides column names with the results. An example of a dictionary-based row factory for SQLite is found here, and you can see a discussion of the sqlite3.Row type below that.
Step 1: Select your engine like pyodbc, SQLAlchemy etc.
Step 2: Establish connection
cursor = connection.cursor()
Step 3: Execute SQL statement
cursor.execute("Select * from db.table where condition=1")
Step 4: Extract Header from connection variable description
headers = [i[0] for i in cursor.description]
print(headers)
Try Pandas .read_sql(), I can't check it right now but it should be something like:
pd.read_sql( Q , connection)
Here is a sample code using cx_Oracle, that should do what is expected:
import cx_Oracle
def test_oracle():
connection = cx_Oracle.connect('user', 'password', 'tns')
try:
cursor = connection.cursor()
cursor.execute('SELECT day_no,area_code ,start_date from dic.b_td_m_area where rownum<10')
#only print head
title = [i[0] for i in cursor.description]
print(title)
# column info
for x in cursor.description:
print(x)
finally:
cursor.close()
if __name__ == "__main__":
test_oracle();
I'm tring to use this code to read all temperature values from a sqlite database column, but the output is showing [(u'29',), (u'29',), (u'29',)] and I'm only storing the numeric value in the database. I would like the output to be [29, 29, 29]
import sqlite3
conn = sqlite3.connect("growll.db")
cursor = conn.cursor()
print "\nHere's a listing of all the records in the table:\n"
cursor.execute("select lchar from GrowLLDados")
print cursor.fetchall()
Try this:
import sqlite3
conn = sqlite3.connect("growll.db")
cursor = conn.cursor()
print "\nHere's a listing of all the records in the table:\n"
cursor.execute("select lchar from GrowLLDados")
print [int(record[0]) for record in cursor.fetchall()]
print [int(i[0]) for i in cursor.fetchall()]
Let me know how you get on.