from bs4 import BeautifulSoup
import requests
import sqlite3
conn = sqlite3.connect('stadiumsDB.db')
c = conn.cursor()
c.executescript('''
DROP TABLE IF EXISTS Stadium;
CREATE TABLE Stadium (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
Stadium TEXT UNIQUE,
Club Text UNIQUE,
Location Text UNIQUE
)
''')
main_site = requests.get('https://en.wikipedia.org/wiki/List_of_Premier_League_stadiums').text
soup = BeautifulSoup(main_site)
column_names = [th.getText() for th in soup.findAll('th')[:9]]
print column_names
stadium_rows = soup.findAll('tr')[1:]
stadium_data = [[td.getText() for td in stadium_rows[i].findAll('td')]
for i in range(len(stadium_rows))]
print stadium_data
I want to build a sqlite database. First row of the table will be the column name and i want them import from my column_names variable. Next rows i want them import from stadium_data variable. Any guidance please !!!!
For your table definition you can insert stadium_data best with executemany like
cn=column_names
stadium=cn.index("Stadium")
club=cn.index("Club")
location=cn.index("Location")
# this is for making sure that column order is correct
c.executemany("insert into stadium(stadium, club, location) values (?,?,?)",
((row[stadium], row[club],row[location]) for row in stadium_data))
This does not put column_names first in the table as this is doesnt seem like a good idea to me but you can do so with (insert this before the executemany)
c.execute("insert into stadium(id,stadium, club, location) values (?,?,?,?)",
(0,cn[stadium], cn[club],cn[location]))
Related
So I'm trying to work with an SQLite database at the moment and was hoping to get some input on the best way of writing a value to a particular row and column (so cell) of a database. I know how to write to the database row by row, so basically appending a row to the end of the database each time, but instead I would like to write the data into the database non sequentially.
I have put together an arbitrary example below to illustrate what I'm trying to do by using apples. In this case I create two tables in a database. The first table will be my ID table and is called apples. This will contain a primary key and two columns for the apple name and the farm it is grown in. The second table keyFeatures will contain the primary key again which refers back to the ID of the apple in the apple table, but also a column for the taste, texture and the colour.
In the example below I have only the taste, texture and colour of the apple Pink Lady from farm 3. I now want to write that information into row 3 of the table keyFeature in the relevant columns before any of the other rows are populated. For the life of me I can't work out how to do this. I assume I need to position the cursor to the correct row, but from the documentation I am not clear on how to achieve this. I'm sure that this is a trivial problem and if anyone can point me in the right direction I would greatly appreciate it!
import sqlite3
dbName = 'test.db'
################# Create the Database File ###################
# Connecting to the database file
conn = sqlite3.connect(dbName)
c = conn.cursor()
#Create the identification table with names and origin
c.execute('''CREATE TABLE apples(appleID INT PRIMARY KEY, Name TEXT,
farmGrown TEXT)''')
#Create the table with key data
c.execute('''CREATE TABLE keyFeature(appleID INT PRIMARY KEY, taste INT,
texture INT, Colour TEXT)''')
#Populate apples table and id in keyFeature table
AppleName = ['Granny Smith', 'Golden Delicious', 'Pink Lady', 'Russet']
appleFarmGrown = ['Farm 1', 'Farm 2', 'Farm 3', 'Farm 4']
id = []
for i in range(len(AppleName)):
id.append(i+1)
c = conn.cursor()
c.execute('''INSERT INTO apples(appleID, Name, farmGrown)
VALUES(?,?,?)''', (id[i], AppleName[i], appleFarmGrown[i]))
c.execute('''INSERT INTO keyFeature(appleID)
VALUES(?)''', (id[i],))
#Current Apple to populate row in keyFeature
appleCurrent = (('Pink Lady','Farm 3'))
tasteCurrent = 4
textureCurrent = 5
colourCurrent = 'red'
#Find ID and write into the database
c.execute("SELECT appleID FROM apples")
appleIDDB = c.fetchall()
c.execute("SELECT name FROM apples")
nameDB = c.fetchall()
c.execute("SELECT farmGrown FROM apples")
farmGrownDB = c.fetchall()
conn.commit()
conn.close()
# I assume that if I close the connection the cursor whould be positioned at the
# first row again but this doesn't appear to be the case
conn = sqlite3.connect(dbName)
for i in range(len(appleIDDB)):
c = conn.cursor()
if ((nameDB[i][0] == appleCurrent[0]) and (farmGrownDB[i][0] == appleCurrent[1])):
idCurrent = appleIDDB[i][0]
print("This apple matches the apple stored with id number " +str(idCurrent))
# This writes into the fifth row of the table
c.execute('''INSERT INTO keyFeature(taste, texture, Colour)
VALUES(?,?,?)''', (tasteCurrent, textureCurrent, colourCurrent))
conn.commit()
conn.close()
You're almost there.
A relational database such as sqlite isn't quite like a table in a speadsheet. Instead of a list of rows with a certain order you just have "a big bag of rows" (technically called a set of tuples) and you can sort them any way you want.
The way we solve your problem, as you've already identified when you created the table, is to make sure every row has a key that allows us to identify it (like your apple ID). When we want this key to represent an ID in another table, that's called a foreign key. So we just need to add a foreign key (called appleID) to the keyFeature table, and use that whenever we add a row to it.
First, get rid of this from your first for loop, we don't need it at this stage, the table can sit empty.
c.execute('''INSERT INTO keyFeature(appleID)
VALUES(?)''', (id[i],))
Next, you don't need to get the whole table to find the apple you want, you can just select the one you are interested in:
c.execute("SELECT appleID FROM apples WHERE name=? AND farm=?",("Pink Lady", "Farm 3"))
idCurrent = c.fetchone()[0]
The real trick is, when adding data to keyFeature, we have to insert all the data in one statement. This way a new tuple (row) is created with the ID and all the other information all at once. As if it were in "the right place" in the table.
c.execute('''INSERT INTO keyFeature(appleID, taste, texture, Colour)
VALUES(?,?,?,?)''', (idCurrent, tasteCurrent, textureCurrent, colourCurrent))
Now we can retrieve information from the keyFeature table using the ID of the apple we're interested in.
c.execute("SELECT taste, texture, Colour FROM keyFeature WHERE apple_id=?", (my_apple_id,))
Final complete code:
import sqlite3
dbName = 'test.db'
################# Create the Database File ###################
# Connecting to the database file
conn = sqlite3.connect(dbName)
c = conn.cursor()
#Create the identification table with names and origin
c.execute('''CREATE TABLE apples(appleID INT PRIMARY KEY, Name TEXT,
farmGrown TEXT)''')
#Create the table with key data
c.execute('''CREATE TABLE keyFeature(appleID INT PRIMARY KEY, taste INT,
texture INT, Colour TEXT)''')
#Populate apples table and id in keyFeature table
AppleName = ['Granny Smith', 'Golden Delicious', 'Pink Lady', 'Russet']
appleFarmGrown = ['Farm 1', 'Farm 2', 'Farm 3', 'Farm 4']
id = []
for i in range(len(AppleName)):
id.append(i+1)
c = conn.cursor()
c.execute('''INSERT INTO apples(appleID, Name, farmGrown)
VALUES(?,?,?)''', (id[i], AppleName[i], appleFarmGrown[i]))
#Current Apple to populate row in keyFeature
appleCurrent = ('Pink Lady','Farm 3')
tasteCurrent = 4
textureCurrent = 5
colourCurrent = 'red'
#Find ID and write into the database
c.execute("SELECT appleID FROM apples WHERE name=? AND farm=?",(appleCurrent[0], appleCurrent[1]))
idCurrent = c.fetchone()[0]
c.execute('''INSERT INTO keyFeature(appleID, taste, texture, Colour)
VALUES(?,?,?,?)''', (idCurrent, tasteCurrent, textureCurrent, colourCurrent))
conn.commit()
conn.close()
Good, what happens is that I have two tables in the same database, the first table I will call patient, the second appointment .... both have the same column that is "cc" .... I look for a date, That match my table in quotes and grab the "cc", then go to the patient table and bring the name in such a way that I print name + cc + date ...... what worries me is how I make that link Between the tables with python, attached images to see the database and part of the code of which I try to join and print the matches of "cc".
Thank you for your cooperation.
Data from the first table
Data from the second table
You didn't mention what library you are/intend to use for MySQL. I will assume pymssql. Here is a simple example to get you started based off their documentation and Hatik's query.
import pymssql
conn = pymssql.connect("localhost", "admin", "password", "database")
cursor = conn.cursor()
cursor.execute("""
SELECT B.NAME, B.CC, C.DATE FROM
APPOINTMENT A LEFT JOIN PATIENT B ON A.CC = B.CC
""")
row = cursor.fetchone()
while row:
print row
row = cursor.fetchone()
conn.close()
I am trying to take a dataframe and convert it into sql. I am creating the table first to set the unique indexing to allow for a rolling update with out having duplicates if there happens to be two A. Rods over time. Though I can't seem to shake this table column error and i don't know why.
import pandas as pd
import sqlite3 as sq
conn = sq.connect('test.db')
c = conn.cursor()
def set_table():
c.execute("""CREATE TABLE IF NOT EXISTS players(
"#" INTEGER,
" " REAL,
"Named" TEXT,
"B/T" TEXT,
"Ht" TEXT,
"Wt" TEXT,
"DOB" TEXT);""")
conn.commit()
def set_index_table():
c.execute(""" CREATE UNIQUE INDEX index_unique
ON players (Named, DOB)""")
conn.commit()
set_table()
set_index_table()
roster_active = pd.read_html('http://m.yankees.mlb.com/roster',index_col=0)
df = roster_active[0]
df = df.rename(columns={'Name': 'Named'})
df.to_sql('players', conn, if_exists='append')
conn.commit()
conn.close()
sqlite3.OperationalError: table players has no column named
Thank you for your time.
So I am not completely sure why this doesn't work but I found how I could get it to work. I believe it had something to do with the dataframe index. So I defined what columns I wanted to select for the dataframe and that worked.
df = df[['Named','B/T', 'Ht','Wt','DOB']]
I'm trying to accomplish a very simple task:
Create a table in SQLite
Insert several rows
Query a single column in the table and pull back each row
Code to create tab:
import sqlite3
sqlite_file = '/Users/User/Desktop/DB.sqlite'
conn = sqlite3.connect(sqlite_file)
c = conn.cursor()
c.execute('''CREATE TABLE ListIDTable(ID numeric, Day numeric, Month
numeric, MonthTxt text, Year numeric, ListID text, Quantity text)''')
values_to_insert = [
(1,16,7,"Jul",2015,"XXXXXXX1","Q2"),
(2,16,7,"Jul",2015,"XXXXXXX2","Q2"),
(3,14,7,"Jul",2015,"XXXXXXX3","Q1"),
(4,14,7,"Jul",2015,"XXXXXXX4","Q1")] #Entries continue similarly
c.executemany("INSERT INTO ListIdTable (ID, Day, Month, MonthTxt,
Year, ListID, Quantity) values (?,?,?,?,?,?,?)", values_to_insert)
conn.commit()
conn.close()
When I look at this table in SQLite DB Browser, everything looks fine.
Here's my code to try and query the above table:
import sqlite3
sqlite_file = '/Users/User/Desktop/DB.sqlite'
conn = sqlite3.connect(sqlite_file)
conn.row_factory = sqlite3.Row
c = conn.cursor()
for row in c.execute('select * from ListIDTable'):
r = c.fetchone()
ID = r['ID']
print (ID)
I should get a print out of 1, 2, 3, 4.
However, I only get 2 and 4.
My code actually uploads 100 entries to the table, but still, when I query, I only get ID printouts of even numbers (i.e. 2, 4, 6, 8 etc.).
Thanks for any advice on fixing this.
You don't need to fetchone in the loop -- The loop is already fetching the values (one at a time). If you fetchone while you're iterating, you'll only see half the data because the loop fetches one and then you immediately fetch the next one (without ever looking at the one that was fetched by the loop):
for r in c.execute('select * from ListIDTable'):
ID = r['ID']
print (ID)
I have a small database which is legacy from an almost defunct project. This database has a "Patients" table with individual personal data and an unique "Id" field, and an "Exams" table with some fields for each exam, one of these fields being "Patient_Id".
What I want is, for a given patient (row from "Pacientes" table) the exams (rows) from "Exames" table whose "Patient_Id" matches that of the given patient.
I am very beginner with SQL, so if the question is very naive, I apologize.
My working code is the following, but I am using a for loop while I would much rather have a SQL query (which is the very point of using databases, I think...):
#!/usr/bin/env python
# coding: utf-8
import os, sqlite3
conn = sqlite3.connect('BDdata.db3')
conn.row_factory = sqlite3.Row
c = conn.cursor()
c.execute('SELECT * FROM Exames')
exams = c.fetchall()
c.execute('SELECT * FROM Pacientes')
for row in c:
Nome = row['nome']
ID = row['Id']
for exam in exams: # I would like to replace this loop
if exam['Id_Paciente'] == ID: # with a SQL query meaning
print exam['File']
============================
An answer to a similar question seems to be what I want, but I have no idea how to do this in Python with sqlite3 module, much less what in this expression is essential and what is incidental, or what is the syntax structure:
Selecting rows from a table by One a field from other table
SELECT i.prof_image
FROM profile_images i
WHERE cat_id = (select max(cat_id) from images_cat)
I think the following should do what you want:
...
c = conn.cursor()
c.execute('SELECT * FROM Pacientes')
for row in c.fetchall():
Nome = row['nome']
ID = row['Id']
c.execute('SELECT File FROM Exames WHERE Id_Paciente=?', [ID])
for exam in c:
print exam['File']