I am creating a database from different CSV files. After doing this I have tried to define the primary key table by table but I got an error.
c.execute("ALTER TABLE patient_data ADD PRIMARY KEY (ID);").fetchall()
OperationalError: near "PRIMARY": syntax error
Maybe the best thing to avoid this error is to define the primary key when the table is create but I dont know how to do that. I have been working with python for a few years but today is my first approach with SQL.
This is the code I use to import a CSV to a table
c.execute('''DROP TABLE IF EXISTS patient_data''')
c.execute(''' CREATE TABLE patient_data (ID, NHS_Number,Full_Name,Gender, Birthdate, Ethnicity, Postcode)''')
patients_admitted.to_sql('patient_data', conn, if_exists='append', index = False)
c.execute('''SELECT * FROM patient_data''').fetchall()
This is too long for a comment.
If your table does not have data, just re-create it with the primary key definition.
If your table does have data, you cannot add a primary key in one statement. Why not? The default value is either NULL or constant. And neither is allowed as a primary key.
And finally, SQLite does not allow you to add a primary key to an existing table. The solution is to copy the data to another table, recreate the table with the structure you want, and then copy the data back in.
Related
I am trying to write some data to a table in a database which I am creating.
However, I am facing with an integrity error like:
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) PRIMARY KEY must be unique
My question is how to avoid these errors as I will run a couple of times the script
Basically you are creating an object with an already existing primary key, and it's not accepted by SQLite. Verify it by querying the db with something like
select * from airport where id = 6256
If the query returns a result, you need to change the id of the airport you are saving. Since you use the autoincrement, you don't need to specify an id and the DBMS will assign the next free id in that table.
I need to create a copy of a table (if it does not already exist), and this copy needs one of the existing columns to become a primary key.
Currently I have the following SQL commands (in a Python script):
CREATE TABLE IF NOT EXISTS FilteredTable AS SELECT * FROM RawTable WHERE <cond>
ALTER TABLE FilteredTable ADD COLUMN IF NOT EXISTS (NewCol1 TEXT, etc TEXT)
ALTER TABLE FilteredTable ADD PRIMARY KEY (ColumnNameFromRaw)
This works fine the first time around, but of course when run again after the table already exists it complains that a primary key already exists. Is there a way to combine the ADD PRIMARY KEY with the CREATE TABLE ... command so that it only attempts to set the primary key when the table needs to be created?
EDIT
Python code can be used.
SQL: 10.1.23-MariaDB
Python: 2.7
You can add extra columns and indexes in the CREATE TABLE command even when coming from a SELECT. So do it all in one step:
CREATE TABLE IF NOT EXISTS FilteredTable (
NewCol1 TEXT, etc TEXT,
PRIMARY KEY (ColumnNameFromRaw)
) AS SELECT * FROM RawTable WHERE <cond>
Note: the column names are matched up by name. That is, if * has a NewCol1, it will go into that TEXT column. If not, then NewCol1 will be empty.
This isn't quite as clean as a purely SQL solution, but it works in the event that you're accessing SQL indirectly like so...
# ...
# (New table creation)
# ...
# Display column data for specified table
curs.execute("SHOW COLUMNS FROM NewTable")
# Collect the data as a tuple of tuples containing chart rows
colsNew = curs.fetchall()
# Iterate through each table column (row in the chart)
for col in colsNew:
# Check to see if Key is not "PRI" for the Field (Column) "DesiredColumn"
if (col[0] == "DesiredColumn") and (col[3] != "PRI"):
# Make DesiredColumn the primary key
curs.execute("ALTER TABLE NewTable ADD PRIMARY KEY (DesiredColumn)")
I'm working with sqlite3 on python 2.7 and I am facing a problem with a many-to-many relationship. I have a table from which I am fetching its primary key like this
current.execute("SELECT ExtensionID FROM tblExtensionLookup where ExtensionName = ?",[ext])
and then i am fetching another primary key from another table
current.execute("SELECT HostID FROM tblHostLookup where HostName = ?",[host])
now what i am doing is i have a third table with these two keys as foreign keys and i inserted them like this
current.execute("INSERT INTO tblExtensionHistory VALUES(?,?)",[Hid,Eid])
The problem is i don't know why but the last insertion is not working it keeps giving errors. Now what i have tried is:
First I thought it was because I have an autoincrement primary id for the last mapping table which I didn't provide, but isn't it supposed to consider itself as it's auto incremented? However I went ahead and tried adding Null,None,0 but nothing works.
Secondly I thought maybe because i'm not getting the values from tables above so I tried printing it out and it shows so it works.
Any suggestions what I am doing wrong here?
EDIT :
When i don't provide primary key i get error as
The table has three columns but you provided only two values
and when i do provide them as None,Null or 0 it says
Parameter 0 is not supported probably because of unsupported type
I tried implementing the #abarnet way but still keeps saying parameter 0 not supported
connection = sqlite3.connect('WebInfrastructureScan.db')
with connection:
current = connection.cursor()
current.execute("SELECT ExtensionID FROM tblExtensionLookup where ExtensionName = ?",[ext])
Eid = current.fetchone()
print Eid
current.execute("SELECT HostID FROM tblHostLookup where HostName = ?",[host])
Hid = current.fetchone()
print Hid
current.execute("INSERT INTO tblExtensionHistory(HostID,ExtensionID) VALUES(?,?)",[Hid,Eid])
EDIT 2 :
The database schema is :
table 1:
CREATE TABLE tblHostLookup (
HostID INTEGER PRIMARY KEY AUTOINCREMENT,
HostName TEXT);
table2:
CREATE TABLE tblExtensionLookup (
ExtensionID INTEGER PRIMARY KEY AUTOINCREMENT,
ExtensionName TEXT);
table3:
CREATE TABLE tblExtensionHistory (
ExtensionHistoryID INTEGER PRIMARY KEY AUTOINCREMENT,
HostID INTEGER,
FOREIGN KEY(HostID) REFERENCES tblHostLookup(HostID),
ExtensionID INTEGER,
FOREIGN KEY(ExtensionID) REFERENCES tblExtensionLookup(ExtensionID));
It's hard to be sure without full details, but I think I can guess the problem.
If you use the INSERT statement without column names, the values must exactly match the columns as given in the schema. You can't skip over any of them.*
The right way to fix this is to just use the column names in your INSERT statement. Something like:
current.execute("INSERT INTO tblExtensionHistory (HostID, ExtensionID) VALUES (?,?)",
[Hid, Eid])
Now you can skip any columns you want (as long as they're autoincrement, nullable, or otherwise skippable, of course), or provide them in any order you want.
For your second problem, you're trying to pass in rows as if they were single values. You can't do that. From your code:
Eid = current.fetchone()
This will return something like:
[3]
And then you try to bind that to the ExtensionID column, which gives you an error.
In the future, you may want to try to write and debug the SQL statements in the sqlite3 command-line tool and/or your favorite GUI database manager (there's a simple extension that runs in for Firefox if you don't want anything fancy) and get them right, before you try getting the Python right.
* This is not true with all databases. For example, in MSJET/Access, you must skip over autoincrement columns. See the SQLite documentation for how SQLite interprets INSERT with no column names, or similar documentation for other databases.
I'm writing a python script that would reset the database to an initial state (some hardcoded entries in every table). The db consists of multiple tables with primary and foreign keys.
Every time I would run the script, it should remove all the old entries in all of the tables, reset the primary key counter and insert the sample entries.
Currently I am trying to achieve this like this:
# Delete all the entries from the tables
cursor.execute("DELETE FROM table1")
cursor.execute("DELETE FROM table2")
cursor.execute("DELETE FROM table3")
# Reset the primary key counter and insert sample entries
cursor.execute("ALTER TABLE table1 AUTO_INCREMENT = 1")
cursor.execute("INSERT INTO table1(username, password) VALUES('user01', '123')")
cursor.execute("ALTER TABLE table2 AUTO_INCREMENT = 1")
cursor.execute("INSERT INTO table2(column1, column2) VALUES('column1_data', 'column2_data')")
This isn't working due to the presence of foreign keys in some tables (it won't let me delete them).
I generate the tables using a models.py script (I also use Django), so I thought I could solve this the following way:
remove the database programatically and create a new one with the same name
call the models.py script to generate empty tables in the db
insert sample data using the script I wrote
Is this a good solution or am I overlooking something?
I use scripts monthly to purge a transaction table, after archiving the contents.
Try using the 'truncate' command, ie.
truncate table [tablename];
It resets the counter (auto-increment) for primary key, automatically.
Then use your insert statements to populate base info.
Also, this preserves all of the table base settings (keys,indexes,.).
I am trying to add an 'id' primary key column to an already existing MySQL table using alembic. I tried the following...
op.add_column('mytable', sa.Column('id', sa.Integer(), nullable=False))
op.alter_column('mytable', 'id', autoincrement=True, existing_type=sa.Integer(), existing_server_default=False, existing_nullable=False)
but got the following error
sqlalchemy.exc.OperationalError: (OperationalError) (1075, 'Incorrect table definition; there can be only one auto column and it must be defined as a key') 'ALTER TABLE mytable CHANGE id id INTEGER NOT NULL AUTO_INCREMENT' ()
looks like the sql statement generated by alembic did not add PRIMARY KEY at the end of the alter statement. Could I have missed some settings?
Thanks in advance!
I spent some time digging through the alembic source code, and this doesn't seem to be supported. You can specify primary keys when creating a table, but not when adding columns. In fact, it specifically checks and won't let you (link to source):
# from alembic.operations.toimpl.add_column, line 132
for constraint in t.constraints:
if not isinstance(constraint, sa_schema.PrimaryKeyConstraint):
operations.impl.add_constraint(constraint)
I looked around, and adding a primary key to an existing table may result in unspecified behavior - primary keys aren't supposed to be null, so your engine may or may not create primary keys for existing rows. See this SO discussion for more info: Insert auto increment primary key to existing table
I'd just run the alter query directly, and create primary keys if you need to.
op.execute("ALTER TABLE mytable ADD id INT PRIMARY KEY AUTO_INCREMENT;")
If you really need cross-engine compatibility, the big hammer would be to (1) create a new table identical to the old one with a primary key, (2) migrate all your data, (3)delete the old table and (4) rename the new table.
Hope that helps.
You have to remove the primary key that is in the table and then create a new one that includes all columns that you want as the primary key.
eg. In psql use \d <table name> to define the schema, then check the primary key constraint.
Indexes:
"enrollments_pkey" PRIMARY KEY, btree (se_crs_id, se_std_id)
then use this information in alembic
def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('enrollments', sa.Column(
'se_semester', sa.String(length=30), nullable=False))
op.drop_constraint('enrollments_pkey', 'enrollments', type_='primary')
op.create_primary_key('enrollments_pkey', 'enrollments', [
'se_std_id', 'se_crs_id', 'se_semester'])
The results after running \d enrollments should be updated to
Indexes:
"enrollments_pkey" PRIMARY KEY, btree (se_std_id, se_crs_id, se_semester)
This solution worked fine for me.