Weird max min results for int column - python

I'm using python 2.7.6 with sqlite 3.3.6.
I have a table with an INT column. When I select max() or min() over the column, some random number from the row is returned that is neither the min nor max. I've tried to reproduce this behavior, manually creating table and inserting the same values, min and max work perfectly. However, after my script, a manual query table gives weird results again.
Here is create table statement
CREATE TABLE customer_score ( DATE INT , SPAUBM INT , "CUSTOMER" TEXT, "SCORE" INT )
I use executemany method from sqlite to fill the table.
cursor.executemany("INSERT INTO customer_score VALUES ( ?, ?, ? , ? )", list_to_put)
Here is example of the row, that is put into table from file
4203 6JASTYMPT 987335
Example of the issue :
sqlite> select * from customer_score where customer='AAA' and date in (20140219);
20140219|9214|AAA|5017262
20140219|9213|AAA|3409363
20140219|9207|AAA|2288238
20140219|9208|AAA|809365
sqlite> select max(score) from customer_score where customer='AAA' and date in (20140219);
809365
sqlite> select min(score) from customer_score where customer='AAAL' and date in (20140219);
2288238
Could it be that executemany method screws up the INT data somehow before insertion into the table?
I have no idea how can I check this.
UPDATE
SQlite manual says that MIN() MAX() are shorcut to ORDER BY LIMIT 1. In my case , ORDER BY also does not work as expected. Rows position looks completely random.
sqlite> select * from customer_score where customer='AAA' and date in (20140218, 20140219) order by score desc ;
20140219|9208|AAA|809365
20140218|9208|AAA|629937
20140219|9214|AAA|5017262
20140218|9214|AAA|3911855
20140219|9213|AAA|3409363
20140219|9207|AAA|2288238
20140218|9213|AAA|2127092
20140218|9207|AAA|1489895

The values in the score column are not numbers but strings.
You have to ensure that the values in list_to_put are actually numbers.
(Please note that SQLite pretty much ignores the column data type; it does not matter whether you declare it as INT or TEXT or FLUFFY BUNNIES.)

Related

Is it possible to assign cursor.fetchall() to a variable?

rows_order = "SELECT COUNT (*) FROM 'Order'"
cursor.execute(rows_order)
ordernum = cursor.fetchall()
connection.commit()
cursor.execute("INSERT INTO 'Order' (OrderNo, CustomerID, Date, TotalCost) VALUES (?,?,?,?)", (
[ordernum], custid_Sorder, now, total_item_price))
This is what I am trying but this error popped up;
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
How do I fix this? I want to make it so the OrderNo is = to the amount of orders before it, hence why I want to assign the orderno to it. (I am using sqlite3)
as you have only one value you need only fetchone
import sqlite3
con = sqlite3.connect("tutorial.db")
cursor = con.cursor()
rows_order = "SELECT COUNT (*) FROM 'Order'"
cursor.execute(rows_order)
ordernum = cursor.fetchone()[0]
cursor.execute("INSERT INTO 'Order' (OrderNo, CustomerID, Date, TotalCost) VALUES (?,?,?,?)", (
ordernum, custid_Sorder, now, total_item_price))
tl;dr Don't do this. Use an auto-incremented primary key.
fetchall returns all rows as a list, even if there is only one row.
Instead, use fetchone. This will return a single tuple which you can then select the first item. ordernum = cursor.fetchone()[0]
However, you appear to be writing a query to get the next ID. Using count(*) is wrong. If there are any gaps in OrderNo, for example if something gets deleted, it can return a duplicate. Consider [1, 3, 4]; count(*) will return 3. Use max(OrderNo) instead.
Furthermore, if you try to insert two orders at the same time you might get a race condition and one will try to duplicate the other.
process 1 process 2
select max(orderNo)
fetchone # 4
select max(orderNo)
fetchone # 4
insert into orders...
insert into orders... # duplicate OrderNo
To avoid this, you have to do both the select and insert in a transaction.
process 1 process 2
begin
select max(orderNo)...
fetchone # 4 begin
select max(orderNo)
fetchone
insert into orders... # wait
commit # wait
# 5
insert into orders...
commit
Better yet, do them as a single query.
insert into "Order" (OrderNo, CustomerID, Date, TotalCost)
select max(orderNo), ?, ?, ?
from "order"
Even better don't do it at all. There is a built-in mechanism to do this use an auto-incremented primary keys.
-- order is a keyword, pluralizing table names helps to avoid them
create table orders (
-- It is a special feature of SQLite that this will automatically be unique.
orderNo integer primary key
customerID int,
-- date is also a keyword, and vague. Use xAt.
orderedAt timestamp,
totalCost int
)
-- orderNo will automatically be set to a unique number
insert into orders (customerID, orderedAt, totalCost) values (...)

How to insert DEFAULT value with MySQLdb string formatting?

I have run into this a few times now, where I'm trying to insert (or bulk insert) into a MySQL table using VALUES without defining the columns explicitly, but there is an auto_increment field I want to let auto_increment, or a generated column that I can't insert a value for.
Specifically, let's say I have a table with three columns, two for numbers and one generated column that's the sum of those numbers:
CREATE TABLE `addition` (
`num_1` int DEFAULT NULL,
`num_2` int DEFAULT NULL,
`sum` int GENERATED ALWAYS AS ((`num_1` + `num_2`)) VIRTUAL
)
If I want to insert values to this database with a MySQLdb cursor object cur, I can't do:
cur.execute('INSERT INTO addition VALUES %s', [(2, 2, 'DEFAULT')])
...because you can't define the value for the generated field "sum", and 'DEFAULT' here is interpreted as the literal string. You'll get MySQL error 3105: The value specified for generated column 'sum' in table 'addition' is not allowed.
But the same error occurs for any value I could think to put in place of 'DEFAULT', for example None or False.
So is there any way to pass a value in the data section (i.e. [(2, 2, <something>)]) to tell MySQL to use the default value for the sum column? Or is the only way to define it in the SQL itself, i.e.
cur.execute('INSERT INTO addition VALUES (%s, %s, DEFAULT)', [2, 2])
This would be helpful when the table structure isn't known, or is prone to change, and you don't want to hard-code which fields should insert as DEFAULT.
--Edit--
Some clarification post-discussion in the comments, if I were to try cur.execute('INSERT INTO addition VALUES %s', [(2, 2, 'DEFAULT')]), this tries to insert the literal string 'DEFAULT', similarly for None, or any other value I could think of. So the question is really a Python question, is there a field (e.g. MySQLdb.DEFAULT()) that I can pass to accomplish this. So the final result would look something like cur.execute('INSERT INTO addition VALUES %s', [(2, 2, MySQLdb.DEFAULT())])
If you want a computed sum column, than handle it on the database side via a generated column:
CREATE TABLE addition (
num_1 INT,
num_2 INT,
sum AS (num_1 + num_2)
);
Then, when you insert two numbers, MySQL will handle the math for you:
INSERT INTO addition (num_1, num2) VALUES (2, 2);
Note that generated columns in MySQL are virtual by default, meaning that the sum won't actually be persisted, by rather would happen at the time you do a select.

Python - How to add a new column on SQL server & insert rows of lists

I want to insert values to SQL Server from python. Here's my code:
for value in rows:
cursor.execute ("""INSERT INTO Table ([ColumnOne]) VALUES (?)""", value)
cnxn.commit()
In rows , it contains lists (iteration) of rows, something like this:
row 1 contains of lists of float numbers
1.0
2.0
1.5
1.75
..... (in total of 1000 values in a row/column),
And it goes the same with row2, row3, and so on.
But, when I tried to run the code, I have this error
pyodbc.ProgrammingError: ('The SQL contains 1 parameter markers, but
1000 parameters were supplied', 'HY000')
Is there any way that I could do so the float values are not treated individually or to fix this problem?
I think maybe I should use ','.join statement to make it as string?
Considering I am not good at explaining and new to python, please correct me if i have some mistakes. Thank you.
When you attempt to insert multiple table rows in one query, you need to supply a list of values for each row.
For example, the following query would insert two rows:
INSERT INTO Table (ColumnOne) VALUES (1.0), (2.0);
So your python code needs to prepare the correct VALUES part of the query:
for row in rows:
values = ", ".join(("(?)",) * len(row))
cursor.execute(f"INSERT INTO Table (ColumnOne) VALUES {values}", row)

Python - Sqlite insert tuple without the autoincrement primary key value

I create a table with primary key and autoincrement.
with open('RAND.xml', "rb") as f, sqlite3.connect("race.db") as connection:
c = connection.cursor()
c.execute(
"""CREATE TABLE IF NOT EXISTS race(RaceID INTEGER PRIMARY KEY AUTOINCREMENT,R_Number INT, R_KEY INT,\
R_NAME TEXT, R_AGE INT, R_DIST TEXT, R_CLASS, M_ID INT)""")
I want to then insert a tuple which of course has 1 less number than the total columns because the first is autoincrement.
sql_data = tuple(b)
c.executemany('insert into race values(?,?,?,?,?,?,?)', b)
How do I stop this error.
sqlite3.OperationalError: table race has 8 columns but 7 values were supplied
It's extremely bad practice to assume a specific ordering on the columns. Some DBA might come along and modify the table, breaking your SQL statements. Secondly, an autoincrement value will only be used if you don't specify a value for the field in your INSERT statement - if you give a value, that value will be stored in the new row.
If you amend the code to read
c.executemany('''insert into
race(R_number, R_KEY, R_NAME, R_AGE, R_DIST, R_CLASS, M_ID)
values(?,?,?,?,?,?,?)''',
sql_data)
you should find that everything works as expected.
From the SQLite documentation:
If the column-name list after table-name is omitted then the number of values inserted into each row must be the same as the number of columns in the table.
RaceID is a column in the table, so it is expected to be present when you're doing an INSERT without explicitly naming the columns. You can get the desired behavior (assign RaceID the next autoincrement value) by passing an SQLite NULL value in that column, which in Python is None:
sql_data = tuple((None,) + a for a in b)
c.executemany('insert into race values(?,?,?,?,?,?,?,?)', sql_data)
The above assumes b is a sequence of sequences of parameters for your executemany statement and attempts to prepend None to each sub-sequence. Modify as necessary for your code.

Using an Alias for 2 Queries - SQLite [duplicate]

I am trying to INSERT INTO a table using the input from another table. Although this is entirely feasible for many database engines, I always seem to struggle to remember the correct syntax for the SQL engine of the day (MySQL, Oracle, SQL Server, Informix, and DB2).
Is there a silver-bullet syntax coming from an SQL standard (for example, SQL-92) that would allow me to insert the values without worrying about the underlying database?
Try:
INSERT INTO table1 ( column1 )
SELECT col1
FROM table2
This is standard ANSI SQL and should work on any DBMS
It definitely works for:
Oracle
MS SQL Server
MySQL
Postgres
SQLite v3
Teradata
DB2
Sybase
Vertica
HSQLDB
H2
AWS RedShift
SAP HANA
Google Spanner
Claude Houle's answer: should work fine, and you can also have multiple columns and other data as well:
INSERT INTO table1 ( column1, column2, someInt, someVarChar )
SELECT table2.column1, table2.column2, 8, 'some string etc.'
FROM table2
WHERE table2.ID = 7;
I've only used this syntax with Access, SQL 2000/2005/Express, MySQL, and PostgreSQL, so those should be covered. It should also work with SQLite3.
To get only one value in a multi value INSERT from another table I did the following in SQLite3:
INSERT INTO column_1 ( val_1, val_from_other_table )
VALUES('val_1', (SELECT val_2 FROM table_2 WHERE val_2 = something))
Both the answers I see work fine in Informix specifically, and are basically standard SQL. That is, the notation:
INSERT INTO target_table[(<column-list>)] SELECT ... FROM ...;
works fine with Informix and, I would expect, all the DBMS. (Once upon 5 or more years ago, this is the sort of thing that MySQL did not always support; it now has decent support for this sort of standard SQL syntax and, AFAIK, it would work OK on this notation.) The column list is optional but indicates the target columns in sequence, so the first column of the result of the SELECT will go into the first listed column, etc. In the absence of the column list, the first column of the result of the SELECT goes into the first column of the target table.
What can be different between systems is the notation used to identify tables in different databases - the standard has nothing to say about inter-database (let alone inter-DBMS) operations. With Informix, you can use the following notation to identify a table:
[dbase[#server]:][owner.]table
That is, you may specify a database, optionally identifying the server that hosts that database if it is not in the current server, followed by an optional owner, dot, and finally the actual table name. The SQL standard uses the term schema for what Informix calls the owner. Thus, in Informix, any of the following notations could identify a table:
table
"owner".table
dbase:table
dbase:owner.table
dbase#server:table
dbase#server:owner.table
The owner in general does not need to be quoted; however, if you do use quotes, you need to get the owner name spelled correctly - it becomes case-sensitive. That is:
someone.table
"someone".table
SOMEONE.table
all identify the same table. With Informix, there's a mild complication with MODE ANSI databases, where owner names are generally converted to upper-case (informix is the exception). That is, in a MODE ANSI database (not commonly used), you could write:
CREATE TABLE someone.table ( ... )
and the owner name in the system catalog would be "SOMEONE", rather than 'someone'. If you enclose the owner name in double quotes, it acts like a delimited identifier. With standard SQL, delimited identifiers can be used many places. With Informix, you can use them only around owner names -- in other contexts, Informix treats both single-quoted and double-quoted strings as strings, rather than separating single-quoted strings as strings and double-quoted strings as delimited identifiers. (Of course, just for completeness, there is an environment variable, DELIMIDENT, that can be set - to any value, but Y is safest - to indicate that double quotes always surround delimited identifiers and single quotes always surround strings.)
Note that MS SQL Server manages to use [delimited identifiers] enclosed in square brackets. It looks weird to me, and is certainly not part of the SQL standard.
Two approaches for insert into with select sub-query.
With SELECT subquery returning results with One row.
With SELECT subquery returning results with Multiple rows.
1. Approach for With SELECT subquery returning results with one row.
INSERT INTO <table_name> (<field1>, <field2>, <field3>)
VALUES ('DUMMY1', (SELECT <field> FROM <table_name> ),'DUMMY2');
In this case, it assumes SELECT Sub-query returns only one row of result based on WHERE condition or SQL aggregate functions like SUM, MAX, AVG etc. Otherwise it will throw error
2. Approach for With SELECT subquery returning results with multiple rows.
INSERT INTO <table_name> (<field1>, <field2>, <field3>)
SELECT 'DUMMY1', <field>, 'DUMMY2' FROM <table_name>;
The second approach will work for both the cases.
To add something in the first answer, when we want only few records from another table (in this example only one):
INSERT INTO TABLE1
(COLUMN1, COLUMN2, COLUMN3, COLUMN4)
VALUES (value1, value2,
(SELECT COLUMN_TABLE2
FROM TABLE2
WHERE COLUMN_TABLE2 like "blabla"),
value4);
Instead of VALUES part of INSERT query, just use SELECT query as below.
INSERT INTO table1 ( column1 , 2, 3... )
SELECT col1, 2, 3... FROM table2
Most of the databases follow the basic syntax,
INSERT INTO TABLE_NAME
SELECT COL1, COL2 ...
FROM TABLE_YOU_NEED_TO_TAKE_FROM
;
Every database I have used follow this syntax namely, DB2, SQL Server, MY SQL, PostgresQL
This can be done without specifying the columns in the INSERT INTO part if you are supplying values for all columns in the SELECT part.
Let's say table1 has two columns. This query should work:
INSERT INTO table1
SELECT col1, col2
FROM table2
This WOULD NOT work (value for col2 is not specified):
INSERT INTO table1
SELECT col1
FROM table2
I'm using MS SQL Server. I don't know how other RDMS work.
This is another example using values with select:
INSERT INTO table1(desc, id, email)
SELECT "Hello World", 3, email FROM table2 WHERE ...
Just use parenthesis for SELECT clause into INSERT. For example like this :
INSERT INTO Table1 (col1, col2, your_desired_value_from_select_clause, col3)
VALUES (
'col1_value',
'col2_value',
(SELECT col_Table2 FROM Table2 WHERE IdTable2 = 'your_satisfied_value_for_col_Table2_selected'),
'col3_value'
);
Simple insertion when table column sequence is known:
Insert into Table1
values(1,2,...)
Simple insertion mentioning column:
Insert into Table1(col2,col4)
values(1,2)
Bulk insertion when number of selected columns of a table(#table2) are equal to insertion table(Table1)
Insert into Table1 {Column sequence}
Select * -- column sequence should be same.
from #table2
Bulk insertion when you want to insert only into desired column of a table(table1):
Insert into Table1 (Column1,Column2 ....Desired Column from Table1)
Select Column1,Column2..desired column from #table2
from #table2
Here is another example where source is taken using more than one table:
INSERT INTO cesc_pf_stmt_ext_wrk(
PF_EMP_CODE ,
PF_DEPT_CODE ,
PF_SEC_CODE ,
PF_PROL_NO ,
PF_FM_SEQ ,
PF_SEQ_NO ,
PF_SEP_TAG ,
PF_SOURCE)
SELECT
PFl_EMP_CODE ,
PFl_DEPT_CODE ,
PFl_SEC ,
PFl_PROL_NO ,
PF_FM_SEQ ,
PF_SEQ_NO ,
PFl_SEP_TAG ,
PF_SOURCE
FROM cesc_pf_stmt_ext,
cesc_pfl_emp_master
WHERE pfl_sep_tag LIKE '0'
AND pfl_emp_code=pf_emp_code(+);
COMMIT;
Here's how to insert from multiple tables. This particular example is where you have a mapping table in a many to many scenario:
insert into StudentCourseMap (StudentId, CourseId)
SELECT Student.Id, Course.Id FROM Student, Course
WHERE Student.Name = 'Paddy Murphy' AND Course.Name = 'Basket weaving for beginners'
(I realise matching on the student name might return more than one value but you get the idea. Matching on something other than an Id is necessary when the Id is an Identity column and is unknown.)
You could try this if you want to insert all column using SELECT * INTO table.
SELECT *
INTO Table2
FROM Table1;
I actually prefer the following in SQL Server 2008:
SELECT Table1.Column1, Table1.Column2, Table2.Column1, Table2.Column2, 'Some String' AS SomeString, 8 AS SomeInt
INTO Table3
FROM Table1 INNER JOIN Table2 ON Table1.Column1 = Table2.Column3
It eliminates the step of adding the Insert () set, and you just select which values go in the table.
This worked for me:
insert into table1 select * from table2
The sentence is a bit different from Oracle's.
INSERT INTO yourtable
SELECT fielda, fieldb, fieldc
FROM donortable;
This works on all DBMS
For Microsoft SQL Server, I will recommend learning to interpret the SYNTAX provided on MSDN. With Google it's easier than ever, to look for syntax.
For this particular case, try
Google: insert site:microsoft.com
The first result will be http://msdn.microsoft.com/en-us/library/ms174335.aspx
scroll down to the example ("Using the SELECT and EXECUTE options to insert data from other tables") if you find it difficult to interpret the syntax given at the top of the page.
[ WITH <common_table_expression> [ ,...n ] ]
INSERT
{
[ TOP ( expression ) [ PERCENT ] ]
[ INTO ]
{ <object> | rowset_function_limited
[ WITH ( <Table_Hint_Limited> [ ...n ] ) ]
}
{
[ ( column_list ) ]
[ <OUTPUT Clause> ]
{ VALUES ( { DEFAULT | NULL | expression } [ ,...n ] ) [ ,...n ]
| derived_table <<<<------- Look here ------------------------
| execute_statement <<<<------- Look here ------------------------
| <dml_table_source> <<<<------- Look here ------------------------
| DEFAULT VALUES
}
}
}
[;]
This should be applicable for any other RDBMS available there. There is no point in remembering all the syntax for all products IMO.
INSERT INTO FIRST_TABLE_NAME (COLUMN_NAME)
SELECT COLUMN_NAME
FROM ANOTHER_TABLE_NAME
WHERE CONDITION;
Best way to insert multiple records from any other tables.
INSERT INTO dbo.Users
( UserID ,
Full_Name ,
Login_Name ,
Password
)
SELECT UserID ,
Full_Name ,
Login_Name ,
Password
FROM Users_Table
(INNER JOIN / LEFT JOIN ...)
(WHERE CONDITION...)
(OTHER CLAUSE)
select *
into tmp
from orders
Looks nice, but works only if tmp doesn't exists (creates it and fills). (SQL sever)
To insert into existing tmp table:
set identity_insert tmp on
insert tmp
([OrderID]
,[CustomerID]
,[EmployeeID]
,[OrderDate]
,[RequiredDate]
,[ShippedDate]
,[ShipVia]
,[Freight]
,[ShipName]
,[ShipAddress]
,[ShipCity]
,[ShipRegion]
,[ShipPostalCode]
,[ShipCountry] )
select * from orders
set identity_insert tmp off
IF you want to insert some data into a table without want to write column name.
INSERT INTO CUSTOMER_INFO
(SELECT CUSTOMER_NAME,
MOBILE_NO,
ADDRESS
FROM OWNER_INFO cm)
Where the tables are:
CUSTOMER_INFO || OWNER_INFO
----------------------------------------||-------------------------------------
CUSTOMER_NAME | MOBILE_NO | ADDRESS || CUSTOMER_NAME | MOBILE_NO | ADDRESS
--------------|-----------|--------- || --------------|-----------|---------
A | +1 | DC || B | +55 | RR
Result:
CUSTOMER_INFO || OWNER_INFO
----------------------------------------||-------------------------------------
CUSTOMER_NAME | MOBILE_NO | ADDRESS || CUSTOMER_NAME | MOBILE_NO | ADDRESS
--------------|-----------|--------- || --------------|-----------|---------
A | +1 | DC || B | +55 | RR
B | +55 | RR ||
If you go the INSERT VALUES route to insert multiple rows, make sure to delimit the VALUES into sets using parentheses, so:
INSERT INTO `receiving_table`
(id,
first_name,
last_name)
VALUES
(1002,'Charles','Babbage'),
(1003,'George', 'Boole'),
(1001,'Donald','Chamberlin'),
(1004,'Alan','Turing'),
(1005,'My','Widenius');
Otherwise MySQL objects that "Column count doesn't match value count at row 1", and you end up writing a trivial post when you finally figure out what to do about it.
If you create table firstly you can use like this;
select * INTO TableYedek From Table
This metot insert values but differently with creating new copy table.
In informix it works as Claude said:
INSERT INTO table (column1, column2)
VALUES (value1, value2);
Postgres supports next:
create table company.monitor2 as select * from company.monitor;

Categories

Resources