Develop Reference

Unable to insert a row in SQL Server table using Python SQLAlchemy (PK not set as IDENTITY) [duplicate]

This question already has answers here:
Prevent SQLAlchemy from automatically setting IDENTITY_INSERT
(4 answers)
Closed last year.
Have this Python Flask SQLAlchemy app that fetch data from a third party SQL Server database.
There is a table with to columns that I need to insert rows:
TABLE [dbo].[TableName](
[Id] [bigint] NOT NULL,
[Desc] [varchar](150) NOT NULL,
CONSTRAINT [PK_Id] PRIMARY KEY CLUSTERED ...
The primary key is not set as IDENTITY
Using SQLAlchemy ORM, if I try to add a new row without an explicit value for Id field, I have this error:
sqlalchemy.exc.IntegrityError: (pyodbc.IntegrityError) ('23000', "[23000] ...
The column not allow Null values* (translated text)
If I explicit an Id value, another error occurs:
sqlalchemy.exc.ProgrammingError: (pyodbc.ProgrammingError) ('42000', '[42000] ...*
It is not possible to find the object "dbo.TableName", because it not exists or you don't have permissions (translated text)
This error is followed by the sentence:
[SQL: SET IDENTITY_INSERT dbo.[TableName] ON]
I'm supposing SQLAlchemy is trying to execute this command, but as Id is not set as IDENTITY, there's no need for that.
Using SQL Server Management Studio, with the same user of pyodbc connection, I'm able to insert new records, choosing whatever value for Id.
I would appreciate any hint.

Your INSERT will fail because a value must be defined for the primary key column of a table, either explicitly in your INSERT or implicitly by way of an IDENTITY property.
This requirement is due to the nature of primary keys and cannot be subverted. Further, you are unable to insert a NULL because the table definition explicitly disallows NULLs in that column.
You must provide a value in your INSERT statement explicitly due to the combination of design factors present.
Based on the documentation (https://docs-sqlalchemy.readthedocs.io/ko/latest/dialects/mssql.html#:~:text=The%20SQLAlchemy%20dialect%20will%20detect%20when%20an%20INSERT,OFF%20subsequent%20to%20the%20execution.%20Given%20this%20example%3A), it appears that SqlAlchemy may be assuming that column is an IDENTITY and is attempting to toggle IDENTITY_INSERT to on. As it is not an identity column, it is encountering an exception.
In your table metadata, check that you have autoincrement=False set for the Id column.
Edit to add: According to comments in an answer on a related question (Prevent SQLAlchemy from automatically setting IDENTITY_INSERT), it appears that SqlAlchemy assumes all integer-valued primary keys to be identity, auto-incrementing as well - meaning that you need to explicitly override that assumption as described here.

Trying to call a stored procedure from mysql in python, keep getting a strange error column not found in fieldlist, any ideas?

I am trying to run a simple stored procedure from python, but keep running into an error that persists no matter how I try to format according to faqs online. When I call the stored procedure and enter the parameters directly in mysql it runs with the same error. I am a mysql newb, so it could be something stupid I don't know. Here is my stored procedure and it being tested in mysql, the table (whose contents are solely the admin admin being tested), the python code, and the error msg.
stored procedure test in mysql python callproc code table referenced python error

Solved it. Changed the query slightly and needed quotations in the comparison parts for some reason. Here is what it looks like now.
CREATE DEFINER=`root`#`localhost` PROCEDURE `userLogin`(
in input_userId varchar(20),
in input_password varchar(20),
out loginMsg char(50))
begin
set #numUsers = 0;
select count(*) into #numUsers
from login x
where x.userId = input_userId and x.pword = input_password;
if #numUsers = 1 then
set loginMsg = "Login successful";
else
set loginMsg = "Invalid username/password";
end if;
end

Incorrect string value error - Python + mariaDB

I am using the Python mysql-connector module to insert unicode character point 128049 (U+1F431) into a mariaDB sql table.
My SQL table is defined as:
show create table t1;
CREATE TABLE `t1` (
`c1` varchar(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
And the python code is:
import mysql.connector as db
conn = db.connect(sql_mode = 'STRICT_ALL_TABLES')
curs = conn.cursor(prepared = True)
curs.execute('insert into t1 (c1) values(%)', chr(128049))
Since this is a plane 1 unicode value it needs 4 bytes, but changing the table and column to utf8mb4 as suggested here didn't work.
The error I'm getting is:
Incorrect string value: '\xF0\x9F\x90\xB1' for column 'c1' at row 1
The string being inserted looks correct when compared to:
chr(128049).encode('utf-8')
The sql_mode for this version of mariadb is not strict by default. While the insert works when I do not specify strict mode, the characters are converted to the default '?' character.
I can't figure out how why SQL thinks this is an invalid string.
I am connecting to mariadb 10.1.9 via mysql-connector 2.1.4 in python 3.6.1.

The connection needs to specify utf8mb4. Or SET NAMES utf8mb4. This is to specify the encoding of the client's bytes.
🐱 is a 4-byte Emoji.
More Python tips: http://mysql.rjweb.org/doc.php/charcoll#python

Rick James answer is correct. From that I was able to create a solution that worked for me.
SET NAMES 'utf8mb4';
Sets 3 global variables as seen here. The only issue is this only sets session variables so you have to issue this command for every connection.
It doesn't appear possible to set those 3 variables in the mysqld group of the my.cnf file (I believe this is because they can not be set at the command line. Note the missing command line detail in the definitions here)
Instead I set the init_file option in the mysqld group of the my.cnf options file.
[mysqld]
init_file=/path/to/file.sql
Within that file I set the 3 variables:
set ##global.character_set_client='utf8mb4';
set ##global.character_set_connection='utf8mb4';
set ##global.character_set_results='utf8mb4';
Setting these globally forced the session variables to the same value. Problem solved.

Django, Postgres - column cannot be cast automatically to type integer

In my database i have column:
currency = models.CharField(max_length=10, blank=True, null=True)
I want to change this column from CharField to IntegerField. So in models.py i change this:
currency = models.IntegerField(blank=True, null=True)
then i made migrations: python manage.py makemigrations and python manage.py migrate. After that actions it rise error:
django.db.utils.ProgrammingError: column "currency" cannot be cast automatically to type integer
HINT: Specify a USING expression to perform the conversion.
After that in pgAdmin3 console i made this changes:
ALTER TABLE my_table ALTER COLUMN currency TYPE integer USING (currency::integer);
But i still got that error, I tried to change all back, but error doesn't disappear. What i have to do to escape this error. Thank you

I think django migrations does not perform casting, I looked in the documentation but I did not find any thing about column casting.
if the existing data is not that important for you, you can delete the column and create a new one
first step remove currency from you model and apply migration
add again the currency with the new definition and apply again the migration
if you want to keep your data, you need to give your new column a different name and use the old column as a temporary column to hold the data during the transition.
Important: Postgresql is more strongly typed in recent versions, and as explained here some casting may not work in PosgreSQL unless it's explicitly done. And it required to be more specific about the type. So you have to make the right choice based on your values:
alter table my_table alter column currency type bigint using currency::bigint
or maybe:
alter table my_table alter column currency type numeric(10,0) using currency::numeric

It is a PSQL issue when changing from certain data types to others... I had a similar problem an I did something a bit hackey but it worked ... but only because I didn't have any important data in the column
1) delete the most recent migration for that app
2) delete/comment out the "column" on the object
3) migrate the app with the missing column
4) reinstate/uncomment the offending "column"
5) migrate again
this is all a long way to delete and recreate the column on the actual db without using sql ... figured I would share in case it might help someone down the road

Issue with SQLite DROP TABLE statement

EDIT: At this point, I found the errant typo that was responsible, and my question has become "How did the typo that I made cause the error that I received" and "How might I have better debugged this in the future?"
I've setup a database script for SQLite (through pysqlite) as follows:
DROP TABLE IF EXISTS LandTerritory;
CREATE TABLE LandTerritory (
name varchar(50) PRIMARY KEY NOT NULL UNIQUE,
hasSC boolean NOT NULL DEFAULT 0
);
I'm expecting this to always run without error. However, if I run this script twice, (using the sqlite.Connection.executescript method) I get this error:
OperationalError:table LandTerritory already exists
Trying to debug this myself, I run DROP TABLE LandTerritory on its own and get:
sqlite3.OperationalError: no such table: main.LandTerrito
I'm guessing this has something to do with the "main." part, but I'm not sure what.
EDIT:
Okay PRAGMA foreign_keys=ON is definitely involved here, too. When I create my connection, I turned on foreign_keys. If I don't turn that on, I don't seem to get this error.
And I should have mentioned that there's more to the script, but I had assumed the error was occurring in these first 2 statements. The rest of the script just does the same, drop table, define table. A few of the tables have foreign key references to LandTerritory.
Is there a way to get something like line number information about the sqlite errors? That would be really helpful.
EDIT 2:
Okay, here's another table in the script that references the first.
DROP TABLE IF EXISTS LandAdjacent;
CREATE TABLE LandAdjacent (
tname1 varchar(50) NOT NULL,
tname2 varchar(50) NOT NULL,
PRIMARY KEY (tname1, tname2),
/* Foreign keys */
FOREIGN KEY (tname1)
REFERENCES LandTerrito
ON DELETE CASCADE
ON UPDATE CASCADE,
FOREIGN KEY (tname2)
REFERENCES LandTerritory(name)
ON DELETE CASCADE
ON UPDATE CASCADE
);
Looking at this, I found were the "LandTerrito" came from, somehow a few characters got cut off. I'm guessing fixing this may fix my problem.
But I'm really confused how a broken line in this table led to the script running correctly the first time, and then giving me an error related to a different table when I run it the second time, and how foreign keys played into this.
I guess, to reiterate from above, is there a better way to debug this sort of thing?

The source of the error is your typo
REFERENCES LandTerrito
in line 8 of your script. This leads to the "missing" table LandTerrito in the CREATE TABLE LandAdjacent statement.
If you run your two CREATE TABLE statements Sqlite wont complain. But if you have PRAGMA foreign_keys=ON; and try to run an INSERT or DELETE statement on the table LandAdjacent, you'll get the error no such table: main.LandTerrito.
Because of the foreign key constraints DROP TABLE on LandTerritory however will result in a DELETE on the table LandAdjacent, which triggers the error.
The following things will avoid the error
set PRAGMA foreign_keys=ON; before you drop the table (tested) or
add a dummy table LandTerrito (tested) or
drop LandAdjacent first, then LandTerritory (tested) or
dont use ON DELETE CASCADE (not tested)
and of course correcting the original typo.

Put a "GO" (or whatever equivalent is used in SQLlite) to terminate a batch between the drop table statement and the create statement