I took a look at the tutorial here: http://www.pythoncentral.io/introductory-tutorial-python-sqlalchemy/
The decisive parts for my question are first:
import os
import sys
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy import create_engine
Base = declarative_base()
class Person(Base):
__tablename__ = 'person'
# Here we define columns for the table person
# Notice that each column is also a normal Python instance attribute.
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
class Address(Base):
__tablename__ = 'address'
# Here we define columns for the table address.
# Notice that each column is also a normal Python instance attribute.
id = Column(Integer, primary_key=True)
street_name = Column(String(250))
street_number = Column(String(250))
post_code = Column(String(250), nullable=False)
person_id = Column(Integer, ForeignKey('person.id'))
person = relationship(Person)
# Create an engine that stores data in the local directory's
# sqlalchemy_example.db file.
engine = create_engine('sqlite:///sqlalchemy_example.db')
# Create all tables in the engine. This is equivalent to "Create Table"
# statements in raw SQL.
Base.metadata.create_all(engine)
and second:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy_declarative import Address, Base, Person
engine = create_engine('sqlite:///sqlalchemy_example.db')
# Bind the engine to the metadata of the Base class so that the
# declaratives can be accessed through a DBSession instance
Base.metadata.bind = engine
DBSession = sessionmaker(bind=engine)
# A DBSession() instance establishes all conversations with the database
# and represents a "staging zone" for all the objects loaded into the
# database session object. Any change made against the objects in the
# session won't be persisted into the database until you call
# session.commit(). If you're not happy about the changes, you can
# revert all of them back to the last commit by calling
# session.rollback()
session = DBSession()
# Insert a Person in the person table
new_person = Person(name='new person')
session.add(new_person)
session.commit()
# Insert an Address in the address table
new_address = Address(post_code='00000', person=new_person)
session.add(new_address)
session.commit()
If the second part is run (e.g. "python second_part"), won't the import statement
from sqlalchemy_declarative import Address, Base, Person
cause all executable code in the first part to run? (Which would trigger the code which creates tables in a db each time the first part is run)
Best regards
Yes, every time, the first file is imported, the tables are created again.
To prevent this, use the following statements:
if __name__ == '__main__':
Base.metadata.create_all(engine)
Related
I am trying to create an application using SQLAlchemy. It worked fine as long as I only had one file with one Class. Now I want to have multiple classes/tables in different files. I stumbled upon this question, and tried to do it like it was suggested there: I now have three files
base.py
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
blind.py
from sqlalchemy import Column, String
from .base import Base
class Blind(Base):
__tablename__ = 'blinds'
blind = Column(String)
data_processor_uuid = Column(String, primary_key=True)
data_source_uuid = Column(String)
timestamp = Column(String, primary_key=True)
and data.py
from sqlalchemy import Column, Integer, String, Float
from .base import Base
class Datum(Base):
__tablename__ = 'data'
data_source_uuid = Column(Integer, primary_key=True)
sensor_type = Column(String)
timestamp = Column(String, primary_key=True)
value = Column(Float)
I now want to initialize the database in db_setup.py using
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .base import Base
engine = create_engine('sqlite:///test.db', echo=True)
Base.metadata.bind = engine
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
def get_db_session():
return session
This works, however, it does not create the tables in the database as expected. When I try to insert something into the table, I get an error saying "table does not exist". Can someone tell me what I am doing wrong here?
The problem was that I wasn't importing the class definitions for Blinds and Datum anywhere, so they weren't evaluated! Before I split them up into different files, I had imported them to get to Base. Thanks to #IljaEverilä for this answer!
I'm using SQLAlchemy ORM for a Flask project where I want to join across an eagerly loaded model but this leads to two joins to the same intermediary model. If you run the code below you'll see in the generated SQL that there are two joins between the Author model and the Book model. If the lazy=joined bit is removed the sql generated is perfect.
I don't know if I'm doing something wrong or this is by design. How do I get SQLAlchemy to emit the right SQL while maintaining the joinedload in this case?
Note: I have tried this with MySQL and SQLite and it happens with both those dbs.
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
from sqlalchemy import create_engine, Integer, String, Column
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base()
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
name = Column(String)
pseudo = Column(String)
books = relationship("Book", lazy='joined')
def __repr__(self):
return "<User(name='%s', fullname='%s', password='%s')>" % (
self.name, self.fullname, self.password)
class Book(Base):
__tablename__ = 'books'
id = Column(Integer, primary_key=True)
author_id = Column(Integer, ForeignKey('authors.id'))
name = Column(String)
user = relationship("Author", back_populates="books")
pages = relationship("Page")
class Page(Base):
__tablename__ = 'pages'
id = Column(Integer, primary_key=True)
book_id = Column(Integer, ForeignKey('books.id'))
text = Column(String)
book = relationship("Book", back_populates="pages")
Base.metadata.create_all(engine)
session = Session()
print(str(session.query(Author).outerjoin(Author.books, Page)))
It is by design – read The Zen of Joined Eager Loading:
It is critical to understand the distinction that while Query.join() is used to alter the results of a query, joinedload() goes through great lengths to not alter the results of the query, and instead hide the effects of the rendered join to only allow for related objects to be present.
There are multiple somewhat similar questions in sqlalchemy, though couldn't find one that'd fit the bill exactly.
If you manually add a join, and want to use it to eager load a relationship as well, you need contains_eager():
session.query(Author).\
outerjoin(Author.books, Book.pages).\
options(contains_eager(Author.books).contains_eager(Book.pages))
Note that the relationship definitions Author.books and Book.pages would seem to be missing the back_populates= argument.
The (executable) code below causes an IntegrityError because an entity with the same PK is added twice (implicitly) to the session. The session does not know that the entity represents the same object (same PK) and triggers two INSERT statements. I was under the impression that the session was supposed to detect that automatically. I realise that both entities are transient/detached and I would need to perform a merge to get a managed instance. But is this not avoidable?
A few notes concerning the example code:
It's immensely simplified to demonstrate the issue at hand. The real code I have is quite a bit more complex.
An important thing to note is that the main entity is "built" using a function "build_data" which does not have any reference to the session. The session is created on a higher level of the application.
The concept of User and Article are just for illustration. In reality it's different business entities. I replaced them here with "article" and "user" as it's a well-known concept.
Between the creation of article and article2 a lot of other stuff is happening, creating a fairly complex data-structure. I also don't know which line is happening first as a non-deterministic loop is involved (over dictionary keys).
I could likely solve this by either:
Passing the session to the build_data function and merging as required, or
Keep a manual reference to the instances and only create them once
What I would like to know: Can I avoid both approaches above to keep the code simple?
Here's the code in question:
from sqlalchemy import (
create_engine,
Column,
ForeignKeyConstraint,
Unicode,
)
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import (
relationship,
sessionmaker)
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
name = Column(Unicode, nullable=False, primary_key=True)
class Article(Base):
__tablename__ = 'article'
__table_args__ = (
ForeignKeyConstraint(
['user_'],
['user.name'],
ondelete='CASCADE',
onupdate='CASCADE'),
)
user_ = Column(Unicode, nullable=False, primary_key=True,
)
title = Column(Unicode, nullable=False, primary_key=True)
content = Column(Unicode)
user = relationship(User, backref='articles')
# Prepare the session
Session = sessionmaker()
engine = create_engine('sqlite:///:memory:', echo=True)
Session.configure(bind=engine)
Base.metadata.create_all(engine)
# --- The main code -----------------------------------------------------------
def build_data():
user = User(name='JDoe')
article = Article(user=user, title='Hello World', content='Foobar')
print(article)
# More stuff is happening here.
article2 = Article(user=user, title='Hello World', content='Foobar')
print(article2)
return user
session = Session()
entity = build_data()
session.add(entity)
session.flush()
# -----------------------------------------------------------------------------
When I use sqlAlchemy to read data from mysql DB, it work when my class and the search code are in the same py file just like this:
class odd_basis(Base):
__tablename__ = 'odd_basis'
NO = Column(Integer, primary_key=True)
LEAGUE = Column(String(20))
TIME = Column(String(20))
engine = create_engine("......")
session = sessionmaker(bind=engine)
DBsession = session()
basis_data = DBsession.query(odd_basis).filter(odd_basis.NO=='16039').all()
when I put the class definition in another py file, then use the sys.path.append to import, then it will have this error:
sqlalchemy.exc.ArgumentError: SQL expression object or string expected, got object of type instead
my main code:
import sys
sys.path.append("~/analysis/table_class.py")
engine = create_engine("......")
session = sessionmaker(bind=engine)
DBsession = session()
basis_data = DBsession.query(odd_basis).filter(odd_basis.NO=='16039').all()
table_class.py:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, Numeric, ForeignKey, UniqueConstraint, Index
Base = declarative_base()
class odd_basis(Base):
__tablename__ = 'odd_basis'
NO = Column(Integer, primary_key=True)
LEAGUE = Column(String(20))
TIME = Column(String(20))
Sys.path.append does not import anything. It just adds to your search path. You need to import your class from there:
from table_class import odd_basis
and so forth.
Hannu
I am trying to follow this tutorial from SQLAlchemy on how to create entries in and query a MYSQL database in python. When I try and query the database for the first time following along in their adding new objects section to test whether an object has been added to the database (see large code block below), I get the following error: AttributeError: 'Connection' object has no attribute 'contextual_connect'
I can query the database. For example, if I change the final line of code to our_user = session.query(User).filter_by(name='ed') it successfully returns a query object, but I cannot figure out how to get the object I entered into the database out of this query result.
Similarly, if I try to loop over the results as they suggest in their querying
section:
for instance in session.query(User).order_by(User.id):
print instance.name, instance.fullname
I get the same error. How can I fix this particular error and are there any other tutorials on using MYSQL in Python with SQLAlchemy that you could point me to?
My code:
import MySQLdb
from sqlalchemy import create_engine
db1 = MySQLdb.connect(host="127.0.0.1",
user="root",
passwd="****",
db="mydata")
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy import Column, Integer, String
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)
password = Column(String)
def __init__(self, name, fullname, password):
self.name = name
self.fullname = fullname
self.password = password
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
ed_user = User('ed', 'Ed Jones', 'edspassword')
from sqlalchemy.orm import sessionmaker
Session = sessionmaker()
Session.configure(bind=db1)
session = Session()
session.add(ed_user)
our_user = session.query(User).filter_by(name='ed').first()
Update/Working Code:
(1) Change to SQLAlchemy engine as discussed by codeape below.
(2) Remember to create the table: Base.metadata.create_all(engine)
(3) Use the "foolproof" version of the User class from SQLAlchemy's tutorial. Note to SQLAlchemy, we (at least I) feel like a fool and would like you to use to always use the foolproof version in the main body of your tutorial and not as an aside that a busy reader might skip over.
All that yields working code:
import MySQLdb
from sqlalchemy import create_engine
engine = create_engine("mysql://user:password#host/database")
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy import Column, Integer, String, Sequence
class User(Base):
__tablename__ = 'users'
id = Column(Integer, Sequence('user_id_seq'), primary_key=True)
name = Column(String(50))
fullname = Column(String(50))
password = Column(String(12))
def __init__(self, name, fullname, password):
self.name = name
self.fullname = fullname
self.password = password
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
Base.metadata.create_all(engine)
ed_user = User('ed', 'Ed Jones', 'edspassword')
from sqlalchemy.orm import sessionmaker
Session = sessionmaker()
Session.configure(bind=engine)
session = Session()
session.add(ed_user)
our_user = session.query(User).filter_by(name='ed').first()
print(our_user is ed_user)
You must bind the session to a SQLAlchemy engine, not directly to a MySQLDb connection object.
engine = create_engine("mysql://user:password#host/dbname")
Session.configure(bind=engine)
(You can remove your db1 variable.)
From the tutorial:
The return value of create_engine() is an instance of Engine, and it represents the core interface to the database, adapted through a dialect that handles the details of the database and DBAPI in use.
See also https://docs.sqlalchemy.org/en/latest/orm/tutorial.html
from sqlalchemy.orm import sessionmaker
engine = create_engine("mysql://user:password#host/dbname")
Session = sessionmaker(bind=engine)
session = Session()