I have two sql files: data.sql, where is CREATE DATABASE, some CREATE TABLE and also INSERT sentences. Start of the file:
CREATE DATABASE IF NOT EXISTS `network` /*!40100 DEFAULT CHARACTER SET latin1 */;
USE `network`;
I also have a python file, where I want to connect to database:
import pyodbc
def connectToDB():
connection = None
while connection is None:
try:
connection = pyodbc.connect('DSN=data')
except:
print ("\n[DB connector] Error connecting to database. Trying again in 1 sec.")
time.sleep(1)
return connection
This is not my code, I get it on github. But I am a complete beginner with SQL I and have no idea how to use it. Do I have to establish a SQL Server? If so, how? I use PyCharm.
I know it's a dumb question, but I'm really struggling with that.
Any help would be appreciated!
Yes, you need to install a database in your local first.
You can try to use some open source solution like MariaDB.
Maybe it has the GUI to create a instance for setting the connection information (user name, password, ...etc).
Please make sure your database works fine then write your python code.
(Connection test and CRUD)
If I have text that is saved in a Postgresql database is there any way to execute that text as Python code and potentially have it update the same database?
That sounds terrifying.
Yes, plpy
PL/Python is a extension for postgres that lets you write functions in python. You may have to install the extension from your package manager or it may have been bundled in already when you installed postgres (this depends on how you installed postgres apt-get install postgresql-plpython-9.1 in debian).
To enable the extension in your database first use psql to run:
CREATE EXTENSION plpythonu
Now you can specify functions with python so you could write a function to execute that code like:
CREATE FUNCTION eval_python(code text) RETURNS integer AS $$
eval(code)
return 1
$$ LANGUAGE plpythonu;
And execute it for every code field in my_table like:
SELECT eval_python(code) FROM my_table;
Read the docs on PL/python for more details on how to interact with the db from python.
let me see if I understand what you are trying to accomplish:
store ad-hoc user code in a varchar field on a database
read and execute said code
allow said code to affect the database in question, say drop table ...
Assuming that I've got it, you could write something that
reads the table holding the code (use pyodbc or something)
runs an eval on what was pulled from the db - this will let you execute ANY code, including self updating code
are you sure this is what you want to do?
How to check if the table in database exist before creating it by using Python?
I would like to check if it exist then drop the table. If not, then create..
Also, how to display error if there are some errors in the codes of database?
(If in PHP. Codes will be die("....."); for Error Handling.. So what should I use in Python?)
Any advice?
From the MySQL manal use
CREATE table IF NOT EXISTS ...
To create a table if it doesn't exist. That might even be standard SQL (not sure but it will work in sqlite too)!
As for your second question "how to display error if there are some errors in the codes of database?" not sure exactly what you mean. but if MySQL returns an error then the database module will turn it into a python exception - see this StackOverflow question for some discussion
I have a requirement where I need to insert the postgres data into mysql. Suppose I have user table in postgres. I have user table also in mysql. I tried to do something like this:
gts = 'cd '+js_browse[0].js_path #gts prints correct folder name/usr/local/myfolder_name
os.system(gts)
gts_home = 'export GTS_HOME='+js_browse[0].js_path
os.system(gts_home)
tt=gts+'&& sh bin/admin.sh User --input-dir /tmp/import'
#inside temp/import i import store my postgres user table data
#bin is the folder inside myfolder_name
In mysql if I use the command it works perfectly fine:
cd /usr/local/myfolder_name
bin/admin.sh User -account=1 user=hamid -create'
I am unable to store data inside mysql this way. Any help shall be appreciated.
You don't really give us much information. And why would go from postgres to mysql?
But you can use one of these tools - I have seen people talk good about them
pg2mysql or pgs2sql
Hope it works out.
PostgreSQL provides possibility to dump data into the CSV format using COPY command.
The easiest path for you will be to spend time once to copy schema objects from PostgreSQL to MySQL, you can use pg_dump -s for this on the PostgreSQL side. IMHO, it will be the biggest challenge to properly move schemas.
And then you should import CSV-formatted data dumps into the MySQL, check this for reference. Scrolling down to the comments you'll find recipes for Windows also. Something like this should do the trick (adjust parameters accordingly):
LOAD DATA LOCAL INFILE C:\test.csv
INTO TABLE tbl_temp_data
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n';
I have created a Python module that creates and populates several SQLite tables. Now, I want to use it in a program but I don't really know how to call it properly. All the tutorials I've found are essentially "inline", i.e. they walk through using SQLite in a linear fashion rather than how to actually use it in production.
What I'm trying to do is have a method check to see if the database is already created. If so, then I can use it. If not, an exception is raised and the program will create the database. (Or use if/else statements, whichever is better).
I created a test script to see if my logic is correct but it's not working. When I create the try statement, it just creates a new database rather than checking if one already exists. The next time I run the script, I get an error that the table already exists, even if I tried catching the exception. (I haven't used try/except before but figured this is a good time to learn).
Are there any good tutorials for using SQLite operationally or any suggestions on how to code this? I've looked through the pysqlite tutorial and others I found but they don't address this.
Don't make this more complex than it needs to be. The big, independent databases have complex setup and configuration requirements. SQLite is just a file you access with SQL, it's much simpler.
Do the following.
Add a table to your database for "Components" or "Versions" or "Configuration" or "Release" or something administrative like that.
CREATE TABLE REVISION(
RELEASE_NUMBER CHAR(20)
);
In your application, connect to your database normally.
Execute a simple query against the revision table. Here's what can happen.
The query fails to execute: your database doesn't exist, so execute a series of CREATE statements to build it.
The query succeeds but returns no rows or the release number is lower than expected: your database exists, but is out of date. You need to migrate from that release to the current release. Hopefully, you have a sequence of DROP, CREATE and ALTER statements to do this.
The query succeeds, and the release number is the expected value. Do nothing more, your database is configured correctly.
AFAIK an SQLITE database is just a file.
To check if the database exists, check for file existence.
When you open a SQLITE database it will automatically create one if the file that backs it up is not in place.
If you try and open a file as a sqlite3 database that is NOT a database, you will get this:
"sqlite3.DatabaseError: file is encrypted or is not a database"
so check to see if the file exists and also make sure to try and catch the exception in case the file is not a sqlite3 database
SQLite automatically creates the database file the first time you try to use it. The SQL statements for creating tables can use IF NOT EXISTS to make the commands only take effect if the table has not been created This way you don't need to check for the database's existence beforehand: SQLite can take care of that for you.
The main thing I would still be worried about is that executing CREATE TABLE IF EXISTS for every web transaction (say) would be inefficient; you can avoid that by having the program keep an (in-memory) variable saying whether it has created the database today, so it runs the CREATE TABLE script once per run. This would still allow for you to delete the database and start over during debugging.
As #diciu pointed out, the database file will be created by sqlite3.connect.
If you want to take a special action when the file is not there, you'll have to explicitly check for existance:
import os
import sqlite3
if not os.path.exists(mydb_path):
#create new DB, create table stocks
con = sqlite3.connect(mydb_path)
con.execute('''create table stocks
(date text, trans text, symbol text, qty real, price real)''')
else:
#use existing DB
con = sqlite3.connect(mydb_path)
...
Sqlite doesn't throw an exception if you create a new database with the same name, it will just connect to it. Since sqlite is a file based database, I suggest you just check for the existence of the file.
About your second problem, to check if a table has been already created, just catch the exception. An exception "sqlite3.OperationalError: table TEST already exists" is thrown if the table already exist.
import sqlite3
import os
database_name = "newdb.db"
if not os.path.isfile(database_name):
print "the database already exist"
db_connection = sqlite3.connect(database_name)
db_cursor = db_connection.cursor()
try:
db_cursor.execute('CREATE TABLE TEST (a INTEGER);')
except sqlite3.OperationalError, msg:
print msg
Doing SQL in overall is horrible in any language I've picked up. SQLalchemy has shown to be easiest from them to use because actual query and committing with it is so clean and absent from troubles.
Here's some basic steps on actually using sqlalchemy in your app, better details can be found from the documentation.
provide table definitions and create ORM-mappings
load database
ask it to create tables from the definitions (won't do so if they exist)
create session maker (optional)
create session
After creating a session, you can commit and query from the database.
See this solution at SourceForge which covers your question in a tutorial manner, with instructive source code :
y_serial.py module :: warehouse Python objects with SQLite
"Serialization + persistance :: in a few lines of code, compress and annotate Python objects into SQLite; then later retrieve them chronologically by keywords without any SQL. Most useful "standard" module for a database to store schema-less data."
http://yserial.sourceforge.net
Yes, I was nuking out the problem. All I needed to do was check for the file and catch the IOError if it didn't exist.
Thanks for all the other answers. They may come in handy in the future.