Ok first, please assume that this is the first time I will be writing a single alphabet of code in SQL.. so please please please... it would be very kind if you can explain me each step. :)
I am using mac for this development.
So in the desktop, I have a huge (1.5GB) dumpfile of a database.
I have mysqlworkbench (yes I have never worked in that either :( )
And I have installed MySQLdb.
So, basically what I want to do is use the data in the database from python.
So I have python.. mysqldb to connect to db.. huge db and mysqlworkbench..
Now I assume, that this dumpfile is useless as its not in form of database..
so how do I "mount" this dump to a database..(mysqlworkbench)
After that how do I query from python..
(like the port number.. username , password!! ? )
It would be of huge help if anyone can help me.. I have been stuck in this since morning.. :(
In order to use the data in the file, it needs to be imported into the database. (Think of the file as a recipe for MySQL to make a database. Now you need to tell MySQL to make the database from the recipe).
You can import the database using command line, a python script or even MySQL workbench. However, due to the size of the file, using MySQL workbench could be a problem (Note: I am not familiar with MySQL workbench).
To import the database with the command line use the command:
mysql -u <username> -p -h localhost king_tracking < <filename>
To actually use the database with python, there are several step by step tutorials available, easily accessible from a google search. A basic script is (from here)
import MySQLdb
conn = MySQLdb.connect (host = "localhost",
user = "testuser",
passwd = "testpass",
db = "test")
cursor = conn.cursor ()
cursor.execute ("SELECT VERSION()")
row = cursor.fetchone ()
print "server version:", row[0]
cursor.close ()
conn.close ()
This just shows the version of MySQL, however if
"SELECT VERSION()"
is replaced by your own query, like
"SELECT * FROM <tablename> LIMIT 0,30"
you can execute your own queries.
In order to understand the tutorials on interfacing python and MySQL, you should be familiar with both separately. A good SQL tutorial is the one at W3schools.
The problem with creating a table is separate from the original question, but may need to be addressed before trying to import the file into the database.
Related
I have two sql files: data.sql, where is CREATE DATABASE, some CREATE TABLE and also INSERT sentences. Start of the file:
CREATE DATABASE IF NOT EXISTS `network` /*!40100 DEFAULT CHARACTER SET latin1 */;
USE `network`;
I also have a python file, where I want to connect to database:
import pyodbc
def connectToDB():
connection = None
while connection is None:
try:
connection = pyodbc.connect('DSN=data')
except:
print ("\n[DB connector] Error connecting to database. Trying again in 1 sec.")
time.sleep(1)
return connection
This is not my code, I get it on github. But I am a complete beginner with SQL I and have no idea how to use it. Do I have to establish a SQL Server? If so, how? I use PyCharm.
I know it's a dumb question, but I'm really struggling with that.
Any help would be appreciated!
Yes, you need to install a database in your local first.
You can try to use some open source solution like MariaDB.
Maybe it has the GUI to create a instance for setting the connection information (user name, password, ...etc).
Please make sure your database works fine then write your python code.
(Connection test and CRUD)
I want to come up minimal set of queries/loc that extracts the table metadata within a database, on as many versions of database as possible. I'm using PostgreSQl. I'm trying to get this using python. But I've no clue on how to do this, as I'm a python newbie.
I appreciate your ideas/suggestions on this issue.
You can ask your database driver, in this case psycopg2, to return some metadata about a database connection you've established. You can also ask the database directly about some of it's capabilities, or schemas, but this is highly dependent on the version of the database you're connecting to, as well as the type of database.
Here's an example taken from http://bytes.com/topic/python/answers/438133-find-out-schema-psycopg for PostgreSQL:
>>> import psycopg2 as db
>>> conn = db.connect('dbname=billings user=steve password=xxxxx port=5432')
>>> curs = conn.cursor()
>>> curs.execute("""select table_name from information_schema.tables WHERE table_schema='public' AND table_type='BASETABLE'""")
>>> curs.fetchall()
[('contacts',), ('invoicing',), ('lines',), ('task',), ('products',),('project',)]
However, you probably would be better served using an ORM like SQLAlchemy. This will create an engine which you can query about the database you're connected to, as well as normalize how you connect to varying database types.
If you need help with SQLAlchemy, post another question here! There's TONS of information already available by searching the site.
I am writing python code to get movies data from dbpedia and put in MySql database.
presently, my application is having issues and it is hanging in between or if it aborts, then next time I restart, there some issues occur when droping the table and creating table in database.
Is it because last time connection with the database didnt get close?
If so then what can be possible solution to avoid this issue when restarting the application
What python library are you using to access MySQL? If you are using MySQLdb, then to be sure everything is written correctly, you need to use the "close" method of your cursor and the "commit" method of the connection. For example,
import MySQLdb
conn = MySQLdb.connect(user="username",pass="password",db="dbname")
cur = conn.cursor()
# Work with your cur object to do what you want
cur.close()
conn.commit()
I come from a PHP background where MySQL easily works from PHP, and don't the process for getting MySQL to work from Python. From all the research i did and reading of similar yet non-exact questions, it seems to me that there are different ways to achieve this, which makes it even harder for me to wrap my head around. So far I have MySQL-python-1.2.3 installed for python 2.7.1 in Windows XP 32Bit. Can anyone give me an overview of what is necessary to get MySQL working from Python in Windows, or even what is next after my steps all the way to fetching a table row? Thanks in advance.
UPDATE:
#Mahmoud, using your suggestion i have triggered the following:
If you just want to use the DBAPI, then here's a simple snippet explaining how to issue a SELECT query.
import MySQLdb
db = MySQLdb.connect(host="host", user="username", passwd="your-pass", db="the-db-name")
To perform a query, you first need a cursor, and then you can execute queries on it:
cursor = db.cursor()
max_age = 42
cursor.execute("""SELECT name FROM employees WHERE age < %s""", (max_age,))
print cursor.fetchone()
However, you most likely want to use an ORM, I recommend SQLAlchemy. It essentially trivializes database interaction by providing a super-powerful abstraction layer.
I just downloaded sqlite3.exe. It opens up as a command prompt. I created a table test & inserted a few entries in it. I used .backup test just in case. After I exit the program using .exit and reopened it I don't find the table listed under .tables nor can I run any query on it.
I need to quickly run an open source python program that makes use of this table & although I have worked with MySQL, I have no clue about sqlite. I need the minimal basics of sqlite. Can someone guide me through this or at least tell me how to permanently store my tables.
I have put this sqlite3.exe in Python folder assuming that python would then be able to read the sqlite files. Any ideas on this?
sqlite is built in to Python.
You should be able to access your table like this:
import sqlite3
conn = sqlite3.connect('/path/to/my.db')
curs = conn.cursor()
curs.execute("SELECT a_column FROM my_table;").fetchone()
curs.close()
conn.close()
You can execute your DDL statements from Python as well.
Make sure to commit the changes.
curs.execute("CREATE TABLE my_table (a_column text);")
conn.commit()
Why did you download some sqlite3.exe at all? Python should ship with sqlite3 already on board. import sqlite3 is all you need to do as long as you have a recent Python distribution.
To your problem: I would guess that sqlite3 creates a table in memory by default. Using Python, you need to dbConn = sqlite3.connect("somefile") to connect to a database. Then you can use dbCursor = dbConn.cursor() to connect to this file. The cursor can execute SQL commands by calling its execute(command) method. For example:
dbConn.execute("create table test (row text, otherrow real)")
Finally, you need to call dbConn.commit() to save everything you changed in the database.
The Python documentation knows everything else:
http://docs.python.org/library/sqlite3.html
Just execute sqlite3 foo.db? This will permanently store everything you do afterwards in this file. (No need for .backup.)