I have 2 tables, "vector" and "vocab." I'm trying to do this:
c.execute('SELECT value FROM vector WHERE word IN (SELECT word FROM vocab)')
I'm getting the error sqlite3.OperationalError: no such table: vocab Of course, this is because I haven't connected to the vocab table. I only connected to the vector table before:
dbname = "/Users/quantumjuker/NLP/vector.db"
conn = sqlite3.connect(dbname)
c = conn.cursor()
How can I connect to the vocab table as well so I don't receive an error?
Thanks!
So SQLite3 has an ability to read additional SQLite data files. This is done using the ATTACH command. The nice thing about it is that it is used as an sql query. So you do something like:
c.execute("ATTACH 'vocab.db' AS 'vocabulary'");
Note the AS aliases the database to a name, not the table to a name. Once this is done you can run your query against the vocab table as well.
Related
I am trying to access tables from a database using python. There was some code on the website: https://rnacentral.org/help/public-database
import psycopg2.extras
def main():
conn_string = "host='hh-pgsql-public.ebi.ac.uk' dbname='pfmegrnargs' user='reader' password='NWDMCE5xdipIjRrp'"
conn = psycopg2.connect(conn_string)
cursor = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)`
# retrieve a list of RNAcentral databases
query = "SELECT * FROM rnc_database"
cursor.execute(query)
for row in cursor:
print(row)`
When i run this code, i get back a list of databases:
I want to access tables from one of these databases but I don't know what the schema for those tables are or what the values in each list returned represents. I have been looking at 'postgresql to python' resources but all of them are about accessing tables when you know the name of the tables and the columns within.... Is there code for how I can access the table names from the database?
Thank You
Edit: sorry, i thought i linked the website before
The dataset you want to use has schema diagram here https://rnacentral.org/help/public-database
For general purpose I would use something like https://dbeaver.io/ tool it will show you all the schemas in the db and tables inside the schema and so forth. The DBeaver settings to connect to your db would look like this
If you want to keep using python script to explore the db this sql query
SELECT *
FROM pg_catalog.pg_tables
WHERE schemaname != 'pg_catalog' AND
schemaname != 'information_schema';
Should help you.
I have a MySQL database of some measurements taken by a device and I'm looking for a way to retrieve specific columns from it, where the user chooses what columns he needs from a python interface/front end. All the solutions I've seen till now either retrieves all columns or had the columns specified in the code itself.
Is there a possible way I could do this?
Thanks!
Your query can look something like this :
select
table_name, table_schema, column_name
from information_schema.columns
where table_schema in ('schema1', 'schema2')
and column_name like '%column_name%'
order by table_name;
you can definitely pass the column_name as a parameter(fetch it from python code) run it dynamically.
import MySQLdb
#### #GET COLUMN NAME FROM USER PRESENT WITH IN TABLE
column = input()
#### #Open database connection
db = MySQLdb.connect("host","username","password","DB_name" )
#### #prepare a cursor object using cursor() method
cursor = db.cursor()
#### #execute SQL query using execute() method.
cursor.execute("SELECT * FROM TABLE")
# Fetch a all rows using fetchall() method.
result_set = cursor.fetchall()
for row in result_set:
print(row[column])
# disconnect from server
db.close()
OR you can use .execute() to run a specific query with column name.
I usually use R to do SQL queries by using ODBC to link to a SQL database. The code generally looks like this:
library(RODBC)
ch<-odbcConnect('B1P HANA',uid='****',pwd='****')
myOffice <- c(0)
office_clause = ""
if (myOffice != 0) {
office_clause = paste(
'AND "_all"."/BIC/ZSALE_OFF" IN (',paste(myOffice, collapse=", "),')'
)
}
a <- sqlQuery(ch,paste(' SELECT "_all"."CALDAY" AS "ReturnDate FROM "SAPB1P"."/BIC/AZ_RT_A212" "_all"
WHERE "_all"."CALDAY"=20180101
',office_clause,'
GROUP BY "_all"."CALDAY
'))
The workflow is:
odbcConnect is to link R and SQL using ODBC.
myOffice is an array for achieving data from R. Those data will be used as filter conditions in WHERE clause in SQL.
a stores the query result from SQL database.
So, how to do all of these in Python, i.e., do SQL queries in Python by using ODBC to link SQL database and Python? I am new to Python. All I know is like:
import pyodbc
conn = pyodbc.connect(r'DSN=B1P HANA;UID=****;PWD=****')
Then I do not know how to continue. And I cannot find an overall example online. Could anyone help by providing a comprehensive example? From link SQL database in Python unitl retrieving the result?
Execute SQL from python
Instantiate a Cursor and use the execute method of the Cursor class to execute any SQL statement.
cursor = cnxn.cursor()
Select
You can use fetchall, fetchone, and fetchmany to retrieve rows returned from SELECT statements:
import pyodbc
cursor = cnxn.cursor()
cnxn = pyodbc.connect('DSN=myDSN;UID=***;PWD=***')
cursor.execute("SELECT Col1, Col2 FROM MyTable WHERE Col1= 'SomeValue'")
rows = cursor.fetchall()
for row in rows:
print(row.Col1, row.Col2 )
You can provide parameterized queries in a sequence or in the argument list:
cursor.execute("SELECT Col1, Col2, Col3, ... FROM MyTable WHERE Col1 = ?", 'SomeValue',1)
Insert
INSERT commands also use the execute method; however, you must subsequently call the commit method after an insert or you will lose your changes:
cursor.execute("INSERT INTO MyTable (Col1) VALUES ('SomeValue')")
cnxn.commit()
Update and Delete
As with an insert, you must also call commit after calling execute for an update or delete:
cursor.execute("UPDATE MyTable SET Col1= 'SomeValue'")
cnxn.commit()
Metadata Discovery
You can use the getinfo method to retrieve data such as information about the data source and the capabilities of the driver. The getinfo method passes through input to the ODBC SQLGetInfo method.
cnxn.getinfo(pyodbc.SQL_DATA_SOURCE_NAME)
I have a sqlite database that I want to extract specific tables from.
The database has thousands of table names.
I'm interested only in tables that startwith "contact_"
However there are many that are contactOLD, contact#### you name it.
I then need to extract the row data from each "contact_########" table and create a CSV or spreadsheet type document. In total there are 1600 or so with unique names.
I had initially thought I could do this with a sqlite query but could not.
I then tried to write a small script to do this but I could not figure out how to setup conditionals for the cursor.execute to only grab the data from the tables of interest to me.
Any ideas?
Update**
import sqlite3
fname = raw_input("Enter your filename: ")
con = sqlite3.connect(fname)
cursor = con.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
mydata = cursor.fetchall()
for lines in mydata:
print lines
I have been able to get the tables to list. I still need to add a condition to my WHERE for "contact_" When I add it, I get only [] on my print. I think I'm messing something up here.
****Update 2***
Thanks to #Olver W. below who got me on the right track with this.
fname = raw_input("Enter your filename: ")
con = sqlite3.connect(fname)
cursor = con.cursor()
for tablename in cursor.execute("SELECT name FROM sqlite_master WHERE type='table';"):
if tablename[0].startswith('Contacts_'):
tablename = str(tablename[0])
query = "Select * FROM " + tablename
query = str(query)
data = cursor.execute(query)
for items in data:
print items
I'm going to output this to a spreadsheet in some cases or another SQLite database but in my test database it is selecting the appropriate criteria and outputting the rows correctly. I can and will condense it a bit to make it cleaner but it does the trick. Thanks
As mentioned in the comments, you could first query all the table names, then go through them and check if the name condition is fulfilled, which can be done in Python:
for tablename in cur.execute('SELECT name FROM sqlite_master WHERE type="table";'):
if tablename[0].startswith('conn_'):
execute_some_query_using_this_table()
When using mclient it is possible to list all tables in database by issuing command '\d'. I'm using python-monetdb package and I don't know how the same can be accomplished. I've seen example like "SELECT * FROM TABLES;" but I get an error that "tables" table does not exist.
In your query you need to specify that you are looking for the tables table that belongs to the default sys schema, or sys.tables. The SQL query that returns the names of all non-system tables in MonetDB is:
SELECT t.name FROM sys.tables t WHERE t.system=false
In Python this should look something like:
import monetdb.sql
connection = monetdb.sql.connect(username='<username>', password='<password>', hostname='<hostname>', port=50000, database='<database>')
cursor = connection.cursor()
cursor.execute('SELECT t.name FROM sys.tables t WHERE t.system=false')
If you are looking for tables only in a specific schema, you will need to extend your query, specifying the schema:
SELECT t.name FROM sys.tables t WHERE t.system=false AND t.schema_id IN (SELECT s.id FROM sys.schemas s WHERE name = '<schema-name>')
where the <schema-name> is your schema, surrounded by single quotes.