I'm working on a project where I need to get data from my SQL Server, but there is a catch. In total there is around 100.000 rows in the specific column I need the data out of but I only need the last 20.000 - 30.000 rows of it.
I use the casual connection string and stored procedure but is there a way to select a specific row to start from? (for example let it start at row 70.000)
try:
CONNECTION_STRING = 'DRIVER='+driver+';SERVER='+server+';DATABASE='+databaseName+';UID='+username+';PWD='+ password
conn = pyodbc.connect(CONNECTION_STRING)
cursor = conn.cursor()
storedproc = "*"
cursor.execute(storedproc)
row = cursor.fetchone()
while row:
OID = ((int(row[1])))
print(OID)
So my question: is there a way (for example) set cursor.fetchone to row 70.000 instead of 1? Or is there another way to do that?
Thanks in advance!
Related
I have established connection with SQL using below code and have extracted the data from SQL table, converted into dataframe and ran the predictive model. I have the output generated and want to add the values of output column alone in the database based on Unique ID column.
server = 'Servername'
database = 'DBName'
username = 'username'
password = 'password'
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
sql ='SELECT * FROM TableName'
DF= pd.read_sql(sql,cnxn)
I have columns 'UniqueID','Description','Date','Predicted' in dataframe 'DF' which is retrieved from database. I have predicted the output 'Predicted' and is available in my dataframe. I need to overwrite back only the value in 'Predicted' column of the database based on UniqueID.
Please let me know if there is any way out or we can just overwrite complete dataframe to database table.
The best method I've found is to take advantage of an SQL inner join and temporary tables to update the values. This works well if you need to update many records in SQL.
Apologies if there are any errors here as I'm borrowing this from a class I've written.
SQL Cursor
cursor = cnxn.cursor()
# reduce number of calls to server on inserts
cursor.fast_executemany = True
Insert Values into a Temporary Table
# insert only the key and the updated values
subset = DF[['UniqueID','Predicted']]
# form SQL insert statement
columns = ", ".join(subset.columns)
values = '('+', '.join(['?']*len(subset.columns))+')'
# insert
statement = "INSERT INTO #temp_TableName ("+columns+") VALUES "+values
insert = [tuple(x) for x in subset.values]
cursor.executemany(statement, insert)
Update Values in Main Table from Temporary Table
statement = '''
UPDATE
TableName
SET
u.Predicted
FROM
TableName AS t
INNER JOIN
#temp_TableName AS u
ON
u.UniqueID=t.UnqiueID;
'''
cursor.execute(statement)
I'm creating a program where I need to check if a certain cell in a table equals a string value and, if it does not, to not change that value. Here is some snippet of the code for clarification:
if (db.execute("SELECT :rowchosen FROM userboard WHERE column=:columnchosen", rowchosen = rowchosen, columnchosen = columnchosen)) == '-'):
#change value of cell
else:
#go to a new page that displays an error
Yet, whenever I run this code, I always get an error because the value (I believe) prints as a dictionary value, something like {"row" = 'row'} of that sort. Any help/advice as to why this happens?
Are you sure that userboard is the database and not the table?
i think, here is what you want to do
conn = sqlite3.connect(db_file)
cur = conn.cursor()
cur.execute("SELECT * FROM userboard WHERE one=?", (columnchosen,))
rows = cur.fetchall()
for row in rows:
print(row)
now, in the loop for row in rows: you need to perform your check. For all the rows returned, you need to check each row for - in the appropriate column
also check out http://www.sqlitetutorial.net/sqlite-python/sqlite-python-select/
I am attempting to parse a very big MySQL table that potentially may not fit in memory. The approach that I am following is, using pymysql:
db = PyMySQL.connect(**connection_params)
cur = db.cursor()
cur.execute('SELECT * FROM big_table')
for row in cur:
process(row)
What I am observing is that cur.execute() eagerly loads the data into memory. Is it possible to iterate by rows lazily?
I am aware this could be done combining LIMIT and OFFSET clauses, but is it possible to be done in a more transparent way?
You can get the number of results with that (after cur.execute):
numrows = cur.rowcount
Then, you can iterate over them with a simple for:
for num in xrange(0,numrows):
row = cursor.fetchone()
do stuff...
I am new in python so in the pyodbc. Maybe my question is very simple but I could not find any answer refer to my question.
I'm using this select
cursor.execute("SELECT [something] FROM [someone] WHERE [user_name]='John'")
rows = cursor.fetchall()
for row in rows:
print row.something
It prints some parameters for example 4 or 5.How to print only second or only third parameter.
I also used cursor.fetchmany() but I'm having same problem
If you wan't just the 4th row you can do:
rows = cursor.fetchall()
print rows[3].something
But it's better if you do it in the SQL query and avoid fetching all the rows from the database:
SELECT [something] FROM [someone] WHERE [user_name]='John' LIMIT 1 OFFSET 3
Example.
I guess your mean field and not parameter
cursor.execute("SELECT [something] FROM [someone] WHERE [user_name]='John'")
rows = cursor.fetchall()
from row in rows:
print row[1]
As I know, the SELECT syntax is used for getting data from a row instead of column, here I have a column called time in a table, and I just want to select all the data in this time column, put them in an array, and use this array later.
So how can I select a column of data and put them into an array?
The query: SELECT time FROM Table
Use this query to populate an array in python!
db = MySQLdb.connect(user="yourUser",passwd="1337",db="awesomeDB")
cursor = db.cursor()
resultSet = "SELECT time FROM tableX"
cursor.execute(resultSet)
for row in cursor
#do something here, maybe add to an array if you want
arrayList.append(row)
Something like this?