I apologize if my question seems novice, but I have hit a roadblock when it comes to assigning variables based on a string for tuples in python (2.7).
In the past I have had no issues assigning a variable and using a string to give it a name (eg: rowId = '%sDays' %workoutMode). But in a tuple environment, I am having some issues.
I have three different tables I want to pull from a database and apply the same code to them. In this instance I want to pull the data and print it based on the string from the defined list. But I am having a problem assigning variables based on strings. Here is my code:
def workoutCycle():
catagories = 'Legs', 'Arms', 'Back'
for catagory in catagories:
conn = sqlite3.connect('workoutData.db')
c = conn.cursor()
c.execute ('SELECT round(%sDays, 1), round(%sDaysTotal, 1) FROM Profile_%s' %(catagory, catagory, catagory))
originalData = c.fetchall()
('%sDays' %catagory, '%sDaysTotal' %catagory) = tuple(originalData[0])
print originalData
print '%sDays' %catagory
print '%sDaysTotal' %catagory
This code returns:
SyntaxError: invalid syntax
I've tried different modifications and I'm not having luck. Is there a specific format that I am missing for strings/tuples?
**********EDIT**************
It seems like I wasn't very clear with what I was trying to do. Essentially I'm trying to create a loop to replace having to write a code out for each catagory. So for example I want to print the data pertaining to "Arms", this code works:
def armCycle():
conn = sqlite3.connect('workoutData.db')
c = conn.cursor()
c.execute ('SELECT round(ArmsDays, 1), round(ArmsDaysTotal, 1) FROM Profile_Arms')
#originalData = c.fetchall()
originalData = c.fetchall()
(ArmsDays, ArmsDaysTotal) = tuple(originalData[0])
print originalData
print ArmsDays
print ArmsDaysTotal
I'm trying to create a code that is a little more dynamic than just creating a function for each catagory. I'm sure I'm going about this the wrong way. I apologize, I'm new to programming.
It looks like you want to create new variables (ArmsDays, ArmsDaysTotal) that are named on the basis of the variable category that contains the value "Arms". The simplest and usual way to do this is not to create individual variables dynamically, but instead use a dict.
c.execute ('SELECT round(%sDays, 1), round(%sDaysTotal, 1) FROM Profile_%s' %(category, category, category))
originalData = c.fetchall()
my_dynamic_data = {}
my_dynamic_data['%sDays' %category] = originalData[0][0]
my_dynamic_data['%sDaysTotal' %category] = originalData[0][1]
The keys of the dict are formed on exactly the same principle as the way you construct the column names in the select statement. Then, instead of
print '%sDays' %category
print '%sDaysTotal' %category
do
print my_dynamic_data['%sDays' %category]
print my_dynamic_data['%sDaysTotal' %category]
You can create new variables dynamically in Python, but if you need to ask how, you shouldn't be doing that. You need a very good reason not to use a dict for this.
Related
I want to query a sqlite database using a f-string for the query, it is supposed to return a small description given a specific id that i stored in a local variable var. However some fields will be empty for some values of the variable. I'm specially interested in the value 0 has it will always return an empty description and is an isolated case that needs to be addressed differently from other values.
I want to iterate through a range until I find a not empty description field.
Here is an example:
var = str(self.some_global_variable) # In this case consider var = '0'
query = f'my sql query goes here WHERE this_id={var}'
description = self.function_to_fetch_db(query)
while not description and var == '0':
for i in range (1, 31):
var = str(i)
description = self.function_to_fetch_db(query)
print(description, var)
print(query)
The output of this will be something like:
[] 1
my sql query goes here WHERE this_id=0
[] 2
my sql query goes here WHERE this_id=0
[] 3
my sql query goes here WHERE this_id=0
.
.
.
The local variable is updated but the query always keeps the original value from outside the while loop.
I also tried an if...else instead of the while loop but the result is the same.
I don't think the SQLite part of the problem is relevant, it's just to illustrate my specific case, the query works for other values. I'm just having trouble to figure out this local variable and f-string relationship.
There are two answers to your questions: the first one is formal, the second one is correct.
The formal answer: the string is computed once before the loop in your case. If you want it to alter it for every value, move it inside:
for i in range(31):
query = f'my sql query goes here WHERE this_id={i}'
description = self.function_to_fetch_db(query)
The correct answer: use parameterized queries instead, that will look like:
conn.execute('my sql query goes here WHERE this_id=?', (i,))
(The substitution syntax may vary depending on your database / driver.)
you could use a named variable in a string:
var = str(self.some_global_variable) # In this case consider var = '0'
fquery = 'my sql query goes here WHERE this_id={var}'
description = self.function_to_fetch_db(fquery.format(var=var))
while not description and var == '0':
for i in range(1, 31):
query = fquery.format(var=i)
description = self.function_to_fetch_db(query)
print(description, i)
print(query)
I am new to python (as I am sure you can tell from my code) and have basically written a sample script to test some core commands so I can better understand how things work. I have gotten everything running as designed except the last "insert" command -- after hours of googling and experimenting I cannot figure out what's wrong, so I would appreciate it if anyone can show me what needs to be changed and helping me understand why (I'm sure it's basic, but am stumped!).
Below is the line that is giving me trouble:
c.execute("INSERT OR IGNORE INTO {tn} ({cn1}, {cn2}, {cn3}, {cn4}) VALUES ({VID}, {VSnu}, {VIN}, {VName})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, VID=ID, VSnu=Snu, VIN=IN, VName=Name))
Here's the entire script for context, if that helps:
import sqlite3
sqlite_file = '/test_database.sqlite' # name of the sqlite database file
table_name = 'test_table'
column1 = 'my_1st_column'
column2 = 'my_2nd_column'
column3 = 'my_3rd_column'
column4 = 'my_4th_column'
ID = int(123456)
Base = 'Arnold'
Snu = 'test'
conn = sqlite3.connect(sqlite_file)
c = conn.cursor()
c.execute("UPDATE {tn} SET {cn2}=('Snu'), {cn3}=('Muh'), {cn4}=('Arnold_A') WHERE {cn1}=({NID})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, NID=ID))
i = 1
while(i<15):
if i == 1: IN = 'B'
if i == 2: IN = 'C'
if i == 3: IN = 'D'
if i == 4: IN = 'E'
if i == 5: IN = 'F'
if i == 6: IN = 'G'
if i == 7: IN = 'H'
if i == 8: IN = 'I'
if i == 9: IN = 'J'
ID = ID+1
i = i+1
Name = Base + '_' + IN
params = (Snu, IN, Name)
c.execute("INSERT OR IGNORE INTO {tn} ({cn1}, {cn2}, {cn3}, {cn4}) VALUES ({VID}, {VSnu}, {VIN}, {VName})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, VID=ID, VSnu=Snu, VIN=IN, VName=Name))
if(i == 10): break
conn.commit()
conn.close()
It will write the "VID" item just fine if isolated (which is an integer column and the primary key of the table), but everything after that it interprets as a column and I get the "no such column: [value]" error.
I get this error regardless of what comes after "VID" -- be it any variable, as demonstrated with the command above, or if I try to insert a direct string value. These other columns are simply text columns, if that helps.
I know you are just learning, so don't be scared by what I'm going to say next :-)
You are improperly forming SQL commands. There's a special feature known as parameter substitution that is present in many database APIs, including that of SQLite in Python.
You don't want to concatenate values like you are doing, because it opens up room for something called SQL injection. In simple scenarios like yours it may not a problem, but when people do it in services exposed on the internet, it can cause great damage (data theft, data loss, data corruption etc). This comic illustrates the problem: https://xkcd.com/327/
In your case, writing the SQL command with parameter substitution is not only going to make your code safer and easier to read, it will also solve the problem you are seeing.
See the example below:
import sqlite3
conn = sqlite3.connect(":memory:")
c = conn.cursor()
# Here we don't need parameter substitution
c.execute("CREATE TABLE mytable (name text, quantity int)")
# We set the values somewhere else. For example, this could come from a web
# form
entry = ("myname", 2)
# Now we execute the query, with the proper parameter substitution.
# Note the `?` characters. Python will automatically replace those with the
# values in `entry`, making sure that things are right when forming the
# command before passing to SQLite.
c.execute("INSERT OR IGNORE INTO mytable VALUES (?, ?)", entry)
print(c.execute("SELECT * from mytable").fetchall())
The example above assumes you don't need to parametrize the column names. In your case, you are apparently reading all columns, so you don't really need to pass all the names, just do like I did in the example above and it will read all columns. If you really need column names to be parameters to read a subset of data, then you'll have to resort to a concatenation mechanism similar to what you were using. But in that case, as always, be really careful with user input to make sure it's not forming bad queries and commands.
More details here: https://docs.python.org/3/library/sqlite3.html
im very new to Python but want to preform some mathmatic functions using Python's libraries getting interger values from a mysql table i have running,
ive sucessfully established a connection using mysql.connector however im at a loss,
I can select and print Rows and columbs but im unsure of the Syntax to physically define my query as an "x" or "y" in order to preform mathmatic operations with the varible.
Any help would be greatly appreciated.
EDIT
sql_select_Query = "select * from ATABLE"
cursor = mySQLconnection .cursor()
cursor.execute(sql_select_Query)
records = cursor.fetchall()`
and
for row in records:
print("Name = ", row[1], )
print("X_num = ", row[2])
print("Y_num = ", row[3])
print("Signal_Strength = ", row[4], "\n")
cursor.close()
gives me as an example
Name = X,
X_num = Y,
Y_num = Z,
SS = Q
what i would prefer in my selection operation is to define the X , Y, Z, Q to a Global name that i could then use for atleast my application math operations using Numpy libraries for example being able to perform an operator
X*Y-Z+Q
I hope that is a bit clearer
From the gate, I would recommend following the advice of this thread highlighting the use of select *. Turning a field into an integer is possible with your SQL selection statement int the way of CAST or CONVERT. Sort of like this (my daily language is SQL Server; check out the mysql documentation for exacts):
sql_select_Query = "select Name, CAST(X as INT),CAST(Y as BIGINT) from ATABLE"
In my personal experience, SQL tends to age better than Python (tongue in cheek). Aside, if your SQL instance is on a server; I code to the workhorse as error catching is better.
But coming from it in the other direction, if you want these elements to be re-callable later, I'm suggest fetching your feedback into a dictionary.
Information about Python dictionaries can be found here. At least that way, you're pretty much working from a global but fairly structured set of captured data.
It is a bad idea to play with locals() and globals() if you don't exactly know what you're doing. Create a dictionary.
sql_select_Query = "select * from ATABLE"
cursor = mySQLconnection.cursor()
cursor.execute(sql_select_Query)
records = cursor.fetchall()
columns = [item[0] for item in cursor.description] # Grab the table column names
for record in records:
# Create a dictionary {column_name: value, ...} for each row
variable_dict = dict(zip(columns, record))
print("X variable is: ", variable_dict['X'])
# <Calculation here>
You can also configure MySQL to return values as a dictionary but this is probably an easier starting point.
This way, your "variable X" value would just be variable_dict['X'] and there's no need to make any global values other than the dictionary.
I am trying to streamLine queries to SQLITE3. I use it for financial price modelling and so am re-using the same basic query alot, but have to keep changing the hard coding to get out different column queries each time. So I want a generic query where I just write in what I want once, then it spits out the columns as lists. This is a basic version of what I want but basically still hard coded so you can see what I am trying to create.
dbName = 'NASDAQ_Equities'
ticker = 'AAPL'
def pullDataTest(dbPathName, ticker, *args):
datep = []
openp = []
highp = []
db = sqlite3.connect(dbPathName + '.mydb', detect_types=sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES, timeout=3)
cursor = db.cursor()
cursor.execute('''SELECT ''' + str(args) + ''' FROM ''' + ticker)
for row in cursor:
datep.append(row[0])
openp.append(row[1])
highp.append(row[2])
pullData(dbName, ticker, 'datep', 'openp', 'highp')
At the moment I am lost on how to enter an *args into the select statement as it rejects it because of the () brackets. Also what will be an issue is creating empty lists and appending to those lists from from an *args. Would it be better to create a order dict to append to, then brake that into lists at the end somehow? On returning values for use later down the track I was thinking of making them globals? Any suggestions? Thanks
I have a table with three columns, cell, trx and type.
This is the query I'm trying to run:
db.execute("SELECT cell,trx FROM tchdrop").fetchall()
It gives the correct output.
However when I try a = ("cell", "trx") and then
db.execute("SELECT ?,? FROM tchdrop", t).fetchall()
the output is [(u'cell', u'trx'), (u'cell', u'trx')] (which is wrong)
I'm doing this to figure out how to extract columns dynamically which is a part of a bigger problem.
The place holder (?) of python DB-API (like sqlite3) don't support columns names to be passed, so you have to use python string formatting like this:
a = ("cell", "trx")
query = "SELECT {0},{1} FROM tchdrop".format(*a)
db.execute(query)
EDIT:
if you don't know the length of the columns that you want to pass , you can do something like this:
a = ("cell", "trx", "foo", "bar")
a = ", ".join(a)
query = "SELECT {0} FROM tchdrop".format(a)
# OUTPUT : 'SELECT cell, trx, foo, bar FROM tchdrop'
db.execute(query)
The library replaces the specified values ("cell", "trx") with their quoted SQL equivalent, so what you get is SELECT "cell", "trx" FROM tchdrop. The result is correct.
What you are trying to achieve is not possible with the ? syntax. Instead, do string replacement yourself. You can check column names with regular expressions (like ^[a-zA-Z_]$) for more security.
For example:
columns = ",".join(("cell", "trx"))
db.execute("SELECT %s FROM tchdrop" % columns).fetchall()