Develop Reference

sqlite retrieving items from table containing string in field

I am trying to retrieve items from a SQLite db table in Python where there is a match in a particular field. In other words if I search for 'rabbit' I want to retrieve all entries that have the string 'rabbit' in a particular column. My code looks like this:
Python server code for endpoint:
if self.path=='/getOne':
form = cgi.FieldStorage(
fp=self.rfile,
headers=self.headers,
environ={'REQUEST_METHOD':'POST',
'CONTENT_TYPE':self.headers['Content-Type'],
})
value = []
for key in form.keys():
value.append("%" + form.getvalue(key) + "%")
print 'LOOK my value', value
c.execute('select * from appointments where description=?' , value)
res = c.fetchall()
# _json = json.dumps(res)
# print 'This is res from _get_all_appts', res
# print 'From line 18: ', _json
# self.wfile.write(_json)
print "I'm ya huckleberry", res
return
What is printing in console:
From line 18: [["15:01", "asdf", "2020-05-07"], ["14:01", "test",
"2020-04-04"]]
LOOK my value ['%test%']
I'm ya huckleberry []
LOOK my value ['%test%']
I'm ya huckleberry []
As you can see what is printing out on line 18 are the entries on my table.
My value ['%test%] should return the second entry since I want to return any entry that contains the string test in that particular column but I get nothing. I come from a JS background and would easily do this with string interpolation/template strings. Is there anything anyone can suggest that would help bring the desired effect? I would greatly appreciate it. Thank you ahead of time!

For the answer to your first part of the question, the LIKE operator might do the trick:
SELECT *
FROM appointments
WHERE description LIKE '%rabbit%';
If this does not meet your expectations, then you could try looking into full text search with SQLite. One reason the above query might fall short for you is that it would match the substring rabbit occurring anywhere. For example, it would also match rabbits. Full text search would get around most of these edge cases.
To make the term inside the LIKE expression dynamic, you would use Python code along these lines:
param = 'rabbit'
t = ('%'+param+'%',)
c.execute('select * from appointments where description like ?', t)
c.fetchall()

Pass values from python list to Cassandra query

Beginner here.
I have the following circumstances.
A text file with each line containing a name.
A cassandra 3.5 database
A python script
The intention is to have the script read from the file one line (one name) at a time, and query Cassandra with that name.
FYI, everything works fine except for when I try to pass the value of the list to the query.
I current have something like:
#... driver import, datetime imports done above
#...
with open(fname) as f:
content = f.readlines()
# Loop for each line from the number of lines in the name list file
# num_of_lines is already set
for x in range(num_of_lines):
tagname = str(content[x])
rows = session.execute("""SELECT * FROM tablename where name = %s and date = %s order by time desc limit 1""", (tagname, startDay))
for row in rows:
print row.name + ", " + str(row.date)
Everything works fine if I remove the tagname list component and edit the query itself with a name value.
What am I doing wrong here?

Simply building on the answer from #Vinny above, format simply replaces literal value. You need to put quotes around it.
for x in content:
rows = session.execute("SELECT * FROM tablename where name ='{}' and date ='{}' order by time desc limit 1".format(x, startDay))
for row in rows:
print row.name + ", " + str(row.date)

You can simply iterate over content:
for x in content:
rows = session.execute("SELECT * FROM tablename where name = {} and date = {} order by time desc limit 1".format(x, startDay))
for row in rows:
print row.name + ", " + str(row.date)
....
Also, you don't need to have 3 quotes for the string. Single quotes is good enough (3 quotes is used for documentation / multiple line comments in python)
Note that this might end in a different error; but you will be iterating on the lines instead of iterating over an index and reading lines.

Python format array list into string

I'm looking to take an array list and attach it to a string.
Python 2.7.10, Windows 10
The list is loaded from a mySQL table and the output is this:
skuArray = [('000381001238',) ('000381001238',) ('000381001238',) ('FA200513652',) ('000614400967',)]
I'm wanting to take this list and attach it to a separate query
the problem:
query = "SELECT ItemLookupCode,Description, Quantity, Price, LastReceived "
query = query+"FROM Item "
query = query+"WHERE ItemLookupCode IN ("+skuArray+") "
query = query+"ORDER BY LastReceived ASC;"
I get the error:
TypeError: cannot concatenate 'str' and 'tuple' objects
My guess here is that I need to format the string as:
'000381001238', '000381001238', '000381001238', 'FA200513652','000614400967'
Ultimately the string needs to read:
query = query+"WHERE ItemLookupCode IN ('000381001238', '000381001238', '000381001238', 'FA200513652','000614400967') "
I have tried the following:
skuArray = ''.join(skuArray.split('(', 1))
skuArray = ''.join(skuArray.split(')', 1))
Second Try:
skus = [sku[0] for sku in skuArray]
stubs = ','.join(["'?'"]*len(skuArray))
msconn = pymssql.connect(host=r'*', user=r'*', password=r'*', database=r'*')
cur = msconn.cursor()
query ='''
SELECT ItemLookupCode,Description, Quantity, Price, LastReceived
FROM Item
WHERE ItemLookupCode IN { sku_params }
ORDER BY LastReceived ASC;'''.format(sku_params = stubs)
cur.execute(query, params=skus)
row = cur.fetchone()
print row[3]
cur.close()
msconn.close()
Thanks in advance for your help!

If you want to do the straight inline SQL you could use a list comprehension:
', '.join(["'{}'}.format(sku[0]) for sku in skuArray])
Note: You need to add commas between tuples (based on example)
That said, if you want to do some sql, I would encourage you to parameterize your request with ?
Here is an example of how you would do something like that:
skuArray = [('000381001238',), ('000381001238',), ('000381001238',), ('FA200513652',), ('000614400967',)]
skus = [sku[0] for sku in skuArray]
stubs = ','.join(["'?'"]*len(skuArray))
qry = '''
SELECT ItemLookupCode,Description, Quantity, Price, LastReceived
FROM Item
WHERE ItemLookupCode IN ({ sku_params })
ORDER BY LastReceived ASC;'''.format(sku_params = stubs)
#assuming pyodbc connection syntax may be off
conn.execute(qry, params=skus)
Why?
Non-parameterized queries are a bad idea because it leaves you vulnerable to sql injection and is easy to avoid.

Assuming that skuArray is a list, like this:
>>> skuArray = [('000381001238',), ('000381001238',), ('000381001238',), ('FA200513652',), ('000614400967',)]
You can format your string like this:
>>> ', '.join(["'{}'".format(x[0]) for x in skuArray])
"'000381001238', '000381001238', '000381001238', 'FA200513652', '000614400967'"

Inserting data into mysql using python

I'm trying to execute the following code
for counter in range(0, len(weatherData) - 1):
string = 'INSERT INTO weather VALUES(' + str((weatherData[counter])[0:]) +');'
print(string)
cur.execute(string)
All the the values and data are printed correctly and everything seems to work as there is no errors, however when I check the data base its empty.

Do commit after insertion complete.
for counter in range(0, len(weatherData) - 1):
....
connection_object.commit()
BTW, you'd better to use parameter-passing style instead of build query string yourself:
for data in weatherData:
sql = 'INSERT INTO weather VALUES(%s)'
cur.execute(sql, [data])

The loop can be simplified as follows
for counter, row in weatherData:
string = 'INSERT INTO weather VALUES(' + str(row) + ');'
and you'll need to commit afterwards
EDIT - added counter, which is a count of the for loop
EDIT2 - using row
EDIT3 - removed [0:] from end of string which doesn't do anything

combine lines from 2 prints to single line and insert into mysql database

Hello everyone i currently have this:
import feedparser
d = feedparser.parse('http://store.steampowered.com/feeds/news.xml')
for i in range(10):
print d.entries[i].title
print d.entries[i].date
How would i go about making it so that the title and date are on the same line? Also it doesn't need to print i just have that in there for testing, i would like to dump this output into a mysql db with the title and date, any help is greatly appreciated!

If you want to print on the same line, just add a comma:
print d.entries[i].title, # <- comma here
print d.entries[i].date
To insert to MySQL, you'd do something like this:
to_db = []
for i in range(10):
to_db.append((d.entries[i].title, d.entries[i].date))
import MySQLdb
conn = MySQLdb.connect(host="localhost",user="me",passwd="pw",db="mydb")
c = conn.cursor()
c.executemany("INSERT INTO mytable (title, date) VALUES (%s, %s)", to_db)

Regarding your actual question: if you want to join two strings with a comma you can use something like this:
print d.entries[i].title + ', ' + str(d.entries[i].date)
Note that I have converted the date to a string using str.
You can also use string formatting instead:
print '%s, %s' % (d.entries[i].title, str(d.entries[i].date))
Or in Python 2.6 or newer use str.format.
But if you want to store this in a database it might be better to use two separate columns instead of combining both values into a single string. You might want to consider adjusting your schema to allow this.