I'm trying to add the list that is made after it parses through each line. As I go through each code I get different errors
(C:\Users\myname\Desktop\pythonCourse>dblesson2
Enter file name: mbox.txt
['uct.ac.za']
Traceback (most recent call last):
File "C:\Users\myname\Desktop\pythonCourse\dblesson2.py", line 25, in
<module>
#VALUES ( ?, 1 )''', ( email, ) )
sqlite3.OperationalError: near "#VALUES": syntax error)
and I know that it is because I am not passing the correct data to the database but I can't figure this out on my own.
import sqlite
import re
conn = sqlite3.connect('emaildb.sqlite')
cur = conn.cursor()
cur.execute('''
DROP TABLE IF EXISTS Counts''')
cur.execute('''
CREATE TABLE Counts (email TEXT, count INTEGER)''')
fname = raw_input('Enter file name: ')
if ( len(fname) < 1 ) : fname = 'mbox-short.txt'
fh = open(fname)
for line in fh:
if not line.startswith('From: ') : continue
line = line.rstrip()
email = re.findall('#(\S+[a-zA-Z]+)', line)
print email
cur.execute('SELECT count FROM Counts WHERE email = ? ', (email))
row = cur.fetchone()
if row is None:
#cur.execute('''INSERT INTO Counts (email, count)
#VALUES ( ?, 1 )''', ( email, ) )
else :
cur.execute('UPDATE Counts SET count=count+1 WHERE email = ?',
(email, ))
# This statement commits outstanding changes to disk each
# time through the loop - the program can be made faster
# by moving the commit so it runs only after the loop completes
conn.commit()
# https://www.sqlite.org/lang_select.html
sqlstr = 'SELECT email, count FROM Counts ORDER BY count DESC LIMIT 10'
print
print "Counts:"
for row in cur.execute(sqlstr) :
print str(row[0]), row[1]
cur.close()`
You have a number of small errors in your program. Let me try to list them:
re.findall returns a list, but you seem to treat it as a single string. Try email = email[0] to only consider the first element of the list.
Your first SELECT statement has (email). Putting a single item inside parentheses does not make it a tuple. Try (email,) or [email] instead.
The if after the for loop is meant to occur for each iteration of the for loop, so it must be indented by one stop.
The body of the if cannot be empty. Either uncomment that operation, or change it to pass.
The body of your final for loop needs to be indented one stop.
As a courtesy to Stack Overflow readers, please copy-paste entire stand-alone programs, not merely snippets.
Here is the your program after I fixed the problems:
import sqlite3
import re
conn = sqlite3.connect(':memory:')
cur = conn.cursor()
cur.execute('''
DROP TABLE IF EXISTS Counts''')
cur.execute('''
CREATE TABLE Counts (email TEXT, count INTEGER)''')
fname = raw_input('Enter file name: ')
if ( len(fname) < 1 ) : fname = 'mbox-short.txt'
fh = open(fname)
for line in fh:
if not line.startswith('From: ') : continue
line = line.rstrip()
email = re.findall('#(\S+[a-zA-Z]+)', line)
email = email[0]
cur.execute('SELECT count FROM Counts WHERE email = ? ', (email,))
row = cur.fetchone()
if row is None:
cur.execute('''INSERT INTO Counts (email, count)
VALUES ( ?, 1 )''', ( email, ) )
else :
cur.execute('UPDATE Counts SET count=count+1 WHERE email = ?',
(email, ))
# This statement commits outstanding changes to disk each
# time through the loop - the program can be made faster
# by moving the commit so it runs only after the loop completes
conn.commit()
# https://www.sqlite.org/lang_select.html
sqlstr = 'SELECT email, count FROM Counts ORDER BY count DESC LIMIT 10'
print
print "Counts:"
for row in cur.execute(sqlstr) :
print str(row[0]), row[1]
cur.close()
import sqlite3
conn=sqlite3.connect('emaildb.sqlite')
cur=conn.cursor()
cur.execute('''DROP TABLE IF EXISTS counts''')
cur.execute('''CREATE TABLE counts (org TEXT, count INTEGER)''')
f_name=raw_input('Enter file name: ')
if len(f_name)<1 : f_name='mbox.txt'
fn=open(f_name)
for line in fn:
if not line.startswith('From: ') : continue
words = line.split()
email=words[1]
domain=email.split('#')
organiz=domain[1]
print organiz
cur.execute('SELECT count FROM Counts WHERE org=?',(organiz, ))
row=cur.fetchone()
if row==None:
cur.execute('''INSERT INTO counts (org, count) VALUES (?,1)''',
(organiz, ))
else:
cur.execute('''UPDATE counts SET count=count+1 WHERE org=?''',(organiz,
))
conn.commit()
Related
I am trying to create a command line tool that generates a random string(password) of a given length, stores it in a sql db, and can be queried by name. The password generation and storing of it's output by a given name works beautifully, but trying to select only the password element is giving me trouble. I was able to select all from the table but that returns the name and the password. I only want the password returned. I thought about just splicing the output or even using the linux cut command, but I'd rather just get it from the select statement. Is this possible? My current SELECT statement returns: operation parameter must be a str. When I try it without the call to (name) at the end of the SELECT statement like this: query_password = """SELECT * FROM password_table WHERE name = ?"""
I get this error:
File "passbox.py", line 44, in <module>
query_pswd_by_name(name)
File "passbox.py", line 39, in query_pswd_by_name
c.execute(query_password)
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 0 supplied.
BTW I'm sure my query_pswd_by_name function is all wrong, I've been experimenting. When I just create a connection and SELECT statement outside of a function it does return the name and password.
Also note that I've disguised my database file's name with asterisks for the purpose of this post. I am using an actual working db file in practice.
Here is all the code I've written so far:
import secrets
import string
import sqlite3
#CREATE PASSWORD OF GIVEN LENGTH
def get_pass(length):
return "".join(secrets.choice(string.ascii_uppercase + string.ascii_lowercase + string.digits + string.punctuation) for x in range(length))
length = int(input("Enter the length of password: "))
password= get_pass(length)
print(password)
name = str(input("Enter name for password: "))
#CREATE DATABASE CONNECTION
conn = sqlite3.connect("****.db")
#CREATE CURSOR OBJECT
c = conn.cursor()
#CREATE TABLE IN DISK FILE BASED DATABASE
c.execute("""CREATE TABLE IF NOT EXISTS password_table (
name TEXT,
pswd TEXT
)""")
c.execute("INSERT INTO password_table (name, pswd) VALUES (?, ?)", (name, password))
#COMMIT CHANGES
conn.commit()
conn.close()
def query_pswd_by_name(name):
conn = sqlite3.connect('****.db')
c = conn.cursor()
query_password = """SELECT * FROM password_table WHERE name = ?""", (name)
c.execute(query_password)
result = c.fetchall()
for row in result:
print(row[1])
conn.commit()
query_pswd_by_name(name)
#CLOSE CONNECTION
conn.close()```
You need to break up the argument to the execute call.
c.execute(*query_password)
Or
c.execute("""SELECT * FROM password_table WHERE name = ?""", (name))
import re
import sqlite3
from collections import Counter
from string import punctuation
from math import sqrt
# initialize the connection to the database
connection = sqlite3.connect('chatbot.sqlite')
cursor = connection.cursor()
# create the tables needed by the program
create_table_request_list = [
'CREATE TABLE words(word TEXT UNIQUE)',
'CREATE TABLE sentences(sentence TEXT UNIQUE, used INT NOT NULL DEFAULT 0)',
'CREATE TABLE associations (word_id INT NOT NULL, sentence_id INT NOT NULL, weight REAL NOT NULL)',
]
for create_table_request in create_table_request_list:
try:
cursor.execute(create_table_request)
except:
pass
def get_id(entityName, text):
"""Retrieve an entity's unique ID from the database, given its associated text.
If the row is not already present, it is inserted.
The entity can either be a sentence or a word."""
tableName = entityName + 's'
columnName = entityName
cursor.execute('SELECT rowid FROM ' + tableName + ' WHERE ' + columnName + ' = ?', (text,))
row = cursor.fetchone()
if row:
return row[0]
else:
cursor.execute('INSERT INTO ' + tableName + ' (' + columnName + ') VALUES (?)', (text,))
return cursor.lastrowid
def get_words(text):
"""Retrieve the words present in a given string of text.
The return value is a list of tuples where the first member is a lowercase word,
and the second member the number of time it is present in the text."""
wordsRegexpString = '(?:\w+|[' + re.escape(punctuation) + ']+)'
wordsRegexp = re.compile(wordsRegexpString)
wordsList = wordsRegexp.findall(text.lower())
return Counter(wordsList).items()
B = 'Hello!'
while True:
# output bot's message
print('B: ' + B)
# ask for user input; if blank line, exit the loop
H = raw_input('H: ').strip()
if H == '':
break
# store the association between the bot's message words and the user's response
words = get_words(B)
words_length = sum([n * len(word) for word, n in words])
sentence_id = get_id('sentence', H)
for word, n in words:
word_id = get_id('word', word)
weight = sqrt(n / float(words_length))
cursor.execute('INSERT INTO associations VALUES (?, ?, ?)', (word_id, sentence_id, weight))
connection.commit()
# retrieve the most likely answer from the database
cursor.execute('CREATE TEMPORARY TABLE results(sentence_id INT, sentence TEXT, weight REAL)')
words = get_words(H)
words_length = sum([n * len(word) for word, n in words])
for word, n in words:
weight = sqrt(n / float(words_length))
cursor.execute('INSERT INTO results SELECT associations.sentence_id, sentences.sentence, ?*associations.weight/(4+sentences.used) FROM words INNER JOIN associations ON associations.word_id=words.rowid INNER JOIN sentences ON sentences.rowid=associations.sentence_id WHERE words.word=?', (weight, word,))
# if matches were found, give the best one
cursor.execute('SELECT sentence_id, sentence, SUM(weight) AS sum_weight FROM results GROUP BY sentence_id ORDER BY sum_weight DESC LIMIT 1')
row = cursor.fetchone()
cursor.execute('DROP TABLE results')
# otherwise, just randomly pick one of the least used sentences
if row is None:
cursor.execute('SELECT rowid, sentence FROM sentences WHERE used = (SELECT MIN(used) FROM sentences) ORDER BY RANDOM() LIMIT 1')
row = cursor.fetchone()
# tell the database the sentence has been used once more, and prepare the sentence
B = row[1]
cursor.execute('UPDATE sentences SET used=used+1 WHERE rowid=?', (row[0],))
This is a code written for creating a chatbot. When I try running this code on cmd. By using command python chatbot.py, it returns an error saying invalid syntax.
IS there any way i can remove this error and run this code on my system?
it gives error: File "chatbot.py", line 1 syntax: invalid syntax
What version of Python are you running and in what environment? I ran this code on my Python 3.70b4 under Windows and it worked fine except for line 52:
H = raw_input('H: ').strip()
Which you have to change to:
H = input('H: ').strip()
This is probably unrelated directly to your issue, but the code you posted did run fine for me in my environment, after I made that one change (and of course installed any libraries or modules needed).
It can't figure out the fetchone()[0] part and why when I change [0] to [1], [2] etc. my table is not looking so good (every emails counter is 1 and so every email is duplicated if there is more than one of the same email in the file).
import sqlite3
con = sqlite3.connect('db1.sqlite')
cur = con.cursor()
cur.execute('DROP TABLE IF EXISTS TimesSend')
cur.execute('CREATE TABLE TimesSend(email TEXT,times INTEGER)')
file = open('file.txt','r')
for row in file:
if not row.startswith('From: '):
continue
parts = row.split()
mail = parts[1]
print(mail)
cur.execute('SELECT timesFROM TimesSend WHERE email= ?', (mail,))
try:
times = cur.fetchone()[0]
cur.execute('UPDATE TimesSend SET times=times+1 WHERE email=?', (mail,))
except:
cur.execute('INSERT INTO TimesSend (email,puta) VALUES(?,1)', (mail,))
con.commit()
Your code has some issue, please try below:
import sqlite3
con = sqlite3.connect('db1.sqlite')
cur = con.cursor()
cur.execute('DROP TABLE IF EXISTS TimesSend')
cur.execute('CREATE TABLE TimesSend(email TEXT,times INTEGER)')
file = open('file.txt','r')
for row in file:
if not row.startswith('From: '):
continue
parts = row.split()
mail = parts[1]
print(mail)
cur.execute('SELECT times FROM TimesSend WHERE email= ?', (mail,))
try:
# fetchone() will be a tuple (1,),
# then you should use index 0 to select times 1
times = cur.fetchone()[0]
print(times)
# if duplicate, times+1
cur.execute('UPDATE TimesSend SET times=? WHERE email=?', (times+1, mail))
except:
cur.execute('INSERT INTO TimesSend (email,times) VALUES(?,1)', (mail,))
con.commit()
print(cur.execute('SELECT * FROM TimesSend').fetchall())
con.close()
Print output will be:
testa#gmail.com
testa#gmail.com
1
testc#gmail.com
testd#gmail.com
testb#gmail.com
[(u'testa#gmail.com', 2), (u'testc#gmail.com', 1), (u'testd#gmail.com', 1), (u'testb#gmail.com', 1)]
This application will read the mailbox data (mbox.txt) count up the number email messages per organization (i.e. domain name of the email address) using a database with the following schema to maintain the counts.
CREATE TABLE Counts (org TEXT, count INTEGER)
When you have run the program on mbox.txt upload the resulting database file above for grading.
If you run the program multiple times in testing or with different files, make sure to empty out the data before each run.
You can use this code as a starting point for your application: http://www.pythonlearn.com/code/emaildb.py. The data file for this application is the same as in previous assignments: http://www.pythonlearn.com/code/mbox.txt.
First time to learn Sqlite. I am very confused about this assignment although it seems to be easy. I don't know how can I connect Python codes to Sqlite. It seems that they don't need the code as assignment. All the need is database file. How should I solve this problem. Don't know how to start it. Much appreciated it!
The starting code you've been given is a really good template for what you want to do. The difference is that - in that example - you're counting occurences of email address, and in this problem you're counting domains.
First thing to do is think about how to get domain names from email addresses. Building from the code given (which sets email = pieces[1]):
domain = email.split('#')[1]
This will break the email on the # character, and return the second item (the part after the '#'), which is the domain - the thing you want to count.
After this, go through the SQL statements in the code and replace 'email' with 'domain', so that you're counting the right thing.
One last thing - the template code checks 'mbox-short.txt' - you'll need to edit that as well for the file you want.
import sqlite3
conn = sqlite3.connect('emaildb2.sqlite')
cur = conn.cursor()
cur.execute('''
DROP TABLE IF EXISTS Counts''')
cur.execute('''
CREATE TABLE Counts (org TEXT, count INTEGER)''')
fname = input('Enter file name: ')
if (len(fname) < 1): fname = 'mbox.txt'
fh = open(fname)
list_1 =[]
for line in fh:
if not line.startswith('From: '): continue
pieces = line.split()
email = pieces[1]
dom = email.find('#')
org = email[dom+1:len(email)]
cur.execute('SELECT count FROM Counts WHERE org = ? ', (org,))
row = cur.fetchone()
if row is None:
cur.execute('''INSERT INTO Counts (org, count)
VALUES (?, 1)''', (org,))
else:
cur.execute('UPDATE Counts SET count = count + 1 WHERE org = ?',
(org,))
conn.commit()
# https://www.sqlite.org/lang_select.html
sqlstr = 'SELECT org, count FROM Counts ORDER BY count DESC LIMIT 10'
for row in cur.execute(sqlstr):
print(str(row[0]), row[1])
cur.close()
I am still new here, but I want to thank Stidgeon for pointing me in the right direction. I suspect other Using Databases with Python students will end up here too.
There are two things you need to do with the source code.
domain = email.split('#')[1] http://www.pythonlearn.com/code/emaildb.py
Change from email TEXT to org TEXT when the database is generated.
That should get you on your way.
import sqlite3
conn = sqlite3.connect('emaildb.sqlite')
cur = conn.cursor()
cur.execute('DROP TABLE IF EXISTS Counts')
cur.execute('''
CREATE TABLE Counts (org TEXT, count INTEGER)''')
fname = input('Enter file name: ')
if (len(fname) < 1): fname = 'mbox-short.txt'
fh = open(fname)
for line in fh:
if not line.startswith('From: '): continue
pieces = line.split()
org = pieces[1].split('#')
cur.execute('SELECT count FROM Counts WHERE org = ? ', (org[1],))
row = cur.fetchone()
if row is None:
cur.execute('''INSERT INTO Counts (org, count)
VALUES (?, 1)''', (org[1],))
else:
cur.execute('UPDATE Counts SET count = count + 1 WHERE org = ?',
(org[1],))
conn.commit()
# https://www.sqlite.org/lang_select.html
sqlstr = 'SELECT org, count FROM Counts ORDER BY count DESC LIMIT 10'
for row in cur.execute(sqlstr):
print(str(row[0]), row[1])
cur.close()
print('-----------------done----------------')
Writing a script to clean up some data. Super unoptimized but this cursor is
returning the number of results in the like query rather than the rows what am I doing wrong.
#!/usr/bin/python
import re
import MySQLdb
import collections
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="admin", # your username
passwd="", # your password
db="test") # name of the data base
# you must create a Cursor object. It will let
# you execute all the query you need
cur = db.cursor()
# Use all the SQL you like
cur.execute("SELECT * FROM vendor")
seen = []
# print all the first cell of all the rows
for row in cur.fetchall() :
for word in row[1].split(' '):
seen.append(word)
_digits = re.compile('\d')
def contains_digits(d):
return bool(_digits.search(d))
count_word = collections.Counter(seen)
found_multi = [i for i in count_word if count_word[i] > 1 and not contains_digits(i) and len(i) > 1]
unique_multiples = list(found_multi)
groups = dict()
for word in unique_multiples:
like_str = '%' + word + '%'
res = cur.execute("""SELECT * FROM vendor where name like %s""", like_str)
You are storing the result of cur.execute(), which is the number of rows. You are never actually fetching any of the results.
Use .fetchall() to get all result rows or iterate over the cursor after executing:
for word in unique_multiples:
like_str = '%' + word + '%'
cur.execute("""SELECT * FROM vendor where name like %s""", like_str)
for row in cur:
print row