Saving a string into a sqlite table, retrieving it again and comparing it to the original requires some filters to work and i dont know why exactly.
tl;dr
How can i retrieve string Data from the SQLITE DB without requiring Filter Nr 3 as its dangerous for more complex strings ?
import sqlite3
RAWSTRING = 'This is a DB Teststing'
# create database and table
currentdb = sqlite3.connect('test.db')
currentdb.execute('''CREATE TABLE tickertable (teststring text)''')
# enter RAWSTRING into databasse
currentdb.execute('''INSERT INTO tickertable VALUES(?);''', (RAWSTRING,))
# get RAWSTRING from database
cursorObj = currentdb.cursor()
cursorObj.execute('SELECT * FROM tickertable')
DB_RAWSTRING = cursorObj.fetchall()
currentdb.commit()
currentdb.close()
# Prints This is a DB Teststing
print('originalstring : ', RAWSTRING)
# Prints [('This is a DB Teststing',)]
print('retrieved from DB: ', DB_RAWSTRING)
# Get first entry from List because fetchall gives a list
FILTER1_DB_RAWSTRING = DB_RAWSTRING[0]
# Convert the Listelement to String because its still a listelement and comparing fails to string
FILTER2_DB_RAWSTRING = str(FILTER1_DB_RAWSTRING)
# Remove annoying db extra characters and i dont know why they exist anyway
FILTER3_DB_RAWSTRING = FILTER2_DB_RAWSTRING.replace("'", "").replace("(", "").replace(")", "").replace(",", "")
if RAWSTRING == FILTER3_DB_RAWSTRING:
print('Strings are the same as they should')
else:
print('String are not the same because of db weirdness')
So here's your problem: fetchall returns a list of tuples. This means that casting them to a string puts pesky parenthesis around each row and commas between each element of each row. If you'd like to retrieve the raw information from each column, that can be done by indexing the tuples:
entries = cursorObj.fetchall()
first_row = entries[0]
first_item = first_row[0]
print(first_item)
This ought to print just the content of the first row and column in the DB. If not, let me know!
David
Related
So I'm reading a list of part numbers from excel using Pandas, which can be just about anything, like:
287380274-87
or
ME982394-01
or
HOU8929
that changes randomly based on what the user is looking for and can contain some bad numbers as well. Such as blanks, invalid characters (<, >, or !), as well as phrases, like '12390-01 to 04'. I don't care about filtering the part numbers for all of the random conditions that throw synxtax errors in SQL. But I am attempting to query a SAP database WHERE part number IN (list):
import pandas as pd
from hdbcli import dbapi
userFile = r'T:\H01 Cell\Projects\Part Breakdown Update Spreadsheet Improvements\2022.03.21 Part Breakdown update - VK.xlsm'
# read input from excel for part numbers to WHERE in queries
partNums = pd.read_excel\
(io=userFile, sheet_name='Inputs', usecols=lambda x: 'Unnamed' not in x,\
skiprows=1, dtype={'Part List' : str})
# Open SAP database connection
conn = dbapi.connect(address="server", port=####, user="XXXXX", password="XXXXXXX")
# Function to convert
def listToString(s):
# use list comprehension
listToStr = ', '.join([str(elem) for elem in s])
# return string
return listToStr
partNumStr = listToString(partNums['Part List'].drop_duplicates().tolist())
# GetInvOnHand()
# DISTINCT List
queryIOHlist = [
"PART_NO",
"PLANT",
"LOCATION_DESCRIPTION",
"VALUATION_TYPE"
]
queryIOHstr = listToString(queryIOHlist)
# our SQL query, select all from ' '
queryInvOnHand = (
"SELECT DISTINCT " +
queryIOHstr +
" FROM ZWILLIAMS.ZV_WI_GU_INVENTORY_ON_HAND_IM_THIN INV" +
" WHERE PART_NO IN " +
"(" +
partNumStr +
")"
)
# pandas read SQL to store SQL table in dataframe
inventoryOnHand = pd.read_sql(queryInvOnHand, conn)
conn.close()
I'm running into synxtax errors for my SQL query because of these bad part numbers, such as:
(257, 'sql syntax error: incorrect syntax near "to": line 1 col 8120 (at pos 8120))
where the part number it doesn't like is: 62219-01 to -04
In SQL, is there a way to just skip that number if not found in the Part Numbers column in the table? Ideally, it would just be something like:
if syntaxError:
continue
and then just not record anything in my dataframe for that part number.
I need to search a string and check if it contains numbers in its name. If it does, I want to replace it with nothing. I've started doing something like this but I didn't find a solution for my problem.
table = "table1"
if any(chr.isdigit() for chr in table) == True:
table = table.replace(chr, "_")
print(table)
# The output should be "table"
Any ideas?
You could do this in many different ways. Here's how it could be done with the re module:
import re
table = 'table1'
table = re.sub('\d+', '', table)
This sound like task for .translate method of str, you could do
table = "table1"
table = table.translate("".maketrans("","","0123456789"))
print(table) # table
2 first arguments of maketrans are for replacement character-for-character, as it we do not need this we use empty strs, third (optional) argument is characters to remove.
If you dont want to import any modules you could try:
table = "".join([i for i in table if not i.isdigit()])
table = "table123"
for i in table:
if i.isdigit():
table = table.replace(i, "")
print(table)
I found this works to remove numbers quickly.
table = "table1"
table_temp =""
for i in table:
if i not in "0123456789":
table_temp +=i
print(table_temp)
char_nums = [chr for chr in table if chr.isdigit()]
for i in char_nums:
table = table.replace(i, "")
print(table)
I have to connect to an Oracle database and see if a table exists. While I can get a list of the tables, I'm having trouble seeing if the table I'm looking for is in the list. Some tables have associated table which I'll have to join on, some do not, thus I have to check.
What is in my list: ('NYSDOH_CI_EI_HOSPITAL',)
sql = "SELECT table_name FROM all_tables"
cur.execute(sql)
searchstr = 'NYSDOH_CI_EI_HOSPITAL'
p = re.compile(searchstr)
#create data array to load in SQL results in.
ciDataSet = []
cxRows = cur.fetchall()
for i in cxRows:
#print i # list of tables
if p.match(str(i)):
print i
It doesn't find it, even if I use a wildcard.
fetchall() returns a list of tuples.
So when you do
for i in cxRows:
'i' is of type tuple. In your case, this tuple will have only single value. You can access it using i[0] and match it with p.
Currently you are converting a tuple to string so regular expression will not match.
Corrected code:
sql = "SELECT table_name FROM all_tables"
cur.execute(sql)
searchstr = 'NYSDOH_CI_EI_HOSPITAL'
p = re.compile(searchstr)
#create data array to load in SQL results in.
ciDataSet = []
cxRows = cur.fetchall()
for i in cxRows:
#print i # list of tables
if p.match(str(i[0])):
print i
To improve on the syntax of #vaichidrewar, you could simplify the fetch loop to:
for tabname, in cur:
if p.match(str(tabname)):
print(tabname)
But it's going to be more efficient to do the reg exp matching in the query:
sql = "select table_name from all_tables where regexp_like(table_name, :tn, 'i')"
searchstr = 'EMP'
cur.execute(sql, (searchstr,))
for tabname, in cur:
print(tabname)
The 'i' option does a case-insensitive match. You can adjust the regexp as you like.
I am trying to write user data from a file into a series of insert statements. I feel I am close but just missing one or two things. I am attempting to run a .format, but all I end up with are ?'s
import time, json, sqlite3
def insertsfromfile(file):
results = open(file).readlines()
output = open('UserINSERTFile.txt', 'w')
for rows in results:
jsonobject = json.loads(rows)
userid = jsonobject['user']['id']
name = jsonobject['user']['name']
screenname = jsonobject['user']['screen_name']
description = jsonobject['user']['description']
friendscount = jsonobject['user']['friends_count']
insert = ('INSERT INTO Users VALUES (?,?,?,?,?'.format(userid, name, screenname,description, friendscount)
insert = insert[:-1] + ''
output.write(insert)
output.close()
Thanks
I figured it out after reviewing it. Essentially I was missing that I had to combine the attributes together with my Insert string with the '+'. Also had to convert the variables to str() in case they were int.
When I am retrieving a record from DB getting the record as below
('("2014-02-21 07:10:40",ManualNo,184,vsp,AP10123456,aaaaa,Coconut-Na,5,10)',)
and I need to get the data as tuple like:
("2014-02-21 07:10:40",ManualNo,184,vsp,AP10123456,aaaaa,Coconut-Na,5,10)
without using split function and then want to get the individual values from it.
like
record[0] = 2014-02-21 07:10:40
record[1] = ManualNo
and so on...
You can simply split the string over the comma:
data = ('("2014-02-21 07:10:40",ManualNo,184,vsp,AP10123456,aaaaa,Coconut-Na,5,10)',)
record = data[0].lstrip('(').rstrip(')').split(',')