I use the following bit to make an sql statement using the key:values in the dict
record_number = 627
temp_dict = {
"FOO": 752,
"BAR": "test",
"I": "zzzzz",
"Hate": "tesname",
"SQL": "testsomethingesle",
"SO": "commentlol",
"MCUH": "asadsa",
"FILLING": "zzzzzz",
"NAME": "''",
}
update_query = (
"UPDATE table_name SET {}".format(
", ".join("{} = '?'".format(k) for k in temp_dict)
)
+ " WHERE RECNUM = '"
+ record_number
+ "';"
)
update_values = tuple(temp_dict.values())
cur.execute(update_query, update_values)
the update_query prints out correctly
UPDATE table_name SET FOO = '?', BAR = '?', I = '?', Hate = '?', SQL = '?', SO = '?', MCUH = '?', FILLING = '?', NAME = '?' WHERE RECNUM = '627';
and the update_values also looks right
(752, 'test', 'zzzzz', 'tesname', 'testsomethingesle', 'commentlol', 'asadsa', 'zzzzzz', "''")
but I get back the following error
firebirdsql.OperationalError: conversion error from string "?"
My understanding is that ? is basically a placeholder value and if I put in a tuple or list as the second parameter in the cur.execute() it should replace the ? with the values passed in.
What am I doing wrong?
You're generating a statement that has string literals with a question mark ('?'), not a question mark used as a parameter placeholder (plain ?). This means that when you execute the statement, you're trying to assign the literal value ? to a column, and if that column is not a CHAR, VARCHAR or BLOB, this produces an error, because there is no valid conversion from the string ? to the other data types.
You need to uses "{} = ?" instead (notice the absence of single quotes around the question mark).
Related
Let's say that I have a python list of strings.
the strings are tokens of a C++-like language that I have tokenized them partially. but I am left with some strings that are haven't been tokenized. The problem that I have a set of symbols of the language that I must include in the list.
Example:
class Test
{
method int foo(boolean a, int b) { }
}
The output I need is:
tokens = ['class', 'Test', '{', 'method', 'int', 'foo', '(', 'boolean', 'a', ',', 'int', 'b', ')', '{', '}', '}']
The output I get after I clean the code from whitespaces:
tokens = ['class', 'Test', '{', 'method', 'int', 'foo(boolean', 'a,', 'int', 'b){', '}', '}']
The Code I Use is is using a partial list which is splitted according to white spaces:
def tokenize(self, tokens):
"""
Breaks all tokens into final tokens as needed.
"""
final_tokens = []
for token in tokens:
if not have_symbols(token):
final_tokens.append(token)
else:
current_string = ""
small_tokens = []
for character in token:
if character in SYMBOLS_SET:
if current_string:
small_tokens.append(current_string)
current_string = ""
small_tokens.append(character)
else:
current_string += character
final_tokens = final_tokens + small_tokens
return final_tokens
where SYMBOLS_SET is a set of symbols:
SYMBOLS_SET = {"{", "}", "(", ")", "[", "]", ".", ",", ";", "+", "-", "*", "/", "&", "|", "<", ">", "=", "~"}
and the method have_symbol(token) returns true if token have a symbol from SYMBOL_SET and false otherwise.
I think that it might be a more elegant way to do this, I would be glad for a guidance.
import re
input = r"""
class Test
{
method int foo(boolean a, int b) { }
}"""
SYMBOLS_SET = {"{", "}", "(", ")", "[", "]", ".", ",", ";", "+", "-", "*", "/", "&", "|", "<", ">", "=", "~"}
regexp = r"\s(" + "".join([re.escape(i) for i in SYMBOLS_SET]) + ")"
splitted = re.split(regexp, input)
tokens = [x for x in splitted if x not in [None, ""]]
print(tokens)
gives you:
['class', 'Test', '{', 'method', 'int', 'foo', '(', 'boolean', 'a', ',', 'int', 'b', ')', '{', '}', '}']
Puttin parens around the SYMBOLS makes them a regexp subgroup and thus appearing in the output. The \s (whitespace) we do not want to be included.
I am new to python and am trying to read a file and create a dictionary from it.
The format is as follows:
.1.3.6.1.4.1.14823.1.1.27 {
TYPE = Switch
VENDOR = Aruba
MODEL = ArubaS3500-48T
CERTIFICATION = CERTIFIED
CONT = Aruba-Switch
HEALTH = ARUBA-Controller
VLAN = Dot1q INSTRUMENTATION:
Card-Fault = ArubaController:DeviceID
CPU/Memory = ArubaController:DeviceID
Environment = ArubaSysExt:DeviceID
Interface-Fault = MIB2
Interface-Performance = MIB2
Port-Fault = MIB2
Port-Performance = MIB2
}
The first line OID (.1.3.6.1.4.1.14823.1.1.27 { ) I want this to be the key and the remaining lines are the values until the }
I have tried a few combinations but am not able to get the correct regex to match these
Any help please?
I have tried something like
lines = cache.readlines()
for line in lines:
searchObj = re.search(r'(^.\d.*{)(.*)$', line)
if searchObj:
(oid, cert ) = searchObj.groups()
results[searchObj(oid)] = ", ".join(line[1:])
print("searchObj.group() : ", searchObj.group(1))
print("searchObj.group(1) : ", searchObj.group(2))
You can try this:
import re
data = open('filename.txt').read()
the_key = re.findall("^\n*[\.\d]+", data)
values = [re.split("\s+\=\s+", i) for i in re.findall("[a-zA-Z0-9]+\s*\=\s*[a-zA-Z0-9]+", data)]
final_data = {the_key[0]:dict(values)}
Output:
{'\n.1.3.6.1.4.1.14823.1.1.27': {'VENDOR': 'Aruba', 'CERTIFICATION': 'CERTIFIED', 'Fault': 'MIB2', 'VLAN': 'Dot1q', 'Environment': 'ArubaSysExt', 'HEALTH': 'ARUBA', 'Memory': 'ArubaController', 'Performance': 'MIB2', 'CONT': 'Aruba', 'MODEL': 'ArubaS3500', 'TYPE': 'Switch'}}
You could use a nested dict comprehension along with an outer and inner regex.
Your blocks can be separated by
.numbers...numbers.. {
// values here
}
In terms of regular expression this can be formulated as
^\s* # start of line + whitespaces, eventually
(?P<key>\.[\d.]+)\s* # the key
{(?P<values>[^{}]+)} # everything between { and }
As you see, we split the parts into key/value pairs.
Your "inner" structure can be formulated like
(?P<key>\b[A-Z][-/\w]+\b) # the "inner" key
\s*=\s* # whitespaces, =, whitespaces
(?P<value>.+) # the value
Now let's build the "outer" and "inner" expressions together:
rx_outer = re.compile(r'^\s*(?P<key>\.[\d.]+)\s*{(?P<values>[^{}]+)}', re.MULTILINE)
rx_inner = re.compile(r'(?P<key>\b[A-Z][-/\w]+\b)\s*=\s*(?P<value>.+)')
result = {item.group('key'):
{match.group('key'): match.group('value')
for match in rx_inner.finditer(item.group('values'))}
for item in rx_outer.finditer(string)}
print(result)
A demo can be found on ideone.com.
I am working with Python 2.7 to extract data from a JSON API and push it into a SQL-Server table.
I am having trouble with inserting data into the database where some of the entries returned from the JSON response are missing a section of the dictionary. IE, "CustomFields": 90% of the entries have information, however 10% don't therefore I get an index error
eg
"CustomFields":[
],
vs
"CustomFields":[
{
"Type":"foo",
"Name":"foo",
"Value":"foo"
},
{
"Type":"foo",
"Name":"foo",
"Value":"foo"
},
{
"Type":"foo",
"Name":"foo",
"Value":"foo"
},
What would I change so that if I get a missing index, replace those with 'NULL' entries into the database.
response = '*API URL*'
json_response = json.loads(urllib2.urlopen(response).read())
conn = pypyodbc.connect(r'Driver={SQL Server};Server=*Address*;Database=*DataBase*;Trusted_Connection=yes;')
conn.autocommit = False
c = conn.cursor()
c.executemany("INSERT INTO phil_targetproccess (ResourceType, Id, Name, StartDate, EndDate, TimeSpent, CreateDate, ModifyDate, LastStateChangeDate, ProjectName, EntityStateName, RequestTypeName, AssignedTeamMember#1, Area, SubArea, BusinessTeam) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)" ,
[(x['ResourceType'],
x['Id'],
x['Name'],
(parse_date(x['StartDate'])),
(parse_date(x['EndDate'])),
x['TimeSpent'],
(parse_date(x['CreateDate'])),
(parse_date(x['ModifyDate'])),
(parse_date(x['LastStateChangeDate'])),
x['Project']['Name'],
x['EntityState']['Name'],
x['RequestType']['Name'],
y['GeneralUser']['FirstName']+' '+y['GeneralUser']['LastName'],
x['CustomFields'][0]['Value'],
x['CustomFields'][1]['Value'],
x['CustomFields'][2]['Value'])
for x in json_response['Items']
for y in x['Assignments']['Items']])
Many thanks.
I think your issue is here
x['CustomFields'][0]['Value'],
x['CustomFields'][1]['Value'],
x['CustomFields'][2]['Value']
When CustomFields has no elements
Try
x['CustomFields'][0]['Value'] if len(x['CustomFields']) > 0 else '',
x['CustomFields'][1]['Value'] if len(x['CustomFields']) > 1 else '',
x['CustomFields'][2]['Value'] if len(x['CustomFields']) > 2 else '',
You can use get method to check whether that value in CustomFields
is available if so check the length of the list and then get the value of the dictionary in that list using the same get method.
For example:
customfield_value = (x['CustomFields'][0]).get("Value",None) if len(x['CustomFields'])>0 else None
This will return None if the value is not present in the index 0. you can follow the same for getting values from other 2 indices. If you didn't understand please comment it 'll explain further.
Final Script. Thanks for the help!
c.executemany("INSERT INTO phil_targetproccess (ResourceType, Id, Name, StartDate, EndDate, TimeSpent, CreateDate, "
"ModifyDate, LastStateChangeDate, ProjectName, EntityStateName, RequestTypeName, AssignedTeamMember1, "
"AssignedTeamMember2, AssignedTeamMember3, AssignedTeamMember4, Area, SubArea, BusinessTeam) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
[(x['ResourceType'],
x['Id'],
x['Name'],
(parse_date(x['StartDate'])),
(parse_date(x['EndDate'])),
x['TimeSpent'],
(parse_date(x['CreateDate'])),
(parse_date(x['ModifyDate'])),
(parse_date(x['LastStateChangeDate'])),
x['Project']['Name'],
x['EntityState']['Name'],
x['RequestType']['Name'],
x['Assignments']['Items'][0]['GeneralUser']['FirstName'] + ' ' + x['Assignments']['Items'][0]['GeneralUser']['LastName'] if len(x['Assignments']['Items']) > 0 else None,
x['Assignments']['Items'][1]['GeneralUser']['FirstName'] + ' ' + x['Assignments']['Items'][1]['GeneralUser']['LastName'] if len(x['Assignments']['Items']) > 1 else None,
x['Assignments']['Items'][2]['GeneralUser']['FirstName'] + ' ' + x['Assignments']['Items'][2]['GeneralUser']['LastName'] if len(x['Assignments']['Items']) > 2 else None,
x['Assignments']['Items'][3]['GeneralUser']['FirstName'] + ' ' + x['Assignments']['Items'][3]['GeneralUser']['LastName'] if len(x['Assignments']['Items']) > 3 else None,
x['CustomFields'][0]['Value'] if len(x['CustomFields']) > 0 else '',
x['CustomFields'][1]['Value'] if len(x['CustomFields']) > 1 else '',
x['CustomFields'][2]['Value'] if len(x['CustomFields']) > 2 else '')
for x in json_response['Items']])
This is what I currently have
code
coll = con['X']['Y']
s = "meta http equiv"
m = {'i': s}
n = json.dumps(m)
o = json.loads(n)
coll.insert(o)
data
{
"_id" : ObjectId("58527fe656c7a95cfaf40a15"),
"i" : "meta http equiv"
}
Now in the next iteration, s will change(as per my computations) and I want to append the value of s to same key
let's say in next iteration s becomes sample test data and on same key i
So I want this
{
"_id" : ObjectId("58527fe656c7a95cfaf40a15"),
"i" : "meta http equiv sample test data and"
}
How to achieve this?
Change the way you have formed s:
s = "meta http equiv"
s = (coll.get('i', '') + ' ' + s) if coll.get('i', '') else s
If coll isn't a dict object use getattr instead:
s = "meta http equiv"
s = (getattr(coll, 'i', '') + ' ' + s) if getattr(coll, 'i', '') else s
Using Python's string.Template class - how might I utilize the ${} for fields in a dictionary that contain spaces?
E.g.
t = string.Template("hello ${some field}")
d = { "some field": "world" }
print( t.substitute(d) ) # Returns "invalid placeholder in string"
Edit: Here's the closest I could get, with the caveat being that all variables need to be wrapped in a brackets (otherwise all space separated words would be matched).
class MyTemplate(string.Template):
delimiter = '$'
idpattern = '[_a-z][\s_a-z0-9]*'
t = MyTemplate("${foo foo} world ${bar}")
s = t.substitute({ "foo foo": "hello", "bar": "goodbye" })
# hello world goodbye
Just in case this might be helpful to somebody else. In python 3 you can use format_map:
t = "hello {some field}"
d = { "some field": "world" }
print( t.format_map(d) )
# hello world
From Doc it says that we could use Template option
https://docs.python.org/dev/library/string.html#template-strings
import string
class MyTemplate(string.Template):
delimiter = '%'
idpattern = '[a-z]+ [a-z]+'
t = MyTemplate('%% %with_underscore %notunderscored')
d = { 'with_underscore':'replaced',
'notunderscored':'not replaced',
}
print t.safe_substitute(d)