Develop Reference

SQL Query output formatting URL in python

I am new to python. I am writing a script which queries the database for a URL string. Below is my snippet.
db.execute('select sitevideobaseurl,videositestring '
'from site, video '
'where siteID =1 and site.SiteID=video.VideoSiteID limit 1')
result = db.fetchall()
for row in result:
videosite= row[0:2]
print videosite
It gives me baseURL from a table and the video site string from another table.
output: ('http://www.youtube.com/watch?v={0}', 'uqcSJR_7fOc')
I wish to format the output by removing the braces, quotes and commas and replace the {0} from baseURL with sitestring: uqcSJR_7fOc.
something like: https://www.youtube.com/watch?v=uqcSJR_7fOc in the final output and wish to write this to a file.
Thanks for your time and help in advance.

Use str.format.
db.execute('select sitevideobaseurl,videositestring '
'from site, video '
'where siteID =1 and site.SiteID=video.VideoSiteID limit 1')
result = db.fetchall()
for row in result:
videosite= row[0:2]
print videosite[0].format(videosite[1])

You can use a "replace" command in either Sql or Python.
str_url = str_url.replace('[','')
str_url = str_url.replace(']','')
str_url = str_url.replace('?','')
Something like this in Python which is the easier of the two to do.
Repeat for as many characters as you want to chop out
My str_url is your "videosite".

Populating Insert Statements from a local file

I am trying to write user data from a file into a series of insert statements. I feel I am close but just missing one or two things. I am attempting to run a .format, but all I end up with are ?'s
import time, json, sqlite3
def insertsfromfile(file):
results = open(file).readlines()
output = open('UserINSERTFile.txt', 'w')
for rows in results:
jsonobject = json.loads(rows)
userid = jsonobject['user']['id']
name = jsonobject['user']['name']
screenname = jsonobject['user']['screen_name']
description = jsonobject['user']['description']
friendscount = jsonobject['user']['friends_count']
insert = ('INSERT INTO Users VALUES (?,?,?,?,?'.format(userid, name, screenname,description, friendscount)
insert = insert[:-1] + ''
output.write(insert)
output.close()
Thanks

I figured it out after reviewing it. Essentially I was missing that I had to combine the attributes together with my Insert string with the '+'. Also had to convert the variables to str() in case they were int.

Using a 'one key to many values' entry in SQLite3 database with Python

I have a text file that contains many different entries. What I'd like to do is take the first column, use each unique value as a key, and then store the second column as values. I actually have this working, sort of, but I'm looking for a better way to do this. Here is my example file:
account_check:"login/auth/broken"
adobe_air_installed:kb_base+"/"+app_name+"/Path"
adobe_air_installed:kb_base+"/"+app_name+"/Version"
adobe_audition_installed:'SMB/Adobe_Audition/'+version+'/Path'
adobe_audition_installed:'SMB/Adobe_Audition/'+version+'/ExePath'
Here is the code I'm using to parse my text file:
val_dict = {}
for row in creader:
try:
value = val_dict[row[0]]
value += row[1] + ", "
except KeyError:
value = row[1] + ", "
val_dict[row[0]] = value
for row in val_dict.items():
values = row[1][:-1],row[0]
cursor.execute("UPDATE 'plugins' SET 'sets_kb_item'= ? WHERE filename= ?", values)
And here is the code I use to query + format the data currently:
def kb_item(query):
db = get_db()
cur = db.execute("select * from plugins where sets_kb_item like ?", (query,))
plugins = cur.fetchall()
for item in plugins:
for i in item['sets_kb_item'].split(','):
print i.strip()
Here is the output:
kb_base+"/Installed"
kb_base+"/Path"
kb_base+"/Version"
It took me many tries but I finally got the output the way I wanted it, however I'm looking for critique. Is there a better way to do this? Could my entire for item in plugins.... print i.strip() be done in one line and saved as a variable? I am very new to working with databases, and my python skills could also use refreshing.
NOTE I'm using csvreader in this code because I originally had a .csv file - however I found it was just as easy to use the .txt file I was provided.

MongoDB field value to variable in python

I am taking entries from MongoDB and I want to do some modifications, data crunching etc and updating. In this particular example Iam trying for every document in collection
{u'time': 1405694995.310651, u'text': u'HOHO,r\u012bt ar evitu uz positivus ar vip bi\u013ceti kabat\u0101:)', u'_id': ObjectId('53cd621d51f4fbe9f6e04da4'), u'name': u'Madara B\u013cas\u0101ne', u'screenName': u'miumiumadara'} take its text value as a string, count its keyword values and after add to exact particular document field with keyword value.
I am struggling with taking text field as string so it can be operated. And also I havent found solution in python how to add new field to document with count variable. In a Mongo shell comands are easy, but here i dont know. Anything for me to look for?
db = conn.posit2014
collection = db.ceturtdiena
cursor = db.all.find()
for text_fromDB in cursor:
print text_fromDB
source_text = text_fromDB.translate(None, '#!#£$%^&*()_:""?><.,/\|+-')
source_text = source_text.lower()
source_words = source_text.split()
count = 0
word_list = []
with open('pozit.txt') as inputfile:
for line in inputfile:
word_list.append(line.strip())
for word in word_list:
if word in source_words:
count += 1
#add count variable to each document
# {$set : {value:'count'}}

AFAIK text_fromDB is just a dict so you can do this. (If you mean to update document)
text_fromDB['count'] = value
collection.update({'_id':text_fromDB['_id']}, {"$set": text_fromDB})

I'm not sure if I understand everything you're ask. Let's go one piece at a time. To get the text field from your collection as a normal string try this:
collection = db.centurtdiena
for doc in collection.find():
text = str(doc['text'])
print(text)

Find string and replace the next few lines with something

I am writing a Python script that will ask for a file and a name (e.g. "John").
The file contains a whole bunch of lines like this:
...
Name=John
Age=30
Pay=1000
Married=1
Name=Bob
Age=25
Pay=500
Married=0
Name=John
Age=56
Pay=3000
Married=1
...
I want to open this file, ask the user for a name, and replace the pay value for all entries that match that name. So, for example, the user inputs "John", I want to change the Pay for all "John"s to be, say, 5000. The Pay value for other names don't change.
So far, I've opened up the file and concatenated everything into one long string to make things a bit easier:
for line in file:
file_string += line
At first, I was thinking about some sort of string replace but that didn't pan out since I would search for "John" but I don't want to replace the "John", but rather the Pay value that is two lines down.
I started using regex instead and came up with something like this.
# non-greedy matching
re.findall("Name=(.*?)\nAge=(.*?)\nPay=(.*?)\n", file_string, re.S)
Okay, so that spits out a list of 3-tuples of those groupings and it does seem to find everything fine. Now, to do the actual replacement...
I read on another question here on StackOverflow that I can set the name of a grouping and use that grouping later on...:
re.sub(r'Name=(.*?)\nAge=(.*?)\nPay=', r'5000', file_string, re.S)
I tried that to see if it would work and replace all Names with 5000, but it didn't. If it would then I would probably do a check on the first group to see if it matched the user-inputed name or something.
The other problem is that I read on the Python docs that re.sub only replaces the left-most occurrence. I want to replace all occurrences. How do I do that?
Now I am a bit loss of what to do so if anyone can help me that would be great!

I don't think that regex is the best solution to this problem. I prefer more general solutions. The other answers depend on one or more of the following things:
There are always 4 properties for a person.
Every person has the same properties.
The properties are always in the same order.
If these are true in your case, then regex could be ok.
My solution is more verbose, but it isn't depending on these. It handles mixed/missing properties, mixed order, and able to set and get any property value. You could even extend it a little, and support new property or person insertion if you need.
My code:
# i omitted "data = your string" here
def data_value(person_name, prop_name, new_value = None):
global data
start_person = data.find("Name=" + person_name + "\n")
while start_person != -1:
end_person = data.find("Name=", start_person + 1)
start_value = data.find(prop_name + "=", start_person, end_person)
if start_value != -1:
start_value += len(prop_name) + 1
end_value = data.find("\n", start_value, end_person)
if new_value == None:
return data[start_value:end_value]
else:
data = data[:start_value] + str(new_value) + data[end_value:]
start_person = data.find("Name=" + person_name + "\n", end_person)
return None
print data_value("Mark", "Pay") # Output: None (missing person)
print data_value("Bob", "Weight") # Output: None (missing property)
print data_value("Bob", "Pay") # Output: "500" (current value)
data_value("Bob", "Pay", 1234) # (change it)
print data_value("Bob", "Pay") # Output: "1234" (new value)
data_value("John", "Pay", 555) # (change it in both Johns)

Iterate 4 lines at a time. If the first line contains 'John' edit the line that comes two after.
data = """
Name=John
Age=30
Pay=1000
Married=1
Name=Bob
Age=25
Pay=500
Married=0
Name=John
Age=56
Pay=3000
Married=1
"""
lines = data.split()
for i, value in enumerate(zip(*[iter(lines)]*4)):
if 'John' in value[0]:
lines[i*4 + 2] = "Pay=5000"
print '\n'.join(lines)

The following code will do what you need:
import re
text = """
Name=John
Age=30
Pay=1000
Married=1
Name=Bob
Age=25
Pay=500
Married=0
Name=John
Age=56
Pay=3000
Married=1
"""
# the name you're looking for
name = "John"
# the new payment
pay = 500
print re.sub(r'Name={0}\nAge=(.+?)\nPay=(.+?)\n'.format(re.escape(name)), r'Name=\1\nAge=\2\nPay={0}\n'.format(pay), text)