I have a Python application which outputs an SQL file:
sql_string = "('" + name + "', " + age + "'),"
output_files['sql'].write(os.linesep + sql_string)
output_files['sql'].flush()
This is not done in a for loop, it is written as data becomes available. Is there any way to 'backspace' over the last comma character when the application is done running, and to replace it with a semicolon? I'm sure that I could invent some workaround by outputting the comma before the newline, and using a global Bool to determine if any particular 'write' is the first write. However, I think that the application would be much cleaner if I could just 'backspace' over it. Of course, being Python maybe there is such an easier way!
Note that having each insert value line in a list and then imploding the list is not a viable solution in this use case.
Use seek to move your cursor one byte (character) backwards, then write the new character:
f.seek(-1, os.SEEK_CUR)
f.write(";")
This is the easiest change, maintaining your current code ("working code" beats "ideal code") but it would be better to avoid the situation.
How about adding the commas before adding the new line?
first_line = True
...
sql_string = "('" + name + "', " + age + "')"
if not first_line:
output_files['sql'].write(",")
first_line = False
output_files['sql'].write(os.linesep + sql_string)
output_files['sql'].flush()
...
output_files['sql'].write(";")
output_files['sql'].flush()
You did mention this in your question - I think this is a much clearer to a maintainer than seeking commas and overwriting them.
EDIT: Since the above solution would require a global boolean in your code (which is not desirable) you could instead wrap the file writing behaviour into a helper class:
class SqlFileWriter:
first_line = True
def __init__(self, file_name):
self.f = open(file_name)
def write(self, sql_string):
if not self.first_line:
self.f.write(",")
self.first_line = False
self.f.write(os.linesep + sql_string)
self.f.flush()
def close(self):
self.f.write(";")
self.f.close()
output_files['sql'] = SqlFileWriter("myfile.sql")
output_files['sql'].write("('" + name + "', '" + age + "')")
This encapsulates all the SQL notation logic into a single class, keeping the code readable and at the same time simplifying the caller code.
Try opening the file to write as binary: 'wb' instead of 'w'.
Use generators, e.g.:
def with_separator(data, sep):
first = True:
for datum in data:
if first:
first = False
else:
yield sep
yield datum
with open("sdfasdfas", "w") as outf:
for x in with_separator(sql_get_rows(), ",\n"):
outf.write(x)
# flush if needed
For hardcore iterator use, this should get you started:
In [11]: list( itertools.imap("".join, itertools.izip(itertools.chain([""], itertools.repeat(",\n")), "abc")) )
Out[11]: ['a', ',\nb', ',\nc']
If your data uses imperative API, that is not iterable, send() your data to generator:
def write_with_separator(filename, sep):
with file(filename, "w"):
first = True
yield None
while True:
datum = yield None
if first:
first = False
else:
fout.write(sep)
fout.write(datum)
# flush if needed
writer = write_with_separator("somefilename", ",\n")
writer.next() # can't send to just-started generator
# to be called when you get data
for row in sql_get_rows():
writer.send(row)
Related
class Student:
def __init__(self, name, hours, qpoints):
self.name = name
self.hours = float(hours)
self.qpoints = float(qpoints)
def getName(self):
return self.name
def getHours(self):
return self.hours
def getQPoints(self):
return self.qpoints
def gpa(self):
return self.qpoints/self.hours
def makeStudent(infoStr):
name, hours, qpoints = infoStr.split("\t")
return Student(name, hours, qpoints)
def main():
fileName = input("Enter file name: ")
infile = open(fileName, "r")
best = makeStudent(infile.readline())
for line in infile:
s = makeStudent(line)
if s.gpa() > best.gpa():
best = s
infile.close()
print("The best student is:", best.getName())
print("hours:", best.getHours())
print("GPA:", best.gpa())
if __name__ == '__main__':
main()
I want to read line from a text file, split it in by "\t" or "," so I can assign it to variables, and I get "ValueError: not enough values to unpack (expected 3, got 1) in makeStudent(infoStr) function. File I use is written correctly, I get same error if I edit file and code to "," instead of "\t". Why is that happening? Edit: Issue was in skipping lines in text. Solved.
Sometimes the infoStr line may not contain the character you're splitting on (e.g. a blank line ''). Wrap this in a try block and you should be fine.
try:
name, hours, qpoints = infoStr.split('\t')
except ValueError:
name, hours, qpoints = None, None, None
You'll then need to handle the None case before instantiating Student.
I bet this is a classic tabs vs. spaces problem. Your file might actually be space separated due to IDE formatting or search and replace going haywire.
Try this:
def makeStudent(infoStr):
FAKE_TAB = ' '
name, hours, qpoints = infoStr.split(FAKE_TAB)
return Student(name, hours, qpoints)
If that doesn't work, determine how many spaces are between each value in each line manually and then replace FAKE_TAB with that. Admittedly, its a slightly sketchy patch...
Notice that you are already iterating over the file lines with the block starting in for line in infile, so there's no need to do infile.readline() within.
Also you could check your line format before sending it to your function (or checking the format in the fuction, whatever you prefer).
{truncated code}
# This loop will put on each iteration the next line of the file in the "line" var.
for line in infile:
# You need two commas in your line to be able to split it in 3 values.
if line.count(",") != 2:
print("WARN: Invalid format in line: "+line)
# Of course that you could implement some counter to identify
# the problematic line location within the file...
quit()
s = makeStudent(line)
if s.gpa() > best.gpa():
best = s
{truncated code}
Essentially I have a python script that loads in a number of files, each file contains a list and these are used to generate strings. For example: "Just been to see $film% in $location%, I'd highly recommend it!" I need to replace the $film% and $location% placeholders with a random element of the array of their respective imported lists.
I'm very new to Python but have picked up most of it quite easily but obviously in Python strings are immutable and so handling this sort of task is different compared to other languages I've used.
Here is the code as it stands, I've tried adding in a while loop but it would still only replace the first instance of a replaceable word and leave the rest.
#!/usr/bin/python
import random
def replaceWord(string):
#Find Variable Type
if "url" in string:
varType = "url"
elif "film" in string:
varType = "film"
elif "food" in string:
varType = "food"
elif "location" in string:
varType = "location"
elif "tvshow" in string:
varType = "tvshow"
#LoadVariableFile
fileToOpen = "/prototype/default_" + varType + "s.txt"
var_file = open(fileToOpen, "r")
var_array = var_file.read().split('\n')
#Get number of possible variables
numberOfVariables = len(var_array)
#ChooseRandomElement
randomElement = random.randrange(0,numberOfVariables)
#ReplaceWord
oldValue = "$" + varType + "%"
newString = string.replace(oldValue, var_array[randomElement], 1)
return newString
testString = "Just been to see $film% in $location%, I'd highly recommend it!"
Test = replaceWord(testString)
This would give the following output: Just been to see Harry Potter in $location%, I'd highly recommend it!
I have tried using while loops, counting the number of words to replace in the string etc. however it still only changes the first word. It also needs to be able to replace multiple instances of the same "variable" type in the same string, so if there are two occurrences of $film% in a string it should replace both with a random element from the loaded file.
The following program may be somewhat closer to what you are trying to accomplish. Please note that documentation has been included to help explain what is going on. The templates are a little different than yours but provide customization options.
#! /usr/bin/env python3
import random
PATH_TEMPLATE = './prototype/default_{}s.txt'
def main():
"""Demonstrate the StringReplacer class with a test sting."""
replacer = StringReplacer(PATH_TEMPLATE)
text = "Just been to see {film} in {location}, I'd highly recommend it!"
result = replacer.process(text)
print(result)
class StringReplacer:
"""StringReplacer(path_template) -> StringReplacer instance"""
def __init__(self, path_template):
"""Initialize the instance attribute of the class."""
self.path_template = path_template
self.cache = {}
def process(self, text):
"""Automatically discover text keys and replace them at random."""
keys = self.load_keys(text)
result = self.replace_keys(text, keys)
return result
def load_keys(self, text):
"""Discover what replacements can be made in a string."""
keys = {}
while True:
try:
text.format(**keys)
except KeyError as error:
key = error.args[0]
self.load_to_cache(key)
keys[key] = ''
else:
return keys
def load_to_cache(self, key):
"""Warm up the cache as needed in preparation for replacements."""
if key not in self.cache:
with open(self.path_template.format(key)) as file:
unique = set(filter(None, map(str.strip, file)))
self.cache[key] = tuple(unique)
def replace_keys(self, text, keys):
"""Build a dictionary of random replacements and run formatting."""
for key in keys:
keys[key] = random.choice(self.cache[key])
new_string = text.format(**keys)
return new_string
if __name__ == '__main__':
main()
The varType you are assigning will be set in only one of your if-elif-else sequence and then the interpreter will go outside. You would have to run all over it and perform operations. One way would be to set flags which part of sentence you want to change. It would go that way:
url_to_change = False
film_to_change = False
if "url" in string:
url_to_change = True
elif "film" in string:
film_to_change = True
if url_to_change:
change_url()
if film_to_change:
change_film()
If you want to change all occurances you could use a foreach loop. Just do something like this in the part you are swapping a word:
for word in sentence:
if word == 'url':
change_word()
Having said this, I'd reccomend introducing two improvements. Push changing into separate functions. It would be easier to manage your code.
For example function for getting items from file to random from could be
def load_variable_file(file_name)
fileToOpen = "/prototype/default_" + file_name + "s.txt"
var_file = open(fileToOpen, "r")
var_array = var_file.read().split('\n')
var_file.clos()
return var_array
Instead of
if "url" in string:
varType = "url"
you could do:
def change_url(sentence):
var_array = load_variable_file(url)
numberOfVariables = len(var_array)
randomElement = random.randrange(0,numberOfVariables)
oldValue = "$" + varType + "%"
return sentence.replace(oldValue, var_array[randomElement], 1)
if "url" in sentence:
setnence = change_url(sentence)
And so on. You could push some part of what I've put into change_url() into a separate function, since it would be used by all such functions (just like loading data from file). I deliberately do not change everything, I hope you get my point. As you see with functions with clear names you can write less code, split it into logical, reusable parts, no needs to comment the code.
A few points about your code:
You can replace the randrange with random.choice as you just
want to select an item from an array.
You can iterate over your types and do the replacement without
specifying a limit (the third parameter), then assign it to the same object, so you keep all your replacements.
readlines() do what you want for open, read from the file as store the lines as an array
Return the new string after go through all the possible replacements
Something like this:
#!/usr/bin/python
import random
def replaceWord(string):
#Find Variable Type
types = ("url", "film", "food", "location", "tvshow")
for t in types:
if "$" + t + "%" in string:
var_array = []
#LoadVariableFile
fileToOpen = "/prototype/default_" + varType + "s.txt"
with open(fname) as f:
var_array = f.readlines()
tag = "$" + t + "%"
while tag in string:
choice = random.choice(var_array)
string = string.replace(tag, choice, 1)
var_array.remove(choice)
return string
testString = "Just been to see $film% in $location%, I'd highly recommend it!"
new = replaceWord(testString)
print(new)
I'm making a dynamic phone-book where you can save name, number and alias. When you are done you can choose too save everything to a file, this is where the problem comes. I've figured out how to save my names and numbers, but not the aliases.
this is a piece of the function where i save my alias:
def main()
..stuff
def alias(person_list, input_list):
..stuff..
for persons in list(person_list):
..stuff..
person_list[person_list.index(persons)].addAlias(newname)
print "Alias saved"
the methods i use:
class person():
..stuff..
def addAlias(self, alias):
self.alias.append(alias)
def hasAlias(self, alias):
if alias in self.alias:
return True
else:
return False
this is the function where i want to save everything:
def save(input_list, person_list):
filename = input_list[1]
f = open(filename, "w")
for i in range(0, len(person_list)):
line = person_list[i].number + ";" + person_list[i].name + ";" + "\n"
f.write(line,)
f.close
I can find out if the element person_list[i] has an alias with the method hasAlias, but I can't figure out how I can get out the value alias, not just True and False, and print that together with the name and number.
Your problem description isn't quite clear enough for me to be sure what you want. Please refer to MCVE.
I think that you're merely missing a method to retrieve the alias, such as
getAlais(self):
return self.alias
Then you simply include that call in your output line. If this isn't what you mean by "get out the value alias", then please clarify.
I'm pretty new with python (20days) but I already created few stuff in maya, for example pickers, ik-fk snap, and few more things. Now I'm trying to create a button to mirror the pose.
the problem is that i must give 2 conditions to my if cycle but maya is ignoring the second condition
import maya.cmds as cmds
cmds.select('arm_lf_FK_ctrl1', 'arm_lf_FK_ctrl2', 'arm_lf_FK_ctrl3')
baseOBJ = cmds.ls(sl=True)
cmds.select('arm_rt_FK_ctrl1', 'arm_rt_FK_ctrl2', 'arm_rt_FK_ctrl3')
targetOBJ = cmds.ls(sl=True)
attr = ['translateX', 'translateY', 'translateZ', 'rotateX', 'rotateY', 'rotateZ', 'IK' ]
for i in range(len (attr) ):
for x in range(len (targetOBJ) ):
if (cmds.attributeQuery(attr[i], node = targetOBJ[x], exists = True) \
and cmds.getAttr(targetOBJ[x] + '.' + attr[i], lock = False)):
newValue = cmds.getAttr(baseOBJ[x] + '.' + attr[i])
cmds.setAttr(baseOBJ[x] + '.' + attr[i], newValue)
else:
pass
the error is:
Error: RuntimeError: file <maya console> line 17: setAttr: The attribute 'arm_lf_FK_ctrl1.translateX' is locked or connected and cannot be modified. #
but in the if cycle I wrote: cmds.getAttr(targetOBJ[x] + '.' + attr[i], lock = False)
any hint?
EDIT SOLUTION:
here is the code fixed
import maya.cmds as cmds
cmds.select('arm_lf_FK_ctrl1', 'arm_lf_FK_ctrl2', 'arm_lf_FK_ctrl3')
baseOBJ = cmds.ls(sl=True)
cmds.select('arm_rt_FK_ctrl1', 'arm_rt_FK_ctrl2', 'arm_rt_FK_ctrl3')
targetOBJ = cmds.ls(sl=True)
attr = ['translateX', 'translateY', 'translateZ', 'rotateX', 'rotateY', 'rotateZ', 'IK' ]
for i in range(len (attr) ):
for x in range(len (baseOBJ) ):
if (cmds.attributeQuery(attr[i], node = baseOBJ[x], exists = True) \
and cmds.getAttr(baseOBJ[x] + '.' + attr[i], lock = False)):
newValue = cmds.getAttr(baseOBJ[x] + '.' + attr[i])
cmds.setAttr(targetOBJ[x] + '.' + attr[i], newValue)
else:
pass
You need to specify
cmds.getAttr(item + attribute, lock=True)
even if you are checking for an attribute you expect to be locked: the 'lock = true' says 'tell me the lock state', not 'tell me if lock is true'.
You can do this a little more simply using three common python tricks (and also by not adding the extra selections, which will just duplicate the lists you've passed in)
The first is to use a foreach loop -- getting values directly out of the list -- instead of using array indices. This is the standard method for doing loops in python. So instead of
for index in range(len(list_of_things)):
do_something(list_of_things[index])
you just do
for item in list_of_things:
do_something(item)
The second is to use zip() to match up to lists and loop over them as pairs: This makes it much easier to write loops that read cleanly as you keep values in sync.
The final thing is to use try...except and allow some kinds of errors to happen rather than pre-checking. This is a common python trick since exceptions are not expensive and the resulting code is often much more readable.
Putting these together you could do the same code like this:
sources = ('arm_lf_FK_ctrl1', 'arm_lf_FK_ctrl2', 'arm_lf_FK_ctrl3')
targets = ('arm_rt_FK_ctrl1', 'arm_rt_FK_ctrl2', 'arm_rt_FK_ctrl3')
attr = ('.translateX', '.translateY', '.translateZ', '.rotateX', '.rotateY', '.rotateZ', '.IK' )
for source, target in zip(sources, targets):
for attrib in attr:
try:
val = cmds.getAttr(source + attrib)
cmds.setAttr(target + attrib, val)
except Exception as e:
print 'skipped', source + attrib, target + attrib
In this case Maya will throw a RuntimeError if you pass it a bad object, a bad attribute, or if you try to set a locked attribute. You'll really want to be more careful with the check than I was here, depending on what you wish to do when the system tries to do something impossible.
One last trick that will make your life easier is to separate out your condition checks from the logic. Instead of
if (cmds.attributeQuery(attr[i], node = baseOBJ[x], exists = True) \
and cmds.getAttr(baseOBJ[x] + '.' + attr[i], lock = False)):
You may find it easier in the long run to do :
exists, locked = False
try:
exists = cmds.ls(object + attrib) is not None
locked = cmds.getAttr(object + attrib, lock = True)
except:
pass # if the object or attrib is missing, both vals will still be false
if exists and not locked:
#do something
writing it this way makes it easier to insert debug printouts when things go wrong.
I do not know maya, but it looks like the issue is happening in the first condition itself, thus the second is being ignored. moreover the exception says issue while running setAttr. This function will be call when you run an attributeQuery, saying exists = True, which would essentially mean you will end up adding the attribute if not already present.
I'm still relatively new to python, 1-2 years of solo-learning, and am trying to improve my code structure so I'm refactoring some old programs I wrote. In one program, I defined a couple of methods for writing files. The first uses, "write" to dump a huge http-response. The second uses "writelines" to dump various derived lists, e.g. lists of links, or forms, or other extracted data.
I'd originally factored out the naming of the file:
#property
def baseFilename(self):
unacceptable = re.compile(r'\W+')
fname = re.sub(unacceptable,'-',self.myUrl)
t = datetime.datetime.now()
dstring = "%s%s%s%s%s%s" % (t.year, t.month, t.day, t.hour, t.minute, t.second)
fullname = fname + '_' + dstring + '.html'
return fullname
But I have a large redundant block of code in each write method:
def writeFile(self, someHtml, writeMethod=write, prefix="RESPONSE_"):
'''The calling functions will supply only the data to be written and
static prefixes, e.g. "full_" for the entire http-response.
'''
fullpath = self.myDump + prefix + self.baseFilename
with open(fullpath, 'w') as h:
h.write(someHtml)
h.close()
print "saved %s" % fullpath
return fullpath
def writeList(self, someList, prefix="mechList_"):
'''Like write file but for one of the many lists outputted.
How do I refactor this, since redundant?
'''
fullpath = self.myDump + prefix + self.baseFilename
with open(fullpath, 'w') as h:
h.writelines(someList)
h.close()
print "saved %s" % fullpath
return fullpath
I'd like to be able to add a variable to each function that specifies the write method to use, e.g. (writeMethod=writelines). I considered just passing in a string and using one of the black-magic functions-- exec() I guess-- but that can't possibly be right since no one ever seems to use those functions. This whole example may be relatively silly, since I could just work around it, but I decided I'd benefit from knowing how to pass these sorts of instance-methods (is that the right term?). Is this related to binding and unbinding? All I need for a good answer is the syntax required to pass 'write,' 'writelines' etc. Could be simple as: writeMethod = insert_your_syntax_here. Would love additional explanation or guidance though. Thanks.
You can get a "bound method" from an object, which is then callable as a function without having a reference to the object.
f = obj.method
f(args)
# is equivalent to
obj.method(args)
However, that's not useful for you, as you create the object you want to use only in the method - you can't pass it in there as bound method. You can factor out the creation of fullpath, although this only saves you half of the redundancy. One option, which I'd consider overkill, would be passing a callback which return the function to use for writing.
Another option would be a decorator to factor out all the common parts and push the rest into a callback, the decorated function:
def uses_file(prefix_default):
def decorator(f):
#functools.wraps(f)
def decorated(self, data, prefix=prefix_default):
fullpath = obj.myDump + prefix + obj.baseFilename
with open(fullpath, 'w') as h:
f(h, data, prefix)
print "saved", % fullpath
return fullpath
return decorated
return decorator
# ...
#uses_file(default_prefix="RESPONE_")
def writeFile(self, someHtml, prefix):
'''...'''
h.write(someHtml)
#uses_file(default_prefix="mechList_")
def writeList(self, someList, prefix):
'''...'''
h.writelines(someList)
There are different ways of doing this, for instance using lambdas:
def writeFile(self, someHtml, writeMethod=lambda f, data: f.write(data),
prefix="RESPONSE_"):
'''The calling functions will supply only the data to be written and
static prefixes, e.g. "full_" for the entire http-response.
'''
fullpath = self.myDump + prefix + self.baseFilename
with open(fullpath, 'w') as h:
writeMethod(h, someHtml)
h.close()
print "saved %s" % fullpath
return fullpath