I want to refer to an element (mem[0]) of a list (mem) with a different name (fetch):
mem = [0]
f = open("File.lx", "rb").read()
for b in f: mem += [b]
size = len(mem)
while mem[0] < size: #using mem[0]
char = (mem[0]*2)+1
source = mem[char]
target = mem[char + 1]
mem[0] += 1
mem[target] = mem[source]
And I tried that with the with statement:
mem = [0]
f = open("File.lx", "rb").read()
for b in f: mem += [b]
size = len(mem)
with mem[0] as fetch: #with statement
while fetch < size: #using mem[0] as fetch
char = (fetch*2)+1
source = mem[char]
target = mem[char + 1]
fetch += 1
mem[target] = mem[source]
But I got an error:
Traceback (most recent call last):
File "C:\documents\test.py", line 6, in <module>
with mem[0] as fetch:
AttributeError: __enter__
I thought this would be the way because that's how it's done with file objects:
with open("File.lx", "rb") as file:
fileBytes = file.read()
I read the docs for the with statement and it says that the __exit()__ and __enter()__ methods are loaded. According to what I understood after reading that and from the AttributeError, my guess is that sequence elements (mem[0]) do not have an __enter()__ method.
as the comments already mentioned, mem[0] is a literal integer, which doesn't have __enter__ and __exit__ which are required for the as keyword to work and it would be indeed simpler if you just used mem[0]
but that would be too easy, what you CAN do (as an exercise don't actually do this)
is extend the int class and add __enter__ and __exit__ like so:
class FancyInt(int):
def __enter__(self):
return self
def __exit__(self, *args):
pass
mem = [FancyInt(0)]
with mem[0] as fetch:
print(fetch)
this is neat but fetch is an alias to a LITERAL! if you change fetch, mem[0] will not change!
You seem to want a mutable object which functions as an alias for a specific location in a list. I could see some utility in that (since explicit indices are somewhat ugly in Python). You could create such a class. Here is a proof of concept, implementing the three things that you tried to do with fetch in your code:
class Fetcher:
def __init__(self,target_list, index):
self._list = target_list
self._i = index
def __iadd__(self,v):
self._list[self._i] += v
return self
def __mul__(self,v):
return self._list[self._i] * v
def __lt__(self,v):
return self._list[self._i] < v
For example,
mem = [0,1,2]
fetch = Fetcher(mem,0)
print(fetch < 2) #true
mem[0] = 1
print(fetch < 2) #still true
fetch += 1
print(fetch < 2) #false!
print(mem[0]) #2, showing that mem[0] was changed
print(fetch*2) #4 -- but 2*fetch won't work!
The last line shows that there is a limit to what you could achieve here. To make this really useful, you would want to implement many more magic methods (beyond __iadd__ etc.). Whether or not all this is useful just to avoid [0], you be the judge.
Related
I am writing a code to parallelize a task using Python 3.x multiprocessing library.
I have two classes, (Content of which is too large to be pasted), and two functions like so:
def abc_1(objekt, raw):
return raw + "changed"
def file_task(file_name):
con_fig = Some_Config()
obj_ect = Some_other_class(con_fig)
pool = Pool(4)
with open(file_name, 'r', encoding='utf-8') as fh:
res = pool.map(partial(abc_1, con_fig), fh, 4)
return res
When I run above code I get this error:
cls = <class 'multiprocessing.reduction.ForkingPickler'>
obj = (0, 0, <function mapstar at 0x10abec9d8>, ((functools.partial(<function abc_1 at 0x10ad98620>, <project.some_file.... 'Indeed, Republican lawyers identified only 300 cases of electoral fraud in the United States in a decade.\n')),), {})
protocol = None
#classmethod
def dumps(cls, obj, protocol=None):
buf = io.BytesIO()
cls(buf, protocol).dump(obj)
TypeError: 'bool' object is not callable
However, If I change my two functions like so, where I don't need to send object of a class to my function abc_1 which I want parallelized, it works fine:
def abc_1(raw):
return raw + "changed"
def file_task(file_name):
con_fig = Some_Config()
obj_ect = Some_other_class(con_fig)
pool = Pool(4)
with open(file_name, 'r', encoding='utf-8') as fh:
res = pool.map(abc_1, fh, 4)
return res
I would think "partial" is the culprit but even this works:
def file_task(file_name):
con_fig = Some_Config()
obj_ect = Some_other_class(con_fig)
pool = Pool(4)
with open(file_name, 'r', encoding='utf-8') as fh:
res = pool.map(partial(abc_1), fh, 4)
return res
Since, I haven't provided the content that I have in my classes, I understand his makes answering my question, hard.
I have gone through several such posts: TypeError: 'bool' object is not callable while creating custom thread pool
Nothing of this sort is happening in my code, not even in my classes.
Could my initializations in my class be leading to the error I am getting ? What else could be causing this ? What are some things to keep in mind which passing class instance to a function during multiprocessing ?
It will really help if someone can point me in right direction.
Hey I've got a really poorly labelled element I need to get the text from. It's surrounded by other elements that I don't want.
I can easily target the sibling of the element I want.
How can I target the "Next Sibling" of?
dlg4.window(title='WindowTitleThing',class_name='Text').draw_outline()
Can I target first child of?
Using UIA backend. Thanks!
Found this, but can't get it to work:
# Monkey patch for pywinauto that adds a first_only parameter to the window function
# for the UIA backend, in order to work around slow FindAll calls (win10 bug?)
# First copy paste this code on your REPL, then do:
# d1 = pywinauto.Desktop("uia")
# d1.window(first_only=True, title="Calculator").window_text()
# Currently only title is supported, but its easy to implement all the others as well,
# most importantly title_re
def first_child(self):
print("Getting first child")
child = pywinauto.uia_defines.IUIA().iuia.RawViewWalker.GetFirstChildElement(self._element)
if child:
return pywinauto.uia_element_info.UIAElementInfo(child)
else:
return None
def next_sibling(self):
print("Getting sibling")
sibling = pywinauto.uia_defines.IUIA().iuia.RawViewWalker.GetNextSiblingElement(self._element)
if sibling:
return pywinauto.uia_element_info.UIAElementInfo(sibling)
else:
return None
def find_first_element(first_only=None,
title=None,
title_re=None,
top_level_only=True,
backend=None
):
if backend is None:
backend = pywinauto.backend.registry.active_backend.name
backend_obj = pywinauto.backend.registry.backends[backend]
if not top_level_only:
raise NotImplementedError # or can we actually accept this?
rootElement = backend_obj.element_info_class()
element = None
child = rootElement.first_child()
while child is not None:
print(child.name + " ?= " + title)
if child.name == title:
# TODO all the other conditions..
# class_name(_re)
# title_re
# process
# visible / enabled / handle / predicate_func / active_only / control_id / control_type / auto_id / framework_id
break
child = child.next_sibling()
return child
def new_find_element(**kwargs):
if 'first_only' in kwargs and kwargs['first_only'] is True:
print("Using patched function to get only the first match")
el = pywinauto.findwindows.find_first_element(**kwargs)
if el is None:
raise pywinauto.findwindows.ElementNotFoundError(kwargs)
else:
return el
else:
print("Using original function")
return pywinauto.findwindows.original_find_element(**kwargs)
import pywinauto
pywinauto.uia_element_info.UIAElementInfo.first_child = first_child
pywinauto.uia_element_info.UIAElementInfo.next_sibling = next_sibling
pywinauto.findwindows.find_first_element = find_first_element
pywinauto.findwindows.original_find_element = pywinauto.findwindows.find_element
pywinauto.findwindows.find_element = new_find_element
>>> first_child(dlg4)
Getting first child
Using original function
Using original function
Using original function
Using original function
Using original function
Using original function
Using original function
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in first_child
ctypes.ArgumentError: argument 1: <class 'pywinauto.findbestmatch.MatchError'>: Could not find '_element' in 'dict_keys(
# coding=utf-8
def compare(arr1, arr2):
arr1 = arr1.strip()
arr2 = arr2.strip()
arr1 = arr1.split('\t')
arr2 = arr2.split('\t')
# print arr1[0], arr1[1]
arr1min = min(long(arr1[0]), long(arr1[1]))
arr1max = max(long(arr1[0]), long(arr1[1]))
arr2min = min(long(arr2[0]), long(arr2[1]))
arr2max = max(long(arr2[0]), long(arr2[1]))
# print arr1max, arr2max, arr1min, arr1min
if (arr1min < arr2min):
return -1
elif (arr1min > arr2min):
return 1
else:
if (arr1max < arr2max):
return -1
elif (arr1max > arr2max):
return 1
else:
return 0
f = open('er1000000new.txt')
fwrite = open('erzhesorted.txt', 'w')
lines = f.readlines()
lines.sort(compare)
for line in lines:
# fwrite.write(str(line))
print line
f.close()
fwrite.close()
the compare is custom sort function.
for example, when the result printed on screen, the result is
752555452697747457\t752551879448547328\t1468258301659\n
752563934733873152\t752561055289577472\t1468260508664\n
but the result printed on file is
6782762\t12\t1468248110665\n
2660899225\t12\t1468229395665\n
The two results are different, why?
The wb option when you open a file, makes python waiting for a byte object to write in that file.
Your (commented) line fwrite.write(str(line)) sends a string object.
Doesn't it produce the following error?
TypeError: a bytes-like object is required, not 'str'
Also it may come from your compare attribute in sort method.
What is it?
→ Removing the b option and removing the compare attribute produces the same outputs in term and file outputs.
This is my code:
class Parser(object):
def __init__(self, inputFile): # initalizer / constructor
#open input file and gets ready to parse it
f = open(inputFile, "r")
self.commands = list(f)
f.close()
print(self.commands)
self.currentCommand = 0
self.index = 0
def hasMoreCommands(self):
#are there any more commands in the input
#returns boolean
if (self.commands[self.currentCommand][self.index] == "\\") and (self.commands[self.currentCommand][self.index+1] == "n"): # checks for "/n", alluding that the command has ended and we can advance to the next command
return True
else:
return False
def advance(self):
#reads next command and makes it current command
#called only if hasMoreCommands is true
if self.hasMoreCommands():
self.currentCommand += 1
def commandType(self):
#returns type of current command A_COMMAND, C_COMMAND, L_COMMAND
#C A or L(psuedo command for (XxX))
#dest=comp; jmp, #, ()
self.type = self.commands[self.currentCommand][0]
if self.type == "#":
return "A_COMMAND"
elif self.type == "(":
return "L_COMMAND"
else:
return "C_COMMAND"
def dest(self):
#returns dest mnemoic of current C instruction - 8 Poss
#called when command type is C
#return string
if (self.commandType() == "C_COMMAND") and ("=" in self.commands[self.currentCommand]):
return self.commands[self.currentCommand][0:(self.commands[self.currentCommand].index("="))]
def main(inputFile):
d = Parser(inputFile)
d.commandType = "C_COMMAND"
d.commands = ["D=A+2\\n", "AMD=A+5\\n"]
d.currentCommand = 0
print(d.dest())
main("/Users/user1/Desktop/filelocation/projects/06/add/add.asm")
The file in question:
// This file is part of www.nand2tetris.org
// and the book "The Elements of Computing Systems"
// by Nisan and Schocken, MIT Press.
// File name: projects/06/add/Add.asm
// Computes R0 = 2 + 3
#2
D=A
#3
D=D+A
#0
M=D
Error returned:
['// This file is part of www.nand2tetris.org\n', '// and the book "The Elements of Computing Systems"\n', '// by Nisan and Schocken, MIT Press.\n', '// File name: projects/06/add/Add.asm\n', '\n', '// Computes R0 = 2 + 3\n', '\n', '#2\n', 'D=A\n', '#3\n', 'D=D+A\n', '#0\n', 'M=D\n']
Traceback (most recent call last):
File "/Users/user1/Desktop/Python/filelocation/assembler.py", line 104, in <module>
main("/Users/user1Desktop/filelocation/projects/06/add/add.asm")
File "/Users/user1/Desktop/Python/filelocation/assembler.py", line 99, in main
print(d.dest())
File "/Users/user1/Desktop/Python/filelocation/assembler.py", line 50, in dest
if (self.commandType() == "C_COMMAND") and ("=" in self.commands[self.currentCommand]):
TypeError: 'str' object is not callable
[Finished in 0.1s with exit code 1]
I was attempting to test dest.
This is a part of the Nand 2 Tetris / Elements of Computing Systems curriculum at Chapter 6.
In your main, you are replacing the method def commandType(self) with d.commandType = "C_COMMAND", which is a str and therefore cannot be called like a method.
For your class....
class Parser(object):
def __init__(self, inputFile): # initalizer / constructor
#open input file and gets ready to parse it
f = open(inputFile, "r")
self.commands = list(f)
f.close()
You already set self.commands from the file
And note: index and currentCommand appear to serve the exact same purpose
Your function uses that list.
def commandType(self):
#returns type of current command A_COMMAND, C_COMMAND, L_COMMAND
#C A or L(psuedo command for (XxX))
#dest=comp; jmp, #, ()
self.type = self.commands[self.currentCommand][0]
Therefore, you do not need these lines
def main(inputFile):
d = Parser(inputFile)
# d.commandType = "C_COMMAND"
# d.commands = ["D=A+2\\n", "AMD=A+5\\n"]
# d.currentCommand = 0
So you only need this main assuming the input file is correct.
def main(inputFile):
d = Parser(inputFile)
print(d.dest())
Your error is that d.commandType = "C_COMMAND" cannot be "called" by "C_COMMAND()", (i.e. d.commandType())
I'm trying to multiprocess a function that does multiple actions for a large file but I'm getting the knownle pickling error eventhough Im using partial.
The function looks something like this:
def process(r,intermediate_file,record_dict,record_id):
res=0
record_str = str(record_dict[record_id]).upper()
start = record_str[0:100]
end= record_str[len(record_seq)-100:len(record_seq)]
print sample, record_id
if r=="1":
if something:
res = something...
intermediate_file.write("...")
if something:
res = something
intermediate_file.write("...")
if r == "2":
if something:
res = something...
intermediate_file.write("...")
if something:
res = something
intermediate_file.write("...")
return res
The way im calling it is the following in another function:
def call_func():
intermediate_file = open("inter.txt","w")
record_dict = get_record_dict() ### get infos about each record as a dict based on the record_id
results_dict = {}
pool = Pool(10)
for a in ["a","b","c",...]:
if not results_dict.has_key(a):
results_dict[a] = {}
for b in ["1","2","3",...]:
if not results_dict[a].has_key(b):
results_dict[a][b] = {}
results_dict[a][b]['res'] = []
infile = open(a+b+".txt","r")
...parse the file and return values in a list called "record_ids"...
### now call the function based on for each record_id in record_ids
if b=="1":
func = partial(process,"1",intermediate_file,record_dict)
res=pool.map(func, record_ids)
## append the results for each pair (a,b) for EACH RECORD in the results_dict
results_dict[a][b]['res'].append(res)
if b=="2":
func = partial(process,"2",intermediate_file,record_dict)
res = pool.map(func, record_ids)
## append the results for each pair (a,b) for EACH RECORD in the results_dict
results_dict[a][b]['res'].append(res)
... do something with results_dict...
The idea is that for each record inside the record_ids, I want to save the results for each pair (a,b).
I'm not sure what is giving me this error:
File "/code/Python/Python-2.7.9/Lib/multiprocessing/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/code/Python/Python-2.7.9/Lib/multiprocessing/pool.py", line 558, in get
raise self._value
cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function faile
d
func is not defined at the top level of the code so it can't be pickled.
You can use pathos.multiprocesssing which is not a standard module but it will work.
Or, use something diferent to Pool.map maybe a Queue of workers ?
https://docs.python.org/2/library/queue.html
In the end there is an example you can use, it's for threading but is very similar to the multiprocessing where there is also Queues...
https://docs.python.org/2/library/multiprocessing.html#pipes-and-queues