I am getting input from an external device connected via Ethernet, and it is passing several values of string type e.g. value = '(2,2)\n'. I would like to assign these values to a list or tuple variable e.g. final_value = (2,2).
The code I am using is the following:
import socket
sock = socket.socket()
value =sock.recv(buffersize=2048)
formatted_value = eval(value)
I read that the eval function I am using at this moment to get the list is not a very safe approach, as the external device could pass a dangerous script. So, I would like to know if there is any alternative, similar to the function int(), which can be used to get an integer from a string.
Use ast module literal_eval method for a safer eval
import ast
formatted_value = ast.literal_eval(value)
If you know the input contains a tuple
from ast import literal_eval as make_tuple
make_tuple(value)
Well to give the alternative approach you can do.
s = '(2, 2)\n'
s = s.strip()
if s.startswith('(') and s.endswith(')'):
tup = tuple(int(i) for i in s[1:-1].split(','))
print(tup)
Or if you want a list
s = '(2, 2)\n'
s = s.strip()
if s.startswith('(') and s.endswith(')'):
lst = [int(i) for i in s[1:-1].split(',')]
print(lst)
Related
I am trying to convert a column, which looks something like this
cleaned
['11-111']
['12-345', '67-789']
['00-000', '01-234']
into a list, since I read that Pandas initially interprets list as strings from this article:
https://towardsdatascience.com/dealing-with-list-values-in-pandas-dataframes-a177e534f173
I using the function mentioned in the article
master["cleaned"] = master["cleaned"].apply(eval)
but I am getting this error
eval() arg 1 must be a string, bytes or code object
I tried looking them up, but I can't figure it out
df.cleaned = pd.eval(df.cleaned)
There doesn't appear to be a built it to deal with failures, so you can make it yourself:
def try_eval(x):
try:
return eval(x)
except:
return x
df.cleaned = df.cleaned.apply(try_eval)
Then you can look for the ones that didn't convert by doing:
df.cleaned[df.cleaned.apply(lambda x: isinstance(x, str))]
I want to use a string as an already exists list function.
For example i got from a user as input: 'append'
How can i use this string as a function directly?
For example:
function_name = str(input(n)) # let say it is append
arr = []
arr.function_name(9) #Of course it is not working because it is string. Not a function.
Use getattr.
>>> a = []
>>> getattr(a, 'append')(1)
>>> a
[1]
That being said, it's not the best idea to execute arbitrary code based on user input. You should check the input against a collection of allowed methods first.
All functions are attributes of the respective object. You can use the getattr function to get the function callable and then call it.
For Eg. ,
function_name = str(input("")) # let say it is append
arr = []
#Get the attribute and call it
# You can check if callable(function_to_call) to make sure its a function as well.
function_to_call = getattr(arr, function_name)
function_to_call(10)
print(arr)
The method you are using is not possible.
But you can use this method instead...
function_name = str(input(n))
arr = []
if function_name == "append":
arr.append(9)
or you can use getattr:
function_name = str(input(n))
arr = []
getattr(arr, fucntion_name)(9)
I'm not sure why I'm seeing this error message: AttributeError: 'generator' object has no attribute 'replace' (on line: modified_file = hex_read_file.replace(batch_to_amend_final, batch_amendment_final).
import binascii, os, re, time
os.chdir(...)
files_to_amend = os.listdir(...)
joiner = "00"
# Allow user to input the text to be replaced, and with what
while True:
batch_to_amend3 = input("\n\nWhat number would you like to amend? \n\n >>> ")
batch_amendment3 = input("\n\nWhat is the new number? \n\n >>> ")
batch_to_amend2 = batch_to_amend3.encode()
batch_to_amend = joiner.encode().join(binascii.hexlify(bytes((i,))) for i in batch_to_amend2)
batch_amendment2 = batch_amendment3.encode()
batch_amendment = joiner.encode().join(binascii.hexlify(bytes((i,))) for i in batch_amendment2)
# Function to translate label files
def lbl_translate(files_to_amend):
with open(files_to_amend, 'rb') as read_file:
read_file2 = read_file.read()
hex_read_file = (binascii.hexlify(bytes((i,))) for i in read_file2)
print(hex_read_file)
modified_file = hex_read_file.replace(batch_to_amend, batch_amendment)
with open(files_to_amend, 'wb') as write_file:
write_file.write(modified_file)
write_file.close()
print("Amended: " + files_to_amend)
# Calling function to modify labels
for label in files_to_amend:
lbl_translate(label)
hex_read_file is a generator comprehension (note the round brackets around the statement) defined here:
hex_read_file = (binascii.hexlify(bytes((i,))) for i in read_file2)
As many already pointed out in the comments, comprehesions don't have a replace method as strings have, so you have two possibilities, depending on your specific use-case:
Turn the comprehension in a bytestring and call replace on that (considering how you use write_file.write(modified_file) afterwards, this is the option that would work with that directly):
hex_read_file = bytes(binascii.hexlify(bytes((int(i),))) for i in read_file2) # note: I added th eadditional int() call to fix the issue mentioned in the comments
Filter and replace directly in the comprehension (and modify how you write out the result):
def lbl_translate(files_to_amend, replacement_map):
with open(files_to_amend, 'rb') as read_file:
read_file2 = read_file.read()
hex_read_file = ( replacement_map.get(binascii.hexlify(bytes((int(i),))), binascii.hexlify(bytes((int(i),)))) for i in read_file2) # see Note below
with open(files_to_amend, 'wb') as write_file:
for b in hex_read_file:
write_file.write(b)
print("Amended: " + files_to_amend)
where replacement_map is a dict that you fill in with the batch_to_amend as key and the batch_amendment value (you can speficy multiple amendments too and it will work just the same). The call would then be:
for label in files_to_amend:
lbl_translate(label,{batch_to_amend:batch_amendment})
NOTE: Using standard python dicts, because of how comprehensions work, you need to call binascii.hexlify(bytes((int(i),))) twice here. A better option uses collections.defaultdict
A better option would use defaultdict, if they were implemented in a sensible way (see here for more context on why I say that). defaltdicts expect a lambda with no parameters generating the value for unknown keys, instead you need to create your own subclass of dict and implement the __missing__ method to obtain the desired behaviour:
hex_read_file = ( replacement_map[binascii.hexlify(bytes((int(i),)))] for i in read_file2) # replacement_map is a collections.defaultdict
and you define replacement_map as:
class dict_with_key_as_default(dict): # find a better name for the type
def __missing__(self, key):
'''if a value is not in the dictionary, return the key value instead.'''
return key
replacement_map = dict_with_key_as_default()
replacement_map[batch_to_amend] = batch_amendment
for label in files_to_amend:
lbl_translate(label, replacement_map)
(class dict_with_key_as_default taken from this answer and renamed for clarity)
Edit note: As mentioned in the comments, the OP has an error in the comprehension where they call hexlify() on some binary string instead of integer values. The solution adds a cast to int for the bytes where relevant, but it's far from the best solution to this problem. Since the OP's intent is not clear, I left it as close to the original as possible, but an alternative solution should be used instead.
My Input is:
input = ['(var1, )', '(var2,var3)']
Expected Output is:
output = [('var1', ), ('var2','var3')]
Iterating over input and using eval/literal_eval on the tuple-strings is not possible:
>>> eval('(var1, )')
>>> NameError: name 'var1' is not defined
How can I convert an item such as '(var1, )' to a tuple where the inner objects are treated as strings instead of variables?
Is there a simpler way than writing a parser or using regex?
For each occurrence of a variable, eval searches the symbol table for the name of the variable. It's possible to provide a custom mapping that will return the key name for every missing key:
class FakeNamespace(dict):
def __missing__(self, key):
return key
Example:
In [38]: eval('(var1,)', FakeNamespace())
Out[38]: ('var1',)
In [39]: eval('(var2, var3)', FakeNamespace())
Out[39]: ('var2', 'var3')
Note: eval copies current globals to the submitted globals dictionary, if it doesn't have __builtins__. That means that the expression will have access to built-in functions, exceptions and constants, as well as variables in your namespace. You can try to solve this by passing FakeNamespace(__builtins__=<None or some other value>) instead of just FakeNamespace(), but it won't make eval 100% safe (Python eval: is it still dangerous if I disable builtins and attribute access?)
Try this:
tuples = [tuple(filter(None, t.strip('()').strip().split(','))) for t in input]
For example:
In [16]: tuples = [tuple(filter(None, t.strip('()').strip().split(','))) for t in input]
In [17]: tuples
Out[17]: [('var1',), ('var2', 'var3')]
We're iterating through our list of tuple strings, and for each one, removing the ()s, then splitting our string into a list by the ,, and then converting our list back into a tuple. We use filter() to remove empty elements.
I like vaultah's solution. Here's another one with ast.literal_eval and re if eval is not an option:
>>> import re
>>> from ast import literal_eval
>>> [literal_eval(re.sub('(?<=\(|,)(\w+)(?=\)|,)', r'"\1"', x)) for x in input]
[('var1',), ('var2', 'var3')]
Can I convert a string to arguments list in python?
def func(**args):
for a in args:
print a, args[a]
func(a=2, b=3)
# I want the following work like above code
s='a=2, b=3'
func(s)
I know:
list can, just use *list, but list can't have an element like: a=2
and eval can only evaluate expression
which would be like:
def func2(*args):
for a in args:
print a
list1=[1,2,3]
func2(*list1)
func2(*eval('1,2,3'))
You could massage the input string into a dictionary and then call your function with that, e.g.
>>> x='a=2, b=3'
>>> args = dict(e.split('=') for e in x.split(', '))
>>> f(**args)
a 2
b 3
You want a dictionary, not an 'argument list'. You also would be better off using ast.literal_eval() to evaluate just Python literals:
from ast import literal_eval
params = "{'a': 2, 'b': 3}"
func(**literal_eval(params))
Before you go this route, make sure you've explored other options for marshalling options first, such as argparse for command-line options, or JSON for network or file-based transfer or persistence.
You can use the string as an argument list directly in an call to eval, e.g.
def func(**args):
for a in args:
print( a, args[a])
s = 'a=2, b=3'
eval('func(' + s + ')')
>>>b 3
>>>a 2
Note that func needs to be in the namespace for the eval call to work like this.