This question already has answers here:
Python strip unexpected behavior
(2 answers)
Closed 4 years ago.
value = div.xpath('normalize-space(.)').extract()[0].strip('{}:'.format(key)).strip()
The code above sometimes strips the last character from the word. After removing the code after extract() all the data came back fine but in a list.
Example :
Unknown from Duration: Unknown turns into unknow
Movie from Type: Movie turns into Movi
Why does this happen?
I tried this in Python shell and it also strips the last characters
>>> value = ['Type: Movie']
>>> value[0].strip('{}:'.format('Type')).strip()
'Movi'
I expect it to return Movie instead of e getting stripped.
It seems that this .strip('{}:'.format('Type')) is responsible. I removed the last strip() it only return data with spaces.
Edit: It seems that strip() takes characters in inputted string and remove them instead of removing exact strings. That is why the data came out broken. I think a string split then slice is good.
Edit 2:
Seems like answers by Austin and Pankaj Singhal is good and bug free for my use case.
Use a split on 'Type: ' and take the second item:
value = ['Type: Movie']
print(value[0].split('Type: ')[1])
# Movie
Talking about your code, strip is not meant for what you are trying to do. strip only removes characters from at the ends.
You could use lstrip (which returns a copy of the string with only leading characters removed), instead of strip (which returns a copy of the string with leading and trailing characters removed):
>>> 'Type: Movie'.lstrip("Type:").strip()
'Movie'
>>> 'Type: Something with Type'.lstrip("Type:").strip()
'Something with Type'
>>> 'Type: Something with Type:'.lstrip("Type:").strip()
'Something with Type:'
>>>
OR:
>>> value = ['Type: Movie']
>>> value[0][value[0].find(':')+2:]
'Movie'
>>>
And of course, this is another option similar to the first one, just using lstrip:
>>> value[0][value[0].find(':')+1:].lstrip()
'Movie'
>>>
OR:
>>> value[0].lstrip(value[0][:value[0].find(':')+2])
'Movie'
Note: here find can be replaced with index
str.strip does not strip that exact string, but each character in that string, i.e. strip("Type:") will remove each T, y, p, etc. from the beginning and end of the string.
Instead, you could use a regular expression with the ^ anchor to only match substrings at the beginning of the string.
>>> value = ['Type: Movie with Type: in its name']
>>> key = "Type"
>>> re.sub(r"^{}: ".format(key), "", value[0])
'Movie with Type: in its name'
I have characters in the middle of a string that I want to get rid of. These characters are =, p,, and H. Since they are not the leftmost and the rightmost characters in the string, I cannot use strip(). Is there a function that gets rid of a certain character in any location in a string?
The usual tool for this job is str.translate
https://docs.python.org/2/library/stdtypes.html#str.translate
>>> 'hello=potato'.translate(None, '=p')
'hellootato'
Check the .replace() function:
> 'aaba'.replace('a','').replace('b','')
< ''
My usual tool for this is the regular expression.
>>> import re
>>> invalidCharacters = r'[=p H]'
>>> mystring = re.sub(invalidCharacters, '', ' poH==hHoPPp p')
'ohoPP'
If you need to constrain the number (i.e., the count) of characters you remove, see the count argument.
After initializing a variable x with the content shown in below, I applied strip with a parameter. The result of strip is unexpected. As I'm trying to strip "ios_static_analyzer/", "rity/ios_static_analyzer/" is getting striped.
Kindly help me know why is it so.
>>> print x
/Users/msecurity/Desktop/testspace/Hy5_Workspace/security/ios_static_analyzer/
>>> print x.strip()
/Users/msecurity/Desktop/testspace/Hy5_Workspace/security/ios_static_analyzer/
>>> print x.strip('/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/security/ios_static_analyzer
>>> print x.strip('ios_static_analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/secu
>>> print x.strip('analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/security/ios_static_
>>> print x.strip('_analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/security/ios_static
>>> print x.strip('static_analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/security/io
>>> print x.strip('_static_analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/security/io
>>> print x.strip('s_static_analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/security/io
>>> print x.strip('os_static_analyzer/')
Users/msecurity/Desktop/testspace/Hy5_Workspace/secu
Quoting from str.strip docs
Return a copy of the string with the leading and trailing characters
removed. The chars argument is a string specifying the set of
characters to be removed. If omitted or None, the chars argument
defaults to removing whitespace. The chars argument is not a prefix or
suffix; rather, all combinations of its values are stripped:
So, it removes all the characters in the parameter, from both the sides of the string.
For example,
my_str = "abcd"
print my_str.strip("da") # bc
Note: You can think of it like this, it stops removing the characters from the string when it finds a character which is not found in the input parameter string.
To actually, remove the particular string, you should use str.replace
x = "/Users/Desktop/testspace/Hy5_Workspace/security/ios_static_analyzer/"
print x.replace('analyzer/', '')
# /Users/msecurity/Desktop/testspace/Hy5_Workspace/security/ios_static_
But replace will remove the matches everywhere,
x = "abcd1abcd2abcd"
print x.replace('abcd', '') # 12
But if you want to remove words only at the beginning and ending of the string, you can use RegEx, like this
import re
pattern = re.compile("^{0}|{0}$".format("abcd"))
x = "abcd1abcd2abcd"
print pattern.sub("", x) # 1abcd2
What you need, I think, is replace:
>>> x.replace('ios_static_analyzer/','')
'/Users/msecurity/Desktop/testspace/Hy5_Workspace/security/'
string.replace(s, old, new[, maxreplace])
Return a copy of string s with all occurrences of substring old replaced by new.
So you can replace your string with nothing and get the desired output.
Python x.strip(s) remove from the begginning or the end of the string x any character appearing in s ! So s is just a set of characters, not a string being matched for substring.
string.strip removes a set of characters given as an argument. The chars argument is not a prefix or suffix; rather, all combinations of its values are stripped.
strip does not remove the string given as argument from the object; it removes the characters in the argument.
In this case, strip sees the string s_static_analyzer/ as an iterable of characters that needs to be stripped.
i am trying to delete certain portion of a string if a match found in the string as below
string = 'Newyork, NY'
I want to delete all the characters after the comma from the string including comma, if comma is present in the string
Can anyone let me now how to do this .
Use .split():
string = string.split(',', 1)[0]
We split the string on the comma once, to save python the work of splitting on more commas.
Alternatively, you can use .partition():
string = string.partition(',')[0]
Demo:
>>> 'Newyork, NY'.split(',', 1)[0]
'Newyork'
>>> 'Newyork, NY'.partition(',')[0]
'Newyork'
.partition() is the faster method:
>>> import timeit
>>> timeit.timeit("'one, two'.split(',', 1)[0]")
0.52929401397705078
>>> timeit.timeit("'one, two'.partition(',')[0]")
0.26499605178833008
You can split the string with the delimiter ",":
string.split(",")[0]
Example:
'Newyork, NY'.split(",") # ['Newyork', ' NY']
'Newyork, NY'.split(",")[0] # 'Newyork'
Try this :
s = "this, is"
m = s.index(',')
l = s[:m]
A fwe options:
string[:string.index(",")]
This will raise a ValueError if , cannot be found in the string. Here, we find the position of the character with .index then use slicing.
string.split(",")[0]
The split function will give you a list of the substrings that were separated by ,, and you just take the first element of the list. This will work even if , is not present in the string (as there'd be nothing to split in that case, we'd have string.split(...) == [string])
I need to remove all special characters, punctuation and spaces from a string so that I only have letters and numbers.
This can be done without regex:
>>> string = "Special $#! characters spaces 888323"
>>> ''.join(e for e in string if e.isalnum())
'Specialcharactersspaces888323'
You can use str.isalnum:
S.isalnum() -> bool
Return True if all characters in S are alphanumeric
and there is at least one character in S, False otherwise.
If you insist on using regex, other solutions will do fine. However note that if it can be done without using a regular expression, that's the best way to go about it.
Here is a regex to match a string of characters that are not a letters or numbers:
[^A-Za-z0-9]+
Here is the Python command to do a regex substitution:
re.sub('[^A-Za-z0-9]+', '', mystring)
Shorter way :
import re
cleanString = re.sub('\W+','', string )
If you want spaces between words and numbers substitute '' with ' '
TLDR
I timed the provided answers.
import re
re.sub('\W+','', string)
is typically 3x faster than the next fastest provided top answer.
Caution should be taken when using this option. Some special characters (e.g. ø) may not be striped using this method.
After seeing this, I was interested in expanding on the provided answers by finding out which executes in the least amount of time, so I went through and checked some of the proposed answers with timeit against two of the example strings:
string1 = 'Special $#! characters spaces 888323'
string2 = 'how much for the maple syrup? $20.99? That s ridiculous!!!'
Example 1
'.join(e for e in string if e.isalnum())
string1 - Result: 10.7061979771
string2 - Result: 7.78372597694
Example 2
import re
re.sub('[^A-Za-z0-9]+', '', string)
string1 - Result: 7.10785102844
string2 - Result: 4.12814903259
Example 3
import re
re.sub('\W+','', string)
string1 - Result: 3.11899876595
string2 - Result: 2.78014397621
The above results are a product of the lowest returned result from an average of: repeat(3, 2000000)
Example 3 can be 3x faster than Example 1.
Python 2.*
I think just filter(str.isalnum, string) works
In [20]: filter(str.isalnum, 'string with special chars like !,#$% etcs.')
Out[20]: 'stringwithspecialcharslikeetcs'
Python 3.*
In Python3, filter( ) function would return an itertable object (instead of string unlike in above). One has to join back to get a string from itertable:
''.join(filter(str.isalnum, string))
or to pass list in join use (not sure but can be fast a bit)
''.join([*filter(str.isalnum, string)])
note: unpacking in [*args] valid from Python >= 3.5
#!/usr/bin/python
import re
strs = "how much for the maple syrup? $20.99? That's ricidulous!!!"
print strs
nstr = re.sub(r'[?|$|.|!]',r'',strs)
print nstr
nestr = re.sub(r'[^a-zA-Z0-9 ]',r'',nstr)
print nestr
you can add more special character and that will be replaced by '' means nothing i.e they will be removed.
Differently than everyone else did using regex, I would try to exclude every character that is not what I want, instead of enumerating explicitly what I don't want.
For example, if I want only characters from 'a to z' (upper and lower case) and numbers, I would exclude everything else:
import re
s = re.sub(r"[^a-zA-Z0-9]","",s)
This means "substitute every character that is not a number, or a character in the range 'a to z' or 'A to Z' with an empty string".
In fact, if you insert the special character ^ at the first place of your regex, you will get the negation.
Extra tip: if you also need to lowercase the result, you can make the regex even faster and easier, as long as you won't find any uppercase now.
import re
s = re.sub(r"[^a-z0-9]","",s.lower())
string.punctuation contains following characters:
'!"#$%&\'()*+,-./:;<=>?#[\]^_`{|}~'
You can use translate and maketrans functions to map punctuations to empty values (replace)
import string
'This, is. A test!'.translate(str.maketrans('', '', string.punctuation))
Output:
'This is A test'
s = re.sub(r"[-()\"#/#;:<>{}`+=~|.!?,]", "", s)
Assuming you want to use a regex and you want/need Unicode-cognisant 2.x code that is 2to3-ready:
>>> import re
>>> rx = re.compile(u'[\W_]+', re.UNICODE)
>>> data = u''.join(unichr(i) for i in range(256))
>>> rx.sub(u'', data)
u'0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz\xaa\xb2 [snip] \xfe\xff'
>>>
The most generic approach is using the 'categories' of the unicodedata table which classifies every single character. E.g. the following code filters only printable characters based on their category:
import unicodedata
# strip of crap characters (based on the Unicode database
# categorization:
# http://www.sql-und-xml.de/unicode-database/#kategorien
PRINTABLE = set(('Lu', 'Ll', 'Nd', 'Zs'))
def filter_non_printable(s):
result = []
ws_last = False
for c in s:
c = unicodedata.category(c) in PRINTABLE and c or u'#'
result.append(c)
return u''.join(result).replace(u'#', u' ')
Look at the given URL above for all related categories. You also can of course filter
by the punctuation categories.
For other languages like German, Spanish, Danish, French etc that contain special characters (like German "Umlaute" as ü, ä, ö) simply add these to the regex search string:
Example for German:
re.sub('[^A-ZÜÖÄa-z0-9]+', '', mystring)
This will remove all special characters, punctuation, and spaces from a string and only have numbers and letters.
import re
sample_str = "Hel&&lo %% Wo$#rl#d"
# using isalnum()
print("".join(k for k in sample_str if k.isalnum()))
# using regex
op2 = re.sub("[^A-Za-z]", "", sample_str)
print(f"op2 = ", op2)
special_char_list = ["$", "#", "#", "&", "%"]
# using list comprehension
op1 = "".join([k for k in sample_str if k not in special_char_list])
print(f"op1 = ", op1)
# using lambda function
op3 = "".join(filter(lambda x: x not in special_char_list, sample_str))
print(f"op3 = ", op3)
Use translate:
import string
def clean(instr):
return instr.translate(None, string.punctuation + ' ')
Caveat: Only works on ascii strings.
This will remove all non-alphanumeric characters except spaces.
string = "Special $#! characters spaces 888323"
''.join(e for e in string if (e.isalnum() or e.isspace()))
Special characters spaces 888323
import re
my_string = """Strings are amongst the most popular data types in Python. We can create the strings by enclosing characters in quotes. Python treats single quotes the
same as double quotes."""
# if we need to count the word python that ends with or without ',' or '.' at end
count = 0
for i in text:
if i.endswith("."):
text[count] = re.sub("^([a-z]+)(.)?$", r"\1", i)
count += 1
print("The count of Python : ", text.count("python"))
After 10 Years, below I wrote there is the best solution.
You can remove/clean all special characters, punctuation, ASCII characters and spaces from the string.
from clean_text import clean
string = 'Special $#! characters spaces 888323'
new = clean(string,lower=False,no_currency_symbols=True, no_punct = True,replace_with_currency_symbol='')
print(new)
Output ==> 'Special characters spaces 888323'
you can replace space if you want.
update = new.replace(' ','')
print(update)
Output ==> 'Specialcharactersspaces888323'
function regexFuntion(st) {
const regx = /[^\w\s]/gi; // allow : [a-zA-Z0-9, space]
st = st.replace(regx, ''); // remove all data without [a-zA-Z0-9, space]
st = st.replace(/\s\s+/g, ' '); // remove multiple space
return st;
}
console.log(regexFuntion('$Hello; # -world--78asdf+-===asdflkj******lkjasdfj67;'));
// Output: Hello world78asdfasdflkjlkjasdfj67
import re
abc = "askhnl#$%askdjalsdk"
ddd = abc.replace("#$%","")
print (ddd)
and you shall see your result as
'askhnlaskdjalsdk