Develop Reference

How to find word with subscript?

Input: s = "test1 this is a sample subscript o₁"
I've tried: re.compile(r'\b[^\W\d_]{2,}\b').findall(s)
It finds the word with more than 2 chars and doesn't contain number
'this', 'is', 'sample', 'subscript', 'o₁',
but it still has the subscript number.
Is there a way to remove word that contains subscript in it?
Desire output: 'this', 'is', 'sample', 'subscript'

The point is that the Unicode aware \d in Python 3 regex does not match No Unicode category.
If you need to work with ASCII only letter words, use
r'\b[a-zA-Z]{2,}\b'
Or, make the pattern non-Unicode aware by using re.A / re.ASCII flag:
re.compile(r'\b[^\W\d_]{2,}\b', re.A)
See this Python 3 demo.
If you need to work with any Unicode letters you may fix it by either adding all the No characters to the regex negated character class (which might make it a tedious solution), or add a programmatic check after a match is found to see if the match contains any char from the No category.
See this Python 3 demo:
import re, sys, unicodedata
s = "test1 this is a sample subscript o₁"
No = [chr(i) for i in range(sys.maxunicode) if unicodedata.category(chr(i)) == 'No']
print([x for x in re.findall(r'\b[^\W\d_]{2,}\b', s) if not any(y in x for y in No)])
# => ['this', 'is', 'sample', 'subscript']
Make sure you are using the latest Python version to support the latest Unicode standard, or rely on the PyPi regex module:
p = regex.compile(r"\b\p{L}{2,}\b")
print(p.findall(s))

Regex to match capital/special/unicode/vietnamese characters

I'm facing an issue. Indeed, I work with vietnamese texts and I want to find every word containing uppercase(s) (capital letter).
When I use the 're' module, my function (temp) does not catch word like "Đà".
The other way (temp2) is to check each character at a time, it works but it is slow since I have to split the sentences into words.
Hence I would like to know if there is a way of the "re" module to catch all the special capital letter.
I have 2 ways :
def temp(sentence):
return re.findall(r'[a-z]*[A-Z]+[a-z]*', sentence)
lis=word_tokenize(sentence)
def temp2(lis):
proper_noun=[]
for word in lis:
for letter in word:
if letter.isupper():
proper_noun.append(word)
break
return proper_noun
Input:
'nous avons 2 Đồng et 3 Euro'
Expected output :
['Đồng','Euro']
Thank you!

You may use this regex:
\b\S*[AĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴAĂÂÁẮẤÀẰẦẢẲẨÃẴẪẠẶẬĐEÊÉẾÈỀẺỂẼỄẸỆIÍÌỈĨỊOÔƠÓỐỚÒỒỜỎỔỞÕỖỠỌỘỢUƯÚỨÙỪỦỬŨỮỤỰYÝỲỶỸỴA-Z]+\S*\b
Regex Demo

The answer of #Rizwan M.Tuman is correct. I want to share with you the speed of execution of the three functions for 100,000 sentences.
lis=word_tokenize(sentence)
def temp(lis):
proper_noun=[]
for word in lis:
for letter in word:
if letter.isupper():
proper_noun.append(word)
break
return proper_noun
def temp2(sentence):
return re.findall(r'[a-z]*[A-Z]+[a-z]*', sentence)
def temp3(sentence):
return re.findall(capital_letter,sentence)
By this way:
start_time = time.time()
for k in range(100000):
temp2(sentence)
print("%s seconds" % (time.time() - start_time))
Here are the results:
>>Check each character of a list of words if it is a capital letter (.isupper())
(sentence has already been splitted into words)
0.4416656494140625 seconds
>>Function with re module which finds normal capital letters [A-Z] :
0.9373950958251953 seconds
>>Function with re module which finds all kinds of capital letters :
1.0783331394195557 seconds

To match only 1+ letter chunks that contain at least 1 uppercase Unicode letter you may use
import re, sys, unicodedata
pLu = '[{}]'.format("".join([chr(i) for i in range(sys.maxunicode) if chr(i).isupper()]))
p = re.compile(r"[^\W\d_]*{Lu}[^\W\d_]*".format(Lu=pLu))
sentence = 'nous avons 2 Đồng et 3 Ęułro.+++++++++++++++Next line'
print(p.findall(sentence))
# => ['Đồng', 'Ęułro', 'Next']
The pLu is a Unicode letter character class pattern built dynamically using unicodedata. It is dependent on the Python version, use the latest to include as many Unicode uppercase letters as possible (see this answer for more details, too). The [^\W\d_] is a construct matching any Unicode letter. So, the pattern matches any 0+ Unicode letters, followed with at least 1 Unicode uppercase letter, and then having any 0+ Unicode letters.
Note that your original r'[a-z]*[A-Z]+[a-z]*' will only find Next in this input:
print(re.findall(r'[a-z]*[A-Z]+[a-z]*', sentence)) # => ['Next']
See the Python demo
To match the words as whole words, use \b word boundary:
p = re.compile(r"\b[^\W\d_]*{Lu}[^\W\d_]*\b".format(Lu=pLu))
In case you want to use Python 2.x, do not forget to use re.U flag to make the \W, \d and \b Unicode aware. However, it is recommended to use the latest PyPi regex library and its [[:upper:]] / \p{Lu} constructs to match uppercase letters since it will support the up-to-date list of Unicode letters.

Python Regex Picking "not include" word

I am trying to find words in the string that do not contain any "a" characters. I wrote the code below but it does not work. How can I say to regex "do not include"? Can't I use "^" sign as "not"?
import re
string2 = "asfdba12312sssdr1 12şljş1 kf"
t = re.findall(r'([^a]\w*) | \w*[^a] ', string2 )
print(t)
The result of that code is "['sfdba12312sssdr1', '12şljş1']"

You need to use a regex with word boundaries with a re.UNICODE flag:
r = re.compile(ur'\b[^\Wa]+\b', re.UNICODE)
The \W and \b will become Unicode aware then.
See the regex demo
[^\Wa] matches any Unicode letter, digit or inderscore, but not a. Add a re.I flag to make it case-insensitive.
If you do not want to match words with digits, add \d to the char class: [^\W\da].
See Python demo:
# -*- coding: utf-8 -*-
import re
p = re.compile(ur'\b[^\Wa]+\b', re.UNICODE)
s = u"asfdba12312sssdr1 12şljş1 kf"
res = [x.encode('utf8') for x in p.findall(s)]
print(res)

[^a] is the single non-a character. [^a]\w* is a single non-a character followed by any number of word-characters. Note that a space is a non-a character, and word-characters can also include a...
The easiest and the most intuitive way to do this in Python is not using re.findall at all:
[word for word in string2.split() if not 'a' in word]

Python Regex - checking for a capital letter with a lowercase after

I am trying to check for a capital letter that has a lowercase letter coming directly after it. The trick is that there is going to be a bunch of garbage capital letters and number coming directly before it. For example:
AASKH317298DIUANFProgramming is fun
as you can see, there is a bunch of stuff we don't need coming directly before the phrase we do need, Programming is fun.
I am trying to use regex to do this by taking each string and then substituting it out with '' as the original string does not have to be kept.
re.sub(r'^[A-Z0-9]*', '', string)
The problem with this code is that it leaves us with rogramming is fun, as the P is a capital letter.
How would I go about checking to make sure that if the next letter is a lowercase, then I should leave that capital untouched. (The P in Programming)

Use a negative look-ahead:
re.sub(r'^[A-Z0-9]*(?![a-z])', '', string)
This matches any uppercase character or digit that is not followed by a lowercase character.
Demo:
>>> import re
>>> string = 'AASKH317298DIUANFProgramming is fun'
>>> re.sub(r'^[A-Z0-9]*(?![a-z])', '', string)
'Programming is fun'

You can also use match like this :
>>> import re
>>> s = 'AASKH317298DIUANFProgramming is fun'
>>> r = r'^.*([A-Z][a-z].*)$'
>>> m = re.match(r, s)
>>> if m:
... print(m.group(1))
...
Programming is fun

Python 3 regex with diacritics and ligatures,

Names in the form: Ceasar, Julius are to be split into First_name Julius Surname Ceasar.
Names may contain diacritics (á à é ..), and ligatures (æ, ø)
This code seems to work OK in Python 3.3
import re
def doesmatch(pat, str):
try:
yup = re.search(pat, str)
print('Firstname {0} lastname {1}'.format(yup.group(2), yup.group(1)))
except AttributeError:
print('no match for {0}'.format(str))
s = 'Révèrberë, Harry'
t = 'Åapö, Renée'
u = 'C3po, Robby'
v = 'Mærsk, Efraïm'
w = 'MacDønald, Ron'
x = 'Sträßle, Mpopo'
pat = r'^([^\d\s]+), ([^\d\s]+)'
# matches any letter, diacritic or ligature, but not digits or punctuation inside the ()
for i in s, t, u, v, w, x:
doesmatch(pat, i)
All except u match. (no match for numbers in names), but I wonder if there isn't a better way than the non-digit non-space approach.
More important though: I'd like to refine the pattern so it distinquishes capitals from lowercase letters, but including capital diacritics and ligatures, preferably using regex also. As if ([A-Z][a-z]+), would match accented and combined characters.
Is this possible?
(what I've looked at so far:
Dive into python 3 on UTF-8 vs Unicode; This Regex tutorial on Unicode (which I'm not using); I think I don't need new regex but I admit I haven't read all its documentation)

If you want to distinguish uppercase and lowercase letters using the standard library's re module, then I'm afraid you'll have to build a character class of all the relevant Unicode codepoints manually.
If you don't really need to do this, use
[^\W\d_]
to match any Unicode letter. This character class matches anything that's "not a non-alphanumeric character" (which is the same as "an alphanumeric character") that's also not a digit nor an underscore.