Develop Reference

How to backtest portfolio compositions using backtrader?

I have a csv file / pandas dataframe which looks like this. It contains various portfolio compositions for a portfolio which is re-balanced everyday according to my own calculations.
date asset percentage
4-Jan-21 AAPL 12.00%
4-Jan-21 TSM 1.00%
4-Jan-21 IBM 31.00%
4-Jan-21 KO 15.00%
4-Jan-21 AMD 41.00%
5-Jan-21 DELL 23.00%
5-Jan-21 TSM 12.20%
5-Jan-21 IBM 15.24%
5-Jan-21 KO 1.50%
5-Jan-21 NKE 7.50%
5-Jan-21 TSLA 9.50%
5-Jan-21 CSCO 3.30%
5-Jan-21 JPM 27.76%
6-Jan-21 AMD 45%
6-Jan-21 BA 0.50%
6-Jan-21 ORCL 54.50%
7-Jan-21 AAPL 50.00%
7-Jan-21 KO 50.00%
...
I want to test a strategy with a 12 asset portfolio.
AAPL,TSM,IBM,KO,AMD,DELL,NKE,TSLA,CSCO,JPM,BA,ORCL
So let's say on 4Jan2021, the portfolio's composition would be 12% in apple, 1% in TSM.. etc. I want to be able to check the prices and know how many I should be holding.
The next day, 5Jan2021, the composition will change to 23% in Dell.. etc, if the stock isn't in this list means its 0% for that day.
I have been looking at backtrader as a backtesting platform, however, the code I have seen in the repo mostly shows how to do stuff with indicators, like SMA cross over, RSI...
My question is: Is it possible to create and test a portfolio based on these compositions I have so I can check the return of this strategy? It would check this frame, and know how many stocks in a ticker to buy or sell on that particular day.
So the universe of stocks I am buying or sell is AAPL,TSM,IBM,KO,AMD,DELL,NKE,TSLA,CSCO,JPM,BA,ORCL
So on 4-Jan-21 it might look like,
dictionary['4Jan2021'] = {'AAPL':0.12,
'TSM':0.01,
'IBM':0.31,
'KO':0.15,
'AMD':0.41,}
On 5-Jan-21 it will look like,
dictionary['5Jan2021'] = {'DELL':0.23,
'TSM':0.122,
'IBM':0.1524,
'KO':0.015,
'NKE':0.075,
'TSLA':0.095,
'CSCO':0.033,
'JPM':0.2776,}
If the ticker isnt there means its 0%.
The portfolio composition needs to change everyday.

The first thing you will want to do it load your targets with your datas. I like
personally to attach the target to the dataline as I add it to backtrader.
tickers = {"FB": 0.25, "MSFT": 0.4, "TSLA": 0.35}
for ticker, target in tickers.items():
data = bt.feeds.YahooFinanceData(
dataname=ticker,
timeframe=bt.TimeFrame.Days,
fromdate=datetime.datetime(2019, 1, 1),
todate=datetime.datetime(2020, 12, 31),
reverse=False,
)
data.target = target
cerebro.adddata(data, name=ticker)
In next you will want to go through each data, and determine the current allocation. If the current allocation is too far from the desired allocation (threshold) you trade all datas.
Notice there is a buffer variable. This will reduce the overall value of the account for calculating units to trade. This helps avoid margin.
You will use a dictionary to track this information.
def next(self):
track_trades = dict()
total_value = self.broker.get_value() * (1 - self.p.buffer)
for d in self.datas:
track_trades[d] = dict()
value = self.broker.get_value(datas=[d])
allocation = value / total_value
units_to_trade = (d.target - allocation) * total_value / d.close[0]
track_trades[d]["units"] = units_to_trade
# Can check to make sure there is enough distance away from ideal to trade.
track_trades[d]["threshold"] = abs(d.target - allocation) > self.p.threshold
Check all the thresholds to determine if trading. If any of datas need trading, then all need trading.
rebalance = False
for values in track_trades.values():
if values['threshold']:
rebalance = True
if not rebalance:
return
Finally, execute your trades. Always sell first to generate cash in the account and avoid margins.
# Sell shares first
for d, value in track_trades.items():
if value["units"] < 0:
self.sell(d, size=value["units"])
# Buy shares second
for d, value in track_trades.items():
if value["units"] > 0:
self.buy(d, size=value["units"])
Here is the all of the code for your reference.
import datetime
import backtrader as bt
class Strategy(bt.Strategy):
params = (
("buffer", 0.05),
("threshold", 0.025),
)
def log(self, txt, dt=None):
""" Logging function fot this strategy"""
dt = dt or self.data.datetime[0]
if isinstance(dt, float):
dt = bt.num2date(dt)
print("%s, %s" % (dt.date(), txt))
def print_signal(self):
self.log(
f"o {self.datas[0].open[0]:7.2f} "
f"h {self.datas[0].high[0]:7.2f} "
f"l {self.datas[0].low[0]:7.2f} "
f"c {self.datas[0].close[0]:7.2f} "
f"v {self.datas[0].volume[0]:7.0f} "
)
def notify_order(self, order):
""" Triggered upon changes to orders. """
# Suppress notification if it is just a submitted order.
if order.status == order.Submitted:
return
# Print out the date, security name, order number and status.
type = "Buy" if order.isbuy() else "Sell"
self.log(
f"{order.data._name:<6} Order: {order.ref:3d} "
f"Type: {type:<5}\tStatus"
f" {order.getstatusname():<8} \t"
f"Size: {order.created.size:9.4f} Price: {order.created.price:9.4f} "
f"Position: {self.getposition(order.data).size:5.2f}"
)
if order.status == order.Margin:
return
# Check if an order has been completed
if order.status in [order.Completed]:
self.log(
f"{order.data._name:<6} {('BUY' if order.isbuy() else 'SELL'):<5} "
# f"EXECUTED for: {dn} "
f"Price: {order.executed.price:6.2f} "
f"Cost: {order.executed.value:6.2f} "
f"Comm: {order.executed.comm:4.2f} "
f"Size: {order.created.size:9.4f} "
)
def notify_trade(self, trade):
"""Provides notification of closed trades."""
if trade.isclosed:
self.log(
"{} Closed: PnL Gross {}, Net {},".format(
trade.data._name,
round(trade.pnl, 2),
round(trade.pnlcomm, 1),
)
)
def next(self):
track_trades = dict()
total_value = self.broker.get_value() * (1 - self.p.buffer)
for d in self.datas:
track_trades[d] = dict()
value = self.broker.get_value(datas=[d])
allocation = value / total_value
units_to_trade = (d.target - allocation) * total_value / d.close[0]
track_trades[d]["units"] = units_to_trade
# Can check to make sure there is enough distance away from ideal to trade.
track_trades[d]["threshold"] = abs(d.target - allocation) > self.p.threshold
rebalance = False
for values in track_trades.values():
if values['threshold']:
rebalance = True
if not rebalance:
return
# Sell shares first
for d, value in track_trades.items():
if value["units"] < 0:
self.sell(d, size=value["units"])
# Buy shares second
for d, value in track_trades.items():
if value["units"] > 0:
self.buy(d, size=value["units"])
if __name__ == "__main__":
cerebro = bt.Cerebro()
tickers = {"FB": 0.25, "MSFT": 0.4, "TSLA": 0.35}
for ticker, target in tickers.items():
data = bt.feeds.YahooFinanceData(
dataname=ticker,
timeframe=bt.TimeFrame.Days,
fromdate=datetime.datetime(2019, 1, 1),
todate=datetime.datetime(2020, 12, 31),
reverse=False,
)
data.target = target
cerebro.adddata(data, name=ticker)
cerebro.addstrategy(Strategy)
# Execute
cerebro.run()
####################################
############# EDIT ###############
####################################
There was an additional requiest for adding in variable allocations per day per security. The following code accomplishes that.
import datetime
import backtrader as bt
class Strategy(bt.Strategy):
params = (
("buffer", 0.05),
("threshold", 0.025),
)
def log(self, txt, dt=None):
""" Logging function fot this strategy"""
dt = dt or self.data.datetime[0]
if isinstance(dt, float):
dt = bt.num2date(dt)
print("%s, %s" % (dt.date(), txt))
def print_signal(self):
self.log(
f"o {self.datas[0].open[0]:7.2f} "
f"h {self.datas[0].high[0]:7.2f} "
f"l {self.datas[0].low[0]:7.2f} "
f"c {self.datas[0].close[0]:7.2f} "
f"v {self.datas[0].volume[0]:7.0f} "
)
def notify_order(self, order):
""" Triggered upon changes to orders. """
# Suppress notification if it is just a submitted order.
if order.status == order.Submitted:
return
# Print out the date, security name, order number and status.
type = "Buy" if order.isbuy() else "Sell"
self.log(
f"{order.data._name:<6} Order: {order.ref:3d} "
f"Type: {type:<5}\tStatus"
f" {order.getstatusname():<8} \t"
f"Size: {order.created.size:9.4f} Price: {order.created.price:9.4f} "
f"Position: {self.getposition(order.data).size:5.2f}"
)
if order.status == order.Margin:
return
# Check if an order has been completed
if order.status in [order.Completed]:
self.log(
f"{order.data._name:<6} {('BUY' if order.isbuy() else 'SELL'):<5} "
# f"EXECUTED for: {dn} "
f"Price: {order.executed.price:6.2f} "
f"Cost: {order.executed.value:6.2f} "
f"Comm: {order.executed.comm:4.2f} "
f"Size: {order.created.size:9.4f} "
)
def notify_trade(self, trade):
"""Provides notification of closed trades."""
if trade.isclosed:
self.log(
"{} Closed: PnL Gross {}, Net {},".format(
trade.data._name,
round(trade.pnl, 2),
round(trade.pnlcomm, 1),
)
)
def __init__(self):
for d in self.datas:
d.target = {
datetime.datetime.strptime(date, "%d-%b-%y").date(): allocation
for date, allocation in d.target.items()
}
def next(self):
date = self.data.datetime.date()
track_trades = dict()
total_value = self.broker.get_value() * (1 - self.p.buffer)
for d in self.datas:
if date not in d.target:
if self.getposition(d):
self.close(d)
continue
target_allocation = d.target[date]
track_trades[d] = dict()
value = self.broker.get_value(datas=[d])
current_allocation = value / total_value
net_allocation = target_allocation - current_allocation
units_to_trade = (
(net_allocation) * total_value / d.close[0]
)
track_trades[d]["units"] = units_to_trade
# Can check to make sure there is enough distance away from ideal to trade.
track_trades[d]["threshold"] = abs(net_allocation) > self.p.threshold
rebalance = False
for values in track_trades.values():
if values["threshold"]:
rebalance = True
if not rebalance:
return
# Sell shares first
for d, value in track_trades.items():
if value["units"] < 0:
self.sell(d, size=value["units"])
# Buy shares second
for d, value in track_trades.items():
if value["units"] > 0:
self.buy(d, size=value["units"])
if __name__ == "__main__":
cerebro = bt.Cerebro()
allocations = [
("AAPL", "4-Jan-21", 0.300),
("TSM", "4-Jan-21", 0.200),
("IBM", "4-Jan-21", 0.300),
("KO", "4-Jan-21", 0.2000),
("AMD", "4-Jan-21", 0.1000),
("DELL", "5-Jan-21", 0.200),
("TSM", "5-Jan-21", 0.20),
("IBM", "5-Jan-21", 0.1),
("KO", "5-Jan-21", 0.1),
("NKE", "5-Jan-21", 0.15),
("TSLA", "5-Jan-21", 0.10),
("CSCO", "5-Jan-21", 0.050),
("JPM", "5-Jan-21", 0.1),
("AMD", "6-Jan-21", 0.25),
("BA", "6-Jan-21", 0.25),
("ORCL", "6-Jan-21", 0.50),
("AAPL", "7-Jan-21", 0.5000),
("KO", "7-Jan-21", 0.5000),
]
ticker_names = list(set([alls[0] for alls in allocations]))
targets = {ticker: {} for ticker in ticker_names}
for all in allocations:
targets[all[0]].update({all[1]: all[2]})
for ticker, target in targets.items():
data = bt.feeds.YahooFinanceData(
dataname=ticker,
timeframe=bt.TimeFrame.Days,
fromdate=datetime.datetime(2020, 12, 21),
todate=datetime.datetime(2021, 1, 8),
reverse=False,
)
data.target = target
cerebro.adddata(data, name=ticker)
cerebro.addstrategy(Strategy)
cerebro.broker.setcash(1000000)
# Execute
cerebro.run()

Input string recognise text and return date parse

I have a string input where text contains data as "monday of next month" or "4 days before new year" need to convert into dates as suppose today is 25/11/2020, so next month monday is on 07/12/2020, so how should i do it in python (i've tried using datetime, dateutil but didn't help')
next week saturday
4 days before new year
3 days after new year
second wednesday of next month
i need output as
05/12/20
28/12/20
03/01/21
09/12/21
tried reading docs but didn't help can anyone shed some light on it.
here's my python code.
from datetime import datetime, timedelta
from dateutil import parser
from dateparser.search import search_dates
# Create your views here.
def converter_view(request):
form = DateStringForm()
if request.method == 'POST':
form = DateStringForm(request.POST)
if form.is_valid():
input_string = form.cleaned_data['string_date']
list_of_dates = []
if input_string.lower() == "today":
data = datetime.now()
list_of_dates.append(data.strftime('%d/%m/%Y'))
elif input_string.lower() == "tomorrow":
data = datetime.now() + timedelta(1)
list_of_dates.append(data.strftime('%d/%m/%Y'))
elif input_string.lower() == "yesterday":
data = datetime.now() - timedelta(1)
list_of_dates.append(data.strftime('%d/%m/%Y'))
else:
try:
# Using search_dates method we were extracting the keywords related to date, day or month
# So search_dates is going to return a "list of tuple" where 0th index is string and 1st is datetime
data = search_dates(input_string)
except Exception as msg:
print('Exception :', msg)
else:
try:
for i in data:
# So as we require to convert that date back in string format, we parse the extracted data in data
# where parser.parse() is taking 0th index which is string, and converting it to date and by using
# strftime function we get the required format
list_of_dates.append(parser.parse(i[0]).strftime('%d/%m/%Y'))
print(list_of_dates)
except TypeError as msg:
print("String does not contain any day or dates")
values = DateStringConvert(string_date=input_string, date=list_of_dates)
values.save()
return HttpResponseRedirect('/dateconverter')
return render(request, 'DateTimeConverter/index.html', {'form':form})

It's impossible to facilitate all possible combinations and pieces of text. However, you can cover most cases with a few basic patterns.
import re
import datetime
queries = [
"next week saturday",
"4 days before new year",
"3 days after new year",
"second wednesday of next month",
]
def parse_relative_string_date(text: str, start_date: datetime.datetime = datetime.datetime.now()) -> datetime.datetime:
def _next_week(weekday: str):
weekdays = {
"monday": 0,
"tuesday": 1,
"wednesday": 2,
"thursday": 3,
"friday": 4,
"saturday": 5,
"sunday": 6,
}
next_week = start_date + datetime.timedelta(weeks=1)
return next_week + datetime.timedelta((weekdays[weekday.lower()] - next_week.weekday()) % 7)
def _new_year(n_days: str, direction: str):
new_years = datetime.datetime(year=start_date.year + 1, month=1, day=1)
directions = {
"before": -1,
"after": 1,
}
return new_years + datetime.timedelta(directions[direction.lower()] * int(n_days))
patterns = {
r"next week (?P<weekday>\w+)": _next_week,
r"(?P<n_days>\d+) days (?P<direction>\w+) new year": _new_year,
# Add more patterns here
}
for pattern, callback in patterns.items():
found = re.search(pattern, text)
if found is not None:
return callback(**dict(found.groupdict()))
for query in queries:
print(parse_relative_string_date(query))

how dynamically update listbox entry in Tkinter?

I am trying to update a listbox line, based on a condition, but get this error:
_tkinter.TclError: bad listbox index "Wednesday 15 April 2020 ": must be active, anchor, end, #x,y, or a number
These are the script lines. I get the same error when I use i, or tk.END with listboxMain.delete
for i, dt in enumerate(daterange(date1, date2)):
holiday = us_holidays.get(dt)
if not holiday:
holiday = ""
day, date, month, year = dt.strftime('%A %d %B %Y').split()
line = f'{day:10s} {date} {month:>10s} {year:>7} {holiday}'
if (datetime.date.today() - dt).days == 0:
TODAY_INDEX = i
listboxMain.insert(tk.END, line)
for b in list_birthdays:
if b == dt:
listboxMain.delete(i, line) <====== Error is here.
listboxMain.insert(i, line + " Birthday of ")
listboxMain.itemconfig(i, {'bg': '#999966'})
listboxMain.itemconfig(i, {'fg': '#000066'})
break
listboxMain.itemconfig(TODAY_INDEX, {'bg': '#cc9900'})
listboxMain.itemconfig(TODAY_INDEX, {'fg': '#000099'})

Python script error sqlite3.OperationalError: no such column:

I get this error when I run the script and I cannot see the solution. This program is supposed to draw a giveaway from a sqlite3 file which has the number of raffle tickets for a user. And recently the program the gives that creates the sqlite3 file updated some stuff (The script is made by me) and I can figure out the solution.
Traceback (most recent call last):
File "C:\Users\Admin\Desktop\Draw\Test\dave-draw.py", line 244, in <module>
dd = DaveDraw()
File "C:\Users\Admin\Desktop\Draw\Test\dave-draw.py", line 64, in __init__
self.get_viewers()
File "C:\Users\Admin\Desktop\Draw\Test\dave-draw.py", line 215, in
get_viewers
''').fetchall()
sqlite3.OperationalError: no such column: viewer_id
there's the code
#!/usr/bin/env python3
import pdb
import random
import sqlite3
class Viewer(object):
def __init__(self,
viewer_id,
twitch_name,
beam_name,
beam_id,
viewer_type,
rank,
points,
points2,
hours,
raids,
gains_currency,
gains_hours,
in_giveaways,
last_seen,
sub,
entrance_message,
entrance_message_type,
entrance_sfx
):
self.viewer_id = viewer_id
self.twitch_name = twitch_name
self.beam_name = beam_name
self.beam_id = beam_id
self.viewer_type = viewer_type
self.rank = rank
self.points = points
self.points2 = points2
self.hours = hours
self.raids = raids
self.gains_currency = gains_currency
self.gains_hours = gains_hours
self.in_giveaways = in_giveaways
self.last_seen = last_seen
self.sub = sub
self.entrance_message = entrance_message
self.entrance_message_type = entrance_message_type
self.entrance_sfx = entrance_sfx
def win_chance(self, total_tickets):
"""
Takes the total tickets (points) as a paramter and works
out the percentage chance that the viewer has of winning.
Returns the viewers win chance in percent.
"""
percent = total_tickets / 100.00
return self.points2 / percent
class DaveDraw(object):
def __init__(self):
self.debug = False
self.database_path = 'Viewers3DB.sqlite'
self.db_conn = sqlite3.connect(self.database_path)
self.get_viewers()
self.calculate_total_points()
self.assign_tickets()
def assign_tickets(self):
"""
Assigns each user a number range based on the number of
tickets they have.
e.g.
10 1-10
10 11-20
30 21-50
1 51
"""
self.tickets = {}
latest_ticket = 0
for viewer in self.viewers:
# skip anyone with no points
if viewer.points2 == 0:
continue
ticket_range_beg = latest_ticket + 1
ticket_range_end = latest_ticket + 1 + viewer.points2
latest_ticket = ticket_range_end
viewer.tickets = range(ticket_range_beg, ticket_range_end)
# assign a range of tickets:
if self.debug:
print("Assigning viewer twitch: %s beam: %s tickets %i-%i" % (viewer.twitch_name, viewer.beam_name, viewer.tickets.start, viewer.tickets.stop))
if ticket_range_beg == ticket_range_end:
if self.debug:
print("Assigning ticket {} to {}".format(ticket_range_beg, viewer.twitch_name))
self.tickets[ticket_range_beg] = viewer
next
for ticket in viewer.tickets:
if self.debug:
print("Assigning ticket {} to {}".format(ticket, viewer.twitch_name))
self.tickets[ticket] = viewer
def calculate_total_points(self):
"""
Gets the total amount of points awarded to all
viewers.
"""
self.total_points = 0
for viewer in self.viewers:
self.total_points += viewer.points2
self.total_points_percent = self.total_points / 100
print("Total points awarded (total tickets): %s" % self.total_points)
def draw(self):
"""
Picks a random number between 1 and total tickets, finds
the user that has been assigned tickets within that range and
returns the user.
"""
ticket = random.randint(1, self.total_points)
try:
winner = self.tickets[ticket]
except:
pdb.set_trace()
print("\n===== WINNER Twitch: {} / Beam: {} =====\n".format(winner.twitch_name, winner.beam_id))
print("Picked ticket {}\n".format(ticket))
print("Winner win chance: {:f}".format(winner.win_chance(self.total_points)))
print("Winner's ticket range: {}-{}".format(winner.tickets.start, winner.tickets.stop))
print("Winner's ticket amount: {}\n".format(winner.points2))
self.display_viewer(winner)
def display_random_viewer(self):
"""
Displays random viewer.
"""
self.display_viewer(self.get_random_viewer())
def display_viewer(self, viewer):
"""
Outputs the data on all viewers.
"""
print("""Viewer ID: %s\nTwitch Name: %s\nBeam Name: %s\nBeam ID: %s\nRank: %s\nPoints: %s\nPoints2: %s\nHours: %s\nRaids: %s\nGains Currency: %s\nGains Hours: %s\nInGiveaways: %s\nLastSeen: %s\nEntrance Message: %s\nEntranceMsgType: %s\nEntranceSFX: %s"""
% (
viewer.viewer_id,
viewer.twitch_name,
viewer.beam_name,
viewer.beam_id,
viewer.rank,
viewer.points,
viewer.points2,
viewer.hours,
viewer.raids,
viewer.gains_currency,
viewer.gains_hours,
viewer.in_giveaways,
viewer.last_seen,
viewer.entrance_message,
viewer.entrance_message_type,
viewer.entrance_sfx
)
)
def get_random_viewer(self):
"""
Gets a completely random viewer.
"""
return random.choice(self.viewers)
def get_viewers(self):
"""
Gets data on all the viewers in the database and stores
the data in self.viewers.
"""
c = self.db_conn.cursor()
viewers = c.execute('''
SELECT
viewer_id,
TwitchName,
BeamName,
BeamID,
Type,
Rank,
Points,
Points2,
Hours,
Raids,
GainsCurrency,
GainsHours,
InGiveaways,
LastSeen,
Sub,
EntranceMessage,
EntranceMsgType,
EntranceSFX
FROM Viewer
WHERE Type != 1
AND TwitchName NOT IN (
\'treeboydave\',
\'treebotdave\'
);
''').fetchall()
self.viewers = []
for cur_viewer in viewers:
self.viewers.append(
Viewer(
cur_viewer[0],
cur_viewer[1],
cur_viewer[2],
cur_viewer[3],
cur_viewer[4],
cur_viewer[5],
cur_viewer[6],
cur_viewer[7],
cur_viewer[8],
cur_viewer[9],
cur_viewer[10],
cur_viewer[11],
cur_viewer[12],
cur_viewer[13],
cur_viewer[14],
cur_viewer[15],
cur_viewer[16],
cur_viewer[17]
)
)
if __name__ == '__main__':
dd = DaveDraw()
dd.draw()

All your other SQL columns are capitalised, any chance that's why it's not finding the viewer_id column? Maybe it's Viewer_Id or similar?
If you sql execute 'HELP TABLE Viewer' and print what it returns, it will give you an outline of all of the columns in that database table, so you can make sure you have the capitalisation correct, or whether the column actually isn't there at all.

Extract Date and Currency value(separated by comma) from file

Objective:
Extract String data, Currency value , [type of currency] and date.
Content of file:
[["1234567890","Your previous month subscription point is <RS|$|QR|#> 5,200.33.Your current month month subscription point is <RS|$|QR|#> 1,15,200.33, Last Year total point earned <RS|$|QR|#> 5589965.26 and point lost in game is <RS|$|QR|#> 11520 your this year subscription will expire on 19-04-2013. 9. Back"],["1234567890","Your previous month subscription point is <RS|$|QR|#> 5,200.33.Your current month month subscription point is <RS|$|QR|#> 1,15,200.33, Last Year total point earned <RS|$|QR|#> 5589965.26 and point lost in game is <RS|$|QR|#> 11520 your this year subscription will expire on 19-04-2013. 9. Back"]]
What I have done so far:
def read_file():
fp = open('D:\\ReadData2.txt', 'rb')
content = fp.read()
data = eval(content)
l1 = ["%s" % x[1] for x in data]
return l1
def check_currency(l2):
import re
for i in range(l2.__len__()):
newstr2 = l2[i]
val_currency = []
val_currency.extend(re.findall(r'([+-]?\d+(?:\,\d+)*?\d+(?:\.\d+)?)',newstr2))
print " List %s " % val_currency
for i in range(len(val_currency)):
val2 = val_currency[i]
remove_commas = re.compile(r',(?=\d+)*?')
val3 = remove_commas.sub('', val2)
print val3
if __name__=="__main__":main()
EDIT UDP
I am able to extract the currency value but with the currency of -ve value are conflicting with date format(dd-mm-yyyy). And during extracting string value its also extracting [.|,|] how not to read these characters.
Ouput of check_currency:
>List ['5,200.33', '1,15,200.33', '5589965.26', '11520', '19', '-04', '-2013']
>5200.33
>115200.33
>5589965.26
>11520
>19
>-04
>-2013
Expected Ouput of check_currency:
>List ['5,200.33', '1,15,200.33', '5589965.26', '11520']
>5200.33
>115200.33
>5589965.26
>11520

I added this <RS|$|QR|#>\s* at the first part of your regular expression so as
to be used as prefix for the currency value you want to match.
You can change your code to this one:
def check_currency(l2):
import re
for i in range(l2.__len__()):
newstr2 = l2[i]
val_currency = []
val_currency.extend(re.findall(r'<RS|$|QR|#>\s*([+-]?\d+(?:\,\d+)*?\d+(?:\.\d+)?)',newstr2))
# skip empty strings and remove comma characters
val_currency = [v.replace(',', '') for v in val_currency if v]
print " List %s " % val_currency$
for i in range(len(val_currency)):
val2 = val_currency[i]
remove_commas = re.compile(r',(?=\d+)*?')
val3 = remove_commas.sub('', val2)
print val3
Output:
List ['5200.33', '115200.33', '5589965.26', '11520']
5200.33
115200.33
5589965.26
11520
aditions in the code:
val_currency.extend(re.findall(r'<RS|$|QR|#>\s*([+-]?\d+(?:\,\d+)*?\d+(?:\.\d+)?)',newstr2))
val_currency = [v.replace(',', '') for v in val_currency if v]