Develop Reference

No connection adapters were found for using pythons goose3 library to strip simple html code

I am trying to use Python goose3 library to strip simple HTML code from a simple python string.
Is there an easier and better way to do this hopefully not involving beautiful soup as this for some reason does not work on AWS or other cloud platforms?
See the error I get below when I try this
No connection adapters were found
See my code below
from goose3 import Goose
def main():
try:
g = Goose()
str = '<table>xxxxxxx full text below</table>'
article = g.extract(str)
output = article.cleaned_text
print(output)
except Exception as e:
print(str(e))
main()
see the string that I am trying to remove HTML from below
<table border="\"0\"" width="\"100%\"">
<tbody>
<tr>
<td>
<table style="height: 1081px;">
<tbody>
<tr style="height: 54px;">
<td style="width: 88.4531px; height: 54px;">Funding Opportunity ID:</td>
<td style="width: 1862.23px; height: 54px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Opportunity Number:</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Opportunity Title:</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Opportunity Category:</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 54px;">
<td style="width: 88.4531px; height: 54px;">Opportunity Category Explanation:</td>
<td style="width: 1862.23px; height: 54px;"> </td>
</tr>
<tr style="height: 54px;">
<td style="width: 88.4531px; height: 54px;" valign="\"top\"">Funding Instrument Type:</td>
<td style="width: 1862.23px; height: 54px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 54px;">
<td style="width: 88.4531px; height: 54px;" valign="\"top\"">Category of Funding Activity:</td>
<td style="width: 1862.23px; height: 54px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;" valign="\"top\"">Category Explanation:</td>
<td style="width: 1862.23px; height: 36px;"> </td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;" valign="\"top\"">CFDA Number(s):</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;" valign="\"top\"">Eligible Applicants:</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx)</td>
</tr>
<tr style="height: 72px;">
<td style="width: 88.4531px; height: 72px;" valign="\"top\"">Additional Information on Eligibility:</td>
<td style="width: 1862.23px; height: 72px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;" valign="\"top\"">Agency Code:</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;" valign="\"top\"">Agency Name:</td>
<td style="width: 1862.23px; height: 36px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Posted Date:</td>
<td style="width: 1862.23px; height: 36px;">Mar 21, 2022</td>
</tr>
<tr style="height: 18px;">
<td style="width: 88.4531px; height: 18px;">Close Date:</td>
<td style="width: 1862.23px; height: 18px;">May 16, 2022</td>
</tr>
<tr style="height: 54px;">
<td style="width: 88.4531px; height: 54px;">Last Updated Date:</td>
<td style="width: 1862.23px; height: 54px;">Mar 20, 2022</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Award Ceiling:</td>
<td style="width: 1862.23px; height: 36px;">$150,000</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Award Floor:</td>
<td style="width: 1862.23px; height: 36px;">$50,000</td>
</tr>
<tr style="height: 72px;">
<td style="width: 88.4531px; height: 72px;">Estimated Total Program Funding:</td>
<td style="width: 1862.23px; height: 72px;"> </td>
</tr>
<tr style="height: 54px;">
<td style="width: 88.4531px; height: 54px;">Expected Number of Awards:</td>
<td style="width: 1862.23px; height: 54px;">5</td>
</tr>
<tr style="height: 145px;">
<td style="width: 88.4531px; height: 145px;">Description:</td>
<td style="width: 1862.23px; height: 145px;">xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</td>
</tr>
<tr style="height: 18px;">
<td style="width: 88.4531px; height: 18px;">Version:</td>
<td style="width: 1862.23px; height: 18px;">2</td>
</tr>
<tr style="height: 36px;">
<td style="width: 88.4531px; height: 36px;">Modification Comments:</td>
<td style="width: 1862.23px; height: 36px;"> </td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<p>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</p>
<p> </p>
Any helpful suggestions are welcome

Problem extracting text of td from table row (tr) with scrapy

I am parsing data table from the following URL:
https://www.signalstart.com/search-signals
In particular, I am trying to extract the data from the table rows.
The table row has a series of table-data cells:
<table class="table table-striped table-bordered dataTable table-hover" id="searchSignalsTable">
<thead>
<tr>
<th class="sorting sorting_asc">Rank</th>
<th class="sorting ">Name</th>
<th class="sorting ">Gain</th>
<th class="sorting ">Pips</th>
<th class="sorting ">DD</th>
<th class="sorting ">Trades</th>
<th class="sorting ">Type</th>
<th>Monthly</th>
<th>Chart</th>
<th class="sorting ">Price</th>
<th class="sorting " style="width: 40px">Age</th>
<th class="sorting " style="width: 70px">Added</th>
<th>Action</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/joker-1k/110059">Joker 1k</a> </td>
<td><span class="red">-9.99%</span></td>
<td><span class="green">2,092.3</span></td>
<td>15.3%</td>
<td>108</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark110059"><canvas width="12" height="25" style="display: inline-block; vertical-align: top; width: 12px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark110059"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$30</td>
<td>
1m 24d
</td>
<td>
Mar 29, 2020
</td>
<td><a onclick="getMasterPricingData('110059');" data-toggle="modal"><button id="subscribeToMasterBtn110059" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="110059" value="-1.78,-3.68,-4.86">
<input type="hidden" class="dailyGrowthData" oid="110059" value="0.00,-0.03,-1.78,-5.69,-6.75,-5.59,-7.61,-5.31,-6.20,-3.81,-4.40,-8.00,-2.88,-3.78,-4.38,-0.20,-5.40,-10.66,-13.69,-12.51,-13.23,-9.99">
<input type="hidden" class="dailyEquityData" oid="110059" value="0.00,-0.23,-1.41,-5.02,-6.25,-4.29,-6.68,-3.91,-5.37,-4.10,-4.40,-3.59,-1.78,-1.75,-2.65,-0.21,-4.87,-10.76,-13.90,-11.58,-13.23,-10.18">
</div>
</td>
</tr>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/fxabakus/56043">FXabakus</a> </td>
<td><span class="red">-19.57%</span></td>
<td><span class="red">-8,615.2</span></td>
<td>42%</td>
<td>1642</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark56043"><canvas width="80" height="25" style="display: inline-block; vertical-align: top; width: 80px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark56043"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$30</td>
<td>
1y 7m
</td>
<td>
May 4, 2019
</td>
<td><a onclick="getMasterPricingData('56043');" data-toggle="modal"><button id="subscribeToMasterBtn56043" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="56043" value="1.22,1.35,3.92,1.35,-1.57,1.77,2.01,1.11,0.38,-14.89,-14.70,-5.21,5.97,7.03,-17.54,2.92,3.11,-8.94,13.38,1.77">
<input type="hidden" class="dailyGrowthData" oid="56043" value="-27.87,-29.29,-29.01,-26.76,-25.76,-25.59,-30.57,-30.13,-29.78,-29.60,-29.25,-28.34,-28.07,-27.89,-25.20,-25.08,-23.66,-23.46,-21.54,-21.02,-21.62,-20.28,-18.31,-26.97,-27.48,-27.00,-28.21,-24.20,-23.46,-30.04,-31.37,-34.62,-33.84,-32.87,-32.20,-30.99,-30.43,-30.30,-29.75,-27.64,-27.45,-24.34,-24.71,-24.09,-24.15,-21.48,-21.08,-20.97,-19.54,-19.57">
<input type="hidden" class="dailyEquityData" oid="56043" value="-27.87,-29.29,-28.89,-26.76,-25.76,-28.10,-34.47,-32.34,-31.54,-40.80,-32.76,-32.90,-33.50,-30.65,-25.37,-25.05,-22.88,-23.29,-21.54,-21.02,-21.54,-20.90,-19.11,-27.76,-35.15,-29.17,-27.79,-24.20,-26.23,-34.32,-35.95,-51.20,-33.84,-32.76,-32.71,-31.62,-30.43,-39.93,-29.75,-27.64,-28.35,-27.62,-28.41,-24.20,-24.51,-22.06,-21.08,-20.97,-18.82,-30.27">
</div>
</td>
</tr>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/af-investing-pro-final/122603">AF Investing Pro Final</a> </td>
<td><span class="green">56.69%</span></td>
<td><span class="green">29,812</span></td>
<td>8.6%</td>
<td>476</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark122603"><canvas width="8" height="25" style="display: inline-block; vertical-align: top; width: 8px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark122603"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$250</td>
<td>
17d 12h
</td>
<td>
Apr 30, 2020
</td>
<td><a onclick="getMasterPricingData('122603');" data-toggle="modal"><button id="subscribeToMasterBtn122603" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="122603" value="55.18,0.98">
<input type="hidden" class="dailyGrowthData" oid="122603" value="-0.02,0.04,54.78,55.02,55.18,55.82,55.86,55.99,56.06,56.25,56.69">
<input type="hidden" class="dailyEquityData" oid="122603" value="-8.60,16.85,54.86,54.11,55.44,55.85,54.38,52.15,45.00,51.07,56.25">
</div>
</td>
</tr>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/rapid-growth/111340">Rapid growth</a> </td>
<td><span class="green">130.78%</span></td>
<td><span class="green">1,102.9</span></td>
<td>44.3%</td>
<td>126</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark111340"><canvas width="12" height="25" style="display: inline-block; vertical-align: top; width: 12px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark111340"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$31</td>
<td>
2m 8d
</td>
<td>
Apr 1, 2020
</td>
<td><a onclick="getMasterPricingData('111340');" data-toggle="modal"><button id="subscribeToMasterBtn111340" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="111340" value="87.85,18.28,3.87">
<input type="hidden" class="dailyGrowthData" oid="111340" value="0.00,0.64,1.40,1.40,1.90,2.91,7.53,8.21,11.19,11.30,17.60,19.60,23.03,37.74,47.75,54.75,59.91,69.79,73.60,79.36,87.85,93.14,93.40,94.70,95.93,96.01,99.95,100.71,101.85,102.10,102.12,104.36,108.76,110.11,110.14,110.23,112.58,115.10,115.54,117.17,121.24,122.19,123.40,124.18,124.88,124.89,130.09,130.78">
<input type="hidden" class="dailyEquityData" oid="111340" value="-1.80,0.67,0.97,1.91,-0.64,2.58,6.82,6.72,8.65,8.46,16.29,17.71,19.96,34.10,47.24,51.91,59.07,69.79,73.58,79.26,88.01,91.03,93.43,87.85,96.19,95.80,100.29,95.63,98.94,101.71,98.33,104.12,108.26,108.46,86.24,108.42,112.83,114.51,94.42,116.29,120.16,121.93,123.05,115.67,122.81,124.45,130.47,130.14">
</div>
</td>
</tr>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/dream-presentation-1/66543">Dream Presentation 1</a> </td>
<td><span class="red">-99.9%</span></td>
<td><span class="red">-2,724.1</span></td>
<td>99.9%</td>
<td>1612</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark66543"><canvas width="28" height="25" style="display: inline-block; vertical-align: top; width: 28px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark66543"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$30</td>
<td>
6m 13d
</td>
<td>
Nov 8, 2019
</td>
<td><a onclick="getMasterPricingData('66543');" data-toggle="modal"><button id="subscribeToMasterBtn66543" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="66543" value="-100.14,-98.54,-98.79,-91.71,-98.23,-100.00,-88.82">
<input type="hidden" class="dailyGrowthData" oid="66543" value="24.18,-99.90,-99.89,-99.88,-99.88,-99.88,-99.87,-99.87,-99.86,-99.84,-99.83,-99.90,-99.89,-99.90,-99.90,-99.81,-99.81,-99.80,-99.90,-99.90,-99.86,-99.83,-99.79,-99.90,-99.90,-99.90,-99.88,-99.89,-99.89,-99.88,-99.82,-99.74,-99.85,-99.37,-99.88,-99.90,-99.90,-99.90,-99.90,-99.87,-99.83,-99.80,-99.75,-99.64,-99.56,-99.90,-99.90">
<input type="hidden" class="dailyEquityData" oid="66543" value="7.87,-99.90,-99.89,-99.88,-99.88,-99.88,-99.88,-99.87,-99.86,-99.84,-99.83,-99.90,-99.89,-99.90,-99.89,-99.83,-99.88,-99.88,-99.90,-99.90,-99.87,-99.83,-99.84,-99.72,-99.90,-99.90,-99.88,-99.89,-99.88,-99.92,-99.86,-99.74,-99.86,-99.39,-99.88,-99.90,-99.90,-99.90,-99.90,-99.87,-99.83,-99.79,-99.76,-99.63,-99.55,-100.16,-99.83">
</div>
</td>
</tr>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/limerence-ea-suite-3/93679">Limerence EA Suite 3</a> </td>
<td><span class="green">1,246.66%</span></td>
<td><span class="green">199.8</span></td>
<td>34.2%</td>
<td>8</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark93679"><canvas width="20" height="25" style="display: inline-block; vertical-align: top; width: 20px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark93679"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$75</td>
<td>
7m 11d
</td>
<td>
Feb 11, 2020
</td>
<td><a onclick="getMasterPricingData('93679');" data-toggle="modal"><button id="subscribeToMasterBtn93679" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="93679" value="95.40,82.01,94.38,87.49,3.90">
<input type="hidden" class="dailyGrowthData" oid="93679" value="0.00,95.40,255.64,591.28,552.49,1234.12,1196.10,1246.66">
<input type="hidden" class="dailyEquityData" oid="93679" value="0.00,95.40,255.64,591.28,1034.76,1234.12,1196.10,1246.66">
</div>
</td>
</tr>
<tr>
<td style="text-align: center;"> - </td>
<td><a class="pointer" target="_blank" href="https://www.signalstart.com/analysis/easy-money/31727">Easy Money</a> </td>
<td><span class="red">-99.9%</span></td>
<td><span class="green">2,430.6</span></td>
<td>100%</td>
<td>1095</td>
<td>Real</td>
<td><span class="monthlySparkline" id="monthlySpark31727"><canvas width="96" height="25" style="display: inline-block; vertical-align: top; width: 96px; height: 25px;"></canvas></span></td>
<td><span class="dayliSparkline" id="dayliSpark31727"><canvas width="100" height="25" style="display: inline-block; vertical-align: top; width: 100px; height: 25px;"></canvas></span></td>
<td>$30</td>
<td>
2y 2m
</td>
<td>
Apr 1, 2018
</td>
<td><a onclick="getMasterPricingData('31727');" data-toggle="modal"><button id="subscribeToMasterBtn31727" class="btn btn-circle btn-sm green" type="button">Copy</button></a>
<div style="display: none;">
<input type="hidden" class="monthlyData" oid="31727" value="6.22,-6.15,22.04,-5.08,0.08,12.08,-69.31,-99.82,245.26,88.44,113.73,52.29,25.38,77.72,-29.07,-24.73,-86.48,-89.27,195.77,-7.65,-99.98,278.89,-69.98,-65.48">
<input type="hidden" class="dailyGrowthData" oid="31727" value="-99.66,-99.69,-99.72,-99.73,-99.77,-99.77,-99.78,-99.81,-99.90,-99.90,-99.89,-99.84,-99.83,-99.82,-99.81,-99.75,-99.78,-99.77,-99.79,-99.78,-99.77,-99.48,-99.46,-99.36,-99.34,-99.33,-99.33,-99.31,-99.33,-99.34,-99.40,-99.45,-99.33,-99.58,-99.65,-99.73,-99.71,-99.70,-99.68,-99.68,-99.69,-99.68,-99.71,-99.68,-99.80,-99.80,-99.77,-99.81,-99.84,-99.90">
<input type="hidden" class="dailyEquityData" oid="31727" value="-99.66,-99.69,-99.73,-99.70,-99.85,-99.89,-99.95,-99.77,-99.85,-99.90,-99.88,-99.84,-99.83,-99.82,-99.79,-99.75,-99.78,-99.77,-99.70,-99.68,-99.59,-99.48,-99.46,-99.36,-99.34,-99.33,-99.32,-99.25,-99.30,-99.34,-99.37,-99.37,-99.35,-99.58,-99.61,-99.73,-99.71,-99.69,-99.68,-99.68,-99.68,-99.68,-99.71,-99.68,-99.80,-99.76,-99.73,-99.79,-99.80,-99.89">
</div>
</td>
</tr>
</tbody>
</table>
My code successfully extracts the data from the first table-data cell (the rank). But it is showing as blank for the second table data cell (the name). What is wrong with this source code:
import scrapy
from behold import Behold
class SignalStartSpider(scrapy.Spider):
name = 'signalstart'
start_urls = [
'https://www.signalstart.com/search-signals',
]
def parse(self, response):
for provider in response.xpath("//div[#class='row']//tr"):
yield {
'rank': provider.xpath('td[1]/text()').get(),
'name': provider.xpath('td[2]/text()').get(),
}
UPDATE
I am now iterating over the td cells within tr and getting the td cells, but my final problem is: how to get the text from the td cells that I have?
import scrapy
from behold import Behold
class SignalStartSpider(scrapy.Spider):
name = 'signalstart'
start_urls = [
'https://www.signalstart.com/search-signals',
]
def parse(self, response):
cols = "rank name gain pips drawdown trades type monthly chart price age added action"
skip = [9,13]
td = dict()
for i, col in enumerate(cols.split()):
td[i] = col
Behold().show('td')
for provider in response.xpath("//div[#class='row']//tr"):
data_row = dict()
for i, datum in enumerate(provider.xpath('td')):
if i in skip:
continue
data_row[td[i]] = datum
# Behold().show('datum')
yield data_row

The correct answer was provided by gallaecio_ in the Scrapy IRC channel - here is the code:
import scrapy
from behold import Behold
class SignalStartSpider(scrapy.Spider):
name = 'signalstart'
start_urls = [
'https://www.signalstart.com/search-signals',
]
def parse(self, response):
cols = "rank name gain pips drawdown trades type monthly chart price age added action"
skip = [9,13]
td = dict()
for i, col in enumerate(cols.split()):
td[i] = col
Behold().show('td')
for provider in response.xpath("//div[#class='row']//tr"):
data_row = dict()
for i, datum in enumerate(provider.xpath('td/text()')):
if i in skip:
continue
data_row[td[i]] = datum.get()
# Behold().show('datum')
yield data_row
for more involved cases you may need https://github.com/TeamHG-Memex/html-text

Buttons on second html file not working for Flask

Ok so I have two html files. base.html and search_results.html
base.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
<style>
body {
font-family: sans-serif;
background-image: url('/static/Flat-Color-Gradient-Triangles.png');
width: 430px;
height: 763px;
align-items: center;
}
h1 {
font-family: "Century Gothic", CenturyGothic, AppleGothic, sans-serif;
font-size: 45px;
font-color: #333333;
font-weight: bold;
margin: 0;
padding: 3px;
text-align: center;
}
h2 {
font-family: "Avant Garde", Avantgarde, "Century Gothic", CenturyGothic, AppleGothic, sans-serif;
font-size: 15px;
font-weight: bold;
margin: 0;
padding: 3px;
border: 1.5px
}
h3 {
font-family: Candara, Calibri, Segoe, "Segoe UI", Optima, Arial, sans-serif;
font-size: 14px;
font-weight: 100;
margin: 0;
border-top: 1px;
border-bottom: 1px;
padding: 3px;
}
.explicit{
color: rgb(200, 0, 0);
}
.rank{
font-weight: bold;
}
table {
border-collapse: collapse;
padding: 4px;
}
table, th, td {
padding: 5px;
border-bottom: 1px solid black;
color: white;
text-align: center;
}
th {
background-color: #121f1f;
border-left: #121f1f;
border-right:#121f1f;
}
td {
background-color: #253f3f;
border-left: #253f3f;
border-right: #253f3f;
}
input[type='text'] {
width: 100%;
padding: 6px 10px;
margin: 6px 0;
box-sizing: border-box;
border: 2px solid #ccc;
font-size: 14px;
border-radius: 10px;
}
input[type='text']:focus {
border: 3px solid red;
}
input[type = 'submit'] {
background-color: #4CAF50;
border: none;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
width: 100%;
}
input[value="Up"] {
background-color: #4CAF50;
border: none;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
}
input[value="Down"] {
background-color: rgb(200,0,0);
border: none;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
}
input[value="Add"] {
border: 3px solid #4CAF50;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
}
div.content {
width: 430px;
align-items: center;
}
</style>
</head>
<body>
<div class="content">
<h1>Public Playlist</h1>
<form action = '/search_results' method="POST">
<input placeholder="Artist:" type = 'text' name='artist'> </input>
<input placeholder="Album:" type = 'text' name='album'> </input>
<input placeholder="Track:" type = 'text' name='track'> </input>
<input type="submit" value="Submit!" id="search"/>
</form>
<table >
<tr>
<th><h2>Rank</h2></th>
<th><h2>Title</h2>
<h2>Author|Album</h2></th>
<th><h2>Score</h2></th>
<th><h2>Vote</h2></th>
</tr>
<tr>
<td><h3 type="bold">1</h3></td>
<td><h3>{{name}}</h3>
<h3>{{artist}}:{{album}}</h3></td>
<td><h3>{{ score }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote" value="Up" id="up">
<input type="button" name= "vote" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">2</h3></td>
<td><h3>{{name1}}</h3>
<h3>{{artist1}}:{{album1}}</h3></td>
<td><h3>{{ score1 }}</h3></td>
<form action="/" method="POST">
<td>
<input type="button" name="vote1" value="Up" id="up">
<input type="button" name= "vote1" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">3</h3></td>
<td><h3>{{name2}}</h3>
<h3>{{artist2}}:{{album2}}</h3></td>
<td><h3>{{ score2 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote2" value="Up" id="up">
<input type="button" name= "vote2" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">4</h3></td>
<td><h3>{{name3}}</h3>
<h3>{{artist3}}:{{album3}}</h3></td>
<td><h3>{{ score3 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote3" value="Up" id="up">
<input type="button" name= "vote3" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">5</h3></td>
<td><h3>{{name4}}</h3>
<h3>{{artist4}}:{{album4}}</h3></td>
<td><h3>{{ score4 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote4" value="Up" id="up">
<input type="button" name= "vote4" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">6</h3></td>
<td><h3>{{name5}}</h3>
<h3>{{artist5}}:{{album5}}</h3></td>
<td><h3>{{ score5 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote5" value="Up" id="up">
<input type="button" name= "vote5" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">7</h3></td>
<td><h3>{{name6}}</h3>
<h3>{{artist6}}:{{album6}}</h3></td>
<td><h3>{{ score6 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote6" value="Up" id="up">
<input type="button" name= "vote6" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">8</h3></td>
<td><h3>{{name7}}</h3>
<h3>{{artist7}}:{{album7}}</h3></td>
<td><h3>{{ score7 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote7" value="Up" id="up">
<input type="button" name= "vote7" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">9</h3></td>
<td><h3>{{name8}}</h3>
<h3>{{artist8}}:{{album8}}</h3></td>
<td><h3>{{ score8 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote8" value="Up" id="up">
<input type="button" name= "vote8" value="Down" id="down">
</td>
</form>
</tr>
<tr>
<td><h3 type="bold">10</h3></td>
<td><h3>{{name9}}</h3>
<h3>{{artist9}}:{{album9}}</h3></td>
<td><h3>{{ score9 }}</h3></td>
<form action="/vote" method="POST">
<td>
<input type="button" name="vote9" value="Up" id="up">
<input type="button" name= "vote9" value="Down" id="down">
</td>
</form>
</tr>
</table>
<script>
setTimeout(function () {location.reload() },60000);
</script>
</div>
</body>
</html>
search_results.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Title</title>
<style>
body {
font-family: sans-serif;
background-image: url('/static/Flat-Color-Gradient-Triangles.png');
width: 640px;
height: 1136px;
}
h1 {
font-family: "Century Gothic", CenturyGothic, AppleGothic, sans-serif;
font-size: 45px;
font-color: #333333;
font-weight: bold;
margin: 0;
padding: 3px;
text-align: center;
}
h2 {
font-family: "Avant Garde", Avantgarde, "Century Gothic", CenturyGothic, AppleGothic, sans-serif;
font-size: 15px;
font-weight: bold;
margin: 0;
padding: 3px;
border: 1.5px
}
h3 {
font-family: Candara, Calibri, Segoe, "Segoe UI", Optima, Arial, sans-serif;
font-size: 14px;
font-weight: 100;
margin: 0;
border-top: 1px;
border-bottom: 1px;
padding: 3px;
}
.explicit {
color: rgb(200, 0, 0);
}
.rank {
font-weight: bold;
}
table {
border-collapse: collapse;
padding: 4px;
}
table,
th,
td {
padding: 5px;
border-bottom: 1px solid black;
color: white;
text-align: center;
}
th {
background-color: #121f1f;
border-left: #121f1f;
border-right: #121f1f;
}
td {
background-color: #253f3f;
border-left: #253f3f;
border-right: #253f3f;
}
input[type='text'] {
width: 100%;
padding: 6px 10px;
margin: 6px 0;
box-sizing: border-box;
border: 2px solid #ccc;
font-size: 14px;
border-radius: 10px;
}
input[type='text']:focus {
border: 3px solid red;
}
input[type='submit'] {
background-color: #4CAF50;
border: none;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
width: 100%;
}
input[value="Up"] {
background-color: #4CAF50;
border: none;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
}
input[value="Down"] {
background-color: rgb(200, 0, 0);
border: none;
color: white;
padding: 16px 32px;
text-decoration: none;
margin: 4px 2px;
}
input[name="Add"] {
border: 3px solid #4CAF50;
color: white;
background-color: #253f3f;
padding: 16px 32px;
text-decoration: none;
font-weight: bold;
margin: 4px 2px;
}
</style>
</head>
<body>
<h1>Public Playlist</h1>
<table>
<tr>
<th>
<h2>Rank</h2>
</th>
<th>
<h2>Title</h2>
<h2>Author|Album</h2>
</th>
<th>
<h2>Score</h2>
</th>
<th>
<h2>Add</h2>
</th>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">1</h3>
</td>
<td>
<h3>{{name}}</h3>
<h3>{{artist}}:{{album}}</h3>
</td>
<td>
<h3>{{ score }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add" id="Add">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">2</h3>
</td>
<td>
<h3>{{name1}}</h3>
<h3>{{artist1}}:{{album1}}</h3>
</td>
<td>
<h3>{{ score1 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add1" id="Add1">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">3</h3>
</td>
<td>
<h3>{{name2}}</h3>
<h3>{{artist2}}:{{album2}}</h3>
</td>
<td>
<h3>{{ score2 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add2" id="Add2">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">4</h3>
</td>
<td>
<h3>{{name3}}</h3>
<h3>{{artist3}}:{{album3}}</h3>
</td>
<td>
<h3>{{ score3 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add3" id="Add3">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">5</h3>
</td>
<td>
<h3>{{name4}}</h3>
<h3>{{artist4}}:{{album4}}</h3>
</td>
<td>
<h3>{{ score4 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add4" id="Add4">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">6</h3>
</td>
<td>
<h3>{{name5}}</h3>
<h3>{{artist5}}:{{album5}}</h3>
</td>
<td>
<h3>{{ score5 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add5" id="Add5">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">7</h3>
</td>
<td>
<h3>{{name6}}</h3>
<h3>{{artist6}}:{{album6}}</h3>
</td>
<td>
<h3>{{ score6 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add6" id="Add6">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">8</h3>
</td>
<td>
<h3>{{name7}}</h3>
<h3>{{artist7}}:{{album7}}</h3>
</td>
<td>
<h3>{{ score7 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add7" id="Add7">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">9</h3>
</td>
<td>
<h3>{{name8}}</h3>
<h3>{{artist8}}:{{album8}}</h3>
</td>
<td>
<h3>{{ score8 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add8" id="Add8">
</td>
</form>
</tr>
<tr>
<form action="/add" method="GET">
<td>
<h3 type="bold">10</h3>
</td>
<td>
<h3>{{name9}}</h3>
<h3>{{artist9}}:{{album9}}</h3>
</td>
<td>
<h3>{{ score9 }}</h3>
</td>
<td>
<input type="button" name="Add" value="Add9" id="Add9">
</td>
</form>
</tr>
</table>
</body>
</html>
I then have a python file projectfalcon.py The relevant methods are:
#app.route('/add', methods=['GET'])
def addingSongs():
if request.method == 'GET':
btnID = request.form['btn']
if btnID == 'Add0':
songResults[0].upVote()
addedSongs.append(songResults[0])
songResults.remove(0)
if btnID == 'Add1':
songResults[1].upVote()
addedSongs.append(songResults[1])
songResults.remove(1)
return render_template('base.html')
So the base.html is the first page that loads. You type in an artist name to search for songs and it takes you to the search_results.html where the results are displayed. I am trying to let the user be able to click the add button and the song will then be added to a list in the projectfalcon.py file. After clicking it will also take you back to the base.html file and reload the songs that in the playlist I am not sure what I am doing wrong. I have googled multiple times but may have not found the right wording. Any help is appreciated.

The most obvious issue I can spot is the way you're accessing the btn variable. Since your form sends a GET request, it needs to be accessed like this:
btnID = request.args.get('btn', '')
Apart from that, I don't understand how you're storing addedSongs list. Flask context is limited to requests, and you shouldn't use python global namespace. You should either create a context variable (also a global in flask) or the easiest way would be to just store user data in a database.
You should take a look at this flask tutorial which is a complete guide to making a web app from scratch. Perhaps it may be of some use to get the basics cleared up and allow you to 'find the right wordings' to your problems! :)
Cheers.

Python selenium webdriver dropdown menu how to select items

<div id="isc_3B" class="scrollingMenu" onscroll="return isc_PickListMenu_0.$lh()" style="position: absolute; left: 403px; top: 63px; width: 450px; height: 298px; z-index: 800684; visibility: inherit; padding: 0px; box-sizing: border-box; overflow: hidden;" role="listbox" eventproxy="isc_PickListMenu_0" aria-hidden="false">
<div id="isc_3C" style="position: relative; display: inline-block; box-sizing: border-box; width: 100%; vertical-align: top; visibility: inherit; z-index: 800684; cursor: default;" eventproxy="isc_PickListMenu_0">
<div id="isc_3D" role="toolbar" tabindex="-1" onblur="if(window.isc)isc.EH.blurFocusCanvas(isc_Toolbar_1,true);" onfocus="if(event.target!=this)return;isc.EH.focusInCanvas(isc_Toolbar_1,true);" onscroll="return isc_Toolbar_1.$lh()" style="position: absolute; left: 0px; top: 0px; width: 434px; height: 22px; z-index: 200936; overflow: hidden; box-sizing: border-box; cursor: default; display: inline-block;" eventproxy="isc_Toolbar_1">
<div id="isc_3A" class="pickListMenuBody" tabindex="1439" onblur="if(window.isc)isc.EH.blurFocusCanvas(isc_PickListMenu_0_body,true);" onfocus="if(event.target!=this)return;isc.EH.focusInCanvas(isc_PickListMenu_0_body,true);" onscroll="return isc_PickListMenu_0_body.$lh()" style="position: absolute; left: 0px; top: 22px; width: 434px; height: 276px; z-index: 201026; overflow: hidden; background-color: white; box-sizing: border-box; cursor: default; display: inline-block; outline-style: none;" eventproxy="isc_PickListMenu_0_body">
<div id="isc_3N" style="position:absolute;overflow:visible;z-index:1000;width:432px">
<div id="isc_PickListMenu_0_body$28s" style="width:1px;height:0px;overflow:hidden;display:none;">
<table id="isc_3Atable" class="listTable" width="432" cellspacing="0" cellpadding="2" border="0" style="table-layout:fixed;overflow:hidden;padding-left:0px;padding-right:0px;" role="presentation">
<tbody></tbody>
<colgroup>
<tbody>
<tr id="isc_PickListMenu_0_row_0" aria-posinset="1" aria-setsize="686" role="option" aria-selected="false">
<td class="pickListCell" height="16" align="left" style="padding-top: 0px; padding-bottom: 0px; width: 143px; overflow: hidden;">
<div style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap;WIDTH:139px;" cellclipdiv="true" role="presentation">Pens Stabiliner 808 Ballpoint Fine Black</div>
</td>
<td class="pickListCell" height="16" align="left" style="padding-top: 0px; padding-bottom: 0px; width: 144px; overflow: hidden;">
<div style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap;WIDTH:140px;" cellclipdiv="true" role="presentation">Ea</div>
</td>
<td class="pickListCell" height="16" align="left" style="padding-top: 0px; padding-bottom: 0px; width: 145px; overflow: hidden;">
</tr>
<tr id="isc_PickListMenu_0_row_1" aria-posinset="2" aria-setsize="686" role="option">
<td class="pickListCellDark" height="16" align="left" style="WIDTH:143px;OVERFLOW:hidden;padding-top:0px;padding-bottom:0px;;white-space: nowrap;">
<td class="pickListCellDark" height="16" align="left" style="WIDTH:144px;OVERFLOW:hidden;padding-top:0px;padding-bottom:0px;;white-space: nowrap;">
<td class="pickListCellDark" height="16" align="left" style="WIDTH:145px;OVERFLOW:hidden;padding-top:0px;padding-bottom:0px;;white-space: nowrap;">
</tr>
<tr id="isc_PickListMenu_0_row_2" aria-posinset="3" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_3" aria-posinset="4" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_4" aria-posinset="5" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_5" aria-posinset="6" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_6" aria-posinset="7" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_7" aria-posinset="8" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_8" aria-posinset="9" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_9" aria-posinset="10" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_10" aria-posinset="11" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_11" aria-posinset="12" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_12" aria-posinset="13" aria-setsize="686" role="option">
**<tr id="isc_PickListMenu_0_row_13" aria-posinset="14" aria-setsize="686" role="option" aria-selected="true">
<td class="pickListCellSelectedDark" height="16" align="left" style="padding-top: 0px; padding-bottom: 0px; width: 143px; overflow: hidden;">
<div style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap;WIDTH:139px;" cellclipdiv="true" role="presentation">Adding Machine Roll 57x57mm Lint Free</div>
</td>
<td class="pickListCellSelectedDark" height="16" align="left" style="padding-top: 0px; padding-bottom: 0px; width: 144px; overflow: hidden;">
<td class="pickListCellSelectedDark" height="16" align="left" style="padding-top: 0px; padding-bottom: 0px; width: 145px; overflow: hidden;">
</tr>**
<tr id="isc_PickListMenu_0_row_14" aria-posinset="15" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_15" aria-posinset="16" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_16" aria-posinset="17" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_17" aria-posinset="18" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_18" aria-posinset="19" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_19" aria-posinset="20" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_20" aria-posinset="21" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_21" aria-posinset="22" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_22" aria-posinset="23" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_23" aria-posinset="24" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_24" aria-posinset="25" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_25" aria-posinset="26" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_26" aria-posinset="27" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_27" aria-posinset="28" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_28" aria-posinset="29" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_29" aria-posinset="30" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_30" aria-posinset="31" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_31" aria-posinset="32" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_32" aria-posinset="33" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_33" aria-posinset="34" aria-setsize="686" role="option">
<tr id="isc_PickListMenu_0_row_34" aria-posinset="35" aria-setsize="686" role="option">
</tbody>
</table>
<div id="isc_PickListMenu_0_body$284" style="width:1px;height:10416px;overflow:hidden;">
<table style="position:absolute;top:0px;font-size:1px;height:100%;width:100%;z-index:1;overflow:hidden;visibility:hidden;">
</div>
</div>
<div id="isc_3Q" class="scrollbar" onscroll="return isc_PickListMenu_0_body_vscroll.$lh()" style="position: absolute; left: 434px; top: 22px; width: 16px; height: 276px; z-index: 201027; overflow: hidden; box-sizing: border-box; cursor: default; display: inline-block;" dir="ltr" eventproxy="isc_PickListMenu_0_body_vscroll">
<div id="isc_3R" class="vScrollThumb" aria-label=" " onscroll="return isc_PickListMenu_0_body_vscroll_thumb.$lh()" style="position: absolute; left: 434px; top: 38px; width: 15px; height: 20px; z-index: 201033; overflow: hidden; box-sizing: border-box; cursor: default; display: inline-block;" eventproxy="isc_PickListMenu_0_body_vscroll_thumb">
<div id="isc_3O" class="scrollbarDisabled" onscroll="return isc_PickListMenu_0_body_hscroll.$lh()" style="position: absolute; left: 0px; top: 22px; width: 1px; height: 1px; z-index: 201027; overflow: hidden; box-sizing: border-box; cursor: default; display: inline-block; visibility: hidden;" dir="ltr" eventproxy="isc_PickListMenu_0_body_hscroll" aria-hidden="true">
<div id="isc_3P" class="hScrollThumb" aria-label=" " onscroll="return isc_PickListMenu_0_body_hscroll_thumb.$lh()" style="position: absolute; left: 16px; top: 22px; width: 5px; height: 1px; z-index: 201033; overflow: hidden; box-sizing: border-box; cursor: default; display: inline-block; visibility: hidden;" eventproxy="isc_PickListMenu_0_body_hscroll_thumb" aria-hidden="true">
<div id="isc_3T" aria-label="corner menu" role="button" tabindex="1490" onblur="if(window.isc)isc.EH.blurFocusCanvas(isc_PickListMenu_0_sorter,true);" onfocus="if(event.target!=this)return;isc.EH.focusInCanvas(isc_PickListMenu_0_sorter,true);" onscroll="return isc_PickListMenu_0_sorter.$lh()" style="POSITION:absolute;LEFT:434px;TOP:0px;WIDTH:16px;HEIGHT:22px;Z-INDEX:200942;OVERFLOW:hidden;box-sizing:border-box;CURSOR:default;display:inline-block" eventproxy="isc_PickListMenu_0_sorter">
</div>
</div>
</body>
Do you have any ideas how to select options from that menu using python webdriver? selenium ide is not helpful at all in this case. I was trying to select it by row id, name text and It's not working
every tr is option on the dropdown menu like:
>
Adding Machine Roll 57x57mm Lint Free

Following your exemple, you can use this:
select = Select(driver.find_element_by_xpath("//div[contains(text(), 'Adding Machine Roll 57x57mm Lint Free')]"))

Parsing HTML with BeautifulSoup in Python

I am trying to parse HTML with Python using BeautifulSoup, but I can't manage to get what I need.
This is a little module of a personal app I want to do, and it consists in a web login part with credentials, and once the script is logged in the web, I need to parse some information in order to manage it and process it.
The HTML code after getting logged is:
<div class="widget_title clearfix">
<h2>Account Balance</h2>
</div>
<div class="widget_body">
<div class="widget_content">
<table class="simple">
<tr>
<td>Daily Earnings</td>
<td style="text-align: right; width: 125px; color: #119911; font-weight: bold;">
150
</td>
</tr>
<tr>
<td>Weekly Earnings</td>
<td style="text-align: right; border-bottom: 1px solid #000; color: #119911; font-weight: bold;">
500 </td>
</tr>
<tr>
<td>Monthly Earnings</td>
<td style="text-align: right; color: #119911; font-weight: bold;">
1500 </td>
</tr>
<tr>
<td>Total expended</td>
<td style="text-align: right; border-bottom: 1px solid #000; color: #880000; font-weight: bold;">
430 </td>
</tr>
<tr>
<td>Account Balance</td>
<td style="text-align: right; border-bottom: 3px double #000; color: #119911; font-weight: bold;">
840 </td>
</tr>
<tr>
<td></td>
<td style="padding: 5px;">
<center>
<form id="request_bill" method="POST" action="index.php?page=dashboard">
<input type="hidden" name="secret_token" value="" />
<input type="hidden" name="request_payout" value="1" />
<input type="submit" class="btn blue large" value="Request Payout" />
</form>
</center>
</td>
</tr>
</table>
</div>
</div>
</div>
As you can see, it's not a very well-formatted HTML, but I'd need to extract the elements and their values, I mean, for example: "Daily earnings" and "150" | "Weekly earnings" and "500"...
I think that the "id" attribute may help, but when I try to parse it, it crashes.
The Python code I'm working with is:
def parseo(archivohtml):
html = archivohtml
parsed_html = BeautifulSoup(html)
par = parsed_html.find('td', attrs={'id':'west1'}).string
print par
Where archivohtml is the saved html file after logging in the web
When I run the script, I only get errors.
I've also tried doing this:
def parseo(archivohtml):
soup = BeautifulSoup()
html = archivohtml
parsed_html = soup(html)
par = soup.parsed_html.find('td', attrs={'id':'west1'}).string
print par
But the result is still the same.

The tag with id="west1" is an <a> tag. You are looking for the <td> tag that comes after this <a> tag:
import BeautifulSoup as bs
content = '''<div class="widget_title clearfix">
<h2>Account Balance</h2>
</div>
<div class="widget_body">
<div class="widget_content">
<table class="simple">
<tr>
<td>Daily Earnings</td>
<td style="text-align: right; width: 125px; color: #119911; font-weight: bold;">
150
</td>
</tr>
<tr>
<td>Weekly Earnings</td>
<td style="text-align: right; border-bottom: 1px solid #000; color: #119911; font-weight: bold;">
500 </td>
</tr>
<tr>
<td>Monthly Earnings</td>
<td style="text-align: right; color: #119911; font-weight: bold;">
1500 </td>
</tr>
<tr>
<td>Total expended</td>
<td style="text-align: right; border-bottom: 1px solid #000; color: #880000; font-weight: bold;">
430 </td>
</tr>
<tr>
<td>Account Balance</td>
<td style="text-align: right; border-bottom: 3px double #000; color: #119911; font-weight: bold;">
840 </td>
</tr>
<tr>
<td></td>
<td style="padding: 5px;">
<center>
<form id="request_bill" method="POST" action="index.php?page=dashboard">
<input type="hidden" name="secret_token" value="" />
<input type="hidden" name="request_payout" value="1" />
<input type="submit" class="btn blue large" value="Request Payout" />
</form>
</center>
</td>
</tr>
</table>
</div>
</div>
</div>'''
def parseo(archivohtml):
html = archivohtml
parsed_html = bs.BeautifulSoup(html)
par = parsed_html.find('a', attrs={'id':'west1'}).findNext('td')
print par.string.strip()
parseo(content)
yields
150

I can't tell from your question if this will be applicable to you, but here's another method:
def parseo(archivohtml):
html = archivohtml
parsed_html = BeautifulSoup(html)
for line in parsed_html.stripped_strings:
print line.strip()
which yields:
Account Balance
Daily Earnings
150
Weekly Earnings
500
Monthly Earnings
1500
Total expended
430
Account Balance
840
And if you wanted the data in a list:
data = [line.strip() for line in parsed_html.stripped_strings]
[u'Account Balance', u'Daily Earnings', u'150', u'Weekly Earnings', u'500', u'Monthly Earnings', u'1500', u'Total expended', u'430', u'Account Balance', u'840']