Develop Reference

Change the value of a dataframe column to the value of a second column conditional on the value of a third column in pandas

I have data with current names of companies, old names, and the date of name changes. It looks like this:
name
former_name1
name_change_date1
ACMAT CORP
nan
NaT
ACME ELECTRIC CORP
nan
NaT
ACME UNITED CORP
nan
NaT
COLUMBIA ACORN TRUST
LIBERTY ACORN TRUST
2003-10-20
MULTIGRAPHICS INC
AM INTERNATIONAL INC
1997-03-17
MILLER LLOYD I III
nan
NaT
AFFILIATED COMPUTER SERVICES INC
nan
NaT
ADAMS RESOURCES & ENERGY, INC.
ADAMS RESOURCES & ENERGY INC
2005-04-01
BK Technologies Corp
BK Technologies, Inc.
2019-03-28
I want to figure out what the name of each company was at a particular date. Let's say I want to figure out the name of a company as of January 1st 2002. Then I could create a new column called say, edited_name, which would contain the current name of the company unless the company has changed names since 1/1/2002, in which case it would contain the historical name (i.e. former_name1) of the company. So the output should look something like this:
name
former_name1
name_change_date1
edited_name
ACMAT CORP
nan
NaT
ACMAT CORP
ACME ELECTRIC CORP
nan
NaT
ACME ELECTRIC CORP
ACME UNITED CORP
nan
NaT
ACME UNITED CORP
COLUMBIA ACORN TRUST
LIBERTY ACORN TRUST
2003-10-20
LIBERTY ACORN TRUST
MULTIGRAPHICS INC
AM INTERNATIONAL INC
1997-03-17
MULTIGRAPHICS INC
MILLER LLOYD I III
nan
NaT
MILLER LLOYD I III
AFFILIATED COMPUTER SERVICES INC
nan
NaT
AFFILIATED COMPUTER SERVICES INC
ADAMS RESOURCES & ENERGY, INC.
ADAMS RESOURCES & ENERGY INC
2005-04-01
ADAMS RESOURCES & ENERGY INC
BK Technologies Corp
BK Technologies, Inc.
2019-03-28
BK Technologies, Inc.
In Stata (with which I am much more familiar) this could be easily accomplished with:
gen edited_name = name
replace edited_name = former_name1 if name_change_date_1 > date("2002-01-01", "YMD") & name_change_date_1 != .
Unfortunately I am at a loss of how to accomplish this in Python/Pandas.
Data:
{'name': ['ACMAT CORP', 'ACME ELECTRIC CORP', 'ACME UNITED CORP', 'COLUMBIA ACORN TRUST',
'MULTIGRAPHICS INC', 'MILLER LLOYD I III', 'AFFILIATED COMPUTER SERVICES INC',
'ADAMS RESOURCES & ENERGY, INC.', 'BK Technologies Corp'],
'former_name1': [nan, nan, nan, 'LIBERTY ACORN TRUST', 'AM INTERNATIONAL INC', nan, nan,
'ADAMS RESOURCES & ENERGY INC', 'BK Technologies, Inc.'],
'name_change_date1': [NaT, NaT, NaT, '2003-10-20', '1997-03-17', NaT, NaT,
'2005-04-01', '2019-03-28']}

You could use numpy.where to select values depending on if a name change occurred or not:
import numpy as np
df['edited_name'] = np.where(df['name_change_date1'].notna() &
df['name_change_date1'].gt(pd.to_datetime('1/1/2002')),
df['former_name1'], df['name'])
or with mask:
df['edited_name'] = df['name'].mask(df['name_change_date1'].notna() &
df['name_change_date1'].gt(pd.to_datetime('1/1/2002')),
df['former_name1'])
Output:
name former_name1 \
0 ACMAT CORP NaN
1 ACME ELECTRIC CORP NaN
2 ACME UNITED CORP NaN
3 COLUMBIA ACORN TRUST LIBERTY ACORN TRUST
4 MULTIGRAPHICS INC AM INTERNATIONAL INC
5 MILLER LLOYD I III NaN
6 AFFILIATED COMPUTER SERVICES INC NaN
7 ADAMS RESOURCES & ENERGY, INC. ADAMS RESOURCES & ENERGY INC
8 BK Technologies Corp BK Technologies, Inc.
name_change_date1 edited_name
0 NaT ACMAT CORP
1 NaT ACME ELECTRIC CORP
2 NaT ACME UNITED CORP
3 2003-10-20 LIBERTY ACORN TRUST
4 1997-03-17 MULTIGRAPHICS INC
5 NaT MILLER LLOYD I III
6 NaT AFFILIATED COMPUTER SERVICES INC
7 2005-04-01 ADAMS RESOURCES & ENERGY INC
8 2019-03-28 BK Technologies, Inc.

Use:
import numpy as np
df = pd.DataFrame({'name':['a', 'b', 'c', 'd'], 'fname':[np.nan, 'h', 's', np.nan], 'dc':[np.nan, '2003-10-20', '1997-03-17', np.nan]})
df['dc'] = pd.to_datetime(df['dc'])
df['nname'] = df['fname'][df['dc']>'1/1/2002']
res = df['name'][df['nname'].isna()]
temp = df['fname'][df['nname'].notna()]
res = res.append(temp)
df['res']=res
output:

Creating a dict of list from pandas row?

I have a weird problem. I have a index and bunch of columns in a dataframe. I want the index to be a key and all the other columns to be in a list. Here's a example
df:
0 1 2 3
Barker Minerals Ltd
Blackout Media Corp
Booking Holdings Inc Booking Holdings Inc Booking Holdings Inc 4.10 04/13/2025 BOOKING HOLDINGS INC
Baker Hughes Company Baker Hughes Company BAKER HUGHES A GE COMPANY LLC-3.34%-12-15-2027 BAKER HUGHES A GE COMPANY LLC-3.14%-11-7-2029
Bank of Queensland Limited Bank of Queensland Limited Bank of Queensland Limited FRN 10-MAY-2026 3.50% 05/10/26 Bank of Queensland Limited FRN 26-OCT-2020 1.27% 10/26/20 Bank of Queensland Limited FRN 16-NOV-2021 1.12% 11/16/21
If I do this command it turns everything into a list when I want it to be a dict of a list:
df.to_numpy().tolist()
I want a dict with each key a list of values in the other columns(kind of like this):
{
Barker Minerals Ltd:
Blackout Media Corp:
Booking Holdings Inc: [Booking Holdings Inc ,Booking Holdings Inc 4.10 04/13/2025,BOOKING HOLDINGS INC]
Baker Hughes Company: [Baker Hughes Company ,BAKER HUGHES A GE COMPANY LLC-3.34%-12-15-2027,BAKER HUGHES A GE COMPANY LLC-3.14%-11-7-2029]
Bank of Queensland Limited: [Bank of Queensland Limited ,Bank of Queensland Limited FRN 10-MAY-2026 3.50% 05/10/26,Bank of Queensland Limited FRN 26-OCT-2020 1.27% 10/26/20, Bank of Queensland Limited FRN 16-NOV-2021 1.12% 11/16/21]
}
Is this possible to do?

The easiest answer as pointed out in the comments by Michael Szczesny:
df.T.to_dict(orient="list")
The output:
{'Barker Minerals Ltd': [nan, nan, nan, nan],
'Blackout Media Corp': [nan, nan, nan, nan],
'Booking Holdings Inc': ['Booking Holdings Inc',
'Booking Holdings Inc 4.10 04/13/2025',
'BOOKING HOLDINGS INC',
nan],
'Baker Hughes Company': ['Baker Hughes Company',
'BAKER HUGHES A GE COMPANY LLC-3.34%-12-15-2027',
'BAKER HUGHES A GE COMPANY LLC-3.14%-11-7-2029',
nan],
'Bank of Queensland Limited': ['Bank of Queensland Limited',
'Bank of Queensland Limited FRN 10-MAY-2026 3.50% 05/10/26',
'Bank of Queensland Limited FRN 26-OCT-2020 1.27% 10/26/20',
' Bank of Queensland Limited FRN 16-NOV-2021 1.12% 11/16/21']}
Also, in case you would like to lose all the nans, then the code is as follows:
df = pd.read_csv("df_to_dict.csv", index_col=0)
val = df.T.to_dict(orient="list")
cleaned_val = {}
for i in val:
cleaned_val[i] = [j for j in val[i] if str(j)!="nan"]
cleaned_val
The output is as follows:
{'Barker Minerals Ltd': [],
'Blackout Media Corp': [],
'Booking Holdings Inc': ['Booking Holdings Inc',
'Booking Holdings Inc 4.10 04/13/2025',
'BOOKING HOLDINGS INC'],
'Baker Hughes Company': ['Baker Hughes Company',
'BAKER HUGHES A GE COMPANY LLC-3.34%-12-15-2027',
'BAKER HUGHES A GE COMPANY LLC-3.14%-11-7-2029'],
'Bank of Queensland Limited': ['Bank of Queensland Limited',
'Bank of Queensland Limited FRN 10-MAY-2026 3.50% 05/10/26',
'Bank of Queensland Limited FRN 26-OCT-2020 1.27% 10/26/20',
' Bank of Queensland Limited FRN 16-NOV-2021 1.12% 11/16/21']}
The documentation of to_dict() can be accessed here.

scrappig an HTML tag on the web page using BS

Image description is:
Tag is:
I was looking to get the data in the stock dropdown. I went into the source and found the tag but I can't get the code to access the data. Can someone please help me fix the bug?
url ="http://www.moneycontrol.com/india/fnoquote/reliance-industries/RI/2020-07-30"
from bs4 import BeautifulSoup
import pandas as pd
r = requests.get(url, headers = headers)
soup = BeautifulSoup(r.content, "html.parser")
for i in soup.select("stock_id"):
print(i.text)

You can use #stock_code > option instead of stock_id to get the data in the stock dropdown.You can try it:
url ="http://www.moneycontrol.com/india/fnoquote/reliance-industries/RI/2020-07-30"
headers = {'User-Agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'}
from bs4 import BeautifulSoup
import requests
r = requests.get(url, headers = headers)
soup = BeautifulSoup(r.content, "html.parser")
a = soup.select("#stock_code > option")
for i in a:
print(i.text)
Output will be:
ACC
Adani Enterpris
Adani Ports
Adani Power
Ajanta Pharma
Allahabad Bank
Amara Raja Batt
Ambuja Cements
Apollo Hospital
Apollo Tyres
Arvind
Ashok Leyland
Asian Paints
Aurobindo Pharm
Axis Bank
Bajaj Auto
Bajaj Finance
Bajaj Finserv
Balkrishna Ind
Bank of Baroda
Bank of India
Bata India
BEML
Berger Paints
Bharat Elec
Bharat Fin
Bharat Forge
Bharti Airtel
Bharti Infratel
BHEL
Biocon
Bosch
BPCL
Britannia
Cadila Health
Can Fin Homes
Canara Bank
Capital First
Castrol
Ceat
Century
CESC
CG Power
Chennai Petro
Cholamandalam
Cipla
Coal India
Colgate
Container Corp
Cummins
Dabur India
Dalmia Bharat
DCB Bank
Dewan Housing
Dish TV
Divis Labs
DLF
Dr Reddys Labs
Eicher Motors
EngineersInd
Equitas Holding
Escorts
Exide Ind
Federal Bank
GAIL
Glenmark
GMR Infra
Godfrey Phillip
Godrej Consumer
Godrej Ind
Granules India
Grasim
GSFC
Havells India
HCL Tech
HDFC
HDFC Bank
Hero Motocorp
Hexaware Tech
Hind Constr
Hind Zinc
Hindalco
HPCL
HUL
ICICI Bank
ICICI Prudentia
IDBI Bank
IDFC
IDFC Bank
IFCI
IGL
India Cements
Indiabulls Hsg
Indian Bank
IndusInd Bank
Infibeam Avenue
Infosys
Interglobe Avi
IOC
IRB Infra
ITC
Jain Irrigation
Jaiprakash Asso
Jet Airways
Jindal Steel
JSW Steel
Jubilant Food
Just Dial
Kajaria Ceramic
Karnataka Bank
Kaveri Seed
Kotak Mahindra
KPIT Tech
L&T Finance
Larsen
LIC Housing Fin
Lupin
M&M
M&M Financial
Mahanagar Gas
Manappuram Fin
Marico
Maruti Suzuki
Max Financial
MCX India
Mindtree
Motherson Sumi
MRF
MRPL
Muthoot Finance
NALCO
NBCC (India)
NCC
Nestle
NHPC
NIIT Tech
NMDC
NTPC
Oil India
ONGC
Oracle Fin Serv
Oriental Bank
Page Industries
PC Jeweller
Petronet LNG
Pidilite Ind
Piramal Enter
PNB
Power Finance
Power Grid Corp
PTC India
PVR
Ramco Cements
Raymond
RBL Bank
REC
Rel Capital
Reliance
Reliance Comm
Reliance Infra
Reliance Power
Repco Home
SAIL
SBI
Shree Cements
Shriram Trans
Siemens
South Ind Bk
SREI Infra
SRF
Strides Pharma
Sun Pharma
Sun TV Network
Suzlon Energy
Syndicate Bank
Tata Chemicals
Tata Comm
Tata Elxsi
Tata Global Bev
Tata Motors
Tata Motors (D)
Tata Power
Tata Steel
TCS
Tech Mahindra
Titan Company
Torrent Pharma
Torrent Power
TV18 Broadcast
TVS Motor
Ujjivan Financi
UltraTechCement
Union Bank
United Brewerie
United Spirits
UPL
V-Guard Ind
Vedanta
Vodafone Idea
Voltas
Wipro
Wockhardt
Yes Bank
Zee Entertain
Select
ACC
Adani Enterpris
Adani Ports
Adani Power
Ajanta Pharma
Allahabad Bank
Amara Raja Batt
Ambuja Cements
Apollo Hospital
Apollo Tyres
Arvind
Ashok Leyland
Asian Paints
Aurobindo Pharm
Axis Bank
Bajaj Auto
Bajaj Finance
Bajaj Finserv
Balkrishna Ind
Bank of Baroda
Bank of India
Bata India
BEML
Berger Paints
Bharat Elec
Bharat Fin
Bharat Forge
Bharti Airtel
Bharti Infratel
BHEL
Biocon
Bosch
BPCL
Britannia
Cadila Health
Can Fin Homes
Canara Bank
Capital First
Castrol
Ceat
Century
CESC
CG Power
Chennai Petro
Cholamandalam
Cipla
Coal India
Colgate
Container Corp
Cummins
Dabur India
Dalmia Bharat
DCB Bank
Dewan Housing
Dish TV
Divis Labs
DLF
Dr Reddys Labs
Eicher Motors
EngineersInd
Equitas Holding
Escorts
Exide Ind
Federal Bank
GAIL
Glenmark
GMR Infra
Godfrey Phillip
Godrej Consumer
Godrej Ind
Granules India
Grasim
GSFC
Havells India
HCL Tech
HDFC
HDFC Bank
Hero Motocorp
Hexaware Tech
Hind Constr
Hind Zinc
Hindalco
HPCL
HUL
ICICI Bank
ICICI Prudentia
IDBI Bank
IDFC
IDFC Bank
IFCI
IGL
India Cements
Indiabulls Hsg
Indian Bank
IndusInd Bank
Infibeam Avenue
Infosys
Interglobe Avi
IOC
IRB Infra
ITC
Jain Irrigation
Jaiprakash Asso
Jet Airways
Jindal Steel
JSW Steel
Jubilant Food
Just Dial
Kajaria Ceramic
Karnataka Bank
Kaveri Seed
Kotak Mahindra
KPIT Tech
L&T Finance
Larsen
LIC Housing Fin
Lupin
M&M
M&M Financial
Mahanagar Gas
Manappuram Fin
Marico
Maruti Suzuki
Max Financial
MCX India
Mindtree
Motherson Sumi
MRF
MRPL
Muthoot Finance
NALCO
NBCC (India)
NCC
Nestle
NHPC
NIIT Tech
NMDC
NTPC
Oil India
ONGC
Oracle Fin Serv
Oriental Bank
Page Industries
PC Jeweller
Petronet LNG
Pidilite Ind
Piramal Enter
PNB
Power Finance
Power Grid Corp
PTC India
PVR
Ramco Cements
Raymond
RBL Bank
REC
Rel Capital
Reliance
Reliance Comm
Reliance Infra
Reliance Power
Repco Home
SAIL
SBI
Shree Cements
Shriram Trans
Siemens
South Ind Bk
SREI Infra
SRF
Strides Pharma
Sun Pharma
Sun TV Network
Suzlon Energy
Syndicate Bank
Tata Chemicals
Tata Comm
Tata Elxsi
Tata Global Bev
Tata Motors
Tata Motors (D)
Tata Power
Tata Steel
TCS
Tech Mahindra
Titan Company
Torrent Pharma
Torrent Power
TV18 Broadcast
TVS Motor
Ujjivan Financi
UltraTechCement
Union Bank
United Brewerie
United Spirits
UPL
V-Guard Ind
Vedanta
Vodafone Idea
Voltas
Wipro
Wockhardt
Yes Bank
Zee Entertain

Response object doesn't return the data I want to scrape from a URL

I am trying to scrape the titles, description, partners etc from this search result using requests and BeautifulSoup in Python. But the response object doesn't return the data which I need and which is shown when I visit the URL in the browser
Here is what I have so far:
import requests
from bs4 import BeautifulSoup as bs
url = 'https://partneredge.sap.com/content/partnerfinder/search.html#/search/results?itemsPerPage=10&sortBy=shortname&sortOrder=asc'
count = 0
response = requests.get(url)
if response.ok:
response = response.text
content = bs(response , 'lxml')
results = content.find_all('li',class_='search-results__item')
for each in results:
count+=1
title = each.find('header').find('a').text.strip()
link = each.find('header').find('a').get('href')
print('********************* '+str(count)+' *********************')
print('Title: {}\nLink: {}\n'.format(title,link))

The website is loaded with JavaScript event which render it's data dynamically once the page loads.
requests library will not be able to render JavaScript on the fly. so you can use selenium or requests_html. and indeed there's a lot of modules which can do that.
Now, we do have another option on the table, to track from where the data is rendered. I were able to locate the XHR request which is used to retrieve the data from the back-end API and render it to the users side.
You can get the XHR request by open Developer-Tools and check Network and check XHR/JS requests made depending of the type of call such as fetch
Below is a simple call where the limit is maximum of 600 configured by parameter size
So you will need to loop over by increment 600 each time till you reach total of 4803 which is the max result for sap. Below you do have a valid JSON dict, which you can access it using the keys.
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0",
}
data = "{\"returnCount\":{},\"repository\":\"partnerfinder\",\"type\":\"content\",\"filters\":[{\"field\":\"AUTHOR\",\"values\":[\"PWP\"],\"type\":\"exact\"}],\"returnResults\":{\"sort\":[{\"field\":\"SHORTNAME\",\"order\":\"asc\"}],\"page\":{\"size\":600},\"outputFields\":[\"TITLE\",\"EMAIL\",\"PARTNERID\",\"DESCRIPTION\",\"PHONE\",\"AWARDID\",\"FOCUSAREA\",\"LEVEL\",\"URL\",\"SHORTNAME\"]}}"
def main(url):
with requests.Session() as req:
r = req.post(url, data=data, headers=headers).json()
# print(r.keys())
# humanview = json.dumps(r, indent=4)
for item in r["result"]["results"]["results"]:
print(item["TITLE"])
main("https://partneredge.sap.com/bin/fiji/es/search-results")
Sample of output:
IBM World Trade Corporation - Egypt Branch
Origo hf.
2BM A/S
Accenture (China) Co. Limited
Accenture (UK) Ltd.
ACCENTURE - CONSULTORES DE GESTÃO, S.A.
Accenture AB
Accenture AG
Accenture AS
Accenture Australia Pty Ltd
Accenture B.V.
Accenture Chile Asesorías y Serv. Ltda.
Accenture Company Limited
Accenture do Brasil Ltda.
Accenture GmbH
Accenture GmbH
Accenture Inc.
Accenture Japan Ltd.
Accenture Limited LAGOS
Accenture LLC
Accenture Middle East BV
ACCENTURE OUTSOURCING SERVICES, S.A.
Accenture Pte Ltd
Accenture S.A.
Accenture S.p.A.
Accenture S.R.L.
ACCENTURE SAS
Accenture Saudi Arabia Limited
Accenture Sdn Bhd
Accenture Solutions Private Limited
Accenture Solutions Sdn Bhd
Accenture Sp. z o.o.
Accenture Technology Solutions Oy
Accenture Technology Ventures B.V.branch
Accenture, Inc.
Accenture, S.L.
Accenture
Accenture
addIT Dienstleistungen GmbH & Co KG
advanced business Consulting GmbH
Ageless LLC
All for One Group AG
All for One Steeb GmbH
All Tax Platform - Soluções Tributárias S.A.
AMAZON WEB SERVICES INC
AO Deloitte and Touche CIS
Apigee Corporation
Atlantconsult LLC
Atos (Taiwan) Ltd
Atos AG
Atos Belgium SA
Atos Bilisim Danismanlik ve Müsteri Hizmetleri San. A.S.
Atos Bulgaria Competency Center EOOD
Atos Inc.
Atos India Pvt. Ltd.
Atos Information Technology (S) Pte Ltd
Atos Information Technology GmbH
Atos Information Technology (China) Co., Ltd.
Atos IT Services UK Limited
Atos It Services
Atos IT Solutions and Services A/S
Atos IT Solutions and Services AB
Atos IT Solutions and Services GmbH
ATOS IT SOLUTIONS AND SERVICES IBERIA SL
Atos IT Solutions and Services LLC
Atos IT Solutions and Services Oy
Atos IT Solutions and Services, Inc.
Atos IT Solutions and Services, s.r.o.
Atos IT Solutions and Services s.r.o.
ATOS IT SOLUTIONS AND SERVICES D.O.O. BEOGRAD
Atos IT Solutions and Services
Atos IT Solutions Romania SRL
Atos IT Solutions and Services Limited
Atos Italia S.p.A.
Atos Nederland B.V.
Atos Origin FZ LLC
Atos Polska S.A.
Atos Polska S.A.
Atos Saudi Arabia
Atos Serviços de Tecnologia da Informaçao do Brasil Ltda.
AUGUSTA REEVES
Axxis Consulting (S) Pte Ltd
B4B Solutions GmbH
B4B Solutions GmbH
Bluekey Seidor (Pty) Ltd.
BSG Partners Co., Ltd.
BULL SAS
CAPGEMINI (CHINA) CO., LTD
Capgemini AB
Capgemini Argentina S.A.
Capgemini Australia Pty Limited
Capgemini Belgium N.V.
Capgemini Brasil S.A.
CAPGEMINI ESPAÑA, S.L.
Capgemini Italia S.p.A.
Capgemini Mexico, S. de R.L. de C.V.
Capgemini Nederland B.V.
Capgemini Norge AS
CAPGEMINI PORTUGAL - SERVIÇOS DE CONSULTORIA E INFORMÁTICA, S.A.
Capgemini Services Malaysia Sdn Bhd
Capgemini Singapore Pte Ltd
Capgemini Technologies LLC
Capgemini Technologies LLC
Capgemini Technology Services India Ltd. Block 2&3,Plot no IT3,IT4, 2nd &
CAPGEMINI TECHNOLOGY SERVICES
Capgemini UK Plc
Celonis SE
CenturyLink CenturyTel LLC
CIDEON Software & Services GmbH & Co.KG
Cisco Systems, Inc.
COMPAGNIE IBM FRANCE SAS
CPM BRAXIS TECNOLOGIA LTDA.
Crystal Solutions S.A.
CSC Computer Sciences Brasil S.A.
CTAC België NV
Ctac N.V.
DC Extended Delivery S. de R.L. de C.V.
Deloitte & Co. S.A.
Deloitte & Touche Consulting Group ICS Pte Ltd.
Deloitte & Touche S.R.L.
Deloitte AB
Deloitte Asesores y Consultores Ltda
Deloitte Assessoria e Consultoria Ltda
Deloitte Business Solutions S.A
Deloitte Consulting & Advisory Deloitte Entity
Deloitte Consulting (Pty) Ltd
Deloitte Consulting (SEA) Sdn Bhd
Deloitte Consulting B.V.
DELOITTE CONSULTING CR, S.A.
Deloitte Consulting ehf.
Deloitte Consulting Ltd.
Deloitte Consulting Product Services LLC
Deloitte Consulting Pte Ltd
Deloitte Consulting S.A.
Deloitte Consulting S.r.l
DELOITTE CONSULTING VIETNAM CO., LTD
DELOITTE CONSULTORES, S.A.
Deloitte Inc.
DELOITTE LIMITED
Deloitte LLP
Deloitte Servicios Profesionales Ltda
Deloitte Tax & Consulting
Deloitte Technology Limited
Deloitte Tohmatsu Consulting LLC
Deloitte Touche Tohmatsu
Deloitte
Detay Danismanlik Bilgisayar Hizmetleri San. ve Dis Tic.A.S.
Deutsche Telekom Healthcare and Security Solutions GmbH
DSC Software AG
Dunn Solutions Group
DXC Technologies
DXC Technology (Middle East) FZ LLC
DXC Technology Austria GmbH
DXC Technology Czech Republic s.r.o.
DXC Technology Deutschland GmbH
DXC TECHNOLOGY FRANCE
DXC Technology Japan, Ltd.
DXC Technology Services LLC
DXC Technology Services Singapore Pte. Ltd.
DXC Technology Services Vietnam Company Limited
DXC Technology Spain, S.A.
DXD APPLICATIONS AND IT SOLUTIONS, S.L.
DynaSys Solutions Ltd.
Edenhouse Solutions Limited
Edraky LLC Edraky
EIT Services India Private Limited
Enterprise IT Services MiddleEast FZ LLC
ENTERPRISE SERVICES BRASIL SERVICOS DE TECNOLOGIA LTDA
Enterprise Services d.o.o. Beograd
EntServ Enterprise Services Mexico S. de R.L. de C.V.
ENTSERV PHILIPPINES,INC
EPAM Systems, Inc.
Everis Chile S.A.
Everis México, S. de R.L. de C.V.
Everis Peru Sociedad Anonima Cerrada
Excellence Delivered ExD Pvt. Ltd
EY Brightree Sdn. Bhd.
Fasttrack ERP Solutions Inc.
FastTrack Solutions, Inc.
FUJITSU LIMITED
GE Digital
Hewlett Packard Colombia LTDA.
Hewlett Packard Enterprise Company
Hewlett Packard Enterprise India Private Limited
Hewlett Packard Enterprise Polska sp. z o.o.
HEWLETT PACKARD FRANCE SAS
Hewlett-Packard (M) Sdn Bhd
Hewlett-Packard (Schweiz) GmbH
Hewlett-Packard (Thailand) Ltd.
Hewlett-Packard HK SAR Limited
Hitachi Vantara Corporation
Huawei Technologies Co., Ltd
IBM (China) Company Limited
IBM ARGENTINA SRL
IBM Australia Limited
IBM Belgium B.V.B.A.
IBM Brasil - Indústria, Máquinas e Serviços Limitada.
IBM BULGARIA EOOD
IBM Canada Limited
IBM Ceska republika, spol. s r.o.
IBM China / Hong Kong Limited (Partner)
IBM Corporation International Technical
IBM Danmark ApS
IBM de Chile S.A.C.
IBM De Colombia & Cia. S.C.A.
IBM del Peru S.A.C.
IBM Deutschland GmbH
IBM Eastern Europa/Asia Ltd.
IBM Eesti OÜ
IBM Engineering Technology (Shanghai) Co Ltd.
IBM GLOBAL SERVICES ESPAÑA, S.A.
IBM Global Services ve Teknoloji Hizmetleri
IBM Hellas S.A.
IBM HRVATSKA D.O.O.
IBM India Pvt Ltd
IBM Int. Business Machines AS
IBM International Business Machines doo
IBM Ireland Ltd
IBM Italia S.p.A.
IBM Italia S.p.A
IBM Italia SpA (Pakistan)
IBM Japan Services Company Ltd.
IBM Japan, Ltd.
IBM Korea, INC.
IBM Malaysia Sdn Bhd
IBM Middle East FZ LLC
IBM Middle East FZ-LLC SaudiArabiaBranch
IBM Polska Sp. z o.o.
IBM Qatar SSC
IBM Romania SRL
IBM Schweiz AG
IBM Singapore Pte Ltd
IBM Slovenija d.o.o.
IBM South Africa (Pty) Ltd
IBM Svenska AB
IBM Taiwan Corporation
IBM Thailand Co., Ltd.
IBM United Kingdom Limited
IBM World Trade Corporation
ICM.S S.r.l.
Illumiti Corp.
Illumiti Inc
Illumiti One Inc
Implema AB
In Cloud Solutions Ltd
Intelligroup Saudi Arabia Company Ltd
IPS Co., Ltd.
ISS Consulting (Thailand) Ltd.
Itelis d.o.o.
itelligence a/s
Itelligence AB
Itelligence AG, Niederlassung Wien
itelligence AG
itelligence AG
Itelligence AS
Itelligence Benelux Holding BV
Itelligence Bilgi Sistemleri A.S.
Itelligence Business Solutions (UK) Limited
Itelligence Business Solutions
itelligence Business Solutions Canada, Inc.
itelligence Consulting (Shanghai) Ltd
ITELLIGENCE FRANCE
itelligence Hungary Informatika Kft.
Itelligence India Software Solutions Private Limited
Itelligence LLC
itelligence Outsourcing MSC Sdn Bhd
itelligence Slovakia, s.r.o.
Itelligence Software Solutions
itelligence Sp. z o.o.
itelligence, a.s.
Itelligence, Inc.
itelligence
KBJ S.A.
KWP Austria GmbH
KWP INSIDE HR GmbH
Lenovo Global Technology HK Limited
MIBCON a.s.
MSS Seidor Colombia S.A.S.
MSS Seidor Peru SAC
MSS Seidor, S.L.
Myanmar Information Technology Pte Ltd.
NetApp, Inc.
NTT DATA Business Solutions Malaysia Sdn Bhd
NTT DATA Business Solutions Australia Pty Ltd
NTT DATA Business Solutions Singapore Pte Ltd
NTT DATA Romania SA
NTT DATA VIETNAM CO. LTD
Nutanix Inc
OOO T-Systems CIS
Open Text Corporation
ORBIT Gesellschaft für Applikations- und Informationssysteme mbH
OSC Smart Integration GmbH
OSIsoft, LLC
Oxygen Business Solutions Ltd
Oxygen Business Solutions Pty Ltd
Pearl Norge AS
Price Waterhouse & Co Asesores de Empresas S.R.L
PricewaterhouseCoopers Advisory N.V.
PricewaterhouseCoopers Advisory S.p.A.
PricewaterhouseCoopers Business Solutions SA
PricewaterhouseCoopers Consulting (Australia) Pty Limited
PricewaterhouseCoopers Consulting (Singapore) PTE LTD
PricewaterhouseCoopers Consulting (Thailand) Ltd.
PriceWaterhouseCoopers Consultores Auditores SpA
PricewaterhouseCoopers GmbH Wirtschaftsprüfungsgesellschaft
PricewaterhouseCoopers Inc
PricewaterhouseCoopers LLP
PriceWaterhouseCoopers LLP
PricewaterhouseCoopers LLP
PricewaterhouseCoopers Management Consulting (Shanghai) Limited
Pricewaterhousecoopers Management Consultants SRL
PricewaterhouseCoopers Private Limited
Pricewaterhousecoopers, S.C.
PricewaterhouseCoopers Česká republika s.r.o.
Process Partner AG
Proquire LLC
PT Accenture
PT Deloitte Consulting
PT IBM Indonesia
PT Soltius Indonesia
Pure Storage Inc.
PwC Consulting LLC
PWC Consulting Services (M) Sdn Bhd.
PwC Enterprise Advisory cvba/scrl
PwC Société coopérative
Questionmark Computing Limited
QZing Technology (Beijing) Company Limited
Real Consulting IT Business Solutions SA
Red Hat, Inc.
RED POINT SOFTWARE SOLUTIONS SRL
Redwood Alliances B.V.
SAS Institute Inc.
SAVIC Technologies Private Limited
Seidor Chile S.A.
Seidor Colombia SAS
Seidor Consulting Peru SAC
Seidor Consulting SA
SEIDOR CONSULTING, S.L.
Seidor Crystalis - Tecnologia da Informação S.A.
Seidor Crystalis Costa Rica
Seidor Maroc, SARL
SEIDOR MEXICO SAPI de CV
Seidor Middle East & North Africa FZ-LLC
Seidor Portugal, Lda
Seidor S.A.
Seidor UK Ltd
Seidor Uruguay Informatica S.A
Seidor USA Corp
Servicios Informáticos Itelligence, S.A
Shanghai Acloudear Info. Tech. Co., Ltd.
SOA PEOPLE SA/NV
SOA PEOPLE SAS
Société Conseil Groupe LGS.
Sofigate Business Technologies Oy
SOFTINSA - ENGENHARIA DE SOFTWARE AVANÇADO, LDA
SUSE Software Solutions Germany GmbH
Syniti
Systex Corporation
T-Systems Austria GesmbH
T-Systems do Brasil Ltda
T-Systems International GmbH
T-SYSTEMS ITC IBERIA, S.A.U.
T-Systems Malaysia Sdn. Bhd.
T-Systems Multimedia Solutions GmbH
T-Systems México, S.A. de C.V.
T-Systems Nederland B.V.
T-Systems North America, Inc.
T-Systems P.R.China Ltd.
T-Systems Polska Spolka z o.o.
T-Systems Schweiz AG
T-Systems Singapore Pte Ltd
T-Systems South Africa (Pty) Ltd
TalentChamp Consulting GmbH
TEAMIDEA GROUP LLC
TECNILÓGICA ECOSISTEMAS, S.A.U.
Telekom Deutschland GmbH
Terralink Technologies LLC
The Hackett Group d/b/a Answerthink, Inc.
unit-IT Dienstleistungs GmbH & Co KG
Utopia Global, Inc.
VIEWNEXT S.A.
Vistex, Inc.
Westrocon Seidor (PTY) Ltd
Öhrlings PricewaterhouseCoopers AB
3Hold Technologies, SL
4process AG
Aareon Deutschland GmbH
AB Consulting Group Zrt.
Abacus Cambridge Partners (Middle East) Ltd.
Abacus Cambridge Partners Saudia LLC
Abacus Consulting Technology Pvt Ltd
AbacusConsulting
abat AG
abc Consulting S.A.
ABeam Consulting (Malaysia) Sdn Bhd
ABeam Consulting (Shanghai) Co., Ltd.
ABeam Consulting (Thailand) Limited
ABeam Consulting Ltd.
Abeo Vietnam Co., Ltd
Abide Consult AG
abilis GmbH
ABM Global Solutions, Inc.
Abracon GmbH
Absoft Limited
Acando Consulting AB
Accely Consulting India Private Limited
Acclimation Pty Ltd
ACEteK Software Limited
AchieveIT Solutions, Inc.
ACI Holdings
ACJ Consulting Co., Ltd.
Acorel VAR B.V.
ACRON Bilisim A.S.
Actualisap Consultores Bolivia S.A.
Acuma Solutions Ltd
Adesso SE
ADP Consultores S.R.L.
AdS Consulting, Beratung für angewandte Informationsverarbeitung GmbH
Advanced Applications GmbH
Advanced Business Software
Advanced Business Solutions
Advanced Electronics Company
Advantech Technologies Ltd
Advantic Consultores de sistemas de información S.L.
AEKANSA S.A. SYPSOFT360
AFON Technologies Pte Ltd
AG Consultancy & Apps Ltd
Agentil SA
AGILITA AG
Agion
AICOMP Cloud GmbH
AKT Solutions Ltd
AL BILAD ARABIA CO. LTD.
ALEXANDER MOORE S.A.
Alfa Sistemas de Gestao LTDA
Alfacloud LTD
all4cloud GmbH & Co. KG
Allgeier ES France
Allos S.r.l.
alogis AG
Altab S.A.
Altaflux Corporation
ALTEA UP S.r.l.
Altevie Technologies S.r.l.
ALTIM TECNOLOGIAS DE LA INFORMACIÓN,S.L.
AMS Advanced Management Solutions Ltd
AMS Solutions Limited
AMT - CONSULTING, S.A.
Anda Srl
ANSI Information Systems, Inc.
anthesis GmbH
AO "ECOPSY"
AODYS
APPCENTRIC Solutions Inc.
applied international informatics GmbH
APPTechnology Experts, Inc.
APSIA
apsolut GmbH
Arago Consulting Iberia
ARAGO CONSULTING SAS
Arete Bilgisayar, Otomasyon Egitim ve Danismanlik Hizm. San. Tic. Ltd. Sti.
Arete Global Company Branch FZ-LLC
Arineo GmbH
Arinso Iberica S.A.U.
Arithnea GmbH
Arvato Systems GmbH
AS Emergn
ASAP soft
ASAR AMERICA INC.
Ascarii Ltd
Ascentium Corporation DBA SMITH
AScorpi GmbH
Asecom B.V.
ASG Group Limited
Aspire HR Inc.
ASPN CO.,LTD.
Attune Consulting USA, Inc.
Aubay Italia S.p.A.
avantum Consult AG
Avectris AG
Avtenta, napredne poslovne rešitve, d.o.o.
Axians ICT Austria GmbH
Axians NEO Solutions & Technology GmbH
Axianseu II Digital CONSULTING SA
Axxiom Soluções Tecnológicas S.A
Ayesa Advanced Technologies, S.A.
b1 consulting GmbH
Babiel GmbH
Backoffice Associates
BAITCON S.A.
Baraka IT Solutions (Pty) Ltd
BC SKILLS CONSULTING
BCI Consulting SRL
BCS Business Consulting Services Kft.
BDO Unicon Business Solutions AO
BDO Ziv Haft Consulting # Management Ltd
be one solutions Americas Inc.
be one solutions Deutschland GmbH
be one solutions Japan K.K.
Be1Eye GmbH
BearingPoint GmbH
Beijing AVA Technology Inc.
Beijing Ether Electronics Group Co., Ltd.
Beijing Faujor Technology Co., Ltd.
Beijing Pactera Services Limited
Beijing Shunshiheng Technology Development Co,. Ltd.
BENOY LLC
Bestcom Infotech Corporation
Beyond Technologies Consulting Inc.
BEYOND TECHNOLOGIES
BGP Management Consulting S.p.A.
BH Consulting Co., Ltd.
Bilot Oyj
Birchman Solutions Ltd
BizTech Partners Co., Ltd
Blend IT Consultoria e Serviços em Informática Ltda
Blue Pencil Consulting Pty Ltd
Bluekey Seidor (K) Limited
BluePrint Technologies Private Limited
Bluetree Solutions Pty Ltd
BluLeader Pty Ltd
BMS Global Services LLC
bneXt Inc.
BosCloud Jiangsu Science and Technology Co., Ltd.
Boyum IT A/S
Bramasol, Inc.
Brave New World Consulting Pty Ltd t/as BNW Consulting
bridgX
Bright Business Partners
Britehouse a Division of Dimension Data (pty) Ltd
BS&C
BSGOne Co., Ltd. BSGOne
BTC Bilisim Hizmetleri A.S.
BTC Business Technology Consulting AG
BUSINESS AT WORK
Business Process Solutions SA de CV Xamai
Business Service Center LLC
BXT Solution Co., Ltd
BXTI Soluciones en Tecnología de Información S.A. de C.V.
C2E Teknoloji Servisleri Ticaret Anonim Sirketi
Caleo Consulting GmbH
Camelot ITLab GmbH
CANCOM Managed Services GmbH
CAPLAN Corporation
Castaliaz Technologies Private Limited
CBL CONSULTING
CCelera s.r.l.
CEGB Corporation
Celeritech Mexico SAPI de CV
CEO Consultoría, S.R.L
CEREALOG
CGI Suomi Oy
CGI Sverige AB
Chain Services TI S.A.C. Csti S.A.C.
Chengdu Biz-United Information Technology Co., Ltd.
China National Software and Service Co., Ltd.
Chinasoft Technology (Shenzhen) Corporation Limited
cHReative Consultoria LTDA
Cibernetica, S.A.
Citek Technology Joint Stock Company
Clarex Srl
Clariba Consulting S.L.U.
Clientis AG
Clients First Business Solutions LLC
Clockwork Business Solutions Private Limited
Clockwork Inc.
Cloudera Limited
Cloudway Consulting Private Limited
CM CONSULTING
CNBM Technology Co., Ltd
CNT Management Consulting AG
Codestone Solutions Ltd
CODILOG - ELIANCE
Cogniscient Business Solutions Private Limited
Cognitus Consulting LLC
Columbus Systems GmbH
COMLINE Computer + Softwarelösungen SE
COMMON MANAGEMENT SOLUTIONS, S.L.
Complete Business Solutions
COMPTA - EQUIPAMENTOS E SERVIÇOS DE INFORMÁTICA, S.A.
Compuage Infocom Limited
CompuNet, S.A.
CompuTec S.A.
Computer Systems Pvt Ltd
ComSol AG Commercial Solutions
CON.SE s.r.l.
ConCorn LLC.
CONPLUS Mittelstandslösungen GmbH
CONSEILS PLUS
Conseils-Plus
Consensus International, LLC
Consensus S.A.S.
Consilio GmbH
consolut.gmbh
Consulting 4U, s.r.o.
Consultoría Organizacional S.A.S.
ConVista Consulting AG
Cormeta AG
Corponet Implements, S.A. de C.V.
Corporacion Saratella, S.A. Vivo Consulting

You need to add a header with User-Agent string to your request. You can do that like this:
url = 'https://partneredge.sap.com/content/partnerfinder/search.html#/search/results?itemsPerPage=10&sortBy=shortname&sortOrder=asc'
header = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'
}
response = requests.get(url, headers = header)

You can try this
results = content.find_all('li', {'class': ['search-results__item']})

Inserting a header row for pandas dataframe

I have just started python and am trying to rewrite one of my perl scripts in python. Essentially, I had a long script to convert a csv to json.
I've tried to import my csv into a pandas dataframe, and wanted to insert a header row at the top, since my csv lacks that.
Code:
import pandas
db=pandas.read_csv("netmedsdb.csv",header=None)
db
Output:
0 1 2 3
0 3M CAVILON NO STING BARRIER FILM SPRAY 28ML OTC 0 Rs.880.00 3M INDIA LTD
1 BACTI BAR SOAP 75GM OTC Rs.98.00 6TH SKIN PHARMACEUTICALS PVT LTD
2 KWIKNIC MINT FLAVOUR 4MG CHEW GUM TABLET 30'S NICOTINE Rs.180.00 A S V LABORATORIES INDIA PVT LTD
3 RIFAGO 550MG TABLET 10'S RIFAXIMIN 550MG Rs.298.00 AAREEN HEALTHCARE
4 999 OIL 60ML AYURVEDIC MEDICINE Rs.120.00 AAKASH PHARMACEUTICALS
5 AKASH SOAP 75GM AYURVEDIC PRODUCT Rs.80.00 AAKASH PHARMACEUTICALS
6 GROW CARE OIL 100ML AYURVEDIC MEDICINE Rs.190.00 AAKASH PHARMACEUTICALS
7 GROW CARE OIL 100ML AYURVEDIC MEDICINE Rs.190.00 AAKASH PHARMACEUTICALS
8 RHUNS OIL 30ML AYURVEDIC Rs.50.00 AAKASH PHARMACEUTICALS
9 VILLO CAPSULE 10'S AYURVEDIC MEDICINE Rs.70.00 AAKASH PHARMACEUTICALS
10 VITAWIN FORTE CAPSULE 10'S AYURVEDIC MEDICINE Rs.150.00 AAKASH PHARMACEUTICALS
I wrote the following code to insert the first element at row 0,coloumn 0:
db.insert(loc=0,column='0',value='Brand')
db
Output:
0 0 1 2 3
0 Brand 3M CAVILON NO STING BARRIER FILM SPRAY 28ML OTC 0 Rs.880.00 3M INDIA LTD
1 Brand BACTI BAR SOAP 75GM OTC Rs.98.00 6TH SKIN PHARMACEUTICALS PVT LTD
2 Brand KWIKNIC MINT FLAVOUR 4MG CHEW GUM TABLET 30'S NICOTINE Rs.180.00 A S V LABORATORIES INDIA PVT LTD
3 Brand RIFAGO 550MG TABLET 10'S RIFAXIMIN 550MG Rs.298.00 AAREEN HEALTHCARE
4 Brand 999 OIL 60ML AYURVEDIC MEDICINE Rs.120.00 AAKASH PHARMACEUTICALS
5 Brand AKASH SOAP 75GM AYURVEDIC PRODUCT Rs.80.00 AAKASH PHARMACEUTICALS
6 Brand GROW CARE OIL 100ML AYURVEDIC MEDICINE Rs.190.00 AAKASH PHARMACEUTICALS
7 Brand GROW CARE OIL 100ML AYURVEDIC MEDICINE Rs.190.00 AAKASH PHARMACEUTICALS
8 Brand RHUNS OIL 30ML AYURVEDIC Rs.50.00 AAKASH PHARMACEUTICALS
9 Brand VILLO CAPSULE 10'S AYURVEDIC MEDICINE Rs.70.00 AAKASH PHARMACEUTICALS
10 Brand VITAWIN FORTE CAPSULE 10'S AYURVEDIC MEDICINE Rs.150.00 AAKASH PHARMACEUTICALS
But unfortunately I got the word "Brand" inserted at coloumn 0 in all rows.
I'm trying to add the header coloumns "Brand", "Generic", "Price", "Company"

Need parameter names in read_csv only:
import pandas as pd
temp=u"""a,b,10,d
e,f,45,r
"""
#after testing replace 'pd.compat.StringIO(temp)' to 'netmedsdb.csv'
df = pd.read_csv(pd.compat.StringIO(temp), names=["Brand", "Generic", "Price", "Company"])
print (df)
Brand Generic Price Company
0 a b 10 d
1 e f 45 r