Answer a question

I have a beautifulsoup object as follows

<div class="companyProfileHeader">
<div>Industry<a href="/stock-screener/?sp=country::5|sector::a|industry::146|equityType::a&lt;eq_market_cap;1">Life Sciences Tools &amp; Services</a></div>
<div>Sector<a href="/stock-screener/?sp=country::5|sector::18|industry::a|equityType::a&lt;eq_market_cap;1">Healthcare</a></div>
<div>Employees<p class="bold">17000</p></div>
<div>Equity Type<p class="bold">ORD</p></div>
</div>

I want to convert the above into a Pandas DataFrame as follows

Expected Output

+--------------------------------+------------+-----------+-------------+
|            Industry            |   Sector   | Employees | Equity Type |
+--------------------------------+------------+-----------+-------------+
| Life Sciences Tools & Services | Healthcare |     17000 | ORD         |
+--------------------------------+------------+-----------+-------------+

Suppose that the bs object is named divlist I've extracted the text within using divlist.text but can't slice it appropriately to achieve above data frame.

Answers

I have taken your data as html and you can iterate to specific class using find_all method and i have used list Comprehension to get text and it is separated by ~ symbol

from bs4 import BeautifulSoup
soup=BeautifulSoup(html,"html.parser")
lst=[i.get_text(strip=True,separator="~") for i in soup.find("div",class_="companyProfileHeader").find_all("div")]
final_lst=[i.split("~") for i in lst ]

Now you can transform into DataFrame using final_lst

import pandas as pd
df=pd.DataFrame(final_lst)
df=df.transpose()
df.rename(columns=df.iloc[0], inplace = True)
df.drop(df.index[0], inplace = True)