How to extract text from a websites page? [closed]
Answer a question
I am trying to extract text from some website's page whose HTML code looks like below. Sorry for the sorry looking code as I am a newbie and not aware of how to select a specific block of code in HTML.The code I have written below match somewhat with the actual code block as mentioned above.I am trying to extract the text which inside p tags as well as text inside strong tags under h3 and in a manner that it would retain the order of the text as in the website's page.Upon inspecting each webpage I found that all texts are contained under <div class=td-post-content> but not all of them are just under <p> tags or <strong> tags,there can be other branch as <p><strong><em>text</em></strong></p> . Is there a way I can do this without manually extracting text for each and every webpage?(for reference I am mentioning the screenshot here)
<div class="td-post-content">
<p>some_text</p>
<h3>
<strong>some_text</strong>
</h3>
<p>some_text</p>
<p>some_text</p>
<p>some_text</p>
<h3>
<strong>some_text</strong>
</h3>
<p>some_text</p>
<p>
<strong>
<em>text</em>
</strong>
</p>
<p>'some_text'</p>
<p>'some_text'</p>
</div>
Answers
If I correctly understand your question you can do something like this:
Get all the elements inside <div class="td-post-content"> element and then iterate over each element extracting it text.
In case there are another elements inside <div class="td-post-content">, not only p and h3 containing the relevant texts - more filtration should be added here.
all_elements = driver.find_elements(By.XPATH, "//div[@class='td-post-content']//*")
for element in all_elements:
print(element.text)
更多推荐

所有评论(0)