问题:使用beautifulsoup从维基百科表中获取列

source_code = requests.get('http://en.wikipedia.org/wiki/Taylor_Swift_discography')
soup = BeautifulSoup(source_code.text)
tables = soup.find_all("table")

我正在尝试从Taylor Swift 的唱片的“单曲列表”表中获取歌曲名称列表

该表没有唯一的类或 ID。我能想到的唯一独特之处是“单曲列表...”周围的标题标签

主要艺人单曲列表,包括选定的排行榜位置、销售数据和证书

我试过了:

table = soup.find_all("caption")

但它什么也没返回,我假设标题不是 bs4 中的可识别标签?

解答

它实际上与findAll()和find_all()无关。findAll()在BeautifulSoup3中使用,并留在BeautifulSoup4中_出于兼容性原因_,引用自bs4的源代码:

def find_all(self, name=None, attrs={}, recursive=True, text=None,
             limit=None, **kwargs):
    generator = self.descendants
    if not recursive:
        generator = self.children
    return self._find_all(name, attrs, text, limit, generator, **kwargs)

findAll = find_all       # BS3

而且,还有一种更好的方法来获取单曲列表,它依赖于带有id="Singles"的span元素,它表示Singles段落的开始。然后,使用find_next_sibling()得到span标签的父标签之后的第一个表。然后,用scope="row"获取所有th元素:

from bs4 import BeautifulSoup
import requests


source_code = requests.get('http://en.wikipedia.org/wiki/Taylor_Swift_discography')
soup = BeautifulSoup(source_code.content)

table = soup.find('span', id='Singles').parent.find_next_sibling('table')
for single in table.find_all('th', scope='row'):
    print(single.text)

印刷:

"Tim McGraw"
"Teardrops on My Guitar"
"Our Song"
"Picture to Burn"
"Should've Said No"
"Change"
"Love Story"
"White Horse"
"You Belong with Me"
"Fifteen"
"Fearless"
"Today Was a Fairytale"
"Mine"
"Back to December"
"Mean"
"The Story of Us"
"Sparks Fly"
"Ours"
"Safe & Sound"
(featuring The Civil Wars)
"Long Live"
(featuring Paula Fernandes)
"Eyes Open"
"We Are Never Ever Getting Back Together"
"Ronan"
"Begin Again"
"I Knew You Were Trouble"
"22"
"Highway Don't Care"
(with Tim McGraw)
"Red"
"Everything Has Changed"
(featuring Ed Sheeran)
"Sweeter Than Fiction"
"The Last Time"
(featuring Gary Lightbody)
"Shake It Off"
"Blank Space"