Answer a question

I'm having trouble with a for loop in bs4, see code below:

from urllib.request import urlopen
from bs4 import BeautifulSoup
import request

html = urlopen('https://www.ebay.com/sch/i.html?_from=R40&_nkw=manga&_sacat=0&rt=nc&LH_Sold=1&LH_Complete=1')
soup = BeautifulSoup(html.read(), 'html.parser')

#soldItem = soup.find_all('h3', class_='s-item__title s-item__title--has-tags')
#salePrice = soup.find_all('span', class_='POSITIVE')

itemContainer = soup.find('ul', class_='srp-results srp-list clearfix')

for x in itemContainer.findAll('li'):
    #item = x.get_text()
    item = itemContainer.findAll('h3', class_='s-item__title s-item__title--has-tags').get_text()
    sale = itemContainer.findAll('span', class_='POSITIVE').get_text()
    data = {item:sale}
    print(data)

I'm getting the following error message:

AttributeError: ResultSet object has no attribute 'get_text'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?

I want to be able to iterate over every single listing from ebay in page 1 and get the name item and sale price. I've tried several ways but this error is persistent. I've gotten to the point where this errors does not appear anymore, but my variable iterates over the first line 50 times.

Any ideas on how to generate the data in the following format {item name:sale price}?

Answers

Your problem is exactly what the error message says - you're using findAll when you should be using find. Also, you're finding all of the h3's and spans in the itemContainer instead of each item (x) in the container.

Try replacing

item = itemContainer.findAll('h3', class_='s-item__title s-item__title--has-tags').get_text()
sale = itemContainer.findAll('span', class_='POSITIVE').get_text()

with

item = x.find('h3', class_='s-item__title s-item__title--has-tags').get_text()
sale = x.find('span', class_='POSITIVE').get_text()

Edit:

One more thing: when you do for x in itemContainer.findAll('li'), you're finding all of the li's in the itemContainer. It just so happens that there are more than just items for sale in that list, so it finds some that don't have an h3 with a nested span or text. So that's why it fails. Narrowing down the search with itemContainer.findAll('li', class_='s-item') does the trick.

Here is what the complete code should be now:

from urllib.request import urlopen
from bs4 import BeautifulSoup
import requests

html = urlopen('https://www.ebay.com/sch/i.html?_from=R40&_nkw=manga&_sacat=0&rt=nc&LH_Sold=1&LH_Complete=1')
soup = BeautifulSoup(html.read(), 'html.parser')

#soldItem = soup.find_all('h3', class_='s-item__title s-item__title--has-tags')
#salePrice = soup.find_all('span', class_='POSITIVE')

itemContainer = soup.find('ul', class_='srp-results srp-list clearfix')

for x in itemContainer.find_all('li', class_='s-item'):
    item = x.find('h3', class_='s-item__title s-item__title--has-tags').get_text()
    sale = x.find('span', class_='POSITIVE').get_text()
    data = {item:sale}
    print(data)

Note that I also changed findAll to find_all, which is just another name for the function that more closely follows pep8 naming conventions. The same could be applied in renaming itemContainer to item_container, etc, but it's not totally necessary.