Answer a question

I want to print out all the IDs from a page which has a unique class.

The page what I want to scrape with Beautiful Soup is like this:

<div itemscope itemprop="item" itemtype="http://schema.org/Product" id="12345" class="realestate"> 
<div class="contentArea"> 
<meta itemprop="name" content="Name - 12345 " /> 
<meta itemprop="url" content="https://url12345.hu" />   
<meta itemprop="category" content="category1" />   
</div>
</div>
<div itemscope itemprop="item" itemtype="http://schema.org/Product" id="12346" class="realestate"> 
<div class="contentArea"> 
<meta itemprop="name" content="Name - 12346 " /> 
<meta itemprop="url" content="https://url12346.hu" />   
<meta itemprop="category" content="category1" />   
</div>
</div>

the 'ID' is a unique identifier from the Itemscope DIVs, so somehow I want to extract these uniques IDs and print all of them out (the reson is to attach all other ad information to this ID (like name, URL, etc.) later)

I tried with this python code, but it does not work.

import requests
from bs4 import BeautifulSoup

page = requests.get('searchResultPage.url')
soup = BeautifulSoup(page.text, 'html.parser')
id = soup.find_all('id')
print(id)

It gives back an empty list.

What I expect, and what I want is to get back a list with the ID-s from the divs, this way: 12345 12346

Thanks for your help in advance!

Answers

HS-nebula is correct that find_all looks for tags of a certain type, in your soup id is an attribute not a type of tag. To get a list of all id's in the soup you can use the following one liner

ids = [tag['id'] for tag in soup.select('div[id]')]

this uses CSS selectors instead of bs4's find_all since I find bs4's docs regarding its built-ins lacking.

So what the soup.select does is return a list of all div elements which have an attribute called 'id' then we loop over that list of div tags and add the value of the 'id' attribute to the ids list.