Answer a question

(I'm a beginner in the web scraping) I want to scrape this link: https://www.seloger.com/list.htm?tri=initial&idtypebien=1,2&pxMax=3000000&div=2238&idtt=2,5&naturebien=1,2,4&LISTING-LISTpg=2

when I tried to display repo_list I get that [] and not html code !!!`

import requests
from bs4 import BeautifulSoup

page = requests.get('https://www.seloger.com/list.htm?tri=initial&idtypebien=1,2&pxMax=3000000&div=2238&idtt=2,5&naturebien=1,2,4&LISTING-LISTpg=2')
soup = BeautifulSoup(page.text, 'html.parser')
repo = soup.find(class_="c-wrap")
print(repo)
repo_list = repo.find_all(class_='c-pa-list c-pa-sl c-pa-gold cartouche ')
print(repo_list)

Answers

You can regex out, do a little string cleaning and then pass to json to then print out each product as dictionary containing the info for each listing

import re
import requests
import json

r = requests.get('https://www.seloger.com/list.htm?tri=initial&idtypebien=1,2&pxMax=3000000&div=2238&idtt=2,5&naturebien=1,2,4&LISTING-LISTpg=2', headers = {'User-Agent' : 'Mozilla/5.0'})
p = re.compile('var ava_data =(.*);\r\n\s+ava_data\.logged = logged;', re.DOTALL)
x = p.findall(r.text)[0].strip().replace('\r\n    ','').replace('\xa0',' ').replace('\\','\\\\')
x = re.sub(r'\s{2,}|\\r\\n', '', x)
data = json.loads(x)

for product in data['products']:
    print(product)

Example return (from page 3):

{'idannonce': '142830891', 'idagence': '263765', 'idtiers': '284402', 'typedebien': 'Appartement', 'typedetransaction': ['vente'], 'idtypepublicationsourcecouplage': 'SL', 'position': '0', 'codepostal': '77450', 'ville': 'Esbly', 'departement': 'Seine-et-Marne', 'codeinsee': '770171', 'produitsvisibilite': 'AD:AC:BB:AW', 'affichagetype': [{'name': 'liste', 'value': True}], 'cp': '77450', 'etage': '0', 'idtypechauffage': '0', 'idtypecommerce': '0', 'idtypecuisine': 'séparée équipée', 'naturebien': '1', 'si_balcon': '0', 'nb_chambres': '1', 'nb_pieces': '2', 'si_sdbain': '0', 'si_sdEau': '0', 'nb_photos': '14', 'prix': '139900', 'surface': '44'}

Price for example is:

product['prix']