Answer a question

i use code below to scrape results from bing and when I see the scraped web page it says "There are no results for python". but when I search in the browser there is no problem.

import requests
from bs4 import BeautifulSoup

term = 'python'
url = f'https://www.bing.com/search?q={term}&setlang=en-us'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.prettify())

I searched and I didn't find any similar problem

Answers

You need to pass the user-agent while requesting to get the value.

import requests
from bs4 import BeautifulSoup

term = 'python'
url = 'https://www.bing.com/search?q={}&setlang=en-us'.format(term)
headers = {'User-Agent':'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36'}
response = requests.get(url,headers=headers)
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.prettify())