爬取 google 搜尋結果的第一個頁面標題

In [1]:

    

import requests

from bs4 import BeautifulSoup
from urllib3.exceptions import HTTPError
from urllib.parse import urljoin


    

In [2]:

    

base_url = 'https://www.google.com.tw/search'
query = {'q': 'python'}


    

In [3]:

    

try:
    resp = requests.get(base_url, params=query)
    soup = BeautifulSoup(resp.text, 'lxml')
    print(resp.url)
except HTTPError as err:
    print(err)
except AttributeError as err:
    print(err)


    

    




https://www.google.com.tw/search?q=python

In [4]:

    

# print(soup.prettify())


    

In [5]:

    

search_results = soup.select('div.g > h3.r > a[href^="/url"]')
for search_item in search_results:
    print(search_item.text)
    print(urljoin(base_url, search_item['href']))
    print('='*87)


    

    




Welcome to Python.org
https://www.google.com.tw/url?q=https://www.python.org/&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFggoMAA&usg=AOvVaw348GGzSkqgB-FXPinUSErY
=======================================================================================
Download Python | Python.org
https://www.google.com.tw/url?q=https://www.python.org/downloads/&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFggzMAE&usg=AOvVaw2UHusa0FkZGKEoJRjlxYza
=======================================================================================
Python - 維基百科，自由的百科全書 - Wikipedia
https://www.google.com.tw/url?q=https://zh.wikipedia.org/zh-tw/Python&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFgg5MAI&usg=AOvVaw1gsx_ugnMzjTP2nlH7zARm
=======================================================================================
一小時Python入門-part 1 - - 寫點科普
https://www.google.com.tw/url?q=https://kopu.chat/2017/01/18/%25E4%25B8%2580%25E5%25B0%258F%25E6%2599%2582python%25E5%2585%25A5%25E9%2596%2580-part-1/&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFghEMAM&usg=AOvVaw1BLo112Hj6BBWauFDpnbQN
=======================================================================================
課程介紹- 成為python數據分析達人的第一課(自學課程) | 政治大學磨 ...
https://www.google.com.tw/url?q=http://moocs.nccu.edu.tw/course/123&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFghOMAQ&usg=AOvVaw3RXTAa5ochrAyo-2evVdhI
=======================================================================================
《經濟學人》專文探討：「為什麼Python 是世上最屌的程式語言 ...
https://www.google.com.tw/url?q=https://buzzorange.com/techorange/2018/08/01/python-a-skr-language/&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFghTMAU&usg=AOvVaw2yA2hrrl61qBKnKoEeeTix
=======================================================================================
Python 入門| Django Girls Taipei
https://www.google.com.tw/url?q=http://djangogirlstaipei.herokuapp.com/tutorials/python/&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFghZMAY&usg=AOvVaw0ha-itZMKnVgaSsRQlcutt
=======================================================================================
Python Tutorial: Learn Python For Free | Codecademy
https://www.google.com.tw/url?q=https://www.codecademy.com/learn/learn-python&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFghfMAc&usg=AOvVaw09DURYBaIbVzO6GSXKb0gH
=======================================================================================
Python Tutorial - W3Schools
https://www.google.com.tw/url?q=https://www.w3schools.com/python/&sa=U&ved=0ahUKEwj-8PeJzfvdAhUEa7wKHfNzBOoQFghlMAg&usg=AOvVaw2kfYHx2obM5EhxwIrSMn-4
=======================================================================================