Weather news robot

A simple and stupid news robot written in Python that scrapes tomorrows weather and writes a short text complaining about how cold it is.

1. Import libraries


In [ ]:
# Import all the things!
import urllib.request
from datetime import *
from lxml import html
from bs4 import BeautifulSoup

2. Define scraping functions


In [ ]:
# Scrape all HTML from webpage.
def scrapewebpage(url):
	# Open URL and get HTML.
	web = urllib.request.urlopen(url)

	# Make sure there wasn't any errors opening the URL.
	if (web.getcode() == 200):
		html = web.read()
		return(html)
	else:
		print("Error %s reading %s" % str(web.getcode()), url)

# Helper function that scrape the webpage and turn it into soup.
def makesoup(url):
	html = scrapewebpage(url)
	return(BeautifulSoup(html, "lxml"))

3. Scrape weather

We'll scrape tomorrows weather from http://www.weatherpal.se/vader/Goteborg-2711537/


In [ ]:
weather_soup = makesoup("http://www.weatherpal.se/vader/Goteborg-2711537/")

In [ ]:
# Get minimium and maximum temperatures for the next 6 days.
soup_min_temps = weather_soup.find_all("div", "min-temp")
soup_max_temps = weather_soup.find_all("div", "max-temp")

# Create empty lists to store temperatures.
min_temps = []
max_temps = []

# For each minimum temperature, put it in a list.
for temp in soup_min_temps:
    min_temps.append(temp.get_text())

# For each maxiumum temperature, put it in a list.
for temp in soup_max_temps:
    max_temps.append(temp.get_text())

In [ ]:
# Lets check tomorrows weather.
print("Min celsius: " + min_temps[0])
print("Max celsius: " + max_temps[0])

In [ ]:
# What date is it tomorrow?
tomorrow = datetime.today() + timedelta(days=1)

# Name of weekdays.
weekdays = ["Monday", "Tuesday", "Wednesday", "Thursday", "Saturday", "Sunday"]

In [ ]:
# Use simpler names.
tomorrow_weekday = weekdays[tomorrow.weekday()]
tomorrow_mintemp = int(min_temps[0])
tomorrow_maxtemp = int(max_temps[0])

4. Check for errors

Don't assume that the data is always good. Create tests to make sure.


In [ ]:
# We start with the assumption that the data is news worthy.
IsNewsWorthy = True

# Are there errors in the data? 
if tomorrow_mintemp < -40:
    IsNewsWorthy = False
elif tomorrow_maxtemp > 60:
    IsNewsWorthy = False

In [ ]:
IsNewsWorthy

5. Create text

Basically, it's just a lot of if-statements.

How to make the code easier to read:

  1. Don't nest if-statements inside each other too much. Use elif instead.
  2. Use multiline strings, with """ around them.
  3. Use .format on strings.
text = "My name is {0} and I am {1} years old"
text = text.format(Name, Age)

Which is the same as this:

text = "My name is " + Name + " and I am " + str(Age) + " years old"

In [ ]:
text = ""
if IsNewsWorthy:
    if (tomorrow_mintemp < 0):
        text = "At {0} it will be freezing. The temperature will be {1} celsius, {2} at max."
        text = text.format(tomorrow_weekday, tomorrow_mintemp, tomorrow_maxtemp)
    elif (tomorrow_mintemp == 0):
        text = "At {0} it will be just above freezing point. The temperature will be between {1} and {2} celsius."
        text = text.format(tomorrow_weekday, tomorrow_mintemp, tomorrow_maxtemp)
    elif (tomorrow_mintemp > 0 and tomorrow_mintemp < 7):
        text = "Tomorrow, {0}, we will not freeze our socks off. Fortunately, the temperature will go down to {1} celsius but hopefully rise up to the maximum of {2}."
        text = text.format(tomorrow_weekday, tomorrow_mintemp, tomorrow_maxtemp)
    elif (tomorrow_mintemp > 7):
        text = "Tomorrow, {0}, it will be quite warm. Fortunately, the temperature will be {1} celsius at minimum, {2} maximum."
        text = text.format(tomorrow_weekday, tomorrow_mintemp, tomorrow_maxtemp)
        
    # Add this at the end of all texts.
    text = text + "\n\n" + "Published " + datetime.now().strftime("%Y-%m-%d %H:%M") + " by Ada the news robot"

In [ ]:
# Look at the text
print(text)

6. Save the results to a text file

Lets create a function that saves the text to a file.


In [ ]:
# Function to save the text to a file.
def savefile(filename, text):
    f = open(filename, mode="w")   # Open file for writing (w = writing, a = append, r = reading)
    f.write(text)
    f.close()

In [ ]:
# Only save as text file if there is some text.
if text != "":
    savefile("newsrobot-weather.txt", text)
    print("Text published!")
else:
    print("Text is not published.")

7. Present the results (read from text file)

Lets create a function that reads the text from the text file.


In [ ]:
# Function that reads text from a file.
def readfile(filename):
    f = open(filename, mode="r")   # Open file for reading (w = writing, a = append, r = reading)
    lines = f.read()
    f.close()
    return(lines)

In [ ]:
# Read the file created earlier by the news robot.
text = readfile("newsrobot-weather.txt")

In [ ]:
# Look at the text from the file.
print(text)

Assignment

Work in pairs, or alone if you prefer.

  1. Find some web source you find interesting.
  2. Extract at least 1 piece of data. It has to be able to vary, it cannot be a constant.
  3. Decide how to write the text based on that piece of data.
  4. Save the data to a text file. Don't forget to put your name in the text file!

Note, the quality of the text is not important for this assignment.

You can use all the code we worked with. You just need to modify it!

Deadline: See the study guide.

Format: Send in the Jupyter Notebook, or send in a Python file (.py).