SQLQuest

Catherine Devlin

Ohio LinuxFest 2015, Oct 3

https://github.com/catherinedevlin/sql_quest

Me

  • Database administrator since 1999
  • Python programmer since 2003
  • First chair of PyOhio
  • catherinedevlin.blogspot.com, @catherinedevlin
  • Your employee

You

  • Total SQL n00b. Please!

Adventure Synopsis

  • Basic: CSV
  • Advanced: JSON
  • Expert: Relational

In the beginning...

Into Electronica

All but useless electronically


In [37]:
!libreoffice data/aelfryth.odt

CSV


In [38]:
!libreoffice data/party.ods


In [39]:
!cat data/party.csv





Wigstan,Thief,Runaway thrall,3,14,12,9,15,14,10,cudgel,1,3,0.1,,wood,,cookpot,1,10,0.3,,iron,,dagger,1,1,2,,iron,,torches,3,2,0.02,,wood,

In [40]:
from csv import DictReader
from pprint import pprint
with open('data/party.csv') as infile:
    party = list(DictReader(infile))
    
pprint(party)


[{'charisma': '14',
  'class': 'Fighter',
  'constitution': '15',
  'dexterity': '15',
  'equip 1 cost': '7',
  'equip 1 magic': '',
  'equip 1 materials': 'wood, cloth',
  'equip 1 name': 'shield',
  'equip 1 notes': '',
  'equip 1 quantity': '1',
  'equip 1 weight each': '10',
  'equip 2 cost': '4',
  'equip 2 magic': '',
  'equip 2 materials': 'cloth',
  'equip 2 name': 'Tent',
  'equip 2 notes': 'sleeps 4',
  'equip 2 quantity': '1',
  'equip 2 weight each': '15',
  'equip 3 cost': '30',
  'equip 3 magic': '',
  'equip 3 materials': 'wood',
  'equip 3 name': 'bow',
  'equip 3 notes': '',
  'equip 3 quantity': '1',
  'equip 3 weight each': '2',
  'equip 4 cost': '0.1',
  'equip 4 magic': '',
  'equip 4 materials': 'wood, iron',
  'equip 4 name': 'arrows',
  'equip 4 notes': '',
  'equip 4 quantity': '12',
  'equip 4 weight each': '0.1',
  'intelligence': '11',
  'level': '5',
  'name': 'Aelfryth',
  'role': 'Thane',
  'strength': '16',
  'wisdom': '14'},
 {'charisma': '12',
  'class': 'Fighter',
  'constitution': '16',
  'dexterity': '11',
  'equip 1 cost': '5',
  'equip 1 magic': '+1',
  'equip 1 materials': 'wood, iron',
  'equip 1 name': 'battleaxe',
  'equip 1 notes': '',
  'equip 1 quantity': '1',
  'equip 1 weight each': '7',
  'equip 2 cost': '0.1',
  'equip 2 magic': '',
  'equip 2 materials': 'hemp',
  'equip 2 name': '30’ rope',
  'equip 2 notes': '',
  'equip 2 quantity': '1',
  'equip 2 weight each': '10',
  'equip 3 cost': '0.02',
  'equip 3 magic': '',
  'equip 3 materials': 'wood',
  'equip 3 name': 'torches',
  'equip 3 notes': '',
  'equip 3 quantity': '6',
  'equip 3 weight each': '2',
  'equip 4 cost': '',
  'equip 4 magic': '',
  'equip 4 materials': '',
  'equip 4 name': '',
  'equip 4 notes': '',
  'equip 4 quantity': '',
  'equip 4 weight each': '',
  'intelligence': '9',
  'level': '4',
  'name': 'Godric',
  'role': 'Warband member',
  'strength': '17',
  'wisdom': '7'},
 {'charisma': '11',
  'class': 'Sourceror',
  'constitution': '12',
  'dexterity': '14',
  'equip 1 cost': '50',
  'equip 1 magic': 'Y',
  'equip 1 materials': 'oil, spices',
  'equip 1 name': 'Oil of Revelation',
  'equip 1 notes': 'Burned in any lamp, its light reveals all illusions, '
                   'disguises, invisible',
  'equip 1 quantity': '1',
  'equip 1 weight each': '1',
  'equip 2 cost': '500',
  'equip 2 magic': 'Y',
  'equip 2 materials': 'stone',
  'equip 2 name': 'Unfailing Flint',
  'equip 2 notes': 'Immediately ignites regardless of moisture',
  'equip 2 quantity': '1',
  'equip 2 weight each': '0.1',
  'equip 3 cost': '0.5',
  'equip 3 magic': '',
  'equip 3 materials': 'clay',
  'equip 3 name': 'lamp',
  'equip 3 notes': '',
  'equip 3 quantity': '1',
  'equip 3 weight each': '0.2',
  'equip 4 cost': '',
  'equip 4 magic': '',
  'equip 4 materials': '',
  'equip 4 name': '',
  'equip 4 notes': '',
  'equip 4 quantity': '',
  'equip 4 weight each': '',
  'intelligence': '17',
  'level': '5',
  'name': 'Leofflaed',
  'role': 'Enigma',
  'strength': '8',
  'wisdom': '11'},
 {'charisma': '10',
  'class': 'Thief',
  'constitution': '14',
  'dexterity': '15',
  'equip 1 cost': '0.1',
  'equip 1 magic': '',
  'equip 1 materials': 'wood',
  'equip 1 name': 'cudgel',
  'equip 1 notes': '',
  'equip 1 quantity': '1',
  'equip 1 weight each': '3',
  'equip 2 cost': '0.3',
  'equip 2 magic': '',
  'equip 2 materials': 'iron',
  'equip 2 name': 'cookpot',
  'equip 2 notes': '',
  'equip 2 quantity': '1',
  'equip 2 weight each': '10',
  'equip 3 cost': '2',
  'equip 3 magic': '',
  'equip 3 materials': 'iron',
  'equip 3 name': 'dagger',
  'equip 3 notes': '',
  'equip 3 quantity': '1',
  'equip 3 weight each': '1',
  'equip 4 cost': '0.02',
  'equip 4 magic': '',
  'equip 4 materials': 'wood',
  'equip 4 name': 'torches',
  'equip 4 notes': '',
  'equip 4 quantity': '3',
  'equip 4 weight each': '2',
  'intelligence': '12',
  'level': '3',
  'name': 'Wigstan',
  'role': 'Runaway thrall',
  'strength': '14',
  'wisdom': '9'}]

Answering questions

How much weight are the party's fighters carrying?


In [41]:
total_weight = 0
for character in party:
    if character['class'] == 'Fighter':
        for eq_index in range(1, 4):
            try:
                quantity = int(character['equip %d quantity' % eq_index])
                weight_each = float(character['equip %d weight each' % eq_index])
                total_weight += quantity * weight_each
            except (TypeError, ValueError):
                pass
print(total_weight)


56.0

Save vs. Confusion

  • custom program
  • hardcoded number of equipment slots

Relational (from CSV)


In [42]:
!libreoffice data/party.multisheet.ods


In [43]:
from csv import DictReader
from pprint import pprint
with open('data/party.multisheet/stats-Table 1.csv') as infile:
    stats = list(DictReader(infile))
with open('data/party.multisheet/equipment-Table 1.csv') as infile:
    equipment = list(DictReader(infile))
pprint(stats)


[{'charisma': '14',
  'class': 'Fighter',
  'constitution': '15',
  'dexterity': '15',
  'intelligence': '11',
  'level': '5',
  'name': 'Aelfryth',
  'role': 'Thane',
  'strength': '16',
  'wisdom': '14'},
 {'charisma': '12',
  'class': 'Fighter',
  'constitution': '16',
  'dexterity': '11',
  'intelligence': '9',
  'level': '4',
  'name': 'Godric',
  'role': 'Warband Member',
  'strength': '17',
  'wisdom': '7'},
 {'charisma': '11',
  'class': 'Sourceror',
  'constitution': '12',
  'dexterity': '14',
  'intelligence': '17',
  'level': '5',
  'name': 'Leofflaed',
  'role': 'Enigma',
  'strength': '8',
  'wisdom': '11'},
 {'charisma': '10',
  'class': 'Thief',
  'constitution': '14',
  'dexterity': '15',
  'intelligence': '12',
  'level': '3',
  'name': 'Wigstan',
  'role': 'Runaway Thrall',
  'strength': '14',
  'wisdom': '9'}]

In [44]:
total_weight = 0
for character in stats:
    if character['class'] == 'Fighter':
        for itm in equipment:
            if itm['owner'] == character['name']:
                try:
                    quantity = int(itm['quantity'])
                    weight_each = float(itm['weight each'])
                    total_weight += quantity * weight_each
                except (TypeError, ValueError):
                    pass
print(total_weight)


57.2

Still ouch

JSON

arbitrary structure


In [45]:
!cat data/party.json


[
  {
    "class": "Thief",
    "role": "Runaway Thrall",
    "level": 3,
    "equipment": {
      "torches": {
        "quantity": 3,
        "materials": [
          "wood"
        ],
        "weight each": 2.0,
        "magic": "",
        "notes": "",
        "cost": 2.0
      },
      "cudgel": {
        "quantity": 1,
        "materials": [
          "wood"
        ],
        "weight each": 3.0,
        "magic": "",
        "notes": "",
        "cost": 3.0,
        "damage": "d6"
      },
      "cookpot": {
        "quantity": 1,
        "materials": [
          "iron"
        ],
        "weight each": 10.0,
        "magic": "",
        "notes": "",
        "cost": 10.0
      },
      "dagger": {
        "quantity": 1,
        "materials": [
          "iron"
        ],
        "weight each": 1.0,
        "magic": "",
        "notes": "",
        "cost": 1.0,
        "damage": "d4"
      }
    },
    "name": "Wigstan",
    "stats": {
      "dexterity": 15,
      "charisma": 10,
      "strength": 14,
      "wisdom": 9,
      "intelligence": 12,
      "constitution": 14
    }
  },
  {
    "class": "Fighter",
    "role": "Thane",
    "level": 5,
    "equipment": {
      "bow": {
        "quantity": 1,
        "materials": [
          "wood"
        ],
        "weight each": 2.0,
        "magic": "",
        "notes": "",
        "cost": 2.0,
        "damage": "d6"
      },
      "Tent": {
        "quantity": 1,
        "materials": [
          "cloth"
        ],
        "weight each": 15.0,
        "magic": "",
        "notes": "sleeps 4",
        "cost": 15.0
      },
      "shield": {
        "quantity": 1,
        "materials": [
          "wood",
          "cloth"
        ],
        "weight each": 10.0,
        "magic": "",
        "notes": "",
        "cost": 10.0
      },
      "arrows": {
        "quantity": 12,
        "materials": [
          "wood",
          "iron"
        ],
        "weight each": 0.1,
        "magic": "",
        "notes": "",
        "cost": 0.1
      }
    },
    "name": "Aelfryth",
    "stats": {
      "dexterity": 15,
      "charisma": 14,
      "strength": 16,
      "wisdom": 14,
      "intelligence": 11,
      "constitution": 15
    }
  },
  {
    "class": "Sourceror",
    "role": "Enigma",
    "level": 5,
    "equipment": {
      "Unfailing Flint": {
        "quantity": 1,
        "materials": [
          "stone"
        ],
        "weight each": 0.1,
        "magic": "Y",
        "notes": "Immediately ignites in any weather",
        "cost": 0.1
      },
      "lamp": {
        "quantity": 1,
        "materials": [
          "clay"
        ],
        "weight each": 0.2,
        "magic": "",
        "notes": "",
        "cost": 0.2
      },
      "Oil of Revelation": {
        "quantity": 1,
        "materials": [
          "oil",
          "spices"
        ],
        "weight each": 1.0,
        "magic": "Y",
        "notes": "Burned in any lamp, its light reveals all illusions, disguises, invisible",
        "cost": 1.0
      }
    },
    "name": "Leofflaed",
    "stats": {
      "dexterity": 14,
      "charisma": 11,
      "strength": 8,
      "wisdom": 11,
      "intelligence": 17,
      "constitution": 12
    }
  },
  {
    "class": "Fighter",
    "role": "Warband Member",
    "level": 4,
    "equipment": {
      "torches": {
        "quantity": 6,
        "materials": [
          "wood"
        ],
        "weight each": 2.0,
        "magic": "",
        "notes": "",
        "cost": 2.0
      },
      "30\u2019 rope": {
        "quantity": 1,
        "materials": [
          "hemp"
        ],
        "weight each": 10.0,
        "magic": "",
        "notes": "",
        "cost": 10.0
      },
      "battleaxe": {
        "quantity": 1,
        "materials": [
          "wood",
          "iron"
        ],
        "weight each": 7.0,
        "magic": "+1",
        "notes": "",
        "cost": 7.0,
        "damage": "d8"
      }
    },
    "name": "Godric",
    "stats": {
      "dexterity": 11,
      "charisma": 12,
      "strength": 17,
      "wisdom": 7,
      "intelligence": 9,
      "constitution": 16
    }
  }
]

How much does all the fighters' equipment weigh?


In [46]:
import json
import glob
total_weight = 0
with open('data/party.json') as infile:
    for character in json.load(infile):
        if character['class'] == 'Fighter':
            for itm in character['equipment'].values():
                total_weight += (itm['quantity'] * itm['weight each'])
print(total_weight)


57.2

Database management system

  • Multiple access
  • Performance
  • > memory
  • > 1 machine

Document databases

  • Mongo
  • RethinkDB
  • PostgreSQL

In [48]:
!echo "db.party.drop()" | mongo


MongoDB shell version: 3.0.6
connecting to: test
true
bye

In [49]:
!mongoimport --collection party --jsonArray data/party.json


2015-10-02T21:20:13.954-0400	connected to: localhost
2015-10-02T21:20:13.957-0400	imported 4 documents

In [50]:
!echo "db.party.find()" | mongo


MongoDB shell version: 3.0.6
connecting to: test
{ "_id" : ObjectId("560f2d4d44fdf959bdca0f1a"), "class" : "Sourceror", "role" : "Enigma", "level" : 5, "equipment" : { "Unfailing Flint" : { "quantity" : 1, "materials" : [ "stone" ], "weight each" : 0.1, "magic" : "Y", "notes" : "Immediately ignites in any weather", "cost" : 0.1 }, "lamp" : { "quantity" : 1, "materials" : [ "clay" ], "weight each" : 0.2, "magic" : "", "notes" : "", "cost" : 0.2 }, "Oil of Revelation" : { "notes" : "Burned in any lamp, its light reveals all illusions, disguises, invisible", "cost" : 1, "quantity" : 1, "materials" : [ "oil", "spices" ], "weight each" : 1, "magic" : "Y" } }, "name" : "Leofflaed", "stats" : { "dexterity" : 14, "charisma" : 11, "strength" : 8, "wisdom" : 11, "intelligence" : 17, "constitution" : 12 } }
{ "_id" : ObjectId("560f2d4d44fdf959bdca0f1b"), "class" : "Fighter", "role" : "Thane", "level" : 5, "equipment" : { "arrows" : { "quantity" : 12, "materials" : [ "wood", "iron" ], "weight each" : 0.1, "magic" : "", "notes" : "", "cost" : 0.1 }, "bow" : { "magic" : "", "notes" : "", "cost" : 2, "damage" : "d6", "quantity" : 1, "materials" : [ "wood" ], "weight each" : 2 }, "Tent" : { "quantity" : 1, "materials" : [ "cloth" ], "weight each" : 15, "magic" : "", "notes" : "sleeps 4", "cost" : 15 }, "shield" : { "magic" : "", "notes" : "", "cost" : 10, "quantity" : 1, "materials" : [ "wood", "cloth" ], "weight each" : 10 } }, "name" : "Aelfryth", "stats" : { "charisma" : 14, "strength" : 16, "wisdom" : 14, "intelligence" : 11, "constitution" : 15, "dexterity" : 15 } }
{ "_id" : ObjectId("560f2d4d44fdf959bdca0f1c"), "class" : "Thief", "role" : "Runaway Thrall", "level" : 3, "equipment" : { "dagger" : { "damage" : "d4", "quantity" : 1, "materials" : [ "iron" ], "weight each" : 1, "magic" : "", "notes" : "", "cost" : 1 }, "torches" : { "notes" : "", "cost" : 2, "quantity" : 3, "materials" : [ "wood" ], "weight each" : 2, "magic" : "" }, "cudgel" : { "quantity" : 1, "materials" : [ "wood" ], "weight each" : 3, "magic" : "", "notes" : "", "cost" : 3, "damage" : "d6" }, "cookpot" : { "materials" : [ "iron" ], "weight each" : 10, "magic" : "", "notes" : "", "cost" : 10, "quantity" : 1 } }, "name" : "Wigstan", "stats" : { "strength" : 14, "wisdom" : 9, "intelligence" : 12, "constitution" : 14, "dexterity" : 15, "charisma" : 10 } }
{ "_id" : ObjectId("560f2d4d44fdf959bdca0f1d"), "class" : "Fighter", "role" : "Warband Member", "level" : 4, "equipment" : { "torches" : { "materials" : [ "wood" ], "weight each" : 2, "magic" : "", "notes" : "", "cost" : 2, "quantity" : 6 }, "30’ rope" : { "weight each" : 10, "magic" : "", "notes" : "", "cost" : 10, "quantity" : 1, "materials" : [ "hemp" ] }, "battleaxe" : { "magic" : "+1", "notes" : "", "cost" : 7, "damage" : "d8", "quantity" : 1, "materials" : [ "wood", "iron" ], "weight each" : 7 } }, "name" : "Godric", "stats" : { "dexterity" : 11, "charisma" : 12, "strength" : 17, "wisdom" : 7, "intelligence" : 9, "constitution" : 16 } }
bye

How much weight are the party's fighters carrying?


In [51]:
!echo "db.party.find({class: 'Fighter'}, {'equipment': 1, _id: 0})" | mongo


MongoDB shell version: 3.0.6
connecting to: test
{ "equipment" : { "arrows" : { "quantity" : 12, "materials" : [ "wood", "iron" ], "weight each" : 0.1, "magic" : "", "notes" : "", "cost" : 0.1 }, "bow" : { "magic" : "", "notes" : "", "cost" : 2, "damage" : "d6", "quantity" : 1, "materials" : [ "wood" ], "weight each" : 2 }, "Tent" : { "quantity" : 1, "materials" : [ "cloth" ], "weight each" : 15, "magic" : "", "notes" : "sleeps 4", "cost" : 15 }, "shield" : { "magic" : "", "notes" : "", "cost" : 10, "quantity" : 1, "materials" : [ "wood", "cloth" ], "weight each" : 10 } } }
{ "equipment" : { "torches" : { "materials" : [ "wood" ], "weight each" : 2, "magic" : "", "notes" : "", "cost" : 2, "quantity" : 6 }, "30’ rope" : { "weight each" : 10, "magic" : "", "notes" : "", "cost" : 10, "quantity" : 1, "materials" : [ "hemp" ] }, "battleaxe" : { "magic" : "+1", "notes" : "", "cost" : 7, "damage" : "d8", "quantity" : 1, "materials" : [ "wood", "iron" ], "weight each" : 7 } } }
bye

In [52]:
from pymongo import MongoClient
client = MongoClient()
total_weight = 0.0
for character in client.test.party.find({'class':'Fighter'}):
    for itm in character['equipment'].values():
        total_weight += itm['weight each'] * itm['quantity']
print(total_weight)


57.2

Consistency


In [53]:
party.append( {'name': 'Mickey',
               'class': 'Druid/Gunslinger/Paladin/Illusionist/Assassin',
               'level': 87,
               'psionic level': 19} )

JSON schema enforcement

kwalify, marshmallow, ...

RDBMS

Relational DataBase Management System

  • PostgreSQL
  • SQLite
  • MySQL / MariaDB

RDBMS provides

  • scale
  • data consistency
  • query language: SQL
  • tools

DDL

Data Definition Language

Create a database and connect to it

$ psql template1 dungeonmaster
Password for user dungeonmaster: 
psql (9.3.6)
Type "help" for help.

template1=# create database dnd;
CREATE DATABASE
template1=# \c dnd
You are now connected to database "dnd" as user "dungeonmaster".
dnd=# 

In [54]:
!cat sql/ddl/create_char_tbl.sql


  CREATE TABLE character (
    name TEXT,
    class TEXT,
    role TEXT,
    level INTEGER,
    strength INTEGER,
    intelligence INTEGER,
    wisdom INTEGER,
    dexterity INTEGER,
    constitution INTEGER,
    charisma INTEGER );
dnd=# \i sql/ddl/create_char_tbl.sql 
CREATE TABLE

DML

Data Manipulation Language


In [55]:
!cat sql/dml/ins_single_char.sql


INSERT INTO character (
  name, class, role, level,
  strength, intelligence, wisdom, dexterity,
  constitution, charisma)
VALUES (
  'Aelfryth', 'Fighter', 'Thane', 5,
  16, 11, 14, 15,
  15, 14);
dnd=# \i sql/dml/ins_single_char.sql
INSERT 0 1

SELECT

dnd=# SELECT * FROM character;
   name   |  class  | role  | level | strength | intelligence | wisdom | dexterity | constitution | charisma 
----------+---------+-------+-------+----------+--------------+--------+-----------+--------------+----------
 Aelfryth | Fighter | Thane |     5 |       16 |           11 |     14 |        15 |           15 |       14
(1 row)

Mass insert

dnd=# DELETE FROM character;
DELETE 4
dnd=# \copy character FROM 'data/party.multisheet/stats-Table 1.csv' WITH csv HEADER;
COPY 4

Selective SELECT

dnd=# SELECT name, class, role FROM character;
   name    |   class   |      role      
-----------+-----------+----------------
 Aelfryth  | Fighter   | Thane
 Godric    | Fighter   | Warband Member
 Leofflaed | Sourceror | Enigma
 Wigstan   | Thief     | Runaway Thrall
(4 rows)

WHERE

dnd=# SELECT name, class, role 
dnd-# FROM   character
dnd-# WHERE  class = 'Fighter';
   name   |  class  |      role      
----------+---------+----------------
 Aelfryth | Fighter | Thane
 Godric   | Fighter | Warband Member
(2 rows)

AND

dnd=# SELECT name, class, role, intelligence
dnd-# FROM   character
dnd-# WHERE  class = 'Fighter'
dnd-# AND    intelligence > 10;
   name   |  class  | role  | intelligence 
----------+---------+-------+--------------
 Aelfryth | Fighter | Thane |           11
(1 row)

Level up

dnd=# SELECT name, level FROM character;
   name    | level 
-----------+-------
 Aelfryth  |     5
 Godric    |     4
 Leofflaed |     5
 Wigstan   |     3
(4 rows)

UPDATE

dnd=# UPDATE character
dnd-# SET    level = 4
dnd-# WHERE  name = 'Wigstan';
UPDATE 1

dnd=# SELECT name, level FROM character;                                                                               
 name      | level 
-----------+-------
 Aelfryth  |     5
 Godric    |     4
 Leofflaed |     5
 Wigstan   |     4
(4 rows)

Unique identifiers

dnd=# INSERT INTO character
dnd-# SELECT * FROM character
dnd-# WHERE name = 'Wigstan';
INSERT 0 1
dnd=# SELECT name, class, level FROM character;
   name    |   class   | level 
-----------+-----------+-------
 Aelfryth  | Fighter   |     5
 Godric    | Fighter   |     4
 Leofflaed | Sourceror |     5
 Wigstan   | Thief     |     4
 Wigstan   | Thief     |     4
(5 rows)

Primary Key

numeric

dnd=# ALTER TABLE character ADD
dnd-#   id SERIAL PRIMARY KEY;
ALTER TABLE
dnd=# SELECT id, name, class FROM character;
 id |   name    |   class   
----+-----------+-----------
  1 | Aelfryth  | Fighter
  2 | Godric    | Fighter
  3 | Leofflaed | Sourceror
  4 | Wigstan   | Thief
  5 | Wigstan   | Thief
(5 rows)

dnd=# UPDATE character 
dnd-# SET    name = 'Wigmund'
dnd-# WHERE  id = 5;
UPDATE 1
dnd=# ALTER TABLE character DROP id;
ALTER TABLE

Text PK

dnd=# ALTER TABLE character
dnd-# ADD PRIMARY KEY (name);
ALTER TABLE

In [58]:
!cat sql/dml/godric2.sql


INSERT INTO character (
  name, class, role, level,
  strength, intelligence, wisdom, dexterity,
  constitution, charisma)
VALUES (
  'Godric', 'Ranger', 'Guide', 4,
  14, 15, 13, 15,
  16, 7);
dnd=# \i sql/dml/godric2.sql 
psql:sql/dml/godric2.sql:8: ERROR:  duplicate key value violates unique constraint "character_pkey"
DETAIL:  Key (name)=(Godric) already exists.

Relations


In [59]:
!cat sql/ddl/create_equip_tbl.sql


  CREATE TABLE equipment (
    name TEXT,
    quantity INTEGER,
    weight_each NUMERIC(4,1),
    cost NUMERIC(6,2),
    magic TEXT,
    materials TEXT,
    notes TEXT,
    owner TEXT REFERENCES character (name)
  );
dnd=# \i sql/ddl/create_equip_tbl.sql 
CREATE TABLE
dnd=# \copy equipment FROM 'data/party.multisheet/equipment-Table 1.csv' WITH csv HEADER
COPY 14

Join

All Leofflaed's gear

dnd=# SELECT c.name, e.name, e.magic
FROM   character c
JOIN   equipment e ON (e.owner = c.name)
WHERE  c.name = 'Leofflaed';

   name    |       name        | magic 
-----------+-------------------+-------
 Leofflaed | Oil of Revelation | Y
 Leofflaed | Unfailing Flint   | Y
 Leofflaed | lamp              | 

Weight of all fighters' gear

dnd=# SELECT SUM(quantity * weight_each)
dnd-# FROM   equipment e
dnd-# JOIN   character c ON (e.owner = c.name)
dnd-# WHERE  class = 'Fighter';
 sum  
------
 57.2
(1 row)

Constraints


In [60]:
!cat sql/dml/mickey.sql


INSERT INTO character (
  name, class, role, level,
  psionic_level,
  strength, intelligence, wisdom, dexterity,
  constitution, charisma)
VALUES (
  'Mickey', 'Druid/Gunslinger/Paladin/Illusionist/Assassin', 'Demigod', 87,
  19,
  19, 19, 18, 18,
  19, 17);
dnd# \i sql/dml/mickey.sql
ERROR:  column "psionic_level" of relation "character" does not exist
LINE 3:   psionic_level,

Column constraints

  CREATE TABLE character (
    name TEXT,
    class VARCHAR(14),
    ...

dnd=# ALTER TABLE character ALTER COLUMN class TYPE VARCHAR(14);

INSERT INTO character (
...

ERROR:  value too long for type character varying(14)

Foreign key constraints

Check constraints

  CREATE TABLE character (
    ...
    level INTEGER,
    ...

  dnd=# ALTER TABLE character ADD CONSTRAINT reaonable_level CHECK (level < 15);

ERROR:  new row for relation "character" violates check constraint "character_level_check"
DETAIL:  Failing row contains (Mickey, Druid, Demigod, 87, 19, 19, 18, 18, 19, 17).        

Transactions


In [61]:
!cat sql/dml/thief_inventories.sql


SELECT name, owner
FROM   equipment
WHERE  owner LIKE 'Wig%';
dnd=# \i sql/dml/thief_inventories.sql
  name   |  owner  
---------+---------
 cudgel  | Wigstan
 cookpot | Wigstan
 dagger  | Wigstan
 torches | Wigstan
(4 rows)
dnd=# UPDATE equipment
dnd-# SET    owner = 'Wigmund'
dnd-# WHERE  owner = 'Wigstan'
dnd-# AND    name = 'cudgel';
UPDATE 1
dnd=# \i sql/dml/thief_inventories.sql 
  name   |  owner  
---------+---------
 cookpot | Wigstan
 dagger  | Wigstan
 torches | Wigstan
 cudgel  | Wigmund
(4 rows)
dnd=# BEGIN TRANSACTION;
BEGIN
dnd=# UPDATE equipment
dnd-# SET    owner = 'Wigmund'
dnd-# WHERE  owner = 'Wigstan'
dnd-# AND    name = 'dagger';
UPDATE 1
dnd=# SELECT name, owner
dnd-# FROM   equipment
dnd-# WHERE  owner LIKE 'Wig%';
  name   |  owner  
---------+---------
 cookpot | Wigstan
 torches | Wigstan
 cudgel  | Wigmund
 dagger  | Wigmund
(4 rows)
dnd=# ROLLBACK;
ROLLBACK
dnd=# SELECT name, owner
dnd-# FROM   equipment
dnd-# WHERE  owner LIKE 'Wig%';
  name   |  owner  
---------+---------
 cookpot | Wigstan
 torches | Wigstan
 dagger  | Wigstan
 cudgel  | Wigmund
(4 rows)

opposite is COMMIT

dnd=# update equipment set owner = 'Wigstan' WHERE owner = 'Wigmund';
UPDATE 0
dnd=# 

Aggregate functions

dnd=# SELECT MAX(charisma) FROM character;
 max 
-----
  14
(1 row)

Think line count

dnd=# SELECT name, MAX(charisma) FROM character;
ERROR:  column "character.name" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT name, MAX(charisma) FROM character;

Subquery

dnd=# SELECT name, charisma
dnd-# FROM   character
dnd-# WHERE  charisma = (SELECT MAX(charisma) FROM character);
   name   | charisma 
----------+----------
 Aelfryth |       14
(1 row)

GROUP BY

dnd=# SELECT owner, SUM(cost) 
dnd-# FROM   equipment
dnd-# GROUP BY owner
dnd-# ORDER BY SUM(cost) DESC;
   owner   |  sum   
-----------+--------
 Leofflaed | 550.50
 Aelfryth  |  41.10
 Godric    |   5.12
 Wigstan   |   2.42
(4 rows)

Outer joins

dnd=# SELECT c.name, c.class, SUM(e.cost)
dnd-# FROM   character c
dnd-# JOIN   equipment e ON (e.owner = c.name)
dnd-# GROUP BY c.name, c.class
dnd-# ORDER BY SUM(e.cost) DESC;
   name    |   class   |  sum   
-----------+-----------+--------
 Leofflaed | Sourceror | 550.50
 Aelfryth  | Fighter   |  41.10
 Godric    | Fighter   |   5.12
 Wigstan   | Thief     |   2.42
(4 rows)
dnd=# SELECT c.name, c.class, SUM(e.cost)
FROM   character c
LEFT OUTER JOIN equipment e ON (e.owner = c.name)
GROUP BY c.name, c.class
ORDER BY SUM(e.cost) DESC;
   name    |   class   |  sum   
-----------+-----------+--------
 Wigmund   | Thief     |       
 Leofflaed | Sourceror | 550.50
 Aelfryth  | Fighter   |  41.10
 Godric    | Fighter   |   5.12
 Wigstan   | Thief     |   2.42
(5 rows)

... optionally, NULLS LAST

Traps

The NULL trap (3-value logic)

dnd=# SELECT name, magic 
dnd-# FROM   equipment
dnd-# WHERE  magic IS NOT NULL;
       name        | magic 
-------------------+-------
 battleaxe         | +1
 Oil of Revelation | Y
 Unfailing Flint   | Y
(3 rows)

dnd=# SELECT name, magic
dnd-# FROM   equipment
dnd-# WHERE  magic = NULL;
 name | magic 
------+-------
(0 rows)
dnd=# SELECT 1 = 1;
 ?column? 
----------
 t
(1 row)

dnd=# SELECT 1 = 2;
 ?column? 
----------
 f
(1 row)

dnd=# SELECT NULL = NULL;
 ?column? 
----------

(1 row)

dnd=# SELECT NULL != NULL;
 ?column? 
----------

(1 row)
dnd=# SELECT c.name
dnd-# FROM   character c
dnd-# JOIN   equipment e ON (e.owner = c.name)
dnd-# AND    e.magic != NULL;
 name 
------
(0 rows)

dnd=# SELECT c.name AS owner, e.name                                                           
dnd=# FROM   character c                                                                       
dnd=# JOIN   equipment e ON (c.name = e.owner)                                                 
dnd=# WHERE  magic IS NULL;
   owner   |   name   
-----------+----------
 Aelfryth  | shield
 Aelfryth  | Tent
 Aelfryth  | bow
 Aelfryth  | arrows
 Godric    | 30’ rope
 Godric    | torches
 Leofflaed | lamp
 Wigstan   | cookpot
 Wigstan   | torches
 Wigstan   | dagger
 Wigmund   | cudgel
(11 rows)

Missing WHERE trap

dnd=# SELECT name, level
dnd-# FROM   character;
   name    | level 
-----------+-------
 Aelfryth  |     5
 Godric    |     4
 Leofflaed |     5
 Wigstan   |     5
 Wigmund   |     5
(5 rows)

dnd=# UPDATE character
dnd-# SET    level = 6;
UPDATE 5
dnd=# SELECT name, level
dnd-# FROM   character;
   name    | level 
-----------+-------
 Aelfryth  |     6
 Godric    |     6
 Leofflaed |     6
 Wigstan   |     6
 Wigmund   |     6
(5 rows)

Unqualified DELETE = apocalypse

dnd=# BEGIN TRANSACTION;
BEGIN
dnd=# DELETE FROM equipment;
DELETE 14
dnd=# SELECT * FROM equipment;
 name | quantity | weight_each | cost | magic | materials | notes | owner | damage 
------+----------+-------------+------+-------+-----------+-------+-------+--------
(0 rows)

dnd=# ROLLBACK;
ROLLBACK

Triggers

Programming


In [63]:
import psycopg2
import psycopg2.extras
conn = psycopg2.connect('dbname=dnd user=dungeonmaster password=gygax')
curs = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
curs.execute("SELECT * FROM character")
for character in curs.fetchall():
    print(character)
    if character['class'] == 'Fighter':
        print('{name}, get in front!'.format(**character))
conn.close()


['Aelfryth', 'Fighter', 'Thane', 5, 16, 11, 14, 15, 15, 14]
Aelfryth, get in front!
['Godric', 'Fighter', 'Warband Member', 4, 17, 9, 7, 11, 16, 12]
Godric, get in front!
['Leofflaed', 'Sourceror', 'Enigma', 5, 8, 17, 11, 14, 12, 11]
['Wigstan', 'Thief', 'Runaway Thrall', 4, 14, 12, 9, 15, 14, 10]
['Wigmund', 'Thief', 'Runaway Thrall', 4, 14, 12, 9, 15, 14, 10]

Take 500 XP

Questions?

catherinedevlin.blogspot.com github.com/catherinedevlin/sql_quest

Images (except book covers) from British Library's public release


In [ ]: