notebook.community

Edit and run

Notebook for generating class materials, checking links, and pushing materials into GitHub





In [1]:



    


# Load needed modules
from bs4 import BeautifulSoup
import requests
import sys
import warnings
warnings.filterwarnings("ignore")



    











In [2]:



    


# Set variables
checkFiles = [
    "OILS515_syllabus.html",
    "goalsAndObjectives/goalsAndObjectives.html"
]

Generate syllabus





In [3]:



    


%%bash
/usr/local/bin/pandoc --standalone --toc --latex-engine=pdflatex  -V geometry:margin=1in -V fontsize:11pt -o OILS515_syllabus.pdf OILS515_syllabus.md
/usr/local/bin/pandoc --toc -s --standalone --css=page.css -o OILS515_syllabus.html OILS515_syllabus.md
/usr/local/bin/pandoc --toc -s --standalone --css=page.css -o index.html OILS515_syllabus.md
/usr/local/bin/pandoc -s -o OILS515_syllabus.epub OILS515_syllabus.md

Generate Goals and Objectives





In [7]:



    


%%bash
cd goalsAndObjectives
/usr/local/bin/pandoc --toc -c page.css --standalone -o goalsAndObjectives.html goalsAndObjectives.md
/usr/local/bin/pandoc --standalone --toc --latex-engine=pdflatex  -V geometry:margin=1in -V fontsize:11pt  -o goalsAndObjectives.pdf goalsAndObjectives.md



    











In [8]:



    


goodLinks = []
badLinks = []
for file in checkFiles:
    print("\nProcessing: "+file)
    with open(file) as activeFile:
        soup = BeautifulSoup(activeFile, "lxml")
        for link in soup.find_all('a'):
            myLink = link.get('href')
            if myLink[0] == "#":
                print('0', end='')
            else:
                r = requests.get(myLink, verify=False)
                # print("\t" + str(r.status_code))
                if r.status_code == requests.codes.ok:
                    print('+', end='')
                    goodLinks.append([file,myLink,r.status_code])
                else:
                    print('-', end='')
                    badLinks.append([file,myLink,r.status_code])
        print("\n")

print("\n\n--- Good Links ---")                    
for link in goodLinks:
    print(link)

print("\n\n--- Bad Links ---")                    
for link in badLinks:
    print(link)



    


















    






Processing: OILS515_syllabus.html
00000000000++++++++++++++


Processing: goalsAndObjectives/goalsAndObjectives.html
000000000000000000000000000000000000000000000000000000000000000000000000000000000000++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++



--- Good Links ---
['OILS515_syllabus.html', 'http://karlbenedict.com/OILS515/goalsAndObjectives/goalsAndObjectives.html', 200]
['OILS515_syllabus.html', 'http://karlbenedict.com/OILS515/goalsAndObjectives/goalsAndObjectives.pdf', 200]
['OILS515_syllabus.html', 'http://unm.worldcat.org/title/spatial-data-management/oclc/767844616&referer=brief_results', 200]
['OILS515_syllabus.html', 'https://www.amazon.com/Database-Design-Mere-Mortals-Hands/dp/0321884493/ref=mt_paperback?_encoding=UTF8&me=', 200]
['OILS515_syllabus.html', 'http://karlbenedict.com/OILS515/goalsAndObjectives/goalsAndObjectives.html', 200]
['OILS515_syllabus.html', 'http://karlbenedict.com/OILS515/goalsAndObjectives/goalsAndObjectives.pdf', 200]
['OILS515_syllabus.html', 'http://pathfinder.unm.edu/', 200]
['OILS515_syllabus.html', 'http://pathfinder.unm.edu/campus-policies/academic-dishonesty.html', 200]
['OILS515_syllabus.html', 'http://www.qgis.org/', 200]
['OILS515_syllabus.html', 'http://www.kyngchaos.com/software/frameworks', 200]
['OILS515_syllabus.html', 'http://www.gaia-gis.it/gaia-sins/', 200]
['OILS515_syllabus.html', 'https://policy.unm.edu/university-policies/2000/2740.html', 200]
['OILS515_syllabus.html', 'http://creativecommons.org/licenses/by-sa/4.0/', 200]
['OILS515_syllabus.html', 'http://creativecommons.org/licenses/by-sa/4.0/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0000308', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=74275116&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.sciencedirect.com.libproxy.unm.edu/science/article/pii/S1574954115001004', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=e000xna&AN=350538&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00503a&AN=unm.b7920432&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00503a&AN=unm.b7199537&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=e000xna&AN=225164&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.icsm.gov.au/mapping/about_projections.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://docs.qgis.org/2.2/en/docs/gentle_gis_introduction/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://docs.qgis.org/2.2/en/docs/user_manual/index.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.books24x7.com.libproxy.unm.edu/marc.asp?bookid=37960', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://ipython.org/notebook.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://github.com/jrjohansson', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://github.com/karlbenedict/scientific-python-lectures/blob/master/Lecture-0-Scientific-Computing-with-Python.ipynb', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://github.com/karlbenedict/scientific-python-lectures/blob/master/Lecture-1-Introduction-to-Python-Programming.ipynb', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.books24x7.com.libproxy.unm.edu/marc.asp?bookid=37960', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=e000xna&AN=350538&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00503a&AN=unm.b7920432&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://www.lynda.com/SQL-tutorials/SQL-Essential-Training/139988-2.html?org=unm.edu', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gaia-gis.it/gaia-sins/spatialite-cookbook/index.html#toc', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=e000xna&AN=350538&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00503a&AN=unm.b7920432&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00503a&AN=unm.b7199537&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=e000xna&AN=474337&site=eds-live&scope=site&ebv=EB&ppid=pp_361', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://docs.qgis.org/2.2/en/docs/user_manual/working_with_raster/index.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gdal.org/gdal_utilities.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gdal.org/gdalinfo.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gdal.org/gdal_translate.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gdal.org/gdalwarp.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gdal.org/ogrinfo.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.gdal.org/ogr2ogr.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=e000xna&AN=350538&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://libproxy.unm.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=cat00503a&AN=unm.b7920432&site=eds-live&scope=site', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.digitalpreservation.gov/formats/content/gis.shtml', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.digitalpreservation.gov/partners/ngda.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.ngda.org/docs/ngda-final-report.pdf', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.ngda.org/docs/CINGDAfindingspub121509.pdf', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://escholarship.umassmed.edu/jeslib/vol2/iss2/1/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/appraisal-and-selection', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/metadata', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/preservation-metadata', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/resources/curation-reference-manual/completed-chapters/preservation-scenarios-dh', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/resources/curation-reference-manual/chapters-production/scientific-metadata', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.w3.org/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.w3.org/standards/xml/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.lynda.com/XML-tutorials/XML-Essential-Training/145930-2.html?org=unm.edu', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.aptana.com/products/studio3.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.oxygenxml.com/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://notepad-plus-plus.org/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.barebones.com/products/textwrangler/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.fgdc.gov/metadata/geospatial-metadata-standards', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.fgdc.gov/csdgmgraphical/index.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.fgdc.gov/metadata/documents/workbook_0501_bmk.pdf', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://mrdata.usgs.gov/validation/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.w3.org/RDF/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://geo-ide.noaa.gov/wiki/index.php?title=ISO_FAQ', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://tinyurl.com/p24aon4', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.ncddc.noaa.gov/metadata-standards/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://service.ncddc.noaa.gov/rdn/www/metadata-standards/documents/MD-Metadata.pdf', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://service.ncddc.noaa.gov/rdn/www/metadata-standards/documents/MI-Metadata.pdf', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.fgdc.gov/metadata/geospatial-metadata-standards', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://wiki.dublincore.org/index.php/User_Guide', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://tinyurl.com/pubjloq', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://resources.arcgis.com/en/help/main/10.2/index.html#//003t00000039000000', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://plugins.qgis.org/plugins/metatools/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://data.gulfresearchinitiative.org/metadata-editor/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.icpsr.umich.edu/icpsrweb/content/datamanagement/dmp/elements.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.icpsr.umich.edu/icpsrweb/content/datamanagement/dmp/framework.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.icpsr.umich.edu/icpsrweb/content/datamanagement/dmp/table.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/sites/default/files/documents/resource/DMP_Checklist_2013.pdf', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://dmp.cdlib.org/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://dmptool.org/guidance', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.dcc.ac.uk/resources/data-management-plans/funders-requirements', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://datapub.cdlib.org/2014/01/30/institutional-repositories-part-1/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'https://datapub.cdlib.org/2014/02/20/institutional-repositories-part-2/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.re3data.org', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://www.nap.edu/catalog/11865.html', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://creativecommons.org/about', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://creativecommons.org/licenses/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://wiki.creativecommons.org/Data_and_CC_licenses', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://wiki.creativecommons.org/Data', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://wiki.creativecommons.org/CC0_use_for_data', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://px7gv7gt2n.search.serialssolutions.com.libproxy.unm.edu/?genre=article&issn=08852715&title=High+Technology+Law+Journal&volume=1&issue=2&date=19860901&atitle=IS+THERE+A+PROPERTY+INTEREST+IN+SCIENTIFIC+RESEARCH+DATA%3f&spage=447&pages=447-482&sid=EBSCO:Publisher+Provided+Full+Text+Searching+File&au=JONES%2c+RICHARD+H.', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://creativecommons.org/licenses/by-sa/4.0/', 200]
['goalsAndObjectives/goalsAndObjectives.html', 'http://creativecommons.org/licenses/by-sa/4.0/', 200]


--- Bad Links ---

Commit and Push Updates to Github





In [9]:



    


%%bash
# define commit message
MESSAGE="Updated styles for goals and objectives"
git pull
git add -A
git commit -m "$MESSAGE"
git push



    


















    






Already up-to-date.
[master fe03cf6] Updated styles for goals and objectives
 3 files changed, 159 insertions(+), 7 deletions(-)










    






To https://github.com/karlbenedict/OILS515.git
   7f47880..fe03cf6  master -> master

















In [ ]:

Content source: karlbenedict/OILS515

Similar notebooks:

notebook.community | gallery | about