Publications markdown generator for academicpages

Takes a TSV of publications with metadata and converts them for use with academicpages.github.io. This is an interactive Jupyter notebook (see more info here). The core python code is also in publications.py. Run either from the markdown_generator folder after replacing publications.tsv with one containing your data.

TODO: Make this work with BibTex and other databases of citations, rather than Stuart's non-standard TSV format and citation style.

Data format

The TSV needs to have the following columns: pub_date, title, venue, excerpt, citation, site_url, and paper_url, with a header at the top.

excerpt and paper_url can be blank, but the others must have values.
pub_date must be formatted as YYYY-MM-DD.
url_slug will be the descriptive part of the .md file and the permalink URL for the page about the paper. The .md file will be YYYY-MM-DD-[url_slug].md and the permalink will be https://[yourdomain]/publications/YYYY-MM-DD-[url_slug]

This is how the raw file looks (it doesn't look pretty, use a spreadsheet or other program to edit and create).



In [170]:

    
!cat publications.tsv









    






2020-01-15	Counterfactuals uncover the modular structure of deep generative models	ICLR	Deep generative models such as Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs) are important tools to capture and investigate the properties of complex empirical data. However, the complexity of their inner elements makes their functioning challenging to assess and modify. In this respect, these architectures behave as black box models. In order to better understand the function of such networks, we analyze their modularity based on the counterfactual manipulation of their internal variables. Experiments with face images support that modularity between groups of channels is achieved to some degree within convolutional layers of vanilla VAE and GAN generators. This helps understand the functional organization of these systems and allows designing meaningful transformations of the generated images without further training.	"""@article{besserve2018counterfactuals,
  title={Counterfactuals uncover the modular structure of deep generative models},
  author={Besserve, Michel and Sun, R{\'e}my and Sch{\""o}lkopf, Bernhard},
  journal={arXiv preprint arXiv:1812.03253},
  year={2018}
}"""	counterfactuals_gans	https://arxiv.org/pdf/1812.03253

Import pandas

We are using the very handy pandas library for dataframes.



In [171]:

    
import pandas as pd

Import TSV

Pandas makes this easy with the read_csv function. We are using a TSV, so we specify the separator as a tab, or \t.

I found it important to put this data in a tab-separated values format, because there are a lot of commas in this kind of data and comma-separated values can get messed up. However, you can modify the import statement, as pandas also has read_excel(), read_json(), and others.



In [172]:

    
publications = pd.read_csv("publications.tsv", sep="\t", header=0)
publications









    Out[172]:






  
    
      
      pub_date
      title
      venue
      excerpt
      citation
      url_slug
      paper_url
    
  
  
    
      0
      2013-02-13
      Automatic malaria diagnosis system
      ICRoM
      Malaria Diagnosis is normally accomplished by ...
      Mehrjou, Arash, Tooraj Abbasian, and Morteza I...
      malaria
      https://tinyurl.com/snnf62t
    
    
      1
      2019-10-22
      Kernel-Guided Training of Implicit Generative ...
      Arxiv
      The modern implicit generative models such as ...
      Mehrjou, Arash. (2009). &quot;Kernel-Guided Tr...
      glocad
      https://arxiv.org/abs/1910.14428
    
    
      2
      2016-01-01
      Improved Bayesian information criterion for mi...
      Pattern Recognition Letters
      In this paper, we propose a mixture model sele...
      Mehrjou, Arash, Reshad Hosseini, and Babak Nad...
      bici
      https://www.sciencedirect.com/science/article/...
    
    
      3
      2020-01-15
      Counterfactuals uncover the modular structure ...
      ICLR
      Deep generative models such as Generative Adve...
      "@article{besserve2018counterfactuals,\n  titl...
      counterfactuals_gans
      https://arxiv.org/pdf/1812.03253

Escape special characters

YAML is very picky about how it takes a valid string, so we are replacing single and double quotes (and ampersands) with their HTML encoded equivilents. This makes them look not so readable in raw format, but they are parsed and rendered nicely.



In [173]:

    
html_escape_table = {
    "&": "&amp;",
    '"': "&quot;",
    "'": "&apos;"
    }

def html_escape(text):
    """Produce entities within text."""
    return "".join(html_escape_table.get(c,c) for c in text)

Creating the markdown files

This is where the heavy lifting is done. This loops through all the rows in the TSV dataframe, then starts to concatentate a big string (md) that contains the markdown for each type. It does the YAML metadata first, then does the description for the individual page.



In [174]:

    
import os
for row, item in publications.iterrows():
    
    md_filename = str(item.pub_date) + "-" + item.url_slug + ".md"
    html_filename = str(item.pub_date) + "-" + item.url_slug
    year = item.pub_date[:4]
    
    ## YAML variables
    
    md = "---\ntitle: \""   + item.title + '"\n'
    
    md += """collection: publications"""
    
    md += """\npermalink: /publication/""" + html_filename
    
    if len(str(item.excerpt)) > 5:
        md += "\nexcerpt: '" + html_escape(item.excerpt) + "'"
    
    md += "\ndate: " + str(item.pub_date) 
    
    md += "\nvenue: '" + html_escape(item.venue) + "'"
    
    if len(str(item.paper_url)) > 5:
        md += "\npaperurl: '" + item.paper_url + "'"
    
    md += "\ncitation: '" + html_escape(item.citation) + "'"
    
    md += "\n---"
    
    ## Markdown description for individual page
        
    if len(str(item.excerpt)) > 5:
        md += "\n" + html_escape(item.excerpt) + "\n"
    
    if len(str(item.paper_url)) > 5:
        md += "\n[Download paper here](" + item.paper_url + ")\n" 
        
    md += "\nRecommended citation: " + item.citation
    
    md_filename = os.path.basename(md_filename)
       
    with open("../_publications/" + md_filename, 'w') as f:
        f.write(md)

These files are in the publications directory, one directory below where we're working from.



In [175]:

    
!ls ../_publications/









    



2013-02-13-malaria.md              2019-10-22-glocad.md
2016-01-01-bici.md                 2020-01-15-counterfactuals_gans.md



In [176]:

    
!cat ../_publications/2009-10-01-paper-title-number-1.md









    



cat: ../_publications/2009-10-01-paper-title-number-1.md: No such file or directory



In [ ]:



In [ ]:



In [ ]:

	pub_date	title	venue	excerpt	citation	url_slug	paper_url
0	2013-02-13	Automatic malaria diagnosis system	ICRoM	Malaria Diagnosis is normally accomplished by ...	Mehrjou, Arash, Tooraj Abbasian, and Morteza I...	malaria	https://tinyurl.com/snnf62t
1	2019-10-22	Kernel-Guided Training of Implicit Generative ...	Arxiv	The modern implicit generative models such as ...	Mehrjou, Arash. (2009). "Kernel-Guided Tr...	glocad	https://arxiv.org/abs/1910.14428
2	2016-01-01	Improved Bayesian information criterion for mi...	Pattern Recognition Letters	In this paper, we propose a mixture model sele...	Mehrjou, Arash, Reshad Hosseini, and Babak Nad...	bici	https://www.sciencedirect.com/science/article/...
3	2020-01-15	Counterfactuals uncover the modular structure ...	ICLR	Deep generative models such as Generative Adve...	"@article{besserve2018counterfactuals,\n titl...	counterfactuals_gans	https://arxiv.org/pdf/1812.03253