This draft notebook has been created to accompany the workshop "3GC-HI fidelity" held at Port Alfred, from 24 October 2016 to 4 November 2016 ➞. The workshop consisted of three parts, an introduction in radio interferometry, an introduction in advanced interferometry, and an introduction in HI analysis techniques including HI specific data reduction. The notebook addresses problems in the context of HI analysis. It is an ongoing project which will be carried on beyond the workshop to provide beginners in radio astronomy with a comprehensive description of HI analysis techniques. At its current stage it implements single contributions, which are loosely linked. Please contact Gyula Józsa if you want to contribute to this book.
The concept is taken from Foster, Grobler et al. "Fundamentals of Radio Interferometry" ➞ and slightly modified.
This book is developed and tested with the following software dependencies (a guide for setting up a virtual environment with the current versions is available in the git repository readme):
The very first entry in a notebook will import our current standard modules:
In [ ]:
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from IPython.display import HTML
HTML('../style/course.css') #apply general CSS
Followed by an optional import of any section specific modules, e.g. :
In [ ]:
import matplotlib.image as mpimg
from IPython.display import Image
If a section contains a significant amount of code, for readability it might be useful to suppress the code and only show the output. To do this an additional code block should be added:
In [ ]:
from IPython.display import HTML
HTML('../style/code_toggle.html')
A global glossary defines all mathematical notation and useful definitions.
Adhere to the following general mathematical notation:
Vector, scalar and matrix:
$2\times2$-Polarized vs. $N\times N$-Unpolarized matrices:
Jones versus Jacobian:
Fourier transform:
Subscript to avoid ambiguity:
The general list of symbols can be found in the glossary ➞.
If you want to include a specific definition to a word or phase, then italicize the text the first time you use it in a section or chapter and add the term to the glossary.
Each chapter is contained in a seperate directory, the directory name is formatted like chapter_ii_name, where ii is the chapter number and name is the chapter name, with spaces replaced by underscores, e.g. chapter_6_deconvolution_in_imaging. The directory will contain notebooks, a ii_figures file containing figures (ii being the chapter number), a ii_bibtex.bib file containing bibtex entries. Any data or additional files will be in a seperate data directory. Notice that very large data should not be stored in a github repository. They should be stored at another place, then the students be instructed to download the data into the data directories.
Notebook naming should be prefixed with the chapter number (two digits) and a sequential number (two digits) based on ordering in the chapter. Like directories, the notebook name should be the section name (with underscores to replace spaces), a shortened version of the section is also fine, e.g. the Sky Model section of Chapter 6 would be 06_01_sky_models.ipynb.
Each chapter is made up of multiple sections, each of which is possibly made up of sub-section, et cetera. To keep notebooks a reasonable and consumable size, a notebook should only contain a single section. For long sections it may be reasonable to further break up a section into multiple notebooks.
The beginning of each notebook should with a set of navigation links including a link to the global outline (this notebook), glossary, the chapter specific introduction notebook, the previous section notebook, and the next section notebook. See example section 1.1.
Following the navigation links the standard python modules and any section specific modules should be imported, see 'A Note on Software' above for the current standard module import command. Following these import commands the notebook should start with a heading entry for the notebook with text that corresponds to the outlien text above, see 'Section and Subsection Headings' for sizes below.
At the end of a notebook, include a link to the next section notebook. If at the end of a chapter, provide a link to the next chapter.
Each chapter should contain a short introduction notebook ii_00_introduction.py, which will provide an overview of the topics in the chapter and an outline of the notebooks in the chapter. At the end of the introduction include a list of editors and contributors of the chapter (indicate specific sections).
The final notebook of a chapter should contain a section on literature and further reading ii_references_and_further_reading.ipynb which contains links to papers and books, it may be useful to write a sentence about why a link is useful. All the external references noted in the chapter should be listed here. Further, the conclusion to a chapter is an appendix ii_appendix.ipynb, in which the appendix is put, if required.
In a notebook, section names, e.g. 1.1, should use the heading size 2. While each subsequent sub-section should increase the heading size, e.g. a sub-section will be size 3, a sub-sub-section will be size 4,...
For clarity, it is possible to create emphasized point in the course of a paragraph, or a summary of important concepts at the end of a section/chapter. This relies on the use of a common CSS for every user. The CSS style is defined in course.css and will be applied to one notebook upon calling initcss.css_styling(). Those two files are located in the /style dir in the main course dir.
first issue this python lines to load the CSS file in the main style directory (might change after some housekeeping/discussion)
from IPython.display import HTML
HTML('../style/course.css') ##apply general CSSTo write a "warning" text box, one can use in a markdown:
<div class=warn>
<b>Warning:</b> This relation assumes this particular hypothesis
</div>To write a note "note" or a piece of advice, use:
<div class=advice>
<b>Advice:</b> Check the homogeneity of your equations !!!
</div>To create a green summary block:
<p class=conclusion>
<font size=4> <b>Take-away message</b></font>
<br>
<br>
• <b>Conclusion 1</b>: Important item to remember with a specific <em>emphasized</em> word <br><br>
• <b>Conclusion 2</b>: A second important item to remember with a specific <em>emphasized</em> word.
</p>To create a "Prerequisites"/"To read" header block:
<p class=prerequisites>
<font size=4> <b>Prerequisites</b></font>
<br>
<br>
• <b>Definition of ($u$,$v$,$w$):</b> [Go to 4.1](4_1_The_Baseline.ipynb) <br><br>
• <b>The visibility function:</b> [Go to 4.3](4_3_The_Visibility_Function.ipynb)
</p>One of the limitations of the ipython notebook is the inability to render equation, figure, and table labels properly. For the moment, we have settled on a consistent, but inelegant standard.
Linking to internal (i.e. within the same notebook) and external (i.e. other notebooks in the book) references will use a dual method of using the standard markdown HREF and a LaTeX style so that dynamic links will work in the notebooks and conversion to PDF via LaTeX will contain references.
Links internal to a notebook references are created by adding the down arrow symbol ⤵ (HTML code ⤵) as the link, e.g. [hyperlink text ⤵](#destination). A reference is created by including <a id='destination'></a> where the desired reference desition is to be placed. In addition to the ipython notebook dynamic links LaTeX references should be included with the \label{destination} and \ref{destination} tags. An example of a complete internal reference is
[hyperlink text ⤵](#destination) <!--\ref{destination}-->renders as: hyperlink text ⤵
And a complete internal label is
<a id='destination'></a> <!--\label{destination}-->External links are similar to internal links, but use the right arrow symbol ➞ (HTML code ➞) for a link. An example of a reference to another ipython notebook is [hyperlink text ➞](another_notebook.ipynb#destination) with a LaTeX tag \ref{destination}. An example of a complete external reference is
[hyperlink text ➞](another_notebook.ipynb#destination) <!--\ref{destination}-->renders as: hyperlink text ➞
Note, HTML comment tags are wrapped around the Latex label and ref tags to hide them in the notebook.
Citations to published work is a little tricky in our setup, we want to create two links. One for if we convert the notebook to latex we should be able to auto-generate a \cite{} tag, see the nbconvert citation⤴ example⤴. And, the other as a hyperlink to a abstract or copy of the paper (e.g. a NASA ADS link). To do this we need to create a link and use the HTML <cite data-cite='bibtexRef'> tag where bibtexRef is the name of the reference in the bibtex file in the chapter directory. An up arrow symbol ⤴ (HTML code ⤴) is used to denote an external to the book hyperlink. An example of a complete citation is
[<cite data-cite='1999ASPC..180.....T'>Synthesis Imaging in Radio Astronomy II</cite> ⤴](http://adsabs.harvard.edu/abs/1999ASPC..180.....T)which renders as:
Synthesis Imaging in Radio Astronomy II ⤴
Where there is an entry in the bibtex file
@PROCEEDINGS{1999ASPC..180.....T,
title = "{Synthesis Imaging in Radio Astronomy II}",
booktitle = {Synthesis Imaging in Radio Astronomy II},
year = 1999,
series = {Astronomical Society of the Pacific Conference Series},
volume = 180,
editor = {{Taylor}, G.~B. and {Carilli}, C.~L. and {Perley}, R.~A.},
adsurl = {http://adsabs.harvard.edu/abs/1999ASPC..180.....T},
adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}In order to maintain uniform and informative reference labels we will use a standard naming convention of the form chapterStr:type:uniqueID where chapterStr is a unqiue, descriptive string for a chapter or chapter subsection, type is the type of content beinging labelled, and uniqueID is a unique ID for the content. For example a table in and Imaging chapter which contains information on weights could have the label imaging:tbl:weights. The chapterStr for each chapter is to be defined by the authors. If a section of a chapter is sufficiently large it shoud have its own chapterStr, perhaps with the prefix including the chapter chapterStr. The following are valid strings for type: tbl (table), fig (figure), sec (section), code (code block), eq (equation). The uniqueID is left to the authors, but suggested to contain a simple descriptive string and information on location within the chapter.
| Chapter | chapterStr | ||||
|---|---|---|---|---|---|
| 0. Preface | preface |
||||
| HI basics | bas |
||||
| Introduction to HI science: Milky Way HI and HI in the Local Group | mw_lg |
<!-- | Introduction to HI Science: General properties of galaxies | gal |
--> |
| Introduction to HI science: Extragalactic | egal |
||||
| Obtaining clean HI spectra: removing RFI and continuum emission | rfi_cont |
||||
| HI interferometry: deconvolution | deconv |
|HI basic analysis: source finding, basic source parameterisation |sof_sop|
|HI higher-order analysis: tilted-ring modelling, interpreting residuals |trm|
|Predicting HI detections |pred|
| type | value |
|---|---|
| code | code |
| equation | eq |
| figure | fig |
| section | sec |
| table | tbl |
Though, ideally any figure that is included in a notebook can be generated from the code included in the notebook, this is not always possible. The preferred image type is PDF or SVG because these can be rescaled without aliasing issues, but PNG and JPEG can be used. If a figure or image is generated by some set of code, please include a reference or notes to that code so that if it needs to be regenerated then that will be possible. If a figure or image is made with a graphics progams such as Inkscape, GIMP, et cetera please include the working file in the git repository.
To display figures use the Image function, e.g.
Image(filename='figures/sidereal.png', width=300, height=100)or in HTML in a markdown cell (will center the figure):
<img src='figures/sidereal.png' width=30%>An include a description block below the figure, which can include a label for referencing.
Below each code block or figure include a cell which contains a description of what is presented, use italics, which in markdown means by starting and ending the text with stars, e.g. *this text would be italized in markdown*. In this block one can include a label for referencing the figure or code block.
For a 3D figure include
%matplotlib nbaggin the block to embed the figure in the notebook but allow for interaction.
Equations can be written inline or in individual blocks. If you would like to reference an equation block, follow the label standard defined in 'References, Internal and External' above.
The majority of the code in this notebook is python, so please follow standard Python PEP 8⤴
Notebooks can get very large in size when they contain a number of generated figures. In order to keep the size of the repository down to a reasonable size please clear the output before making a new commit. This is done by selecting Cell > All Output > Clear from the menu at the top of the notebook.
Binary files should be stored in a directory in each chapter, for example images would be stored in a directory called figures.
In calculus, it might be interesting to show simplifications as follow: $\require{cancel}$
$B=\frac{x \cancel{a} y}{b \cancel{a}}$
To do that, you need to write this in a markdown:
$\require{cancel}$
$\frac{x \cancel{a} y}{b \cancel{a}}$Please report to the "cancel" package for other crossing-out styles
As this is a working project there are a number of known issues we would like to resolve. If you have a nicer or more efficient solution then please let us know. For the moment, here is a list of known unknowns.