In [1]:
!date
This notebook is to familiarize you with the UNIX shell in a Linux environment.
I'm using the Z Shell but most of the following will work on any standard UNIX shell (bash, etc.)
In [2]:
from IPython.display import Image
Image("terminal.png", width=600)
Out[2]:
This lesson is written up in a Jupyter notebook because it's an easy way to store a list of commands and their output. In Jupyter, input is treated as a shell command whenever it starts with !. For example
In [3]:
!echo $SHELL
If I forget the ! I get an error because my command is interpreted as a Python command:
In [4]:
echo $SHELL
However, there are some IPython magics that mimic shell commands and work without any quanlifier. The next command moves us to my home directory.
In [5]:
cd ~
To show the present working dir use pwd
In [6]:
pwd
Out[6]:
To list its contents use ls
In [7]:
ls
We could add the ! but it wouldn't make any difference.
In [8]:
!pwd
If you're working directly in the shell then of course you should omit the !
The top of the directory tree looks like this on my machine:
In [24]:
ls /
Some comments
bin and usr/bin directories are where most executables (applications) liveusr/libhome directory is where users store personal filesetc is home to system wide configuration filesvar is where logs are written tomedia is where you'll find your USB stick after you plug it inI have a paper by Lars Hansen on asset pricing somewhere in my file system but I can't remember where. One quick way to find files is to use the locate command.
In [25]:
!locate hansen
For more sophisticated searches I use find.
For example, let's find all Julia files in /home/john/sync_dir and below that contain the phrase cauchy.
In [26]:
!find ~/sync_dir/ -name "*cauchy*.jl"
The next command finds all files ending in "tex" modified within the last week:
In [2]:
!find /home/john/sync_dir/ -mtime -7 -name "*.tex"
When working with specific text files we often use a text editor. However, it's also possible to do a significant amount of work directly from the command line. Here are some examples.
In [50]:
cd ~/temp_dir
In [51]:
ls
We're going to make a text file using the shell redirection operator >
In [52]:
!ls -l ~/sync_dir/books
In [53]:
!ls -l ~/sync_dir/books > list_books.txt
In [54]:
ls
We can read the contents of this text file with cat
In [55]:
!cat list_books.txt
We can search within this file using grep
In [56]:
!grep Dec list_books.txt
We can show just the top of the file using ``head"
In [58]:
!head -2 list_books.txt
We can also append to files using the >> operator
In [59]:
!date > new_file.txt
In [60]:
ls
In [61]:
!cat new_file.txt
In [62]:
!date >> new_file.txt
In [63]:
!cat new_file.txt
We can change "Jan" to "January" using the sed line editor
In [64]:
!sed 's/Jan/January/' new_file.txt
The command line becomes very powerful when we start linking the commands shown above into compound commands. Most often this is done with a pipe. The symbol for a pipe is |.
Let's look at some examples.
The first command searches for files with the phrase hansen and pipes the output to grep, which filters for lines containing sargent
In [65]:
!locate hansen | grep sargent
Let's do the same but print only the first 5 hits
In [66]:
!locate hansen | grep sargent | head -5
As an exercise, let's see how many Python files I have in /home/john/.
In [67]:
!find ~ -name "*.py" | wc -l
Here wc is a program that counts words, lines or characters, and -l requests the number of lines
Let's see if any Python files in my papers directory contain the phrase bellman_operator. To do this we'll use xargs, which sends a list of files or similar consecutively to the filter on its right.
In [68]:
!find /home/john/sync_dir/papers -name "*.py" | xargs grep bellman_operator
One useful trick with bash and zsh is that CTRL-R implements backwards search through command history. Thus we can recall an earlier command like
by typing CTRL-R and then JET or similar
Another comment is that file names starting with . are hidden by default. To view them use ls -a
In [69]:
cd ~
In [70]:
ls
In [71]:
ls -a
Another thing to note is that files have permissions associated with them, so your system can keep track of whether they are executable, who is allowed to read / write to them and so on. To view permissions use ls -l
In [ ]:
ls -l ~/bin
The permissions are the characters on the far left. Here x means executable, r is readable and w is writable, d is directory and l is link. To learn more about permissions try googling linux file permissions. To learn more about links google linux file links.
In [ ]: