Using sos magic within iPython allows you to run sos within an iPython session. It does not provide a full-blown SoS system but on the other hand you can use all the iPython features that you are familiar with.
sos magic is installed by default when you install sos. A profile has been created for you so that you can use it by command
ipython --profile sosYou can also load the extension using command %load_ext sos_magic after ipython starts in its default profile, or edit ~/.ipython/profile_default/ipython_config.py and add sos_magic to c.InteractiveShellApp.extensions so that the extension is loaded automatically to your default profile.
Let us starts with the basic. When you start an ipython interactive shell through ipython command or ipython/Jupyter notebook, you are given a cell that you can enter python expression or statements, or iPython magic. There are two kinds of magics
% or starts at the beginning of line without % if automagic is set to True (default). Line magic takes the words after it as parameter.%% that takes both the words and lines after the magic word.If the sos_magic extension is loaded, it already has a SoS dictionary. You can get the dictionary using line magic %sosdict, for example
In [1]:
%sosdict
Out[1]:
The dictionary is empty because we have not assigned anything to it. Let us run a sos statement
In [2]:
%sos a = 1
In [3]:
%sosdict
Out[3]:
and you can see the sos dictionary contains one item. There are other usages of the %sosdict magic, you can get the dictionary by assigning the dictionary to a variable
In [4]:
d = %sosdict
In [5]:
d.keys()
Out[5]:
If you are interested in only a subset of variables, you can list them after %sosdict
In [6]:
d = %sosdict a
d.keys()
Out[6]:
You can get the keys of the dictionary easier using
In [7]:
%sosdict --keys
Out[7]:
If after a while you would like to reset the dictionary and run a SoS script from fresh, you can reset the dictionary using option --reset
In [8]:
%sosdict --reset
%sosdict
Out[8]:
The SoS dictionary actually contains other items such as all the SoS actions and functions. If you would like to see all of them, use option --all. For example,
In [9]:
%sosdict --keys --all
Out[9]:
In summary, the %sosdict magic accepts
%sosdict [-a|-all] [-k|--keys] [-r|--reset] [var1] [var2] ...where
var1, var2 etc are name of variables. All variables will be displayed if no variable is specified.-a|-all: list all dictionary keys, including SoS functions and variables.-k|--keys: list only keys, not their values-r|--reset: reset the dictionary to its original content (with only SoS internal values)Magic sos can be either a line magic or cell magic. Using sos as a line magic, it simply executes the SoS (python) expression or statement in SoS. For example,
In [10]:
%sos a=10
In [11]:
%sos "a + 100 = ${{a+100}}"
Out[11]:
Here we use SoS string interpolation to evaluate an expression a+100 and return its string representation. The sigil is supposed to be ${ } but it has to be used as ${{ }} because iPython has its own interpolation system that tries to evaluate anything in { }. This can be annoying but you can try to change the SoS default sigil using a %sos_options magic
In [12]:
%sos_options sigil='` `'
In [13]:
%sos "a + 100 = `1+100`"
Out[13]:
In [14]:
%sos b=['file1.txt', 'file2.txt']
In [15]:
%sos "`b!r,`"
Out[15]:
Note that iPython will leave { } alone if it does not understand the content inside, so you are usually ok if the variable is not recognizable by iPython
In [16]:
%sos_options sigil='${ }'
In [17]:
%sos name = 'Bob Kenny'
%sos "My name is ${name}"
Out[17]:
If you would like to execute multi-line SoS statements or scripts, you will need to use the magic in cell mode (with %% prefix), for example,
In [18]:
%%sos
run:
echo "something"
runs a shell script within iPython. Similarly, you can run arbitrary shell, R, perl, python code in ipython/SoS, with string interpolation. Note that ipython already has a magic called %%script that allows you to execute scripts in a cell, while SoS provides a lot more features.
For example, the %%sos treats its content as a complete SoS script and accepts command line arguments.
In [19]:
%%sos --rep 10
resource = '~/resource'
ref_genome ="${resource}/hg19"
parameter: rep = 5
In [20]:
%sos rep
Out[20]:
Because rep is defined as a command line parameter, the command line option --rep 10 overrides its default value 5.
If you are using a terminal based iPython session (instead of the Jupyter notebook with iPython kernel that you are seeing right now), you will soon notice a problem with the cell magic %%sos in that it does not accept blank new lines, which is problematic for large piece of code. In this case, you can use line magic %sospaste to read the content directly from clipboard. Using the same example, you can select the text (with newline), and run
In [5]: sospaste --rep 10
resource = '~/resources'
ref_genome = '${resource}/hg19'
parameter: rep = 5
## -- End pasted text --
This is the most convenient way to execute pieces of SoS script in iPython and is used most frequently.
Magics sosget and sosput are used to exchange variables between ipython and sos. For example, if you defined a variable in SoS and would like to use it in ipython, you can use magic %sosget var1 var2 to copy the value of var1 and var2 from SoS dictionary to ipython namespace. Similarly, %sosput copies variables from the ipython namespace to SoS dictionary. For example,
In [21]:
%sos a = 20
In [22]:
%sos b = "a**2 = `a**2`"
In [23]:
%sosget a b
In [24]:
a
Out[24]:
In [25]:
b
Out[25]:
In [26]:
b = 'something else'
In [27]:
%sosput b
In [28]:
%sosdict b
Out[28]:
If you are tired of entering certain SoS options after magic %%sos or %sospaste, you can set these options persistently using magic sosset. For example, if you set
In [29]:
%sosset -v 3
You can run the magic %sosset without option to get a list of acceptable options.
In [30]:
%sosset
Let us reset the default sigil and try to use iPython sos magics to debug a SoS script.
In [39]:
%sos_options sigil='${ }'
In [40]:
%sosset -v1
The script currently looks like
resource = '~/resources'
ref_genome = '${resource}/hg19'
parameter: rep = 5
[1]
print(rep)
seq = range(rep)
input: for_each='seq'
python:
import time
print('sleep {} seconds.'.format(_seq))
time.sleep(_seq)
To check the global definition, you can copy and paste the definitions to ipython as
In [41]:
%%sos
resource = '~/resources'
ref_genome = '${resource}/hg19'
The statements are executed and you can check the result using
In [42]:
%sos ref_genome
Out[42]:
resource is not interpolated because the expression is quoted using single quotes. To use SoS string interpolation, we will need to quote the string in double quotes
In [43]:
%%sos --rep 3
resource = '~/resources'
ref_genome = "${resource}/hg19"
parameter: rep = 5
In [44]:
%sos ref_genome
Out[44]:
In [45]:
%sos rep
Out[45]:
Now, let us continue, copy/paste or select the next step, run sospaste and .... a big block of errors!
In [46]:
%%sos
[1]
print(rep)
seq = range(rep)
input: for_each='seq'
python:
import time
print('sleep {} seconds.'.format(_seq))
time.sleep(_seq)
Here 10 is printed so rep is valid. This means the rep we set last time is available and correct, then what might be the problem? Let us see what is saved in .sos/default_1.py using a bit magic of ipython
In [47]:
!cat .sos/default_1_0.py
The error seems to be obvious, we need to use ${_rep} for the value to be passed through string interpolation. Let us make some changes to the script and run
In [48]:
%%sos
[1]
print(rep)
seq = range(rep)
input: for_each='seq'
python:
import time
print('sleep ${_seq} seconds.')
time.sleep(${_seq})
Using this method, you can execute your SoS script step by step and make sure everything works, before you execute it from a command line.