Data Gathering

In this notebook we gathered the data for our project and turned it into a GDS object.

Making Project Directories

We began by making the appropriate directories to organize our data and analysis into.



In [1]:

    
%%bash

mkdir ../data 
mkdir ../data/raw
mkdir ../data/cleaned
mkdir ../data/simulated
mkdir ../visualizations
mkdir ../script









    



mkdir: cannot create directory `../data': File exists
mkdir: cannot create directory `../data/raw': File exists
mkdir: cannot create directory `../data/cleaned': File exists
mkdir: cannot create directory `../data/simulated': File exists
mkdir: cannot create directory `../visualizations': File exists
mkdir: cannot create directory `../script': File exists

Choosing a Dataset

We decided to analyze a microDNA expression dataset from the NCBI's (National Center for Biotechnology Information) database of gene expression arrays, the Gene Expression Omnibus (GEO) repository. Eventually we settled on one that contained gene expression information for 76 tissue samples from child patients with a particular form of brain cancer, Medulloblastoma.

Eventually, we decided on the data set from this page:

www.ncbi.nlm.nih.gov/sites/GDSbrowser?acc=GDS4471

Loading Data

Load rmagic:



In [2]:

    
%load_ext rmagic

In the cell below we downloaded the data into the ../data/raw directory, and renamed it bio_data.soft.gz using R and the download link for the dataset provided by the NCBI.



In [3]:

    
%%R

data_url = 'ftp://ftp.ncbi.nlm.nih.gov/geo/datasets/GDS4nnn/GDS4471/soft/GDS4471.soft.gz'
bio_dir = '../data/raw/bio_data.soft.gz'

download.file(data_url, destfile=bio_dir)









    





trying URL 'ftp://ftp.ncbi.nlm.nih.gov/geo/datasets/GDS4nnn/GDS4471/soft/GDS4471.soft.gz'
ftp data connection made, file length 12758799 bytes
opened URL
==================================================
downloaded 12.2 Mb

Creating a GDS Object

In order to analyze the GEO Dataset (GDS) file that our data set is contained in, we decided to use the software the NCBI recommends, Bioconductor, which was specifically designed to work with gene expression data sets. However, Bioconductor requires several packages to work properly, which we have installed below:

Installed libcurl:

sudo apt-get install libcurl4-gnutls-dev

Installed r-cran-xml:

sudo apt-get install r-cran-xml

Please note: This had to be done in the terminal. It could not be done in ipython notebook because the install prompts for user permission.

Once we had installed the correct packages in our oskiboxes, we were able to install Bioconductor and the required Bioconductor packages.

Installed Bioconductor



In [4]:

    
%%R

source("http://bioconductor.org/biocLite.R")
biocLite()









    





Bioconductor version 3.0 (BiocInstaller 1.15.1), ?biocLite for help
BioC_mirror: http://bioconductor.org
Using Bioconductor version 3.0 (BiocInstaller 1.15.1), R version 3.1.0.

Installed GEOquery package of Bioconductor

This allowed us to turn the loaded soft.gz or GDS file in our directory into a "GDS object" which gives us access to meta data and a table of our expression data. These can be accessed through the functions Meta() and Table(), respectively.



In [5]:

    
%%R

biocLite("GEOquery")
library(Biobase)
library(GEOquery)









    





BioC_mirror: http://bioconductor.org
Using Bioconductor version 3.0 (BiocInstaller 1.15.1), R version 3.1.0.
Installing package(s) 'GEOquery'
trying URL 'http://bioconductor.org/packages/3.0/bioc/src/contrib/GEOquery_2.31.0.tar.gz'
Content type 'application/x-gzip' length 13636222 bytes (13.0 Mb)
opened URL
==================================================
downloaded 13.0 Mb


The downloaded source packages are in
	‘/tmp/RtmpdTS0vK/downloaded_packages’
Loading required package: BiocGenerics
Loading required package: parallel

Attaching package: ‘BiocGenerics’

The following objects are masked from ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
    clusterExport, clusterMap, parApply, parCapply, parLapply,
    parLapplyLB, parRapply, parSapply, parSapplyLB

The following object is masked from ‘package:stats’:

    xtabs

The following objects are masked from ‘package:base’:

    anyDuplicated, append, as.data.frame, as.vector, cbind, colnames,
    do.call, duplicated, eval, evalq, Filter, Find, get, intersect,
    is.unsorted, lapply, Map, mapply, match, mget, order, paste, pmax,
    pmax.int, pmin, pmin.int, Position, rank, rbind, Reduce, rep.int,
    rownames, sapply, setdiff, sort, table, tapply, union, unique,
    unlist

Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.

Setting options('download.file.method.GEOquery'='curl')

After loading GEOquery, we used the function getGEO() to turn our GDS file into a GDS object.



In [6]:

    
%%R

data = getGEO(filename=bio_dir)

We then examined the object to ensure that it had loaded correctly, and that both the Meta() and Table() functions performed properly.



In [7]:

    
%%R

print(mode(data))









    





[1] "S4"

This class is consistent with a GDS object.



In [8]:

    
%%R

print(head(Table(data)))
print(Meta(data)$description)









    





     ID_REF IDENTIFIER GSM918603 GSM918641 GSM918580 GSM918593 GSM918625
1 1007_s_at       DDR1   6.76088   7.64798    7.6008   7.72551   7.16511
2   1053_at       RFC2    6.7353   6.20294   5.97406    6.6242   6.16395
3    117_at      HSPA6   4.32546    4.4613    5.3627   4.85359   5.05752
4    121_at       PAX8   5.80061   5.31861   5.78598   5.92265   5.44199
5 1255_g_at     GUCA1A   3.21487   3.21888   3.32863   3.77963   4.30136
6   1294_at       UBA7    5.0689    5.5518    6.1334   4.88658   4.86445
  GSM918638 GSM918642 GSM918643 GSM918619 GSM918621 GSM918582 GSM918649
1    6.7995   6.91909   6.64144   7.93787   7.46043    7.5455   7.46691
2   6.44857   6.60232   6.55365   6.50772   6.48234   6.17877   6.05819
3   3.61092   4.13035   4.30676   4.58802   5.21385   5.79088   4.56331
4   5.77269   5.80814    5.4112   5.59248   5.85993     6.218   5.99396
5   3.71357   3.71601   5.01396   4.02535   3.53223   4.61413   4.59915
6   5.16536   4.95018   4.66815   5.46764   5.34996    5.6771   5.82275
  GSM918651 GSM918607 GSM918609 GSM918608 GSM918606 GSM918620 GSM918628
1   7.75074    6.8776   7.77674   6.94534   7.67359    7.4625   9.29046
2   6.54161   6.37707   6.77696   6.31536   6.10099   6.17337   5.42319
3   4.16356   4.79744   4.71492   4.72562   4.45318     4.445   6.16563
4   6.11545    6.0232   5.79788   5.61568   5.79636   5.79271   6.18683
5    4.0448   3.48738   4.58497   3.24649   3.96272    4.3386   3.90399
6   5.55412   5.92532    5.6269   5.56643   5.56414   5.35753   7.06527
  GSM918586 GSM918594 GSM918600 GSM918601 GSM918612 GSM918614 GSM918629
1   7.79235   6.89942   6.17066    7.3025   7.28756    6.7806   6.71998
2   6.57953   6.59687    6.6901   6.35819   6.73459   6.83098   7.31342
3    4.9075    4.8339   5.28523   4.15418   5.10169   4.94947   4.41763
4   6.21241   6.01859   5.89495   5.74652   5.90536   5.78044   5.66677
5   3.15274   4.61611   3.61631   5.14691   6.03763   5.41566   3.92197
6   5.47269   5.13815   5.33417   4.80729   5.25227   4.42004   5.10957
  GSM918587 GSM918588 GSM918589 GSM918611 GSM918624 GSM918637 GSM918639
1   8.26003   7.33289   7.12898   7.72771   7.56076   6.94235   7.40889
2   7.08087   6.64275     6.012   6.20436   6.79593   6.02562   6.56456
3   5.77362   6.09807   4.79496   4.41522   5.18347   4.53045    5.0733
4   6.28283   6.11036   5.91862   6.06146   5.37851   5.36971   5.89743
5   3.40453     4.483   6.73673   6.07993   5.07829   4.33205   5.86873
6    6.1108   5.41699   4.93303   4.94164   4.98907   5.04665   5.06638
  GSM918640 GSM918636 GSM918590 GSM918610 GSM918615 GSM918616 GSM918632
1   6.75624   7.53652   7.68327   7.22052   7.76798   6.94042   6.74205
2   6.33541     6.337   5.88249   6.27023   6.52986   6.28395   6.29101
3   3.99268   3.50856    4.8227    3.8111   5.87409   4.60817   3.60005
4   5.58086   5.86221   5.73269   5.82452   6.07626   6.02611   5.21058
5   5.46806   6.17607   3.78191   3.37759   4.38328   4.95371   5.30827
6    5.3822    4.7032    5.5626   5.01993   5.00797   5.05433   5.29531
  GSM918647 GSM918578 GSM918579 GSM918581 GSM918584 GSM918591 GSM918592
1   7.64979   7.59594   7.80665   7.40141   7.99686    7.4572   7.56221
2   6.60678    6.1444   6.19154   5.90781   6.88336   6.57758   5.84122
3   3.61362   4.82431   5.24439   4.80566   4.53582   3.48431   4.93807
4   5.86164   5.67092   5.62149   5.97838   5.76988   5.84644   5.82777
5   4.25986   4.14155   3.41773   4.21804   3.53515   3.97029   3.44362
6   5.70511   5.48935   5.70478   5.43677   5.55102   4.99991   5.57215
  GSM918597 GSM918598 GSM918599 GSM918604 GSM918605 GSM918613 GSM918623
1   6.84705   6.98878   7.34769   7.48184    7.6702   7.65809   7.17832
2   6.32131   6.13794   6.11699   6.58369   6.27495   6.95978   5.87493
3   4.28772   4.46706    3.5175   4.78415   4.86522   4.72028   5.14283
4   5.93039   5.95012   5.68731   5.96743   5.74236   5.38816   5.28015
5   4.06732   3.97594   5.74268   4.56017   4.67189   4.78248   4.16976
6    5.3547   5.76707   5.42978   5.17615   5.13756   5.26786   5.15213
  GSM918626 GSM918627 GSM918633 GSM918634 GSM918635 GSM918645 GSM918646
1    7.3879   7.02438   7.07369   7.28592    6.9463   7.80503    7.7513
2   6.29674   6.14804   7.04882   6.53161   6.22496   6.61767   6.37264
3   5.25854   4.74145   3.91601   4.42245   4.99991    4.4987   3.31782
4   5.17953   5.87549   5.78628   5.41254   5.22036   5.66087   5.98015
5   7.48099   4.99315    4.3451   4.61809   3.46261   4.65205   5.87465
6   5.99147   5.28219   5.06386   5.06449   4.76217   4.97673   5.42803
  GSM918648 GSM918650 GSM918652 GSM918653 GSM918622 GSM918583 GSM918585
1    7.3308   7.50064   7.66767   7.75773   6.38604    7.6031    7.8252
2   5.76676   6.38873   6.60611   6.89315   6.60841   6.05279   6.28301
3   4.06732   4.13677   4.42963   3.75654   4.25561   4.05178   4.76303
4   5.93859   5.98419   5.43677   6.09086   5.77113   5.59953   5.27811
5   3.91202   4.17285   4.93087    3.6763   5.26165   3.65325   3.68387
6   5.17728   4.11251   5.23218   5.92586   5.01396   5.15791   5.65249
  GSM918595 GSM918596 GSM918602 GSM918617 GSM918630 GSM918631 GSM918618
1   7.34923   7.69298   7.49109   7.12761   7.15618   7.43296   8.03226
2   6.80128   6.38503   6.90635    6.0066   6.60678   5.92985   6.27363
3    4.3631   5.04857   4.78332   4.80402    4.4403    3.6055   6.57368
4   5.99246   5.66053   6.20577    5.8386   5.62329   5.23111   5.81771
5   3.38439   3.71844    6.0781   4.06217   4.37198   3.21487   6.16374
6   5.65913   5.75542   5.43066    5.6058   4.85203   5.07705   5.67915
  GSM918644
1   8.10259
2     6.663
3    3.5293
4   5.59546
5   6.87078
6   4.93159
 [1] "Analysis of medulloblastomas from children ages 3 to 16 years. Medulloblastoma is a malignant childhood brain tumor comprising four discrete subgroups. Results provide insights into pathogenesis of medulloblastoma and highlight targets for therapeutic development."
 [2] "subgroup: WNT"                                                                                                                                                                                                                                                           
 [3] "subgroup: SHH"                                                                                                                                                                                                                                                           
 [4] "subgroup: SHH outlier"                                                                                                                                                                                                                                                   
 [5] "subgroup: G3"                                                                                                                                                                                                                                                            
 [6] "subgroup: G4"                                                                                                                                                                                                                                                            
 [7] "subgroup: N/A"                                                                                                                                                                                                                                                           
 [8] "male"                                                                                                                                                                                                                                                                    
 [9] "female"                                                                                                                                                                                                                                                                  
[10] "M stage: MB-AN"                                                                                                                                                                                                                                                          
[11] "M stage: MB-CL"                                                                                                                                                                                                                                                          
[12] "M stage: MB-DN"                                                                                                                                                                                                                                                          
[13] "M stage: MB-Myo"

The object is consistent with original file. This means we were able to succesfully create the GDS object from our dataset!

Team members responsible for this notebook:

Team member 1: Yucheng Zhang - explored the possibility of using bash and python to get data into the correct format and wrote markdown.

Team member 2: Rebeccah Lavelle Terhune - discovered way to format data with Excel and wrote markdown.

Team member 3: Philip Hong - converted Excel data into .csv format and wrote markdown.

Team member 4: Renee Rao - explored the possibility of using the Bioconductor software to get data into the correct format and wrote markdown.

We all worked together on data gathering, and we ended up using Renee's method.