Cargue de datos s SciDB

1) Verificar Prerequisitos

Python

SciDB-Py requires Python 2.6-2.7 or 3.3



In [1]:

    
import sys
sys.version_info









    Out[1]:





sys.version_info(major=3, minor=4, micro=3, releaselevel='final', serial=0)

NumPy

tested with version 1.9 (1.13.1)



In [2]:

    
import numpy as np
np.__version__









    Out[2]:





'1.13.1'

Requests

tested with version 2.7 (2.18.1) Required for using the Shim interface to SciDB.



In [3]:

    
import requests
requests.__version__









    Out[3]:





'2.18.4'

Pandas (optional)

tested with version 0.15. (0.20.3) Required only for importing/exporting SciDB arrays as Pandas Dataframe objects.



In [4]:

    
import pandas as pd
pd.__version__









    Out[4]:





'0.20.3'

SciPy (optional)

tested with versions 0.10-0.12. (0.19.0) Required only for importing/exporting SciDB arrays as SciPy sparse matrices.



In [5]:

    
import scipy
scipy.__version__









    Out[5]:





'0.19.0'

2) Importar scidbpy

pip install git+http://github.com/paradigm4/scidb-py.git@devel



In [6]:

    
import scidbpy
scidbpy.__version__









    Out[6]:





'16.9'



In [7]:

    
from scidbpy import connect

conectarse al servidor de Base de datos



In [8]:

    
sdb = connect('http://localhost:8080')

3) Leer archivo con cada una de las ondas



In [9]:

    
import urllib.request  # urllib2 in python2 the lib that handles the url stuff
target_url = "https://www.physionet.org/physiobank/database/mimic2wdb/matched/RECORDS-waveforms"
data = urllib.request.urlopen(target_url) # it's a file like object and works just like a file



In [10]:

    
lines = data.readlines();
line = str(lines[100])

Quitarle caracteres especiales



In [11]:

    
carpeta,onda = line.replace('b\'','').replace('\'','').replace('\\n','').split("/")
onda









    Out[11]:





's00652-2965-06-28-16-10'

4) Importar WFDB para conectarse a physionet



In [ ]:

    
import wfdb



In [ ]:

    
sig, fields = wfdb.srdsamp(onda,pbdir='mimic2wdb/matched/'+carpeta) #, sampfrom=11000



In [ ]:

    
print(sig)
print("signame: " + str(fields['signame']))
print("units: " + str(fields['units']))
print("fs: " + str(fields['fs']))
print("comments: " + str(fields['comments']))
print("fields: " + str(fields))

Busca la ubicacion de la señal tipo II



In [ ]:

    
signalII = None
try:
    signalII = fields['signame'].index("II")
except ValueError:
    print("List does not contain value")
if(signalII!=None):
    print("List contain value")

Normaliza la señal y le quita los valores en null



In [ ]:

    
array = wfdb.processing.normalize(x=sig[:, signalII], lb=-2, ub=2)
arrayNun = array[~np.isnan(array)]
arrayNun = np.trim_zeros(arrayNun)
arrayNun

Cambiar los guiones "-" por raya al piso "_" porque por algun motivo SciDB tiene problemas con estos caracteres Si el arreglo sin valores nulos no queda vacio lo sube al SciDB



In [ ]:

    
ondaName = onda.replace("-", "_")
if arrayNun.size>0 :
    sdb.input(upload_data=array).store(ondaName,gc=False)
#    sdb.iquery("store(input(<x:int64>[i], '{fn}', 0, '{fmt}'), "+ondaName+")", upload_data=array)



In [ ]: