In [1]:
%matplotlib inline
plt.rcParams['figure.figsize'] = (10,10)
from matplotlib.pyplot import subplots
In [2]:
import pandas as pd
df = pd.read_hdf('/home/klay6683/l1a_dark_stats.h5','df')
These are the columns I have available in this dataset:
In [3]:
print(df.columns.values)
In [4]:
def correct_mean_value(row):
return row['SPE_SIZE'] * row['SPA_SIZE']
df['pixelsum'] = df.apply(correct_mean_value, axis=1)
df['corrected_mean'] = df.dark_mean / df.pixelsum
In [6]:
df['DN_per_s'] = df.corrected_mean / (df.INT_TIME/1000)
df.index = df.FILENAME.map(lambda x: io.Filename(x).time)
df.index.name = 'Time'
df.sort_index(inplace=True)
Two different kinds of dark modes have been taken:
Because of reduced reproducability for darks from mode 1, it was decided to use mode 2 for the future darks. I therefore filter out case 1 for now and set it to NAN.
Below one can see that most of the darks have been taken in mode 2 anyway:
In [7]:
df.MCP_VOLT.value_counts()
Out[7]:
In [7]:
import numpy as np
df[df.MCP_VOLT > 0] = np.nan
In [8]:
from iuvs import calib
df.DET_TEMP = calib.iuvs_dn_to_temp(df.DET_TEMP, det_temp=True)
df.CASE_TEMP = calib.iuvs_dn_to_temp(df.CASE_TEMP, det_temp=False)
Checking how the stats look like for calibrated DET_TEMP
In [9]:
df.DET_TEMP.describe()
Out[9]:
In [10]:
cols = 'DN_per_s CASE_TEMP DET_TEMP INT_TIME'.split()
This is a scatter matrix for the above chosen columns.
In [11]:
pd.scatter_matrix(df[cols], figsize=(10,10));
Note that the DN_per_s and DET_TEMP scatter plot still shows different families.
Because longer exposure times allow more cosmic rays to hit, it is to be expected that despite the correction for INT_TIME towards DN_per_s still shows differences per initial INT_TIME.
In [12]:
df.plot(x='INT_TIME', y='DN_per_s', kind='scatter')
Out[12]:
Above we see that the longest INT_TIME shows the highest inherent scatter, most likely due to the CRs.
Let's see how the DN_per_s develop in general over time of observations and how that compares to DET_TEMP.
In [13]:
_, axes = subplots(nrows=2)
df[['DET_TEMP', 'DN_per_s']].plot(secondary_y='DN_per_s',ax=axes[0])
df.CASE_TEMP.plot(ax=axes[1], ylim=(4,6))
Out[13]:
As previously known, we have a strong correlation between DET_TEMP and DN_per_s created.
Looking at a scatter plot, there seem to exist identifiable situations that create different relationships between DET_TEMP and DN_per_s:
In [14]:
df.plot(x='DET_TEMP', y='DN_per_s',kind='scatter')
Out[14]:
I was worried that I have to treat sets of dark images differently for some reason, so I filter out any data that has a valid NAXIS3 and look only at data that has no valid NAXIS3. (i.e. focusing on single dark images, which is the majority anyway, as seen here with the NaN entry).
In [15]:
df.NAXIS3.value_counts(dropna=False)
Out[15]:
In [16]:
subdf = df[df.NAXIS3.isnull()]
In [17]:
subdf.plot(x='DET_TEMP', y='DN_per_s', kind='scatter')
Out[17]:
But the result looks the same, so this is no issue. Nevertheless, to be sure I will use this subframe from now on.
As mentioned before, different integration times offer different probabilities for disturbant factors to happen. So let's focus on particular INT_TIMEs. Here's how the dark INT_TIMEs distribute over the L1A dataset. One also has to divide the MUV and FUV data.
In [18]:
df.INT_TIME.value_counts()
Out[18]:
In [19]:
fuv = subdf[subdf.XUV=='FUV']
muv = subdf[subdf.XUV=='MUV']
In [21]:
inttimes = [14400, 10200, 6000, 4200, 4000, 1400]
fig, axes = subplots(nrows=len(inttimes), figsize=(12,13))
for ax, inttime in zip(axes, inttimes):
muv[muv.INT_TIME==inttime].plot(x='DET_TEMP', y='DN_per_s',kind='scatter', ax=ax, sharex=False)
ax.set_title('INT_TIME = {}'.format(inttime))
fig.suptitle('MUV DN_per_s vs DET_TEMP, sorted by INT_TIME', fontsize=20)
fig.tight_layout()
In [22]:
inttimes = [14400, 10200, 6000, 4200, 4000, 1400]
fig, axes = subplots(nrows=len(inttimes), figsize=(12,13))
for ax, inttime in zip(axes, inttimes):
fuv[fuv.INT_TIME==inttime].plot(x='DET_TEMP', y='DN_per_s',kind='scatter', ax=ax, sharex=False)
ax.set_title('INT_TIME = {}'.format(inttime))
fig.suptitle('FUV DN_per_s vs DET_TEMP, sorted by INT_TIME', fontsize=20)
fig.tight_layout()
Following the interesting consistent separation in families of scatter points, let's look at what the DET_TEMP does over time during these different INT_TIMES.
In [23]:
fig, axes = subplots(nrows=len(inttimes), figsize=(13,14))
for ax,inttime in zip(axes, inttimes):
muv[muv.INT_TIME==inttime]['DET_TEMP'].plot(ax=ax, style='*', sharex=False)
ax.set_title('MUV INT_TIME={}'.format(inttime))
fig.tight_layout()
In [24]:
fig, axes = subplots(nrows=len(inttimes), figsize=(13,14))
for ax,inttime in zip(axes, inttimes):
fuv[fuv.INT_TIME==inttime]['DET_TEMP'].plot(ax=ax, style='*', sharex=False)
ax.set_title('FUV INT_TIME={}'.format(inttime))
fig.tight_layout()
Do we have some flip-flopping state the instrument is in, creating different modes of detector efficiency?