The equation for the expected variance is given by Equation 13 of [OD12]_:
$$ \begin{align*} E[V_Z|\log\ell_{s,a}] = S(Z\ |\ \log\ell_{s}) - \int m(Z\ |\ \log\ell_{s,a})^2 \mathcal{N}\left( \log\ell_a \ \big\vert\ \hat{m}_a, \hat{C}_a\right)\ \mathrm{d}\log\ell_a \end{align*} $$From before, we have:
$$ \begin{align*} m(Z\ |\ \log\ell_{s,a})&=E[m_\ell|x_{s,a}]\\\\ &= h_\ell^2 \mathcal{N}\left(\mathbf{x}_{c,a}\ \big\vert\ \mu, W_\ell + \Sigma\right)K_{\exp(\log\ell)}(\mathbf{x}_{c,a}, \mathbf{x}_{c,a})^{-1}\bar{\ell}(\mathbf{x}_{c,a}) \end{align*} $$For brevity, let:
$$ A_{c,a} = h_\ell^2 \mathcal{N}\left(\mathbf{x}_{c,a}\ \big\vert\ \mu, W_\ell + \Sigma\right)K_{\exp(\log\ell)}(\mathbf{x}_{c,a}, \mathbf{x}_{c,a})^{-1} $$such that $m(Z\ |\ \log\ell_{s,a})=A_{c,a}\bar{\ell}(\mathbf{x}_{c,a})$.
We have that $\int\exp(cy)\mathcal{N}\left( y \ \big\vert\ m, S \right)=\exp(cm + \frac{1}{2}c^2 S)$, which allows us to marginalize out $\bar{\ell}_a$. Thus, we need to break apart $\bar{\ell}(\mathbf{x}_{c,a})=[\bar{\ell}_c; \bar{\ell}_a]$, giving:
$$ \begin{align*} m(Z\ |\ \log\ell_{s,a})^2 &= \left([A_c, A_a][\bar{\ell}_c; \bar{\ell}_a]\right)^2\\\\ &= \left( A_c\bar{\ell}_c + A_a\bar{\ell}_a \right)^2\\\\ &= (A_c\bar{\ell}_c)^2 + 2(A_c\bar{\ell}_c)(A_a\bar{\ell}_a) + (A_a\bar{\ell}_a)^2 \end{align*} $$Plugging this into the expected squared mean:
$$ \begin{align*} E[m_Z^2|\log\ell_{s,a}] &= \int m(Z\ |\ \log\ell_{s,a})^2 \mathcal{N}\left( \log\ell_a \ \big\vert\ \hat{m}_a, \hat{C}_a\right)\ \mathrm{d}\log\ell_a\\\\ &= (A_c\bar{\ell}_c)^2 + \int \left( 2(A_c\bar{\ell}_c)(A_a\bar{\ell}_a) + (A_a\bar{\ell}_a)^2 \right)p(\log\ell_a) \ \mathrm{d}\log\ell_a\\ &= (A_c\bar{\ell}_c)^2 + 2A_c\bar{\ell}_cA_a\int \bar{\ell}_ap(\log\ell_a)\ \mathrm{d}\log\ell_a + A_a^2\int \bar{\ell}_a^2 p(\log\ell_a) \ \mathrm{d}\log\ell_a\\ &= (A_c\bar{\ell}_c)^2 + 2A_c\bar{\ell}_cA_a\int \exp(\log\ell_a)p(\log\ell_a)\ \mathrm{d}\log\ell_a + A_a^2\int \exp(2 \log\ell_a) p(\log\ell_a) \ \mathrm{d}\log\ell_a \end{align*} $$Using the identity $\int\exp(cy)\mathcal{N}\left( y \ \big\vert\ m, S \right)=\exp(cm + \frac{1}{2}c^2 S)$, we have:
$$ E[m_Z^2|\log\ell_{s,a}] = A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right) $$Putting it all together, we obtain:
$$ \begin{align*} E[V_Z|\log\ell_{s,a}] &= S(Z\ |\ \log\ell_{s}) - \left( A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\right)\\\\ &= V_Z + m_Z^2 - \left( A_c^2\bar{\ell}_c^2 + 2A_c\bar{\ell}_cA_a\exp\left(\hat{m}_a + \frac{1}{2}\hat{C}_a\right) + A_a^2\exp\left(2\hat{m}_a + 2\hat{C}_a\right)\right) \end{align*} $$where $S(Z\ |\ \log\ell_s) = V_Z + m_Z^2$ from Equation 9 of [OD12]_.