We have:
$$\mathbb{E}[Z]= \int\ell(x)p(x)\ \mathrm{d}x.$$We approximate this as:
$$\mathbb{E}[Z]\approx \int \bar{\ell}(x)p(x)\ \mathrm{d}x,$$where the function $\bar{\ell}$ is the mean of the Gaussian process over $\exp(\log\ell)$.
We expand the first term to obtain:
$$ \begin{align*} E[m_\ell|x_s] &= \int K_{\exp(\log\ell)}(x,\mathbf{x}_c)K_{\exp(\log\ell)}(\mathbf{x}_c,\mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)p(x)\ \mathrm{d}x\\\\ &= \left(\int K_{\exp(\log\ell)}(x,\mathbf{x}_c)p(x)\ \mathrm{d}x\right)\ K_{\exp(\log\ell)}(\mathbf{x}_c,\mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)\\\\ &= \left(\int K_{\exp(\log\ell)}(x, \mathbf{x}_c)p(x)\ \mathrm{d}x\right)\ \alpha(\mathbf{x}_c) \end{align*} $$where $\alpha(\mathbf{x}_c) = K_{\exp(\log\ell)}(\mathbf{x}_c, \mathbf{x}_c)^{-1}\bar{\ell}(\mathbf{x}_c)$.
Assuming $K_{\exp(\log\ell)}$ is a Gaussian kernel, and $p(x)$ is a Gaussian density with mean $\mu$ and covariance $\Sigma$, the integral can be expressed analytically as follows, from [O13]_:
$$ \begin{align*} \int K_{\exp(\log\ell)}(x, \mathbf{x}_c)p(x)\ \mathrm{d}x&=\int h_\ell^2 \mathcal{N}\left(\mathbf{x}_c\ \big\vert\ x, W_\ell\right)\mathcal{N}\left(x\ \big\vert\ \mu, \Sigma\right)\ \mathrm{d}x\\\\ &= h_\ell^2 \mathcal{N}\left(\mathbf{x}_c\ \big\vert\ \mu, W_\ell + \Sigma\right) \end{align*} $$