4729

$$ o_j = \max(i_j, \theta) $$

We want $do_j/d\theta$ I guess?

So, let's break the formula down into two piece-wise linear parts:

When $i_j < \theta$:

$$ o_j = \theta $$

And therefore:

$$ do_j/d\theta = 1 $$

When $i_j > \theta$:

$$ o_j = i_j $$

And therefore:

$$ do_j/d\theta = 0 $$

So, the differnetial $do_j/d\theta$ is:

$$ do_j/d\theta = \begin{cases}1 \text{ when } i_j < \theta \\ 0 \text{ when } i_j > \theta \end{cases} $$

both threshold and threshold_value can change

So, our formula becomes:

$$ o_j = \begin{cases}\phi\,\text{when}\,i_j > \theta \\ i_j \,\text{when}i_j < \theta \end{cases} $$

Looking case by case, for the piece $i_j > \theta$, we have:

$$ o_j = \phi $$

Therefore:

$$ \frac{\partial o_j}{\partial \theta} = 0 $$

and:

$$ \frac{\partial o_j}{\partial \phi} = 1 $$

For the case $i_j < \theta$, we have:

$$ o_j = i_j $$

Therefore:

$$ \frac{\partial o_j}{\partial \theta} = 0 $$

and:

$$ \frac{\partial o_j}{\partial \phi} = 0 $$