A General Derivative Identity for the Conditional Expectation with Focus on the Exponential Family

Consider a pair of random vectors \( (\mathbf{X},\mathbf{Y}) \) and the conditional expectation operator \( \mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}] \). This work studies analytic properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain \( \mathbf{U} \leftrightarrow \mathbf{X} \leftrightarrow \mathbf{Y} \), a compact expression for the Jacobian matrix of \( \mathbb{E}[\mathbf{U}|\mathbf{Y} = \mathbf{y}] \) is derived. In the second part of the paper, the main identity is specialized to the exponential family. Moreover, via various choices of the random vector \(\mathbf{U}\), the new identity is used to recover and generalize several known identities and derive some new ones. As a first example, a connection between the Jacobian of \(\mathbb{E}[\mathbf{X}|\mathbf{Y}=\mathbf{y}]\) and the conditional variance is established. As a second example, a recursive expression between higher order conditional expectations is found, which is shown to lead to a generalization of the Tweedy's identity. Finally, as a third example, it is shown that the \(k\)-th order derivative of the conditional expectation is proportional to the \((k+1)\)-th order conditional cumulant.