workaround infinite loop in template instantiation (fixes e.g. Eigen code)

some template code trips up our instantiation code leading to an
infinite recursion. this patch adds a depth counter to prevent
this.

furthermore, a unit test is added which reproduces this issues reliably
so someone with more knowledge can write a proper fix

BUG: 269352