So apparently this does not count as a bug, since it is just the letter being too big...

Although it looks like there is enough space and it sometimes creates an empty line not only right above the Umlaut, but also above the prior line that does not contain an Umlaut...
Either way, if anyone ever has an issue like this....what I'm doing now is to use the regular A, U, O in the text and then create a seperate text frame with the Umlaut Ä, Ü, Ö and place it on top of the other letter. Using a different color at first to be able to see that I get the position very exact on top of the other.
Not a very smooth solution, but the only thing I could come up with