Steam Decks run Linux. (The specific DE is KDE Plasma I think.) So you can find answers by searching for “Linux” if searching for “Steam Deck” doesn’t get results.

One way is to enable the “Compose” key which lets you enter special characters or sequences by typing switches of more commonly-available characters. I think the Steamdeck OS has a setting for this; but I don’t have one so I can’t check.

For letters with umlauts you press (and release) Compose, then type a double quote (need to hold shift for this part), then type a vowel.

For reference Wikipedia has a list of Common Compose Combinations

Alternatively if you can map an AltGr key I’ve read you can type umlauts by typing AltGr+[ and then typing a vowel. There might be a setting for this too.