Are you on chromecast? There is a volume setting in the Stadia button menu, I assume you have it turned all the way up?
I've never had volume issues myself with my setup.
Thanks for your input, I do have it turned up all the way. It has only happened 2-3 times, but I am pretty sure it was a server issue.