Sounds like your WiFi isn’t the best wherever your controller is located. Do other wireless devices have issues in that room? Run a speed test from your phone? Connect your controller to a mobile hotspot?
Not sure if this will help but worth a shot. I encountered something similar but it seems to be resolved now. I port forwarded 44700-44899 (TCP and UDP), https://support.google.com/stadia/answer/9595943.
Nothing is resolved now 😞
I have 2000 ms latency. Sometimes worse playing on Chrome with Stadia controller. 47 Mbits download speed and a ping of 14 ms.
Played on 2 different laptops. Same thing. Unplayable.
The laptop was only device connected to WiFi
Played on tablet in hotel (40 Mbits) and it worked fine
I'm going insane here. What the heck is wrong with my router????
Chromecast ultra works fine btw but I don't want to be restricted to my TV. That's not the point
Hey, not 100% sure it will help, but you might try going through this quick guide: https://forms.gle/4cb5j1JWrUh1JfdYA
It's not really specifically directed at controller issues, but most of what is described regarding general WiFi problems should apply to your situation.