Is it possible your ISP is throttling Stadia traffic? That may sound paranoid, but ISPs typically have clauses that allow them to do basically anything they want to your Internet traffic, and if they have noticed you using hundreds of gigabytes of data on Stadia, they may have decided to reduce the bandwidth they allow for that service. That would be a scummy move, but many ISPs are indeed scummy.
While playing, if you press the Stadia button, you can check the network connection quality. I would recommend checking it as soon as the game launches and then, once troubles begin, check it again. If the network connection quality has dropped, and nothing else on your local network has changed, like no one else in the household has started streaming anything, that is suspicious.
What is odd is you mentioned it works fine over Wi-Fi on your Chromecast Ultra, but with degraded quality. You should still be able to get a 4K stream, albeit with a bit more latency and a bit less stability, if your Wi-Fi is good. You do not technically need it to be wired.
If you can possibly rule out all network issues, it could just be some issue with the Ethernet port on the Chromecast Ultra's power adapter.
Is there a chance something is wrong with your ethernet cable? That might be another explanation.