Once I started a game in the Chromecast, the game does not recognize any pressed buttons.
So I am not able to get past the "press A button" in destiny for example. However, once I press the stadia button, I can access the menu, and it recognizes all buttons.
As a result I am not able to play on the Chromecast.
And the fix mentioned here: https://community.stadia.com/t5/The-Stadia-Controller/Controller-not-available-in-game/td-p/5277 worked for me.