Mh I don't have that problem with both of my controllers in Samurai Shodown so maybe you got a defective controller?
What you could try is contacting Stadia support for more troubleshooting steps, but maybe that one has to be repaired or you need a replacement, but they are best to help you with that.