OK, you were sort of right in that I looked for the haptics in the wrong place. I had to go to the stick settings, look at the left stick settings and I found a haptics setting there! It was set to something called “activator” so I set that to off and it all stopped.

Solved! Thank you for the pointers!