A big part of the reason was that Facebook offered game studios a big upfront sum if they made their games work on whatever headset they were selling at the time in standalone mode with no major caveats. The headset only had an anemic mobile GPU, so was only capable of as much as mobile games were doing at the time. A bunch of studios took them up on this offer, and cut back their projects’ scope to be viable under the hardware constraints, so nearly everything that got made was gimmicky mobile-style minigames, and obviously that’s not what makes people want to drop hundreds of dollars on hardware, as they can get their fill by borrowing someone else’s headset for an hour.
Mobile GPUs have improved, so standalone headsets aren’t as terrible now, but we missed the expensive toy for enthusiasts and arcades phase and soured most people’s opinions by making their first VR experience shovelware.