Comment on The waiting is killing me

• HER0@beehaw.org ⁨3⁩ ⁨days⁩ ago

  For them to have things like:

  • high quality wireless video that is decoded on the headset
  • decent tracking without anything external (as opposed to using Lighthouse base stations, as used on the Index)
  • eye tracking, which enables the foveated streaming (and allows for a significant perceived quality boost)
  • on-device compositor with async reprojection (reducing simulation sickness from low frame rate or missed frames)

  There needs to be a decent amount of onboard processing, a full OS, and some memory/storage. Before prices exploded (which was certainly after the hardware was mostly/entirely finalized), it would have been only a small price increase (proportional to the rest of the hardware) to bump the RAM and storage a bit to allow for more games to work standalone.

  The passthrough cameras are really not a focus of the headset, they are a semi-convenient side effect of the tracking cameras. The actually-decent-quality passthrough is a separate add-on.

  If you want an AR screen and only need 3 DoF, would something like the XReal glasses line up better for you?

  source

  • ColeSloth@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago

    You vastly overestimate the amount of ram and storage and OS required for data exchange and foveated streaming.

    I own xreal one pro glasses. They show that your 2nd and 4th point are non issues that can be done with extremely limited on device processing power and no cameras or extra head tracking. The xreals do a fantastic job of those.

    The only issue with the xreals is that they aren’t wireless. Still fantastic for games using my steam deck, but not much of a point in using them to sit right in front of my desktop.

    source

    • HER0@beehaw.org ⁨3⁩ ⁨days⁩ ago

      Thanks for your perspective on this. I am going to go into a bunch of detail on why I disagree.

      Streaming video

      I didn’t say that streaming video requires a lot of RAM or storage, I said that streaming video, decent tracking, eye tracking, and the on-device compositor combined require “a decent amount of onboard processing, a full OS, and some memory/storage.”

      It is true that decoding a relatively low bitrate H.265 video doesn’t require a lot of RAM or storage on its own. It does add a bit to the processing requirements across the whole package, though.

      Tracking

      The XReal One Pro has 3 DoF unless if you add a single camera (XReal Eye) to do 6 DoF SLAM. It does benefit from hardware acceleration of the tracking.

      Still, the single tracking camera produces less data that needs to be processed, and limits the tracking quality. This limited quality is reflected in reviews:

      PC Magazine:

      With the XReal Eye, the XReal One and One Pro can do the same thing, in a very loose sense. Anchoring the virtual screen in front of my TV, I could indeed move closer to the screen and step around it. The picture would constantly move with me to some extent, though, making it shift back and forth a few feet when I walked around my living room. It felt like a half-step between 3DOF and 6DOF, where the head tracking is more accurate than without the Eye, but the position of the screen is taken as a suggestion more than a fixed point. This isn’t surprising, though, considering its calculations are based on the input of a single camera.

      Tom’s Hardware:

      However, the real magic of the Eye comes with its ability to infuse the One Pro with spatial 6 degrees of freedom (6DoF) tracking. With this capability, the virtual screen stays in place while you move around your environment. This feature is usually enabled by using multiple forward-looking cameras, but Xreal is implementing it with just one (monocular SLAM), which limits its performance. In other words, tracking isn’t as steady or jitter-free as devices like the Meta Quest 3 or Quest 3s.

      It may be that you don’t need the higher quality tracking with your needs and in AR glasses. This doesn’t work so well in VR.

      In comparison, the Steam Frame has 4 SLAM cameras (also used for tracking the controllers) and 2 eye tracking cameras, and doesn’t have dedicated hardware to accelerate the tracking. This means the tracking is implemented in software, where it must process 6 cameras (among other sensors) at low latencies and high refresh rates to keep the tracking quality relatively high.

      Compositor

      The XReal One Pro has hardware to smooth motion when displaying a relatively low resolution, 2D image in an apparent fixed position. Like with tracking, this is probably enough for AR glasses.

      Unlike the Steam Frame, XReal glasses aren’t capable of compositing arbitrary 3D scenes with depth buffers and/or motion vectors that are used to reproject the image with reduced artifacts compared to a naive implementation.

      The Steam Frame, again, isn’t benefiting from dedicated hardware for this. It does do some fun tricks to reduce latency of the compositor in the GPU.

      Overall package

      Combined, this is a lot of data to crunch that less powerful hardware like is found in the XReal One Pro cannot handle. The most popular headsets with similar needs use the Snapdragon XR series SoCs, which have hardware acceleration for SLAM tracking with many sensors, and basically requires a full OS (in practice, Android) to function as expected (requiring more storage and RAM as a result).

      The Steam Frame has similar needs to the other wireless VR headsets (with some different trade-offs), and is using a smartphone SoC without dedicated tracking hardware. It could be that it could cover all of its needs (outside of standalone functionality) with bespoke hardware that is a lot less powerful, but this would certainly be more expensive than an off-the-shelf solution, as the requirements are too great and the volume too low.

      It also makes sense to reuse SteamOS and SteamVR, which already had a lot of the functionality they need, instead of creating bespoke firmware. These add some overhead relative to the firmware running on XReal glasses.

      Since it is already using an older high end smartphone SoC to be able to handle all of this, plus all the other components (like displays, optics stack, cooling, audio drivers, IPD mechanism, modular strap, etc.), a bit more memory and storage wouldn’t have been a big price increase before prices exploded. Valve said as much in interviews, including this one: www.youtube.com/watch?v=_Ag2S8yvVak

      Early on, before we had a sense of the compute capabilities, we were picking a lower spec, but as more and more of the catalog came online, it turned into a scenario where we’re like, “well, this chip is powerful enough to run these- this level of game- if we had 16 gigs,” right?

      They landed on having the standalone capability because it was already in reach after choosing a SoC that met their other needs, and adding extra memory was an easy decision to make at the time. It makes sense, then, to increase the storage a bit, too. They could not have predicted that prices would explode, and the resulting pricing is their biggest regret with their latest hardware (as stated in the same interview).

      source

      • ColeSloth@discuss.tchncs.de ⁨2⁩ ⁨days⁩ ago

        Few issues. One, as stated explicitly. Don’t need 6dfov. Only 3dfov. Also, your reviews are old as heck. Firmware updates have completely changed how well they work. If I screen lock in place it stays fantasticly. You’re looking at reviews and data from before firmware version 1.9.

        The ram could be cut in half and the storage could drop to 16GB. Would have saved probably $350 at this point.

        source