Recently the place I work for sent us all VR headsets. I decided to see what it
would take to use that headset to make my camera show a virtual avatar instead
of my meat body face. This is the story of my journey through chaining things
together to make work meetings a bit more fun by using a 3D avatar instead of
myself in some of them.
[This post uses SVG for diagrams to help explain what's going on here. You may
need to use a browser with SVG support in order to get the best experience with
this article. All the diagrams will be explained after the fact so that people
using screen readers are not left out.](conversation://Mara/hacker)
<blockquoteclass="twitter-tweet"><plang="en"dir="ltr">Working at <ahref="https://twitter.com/Tailscale?ref_src=twsrc%5Etfw">@Tailscale</a> is great. They sent us all an Oculus Quest 2! <ahref="https://t.co/dDhbwO9cFd">pic.twitter.com/dDhbwO9cFd</a></p>— Cadey A. Ratio (@theprincessxena) <ahref="https://twitter.com/theprincessxena/status/1362871906597224456?ref_src=twsrc%5Etfw">February 19, 2021</a></blockquote><scriptasyncsrc="https://platform.twitter.com/widgets.js"charset="utf-8"></script>
So, let's cover the basics from a high level. At a high level a webcam is just
a video source that may or may not have a microphone attached to it. So in
order to get my avatar to show up in a video call, I need some way to make some
window on my computer act as a webcam. This will make the overall dependency
list look like this (for those of you using screen readers I will describe
this diagram below):
VRChat renders to the Desktop which is picked up by OBS which has the ability
to pretend to be a webcam, which is finally picked up by Google Meet.
If the VR headset that I got from work was a tethered to the PC kind of VR
headset like the Valve Index or HTC Vive, the next steps would involve full
body tracking or something so that I could have my movements in real life
transfer into movements that my avatar makes.
However, the VR headset we got sent was an Oculus Quest 2. This is a
_standalone_ VR headset that is basically an Android tablet that you strap
to your face. This makes things a bit more technically challenging because
now you need some way to get the video to the headset and the motion tracking
data from the headset and to the computer at 90 times per second. This requires
a bit more cleverness.
The Oculus desktop software ships with a feature called Oculus Link that allows
you to use a gaming PC to render the VR data to your headset by sending the
video streams over USB. I had to dig around for a compatible cable (It needs to
be a specific kind of USB-3 to USB-C-3 cable with at least 5 gigabits per
second of transfer capacity) since the ones that
[Oculus sells](https://www.oculus.com/accessories/oculus-link/) are both at
least CAD$110 and out of stock anywhere I can find them in Canada. The 0.75
meter long cable I had been using was good enough to get me through the first
couple days of experimenting with VR, but it was clear that a better solution
was needed.
I did some digging and found a bit of software called
[ALVR](https://github.com/alvr-org/alvr#readme) that claimed to let me do VR
from my computer wirelessly. So I set it up on the Quest and on my tower,
which brought up the dependency graph to this:
ALVR talks with its counterpart on the Quest. This allows you to stream the VR
video and audio bidirectionally. You also need to bring Virtual Audio Cable
into the setup so that you can hear stuff in the game and so that other people
can hear you using the headset mic. However, ALVR is not available on the Quest
store. You need to install [SideQuest](https://sidequestvr.com/setup-howto) for
that.
[SideQuest lets you sideload Android APK files to your Quest 2 because the
Quest 2 is basically an Android tablet that you strap to your face!](conversation://Mara/happy)
So I used SideQuest to install the ALVR client on my Quest 2, and then I opened
up VRChat and was able to do everything I was able to do with the wired cable.
It worked beautifully until it didn't. I started running into issues with the
video stream just dying. The foveated encoding (tl;dr: attempting to hack the
image quality based on how eyes work so you don't notice the artifacting as
much) could only do so much and it just ended up not working. Even when I was
only doing it for short amounts of time. There is a lot of WiFi noise in my
apartment or something and it was really interfering with ALVR's stream
encoding. The latency was also noticeable after a bit.
However, when it worked it worked beautifully. I had to upgrade to the nightly
build of ALVR in order to get game audio and the headset mic working, but once
it all worked it was really convenient. I could walk around my apartment and
I'd also walk around in-game.
A friend told me that the best experience I could have with wireless VR using a
Quest 2 would be to use [Virtual Desktop](https://www.vrdesktop.net). Apparently
Virtual Desktop has a
[patch that enables SteamVR support](https://sidequestvr.com/app/16), so I
purchased Virtual Desktop on a whim and decided to give it a go.
Virtual Desktop made ALVR look like a tech demo. All of the latency issues were
solved instantly. Virtual Desktop also made it convenient for me to access my
tower's monitors while in VR, and it has the best typing experience in VR that
I've ever used.
This brings the dependency graph up to this:
Now all that was left was to make the camera view look somewhat like it does
when I'm using my work laptop's webcam to make video calls. I started out by taking a picture of my office from about the angle that my laptop sits at.
