VR & AR Development with Unity & Unreal Engine

Choose between Unity and Unreal for your first XR project, then build one small test: a room, the player’s head, and one object that reacts when you look at it.

Your first VR or AR project should not be a game. It should be a test that proves three things work: the headset or phone tracks your head, the scene looks the right size, and your code can tell what the player is looking at.

Once those work, everything else is building on a foundation you trust. This guide helps you pick an engine, then walks you through that one test in either Unity or Unreal.

XR moves fast. Package names, templates, and supported devices change between engine versions. This guide cannot tell you which headsets a given template supports this month. Before you start, check the current XR documentation for your exact engine version and your target device’s developer documentation.

VR or AR first?

VR puts the player inside a fully virtual space with a headset. You control everything they see.

AR places virtual objects over the real world, through a phone camera or a headset with passthrough. You share the frame with the player’s room.

For a first test, pick the one you can run today. If you own a VR headset that works with your computer or supports standalone builds, start with VR. If you only have a recent phone, AR may be the faster path. Do not buy hardware for this test until you have read your engine’s current device support pages.

Choosing Unity or Unreal

Both engines can do serious XR work. The better question is which one fits you.

Lean toward Unity if:

• You already know C# or Unity’s editor.
• You want to target phone AR. Unity’s documentation describes an AR Mobile template built on AR Foundation, set up for iOS and Android.
• You prefer adding features through packages you can inspect and swap.

Lean toward Unreal if:

• You already know Blueprints or C++.
• You want a VR starting point with movement and grabbing already built. Epic’s VR Template includes teleport locomotion and grabbing logic.
• You want high-end visuals on PC VR and are comfortable tuning for them.

If you know neither engine, pick the one with the template closest to your idea, and commit for this test. Switching halfway teaches you less than finishing in either.

The setup in Unity

Unity’s manual lists XR templates in the Unity Hub. The VR template sets up the project with URP, OpenXR, and the XR Interaction Toolkit, plus an example scene. The Mixed Reality template adds AR Foundation alongside those.

Steps:

1. Install the Unity Editor, plus any platform modules your device needs. Unity’s docs note, for example, that Android-based headsets need the Android module.
2. In the Unity Hub, create a new project from the VR template, or the AR Mobile template for phone AR.
3. After the project opens, review the XR Plug-in Management settings and confirm the plug-in for your target device is enabled. The docs point to this system for configuring providers.
4. Open the template’s sample scene and run it on your device to confirm tracking works before you change anything.

Some device makers require a developer account or extra setup. Unity’s docs say this is outside their scope, so check your device maker’s developer pages.

The setup in Unreal

Epic’s documentation says the VR Template uses the OpenXR framework through Unreal’s OpenXR plugin, so its logic works across supported devices without platform-specific checks.

Steps:

1. Create a new project from the VR Template in the project browser.
2. Make sure your headset’s OpenXR runtime is installed and active on your computer, following your device maker’s instructions.
3. Open the template’s map and use the VR Preview play option to test in your headset.
4. If your device needs a vendor plugin on top of OpenXR, follow that vendor’s current Unreal setup guide.

Confirm the template runs before you build anything new.

Build the test room

Now make your own scene. Keep it small.

1. Create a new empty scene or level.
2. Add a floor and four walls, sized like a real room. Use the engine’s default units and your own sense of a room you know.
3. Add one light.
4. Bring in the player rig from the template. In Unity, that is the XR rig the template sets up. In Unreal, it is the template’s VR pawn. This rig moves a camera with your real head.
5. Place one object on a table or pedestal in front of where the player starts. A simple shape, like a cube or a vase, works best.

Put on the headset and look around. Ask yourself:

• Does the floor feel like it is at your feet?
• Does the room feel the size you meant?
• Does the object look like it is at arm’s length or across the room, as you intended?

Scale problems are much easier to feel in a headset than to see on a monitor. If the room feels like a dollhouse or a cathedral, fix the scale now.

Make the object react when you look at it

This is the core of your test. Your code needs to know where the player’s head points.

The method is the same in both engines. Each frame, cast a ray from the camera’s position along the direction it faces. If the ray hits your object, change something about it.

In Unity, write a short C# script on the object or on a manager. Each frame, use a physics raycast from the main camera’s position along its forward direction. If the hit object is your target, set a highlight, such as swapping its material or raising emission. When the ray stops hitting it, turn the highlight off. Make sure your object has a collider, or the ray will pass through.

In Unreal, use a line trace in Blueprints on the pawn or the object. Start the trace at the camera’s world location and end it a set distance along the camera’s forward vector. If the trace hits your object, change its material or scale it slightly. Make sure the object’s collision responds to the trace channel you use.

Add one more thing: a short delay before the highlight triggers. Gaze tends to wander, so a brief hold makes the reaction feel intentional rather than flickery.

Test it. Look away, look back. The object should respond every time, and only when you look at it.

AR version of the same test

If you chose AR, the structure stays the same. Instead of building walls, you let the device detect a surface, like a table or floor. Place your object on it. Then cast a ray from the device camera and highlight the object when the camera points at it.

In Unity, AR Foundation handles surface detection in the AR templates. Check the current AR Foundation docs for which features your target device supports.

Comfort as a design choice

Once the test works, you will want movement. How players move is a design decision with real trade-offs.

Snap turning rotates the view in fixed steps. Smooth turning rotates continuously. Teleporting jumps the player to a new spot. Some players prefer one, some another. Epic’s VR Template includes teleport and snap turn logic you can study.

Offer more than one option when you can, and let players choose. Test with several people, not just yourself.

What to do next

When your test passes, you have a working foundation: tracking, scale, and gaze. Next, try one addition at a time. Grab the object with a controller. Open a door by looking at it. Play a sound when the highlight triggers.

Keep each step small, test it in the headset, and check your engine’s XR docs whenever you update versions.