www.enchantedage.com

When developing a PC program for the XNA framework, you usually find that you need to do mouse picking. This means taking the mouse coordinates, and turning them into a ray that you can then raycast in your world to see what, if anything, is being hit when the user clicks the mouse.

MSDN documents the function Viewport.Unproject(), which allows you to do exactly this. However, the function has gotten a bit of a bum rep, with developers claiming it doesn't do what it's supposed to do. However, it actually works just fine, and I am posting this tutorial to show how to use it.

The program displays a simple grid around the worldspace origin, with 20x20 one-unit squares from -10 to 10 in the X and Z directions. This grid lies along the plane with Y = 0, usually thought of as the "ground plane" in XNA programs. The camera initially is set to look down on this grid from above. However, you can move the camera using the first gamepad, or using WASD for movement and arrow keys for turning. The camera does not turn with the mouse (like an FPS camera would), because the purpose of the program is to show how different mouse coordinates all project perfectly into the world.

The camera movement is all found in the Camera class, the function Update(). Update() takes the gamepad and keyboard state as input, as well as the duration of time for the current time step, and updates the camera position/orientation accordingly.

In the upper left corner, the screen-space coordinates of the mouse (the X and Y in the window) are displayed in red. If the red crosshair that shows the mouse location points at the ground plane (Y = 0), then the coordinates for where the crosshair hits the ground are displayed. If the crosshairs don't hit the ground (because the camera is looking up, say), then that fact is displayed instead.

As you move the mouse over the window, the crosshairs show where the mouse cursor is, and you will see a yellow line rising up from the ground, showing where the mouse ray hit the ground. As you can observe, the registration between the crosshairs and the lower end of this line is perfect, showing how Viewport.Unproject() allows you to easily create a ray to use for raycasting into your world.

The code used for the raycast is as follows:

      //  Unproject the screen space mouse coordinate into model space 
      //  coordinates. Because the world space matrix is identity, this 
      //  gives the coordinates in world space.
      Viewport vp = GraphicsDevice.Viewport;
      //  Note the order of the parameters! Projection first.
      Vector3 pos1 = vp.Unproject(new Vector3(ms.X, ms.Y, 0), camera_.Projection, camera_.View, camera_.World);
      Vector3 pos2 = vp.Unproject(new Vector3(ms.X, ms.Y, 1), camera_.Projection, camera_.View, camera_.World);
      Vector3 dir = Vector3.Normalize(pos2 - pos1);

You can find this code inside Game1.Update(GameTime) at line 105 and forward. Using the current viewport of the graphics device, we put in two coordinates; one with a Z = 0 value (which represents the near clipping plane specified when we create our projection matrix), and one with a Z = 1 value, which represents the far clipping plane. In the current program we pass 1 and 1000 for these values, so the unprojected "pos1" value will be 1 unit in front of the camera, and the unprojected "pos2" value will be 1000 units from the camera along the depth axis.

A ray generally needs a start location and a unit-length direction vector, so we subtract the start location from the end location and normalize it to get the direction. The start is simply the near plane location.

You may ask yourself why we don't use the camera position as the start location? The answer is that objects that are between the camera and the near clip plane would not be drawn, and if we started at the camera position, the user may accidentally click on something he cannot see, which would be confusing.

So, download the sample program, run it, and check it out. Viewport.Unproject(), as provided by the XNA framework, works a treat for figuring out what the user is clicking on, aiming at, or interacting with in your XNA game!