Chapter 7: Mouse and keyboard input

JavaScript uses an event system with callback functions to handle user inputs.

("keydown", () => {
  . = `key: "${.}"`;
});

begin typing...

This creates a bit of a challenge. If our game loop is something like this...

function loop() {
  update();
  draw();
  requestAnimationFrame(loop);
}

loop();

...then where in that loop is the event firing? What is the order of operations when user inputs can come in at any time?

Reveal the answer

The answer is that they don't fire inside the loop at all. Event listeners run in the background, firing between frames whenever the player does something. The loop's job is just to read whatever state those handlers have written by the time fixedUpdate() runs.

Before we begin...

These examples assume a game loop with draw and fixedUpdate functions, as well as a DPI-scaled canvas and its boundingClientRect and CanvasRenderingContext2D. If that sounds unfamiliar to you, take a look at Chapter 3.

Keyboard events

Events update state

To bring keyboard events into the game loop, we need to change them from momentary actions into state that the loop can poll.

Some states span many frames. For example, this demo reads which keys are currently pressed. The state only updates when events fire.

Aside--where did { signal } in the code example come from?

Because many of the demos on this page register events on the global window object, they'll interfere with one another if we don't tear down the event listeners when each preview unmounts.

One way to do this is with removeEventListener:

const  = (: KeyboardEvent) => {
  const { ,  } = ;
  .();
  if ( &&  === "c") {
    ("keydown", );
  }
};
("keydown", );

But because it requires holding onto a reference to the callback function, removeEventListener always feels a bit clunky to use. And in our particular case for these demos, coming up with a way to figure out which event each callback in every demo is tied to sounds awful.

Thankfully, a much nicer alternative is provided by AbortController:

const {
  , // a function you can call anywhere
  , // one signal can abort many things
} = new ();

// log keydown events until "ctrl+c"
(
  "keydown",
  () => {
    const { ,  } = ;
    .();
    if ( &&  === "c") {
      ();
    }
  },
  {  },
);

You can pass the same abort signal to as many event listeners as you want, then call abort() from anywhere. This abort is totally unaware of the event types and fully decoupled from the event handlers, making it great for this use case.

A small downside we'll have to live with is that in each of the demos, you'll see a { signal } passed in to the addEventListener but you won't see where it comes from, as we need a way to call abort() on unmount outside of the demo.

For a deeper dive on this, see Don't Sleep on AbortController by Artem Zakharchenko.

Continuous vs discrete events

The previous demo showed an example of continuous, pollable state. But sometimes we need events to behave like one-shot blips that fire just when the user interacts. How can our pollable state object handle these?

The answer is to use event callbacks to set temporary event-related state (like which keys were just pressed this tick). Then reset() that state at the end of each fixedUpdate().

Mouse events

Let's look at how mouse events are usually handled for DOM elements, like an HTML <button>:

. = () => {
  . = `x: ${.}, y: ${.}`;
};

Unlike this example, the items in canvas games aren't DOM elements and they don't have event handlers. So how can we make them clickable?

Before we get there, let's make the canvas itself capable of responding to ~~mouse~~ pointer events.

Why not mouse events?

Because not everyone clicks stuff using a mouse! Mobile devices tend to use touch events, tablets might use a stylus, and there are also things like trackpads or assistive devices that support multi-finger gestures.

Pointer events are a hardware-agnostic way to model all of these.

There are many events for mouse-ish things the browser responds to. But if you want to focus on just a handful, the ones with links are worth knowing for browser games.

Legacy mouse events
- mousedown, mouseup, mousemove, mouseenter, mouseleave, mouseover, mouseout
Pointer events (the "modern" api for click-like events)
- pointerdown, pointerup, pointermove, pointerenter, pointerleave, pointerover, pointerout, pointercancel, gotpointercapture, lostpointercapture
- click is an odd one--it's a PointerEvent in modern browsers, but a MouseEvent in old ones. It generally fires right after pointerup, but can also be triggered programmatically, or by pressing Enter or Space when an interactive HTML element is focused.
Specialty events
- dblclick, contextmenu, auxclick
Scroll events
- wheel, mousewheel, scroll
Touchscreen/trackpad events
- touchstart, touchmove, touchend, touchcancel
Drag events
- dragstart, drag, dragend, dragenter, dragover, dragleave, drop
Non-standard Apple-only gesture events
- gesturestart, gesturechange, gestureend

Reading mouse state in the game loop

Just like with key presses in the previous section, our fixedUpdate() and draw() code never directly listen to pointer events. Instead there is a single global event listener for each event which sets the state that our fixedUpdate() and draw() can read.

Following the cursor position

The cursor position is continuous, pollable state that persists across many frames, just like keysDown. To set it, listen for pointermove and set (x, y) coordinates offset by the canvas' actual location on the screen. This way (0, 0) is the top left corner.

Hitbox tests

To detect if the user is clicking or hovering a specific in-game item, check if the cursor x/y position is within the bounds of that item at the time of the click. (This is the same logic we used to detect collisions in chapter 5.)

Cursor types

Browsers support dozens of built-in cursors [MDN], including an invisible one. Set canvas.style.cursor to change it.

Using right-click

The event that shows the browser's native right-click menu is called contextmenu. If your game relies on right clicks, then you probably don't want that menu popping up. For this, use preventDefault() on the contextmenu event.

Avoid overriding the right-click behavior without reason, though, since blocking the right-click menu is invasive to the user experience.

Customizing right-click behavior

Every PointerEvent also supports an event.button property. To specifically detect right clicks, listen for pointerdown events where event.button === 2.

("pointerdown", () => {
  if (. === 0) {
    // left click occurred
  }
  if (. === 1) {
    // middle/wheel click occurred
  }
  if (. === 2) {
    // right click occurred
  }
});

Hitbox test cheatsheet

Rectangles:

type  = { : number; : number };
type  = { : number; : number; : number; : number };

function (: , : ) {
  return (
    . >= . &&
    . <= . + . &&
    . >= . &&
    . <= . + .
  );
}

An odd shape defined by a canvas path:

To account for any current transforms, such as the DPI scaling, we need to map CSS pixel coordinates into the canvas' current transformed coordinate space before testing isPointInPath.

type  = { : number; : number };

function (: number, : number) {
  const  = new ();

  .(,  - 36);
  .( + 10,  - 10);
  .( + 38,  - 10);
  .( + 16,  + 6);
  .( + 24,  + 34);
  .(,  + 18);
  .( - 24,  + 34);
  .( - 16,  + 6);
  .( - 38,  - 10);
  .( - 10,  - 10);
  .();

  return ;
}

function (: CanvasRenderingContext2D, : , : Path2D) {
  const  = .();
  const { ,  } = new (., .).();
  return .(, , );
}

Circles:

The formula for this is:

{(x - h)}^{2} + {(y - k)}^{2} \leq r^{2}

where (x,y) are the coordinates for the point and (h,k) are the coordinates for the center of the circle.

type  = { : number; : number };
type  = { : number; : number; : number };

function (: , : ) {
  const {  } = ;
  const  = . - .; // (x – h)
  const  = . - .; // (y – k)
  return  ** 2 +  ** 2 <=  ** 2;
}

Arbitrary polygons:

If you shoot a horizontal ray to the right of the point (x, y) and count the number of intersections it has with each edge, an odd number of intersections means that the point is inside the polygon and an even number means it is outside.

A deeper look into this algorithm

Let's begin with a visualization. The polygon in question contains a point to test (the blue dot). The algorithm begins by casting a horizontal ray to the right of it, also in blue, then counting the number of edges it crosses. Edges whose y range contains the point's y become "relevant" candidates (drawn in yellowred). If the number of edges crossed by this ray is odd, then the point is inside the polygon.

1 / 11

Edge:irrelevantCrossings:0State:outside (even count)

(You can drag the point around to test it!)

Credit to Harald @hg42 for this writeup from which the visualization above is forked. If you'd like to know more about how the ray-casting point-in-polygon algorithm works, I recommend you take a look at that post and the related Wikipedia entry.

type  = { : number; : number };
type  = [number, number][];

function ({ ,  }: , : ) {
  let  = false;
  let  = .(-1)!;

  for (const  of ) {
    const [, ] = ;
    const [, ] = ;

    const  =  >  !==  > ;
    const  = (( - ) * ( - )) / ( - ) + ;
    const  =  &&  < ;

    if () {
       = !;
    }

     = ;
  }

  return ;
}

Pointer lock

Pointer lock [MDN] makes the cursor disappear and allows you to move it beyond the edges of the window. It changes the purpose of the mouse so that instead of looking at its x/y position on the screen, you only read its relative movements.

Why would you want this behavior? Picture a 3D first person shooter. There's no mouse cursor moving about the page--instead the world moves around you as you rotate in place. With pointer lock, you can move your mouse infinitely in any direction rather than being forced to stay in your display's boundaries.

The pointer events we'll be using in the next demo have movementX and movementY properties that track the deltas of the cursor's x and y positions between the current event and the previous event of the same type.

Pointer lock movement

Click the canvas to lock the pointer, then move your mouse to pan around. Press Escape when you're done to release the lock.

Working with world-space coordinates and cameras

In chapter 6 we used transforms to place in-game items in world space using a scaled coordinate system. Then we placed a camera over it to zoom and pan about that space.

In this chapter we'll need to invert those transforms in order to figure out what in-game location is being clicked or hovered. This will involve three transforms:

Event coordinates to screen space
Screen space coordinates to world space
Undo the camera to recover a world-space point

Let's start with the first (and simplest) transform. We need to normalize the event's coordinates into our canvas screen space. We also used this logic in the "Following the mouse cursor" demo above.

// a pointer event gives you window x/y coordinates.
// the canvas bounding rect tells you the position of the canvas in the window.
function (: PointerEvent, : DOMRect) {
  return {
    : . - .,
    : . - .,
  };
}

After that, we can take those coordinates and see where they would fall in world space. If you recall our world space transformation from the previous chapter, this transformation simply inverts it.

Chapter 6

function (
  : CanvasRenderingContext2D,
  : { : number; : number },
  : { : number; : number },
  : () => void,
) {
  const { , ,  } = (, );
  .();
  .(, );
  .(, );
  ();
  .();
}

Inverse

function (
  : { : number; : number },
  : { : number; : number },
  : { : number; : number },
) {
  const { , ,  } = (, );
  const  = new ()
    .(, ) 
    .(, ) 
    .();
  const  = new (., .).();
  return ;
}

Likewise, to get an in-camera point into world space, we can invert the camera.