Marcus Andrey Study
Ler em português Library

Motion and Interaction · Step 3 · 7 min read

Gestures and direct manipulation

When you drag, swipe or pinch, the interface should move with your hand, not after it. What direct manipulation means, the feedback each gesture owes, why a gesture is always a shortcut and never the only way, and what WCAG 2.2 asks for dragging and multi-finger gestures.

In a gesture, the motion isn't feedback on the action. The motion is the action. When you drag a card, it moves because your finger moves; when you let go, it settles. If the card lags behind, jumps or keeps going after you stop, the feeling of holding it is gone. Gestures are where motion design meets the hand, and the rules are stricter than anywhere else.

What you'll be able to do after reading:

  • Explain direct manipulation and why it feels natural
  • Say what feedback a gesture owes during, at the end of, and after the movement
  • Design a gesture as a shortcut with a visible alternative
  • Check a gesture against WCAG 2.5.1, 2.5.2 and 2.5.7

1. Direct manipulation

Ben Shneiderman named the idea in 1983, when most computers were still used by typing commands (Shneiderman, Direct Manipulation: A Step Beyond Programming Languages, IEEE Computer, 1983, doi:10.1109/MC.1983.1654471). NN/g's definition today: an interaction style in which people "act on displayed objects of interest using physical, incremental, and reversible actions whose effects are immediately visible on the screen" (NN/g, Direct Manipulation: Definition, 2016).

Every word of that definition is a motion rule:

  • Physical: you move the thing itself, not a command about it.
  • Incremental: it moves a little for every little move of yours, so you can stop at any point.
  • Reversible: move back, and it goes back.
  • Immediately visible: there's no gap between your hand and the screen.

2. Swipe a row, with no script

  • Ana LimaThe new icons are in the file
    ⋯
    ArchiveMark as unreadDelete
  • Rui CostaCan we move the review to Friday?
    ⋯
    ArchiveMark as unreadDelete

Swipe a row left with a finger or a trackpad, or Tab to it and use the arrow keys. The ⋯ button does the same with one tap.

Swipe to reveal, with no script: each row is a small sideways scroller that snaps to "content" or "action". The row follows your finger exactly and stops where you let go, which is what makes it feel direct. The ⋯ menu is the single-tap way to do the same thing, for anyone who can't or won't swipe.Live CSS: scroll-snap-type, overscroll-behavior, <details>. Demo: Marcus Andrey, CC BY 4.0.

The figure uses the browser's own scrolling: each row is a small sideways scroller that snaps to "content" or to "action" (MDN, CSS scroll snap). No animation is written at all, and that's the lesson. Native scrolling already follows the finger exactly, carries on with the speed you gave it, and settles at a snap point. Custom gesture code often does worse.

These are the rules that make the row:

.mo-b-row {
  display: grid; grid-template-columns: 100% 6.5rem;
  overflow-x: auto; scroll-snap-type: x mandatory; overscroll-behavior-x: contain;
  scrollbar-width: none; border-radius: 10px; background: var(--accent-fill);
}

overscroll-behavior-x: contain stops the swipe from leaking out to the page, so a swipe at the end of the row doesn't start a "back" gesture in the browser (MDN, overscroll-behavior). Under reduced motion nothing changes, because no extra motion was added: the movement belongs to the person's hand.


3. Feedback during, at the end, after

GestureFeedback while it happensWhat if it goes wrongThe one-tap way
TapPressed state at onceSlide off before lifting: nothing happens (WCAG 2.5.2)It is the one-tap way
Long pressSomething grows or lifts, so people know to keep holdingLet go early: nothing happensThe same actions in a visible menu
Swipe a rowThe row follows the finger; the action shows underneathUndo, or a confirm for destructive actionsA ⋯ button with the same actions (WCAG 2.5.1)
Drag to reorderThe item lifts, the others make room as it movesDrop outside: it goes back where it wasMove up and Move down buttons (WCAG 2.5.7)
Pinch to zoomThe content scales under the fingers, in step with themPinch back; a reset button+ and − buttons (WCAG 2.5.1)
Swipe back from the edgeThe screen slides with the finger, the one behind showsLet go early: the screen springs backA Back button, always there
Six common gestures and what each owes the person doing it: feedback that follows the hand, a way out, and a simple alternative. A gesture is a shortcut; the alternative is what makes it fair.Table: Marcus Andrey, CC BY 4.0. Criteria: W3C, WCAG 2.2 (2.5.1 Pointer Gestures, 2.5.2 Pointer Cancellation, 2.5.7 Dragging Movements).

Apple's guidelines describe the job well: as people perform a gesture, give feedback "that helps them predict its results" and, if needed, "communicates the extent and type of movement required to complete the action" (Apple HIG, Gestures). In practice, three moments:

  • During: the object moves with the hand, one to one. A list row slides as far as the finger, a sheet follows a downward drag. Show what will happen if they let go now: the Archive label grows, the colour fills.
  • At the end: it settles. A release past the threshold completes the action; a release before it springs back. A spring is the right tool here, because it can carry on from the speed of the hand (Springs and physics).
  • After: confirm, and offer a way back. Undo beats "Are you sure?" for anything that can be undone.

Apple also asks you to show when a gesture isn't available. If a locked item can't be dragged, it should resist or say so, or people will think the app froze (Apple HIG, Gestures).


4. A gesture is a shortcut, never the only way

Gestures are invisible. Nothing on a list row says "swipe me". And not everyone can perform them: a tremor, a head pointer, a switch device, a mouth stick, or just one hand busy with a coffee. WCAG 2.2 has three criteria for this:

  • 2.5.1 Pointer Gestures (A): anything done with a multi-finger or path-based gesture (pinch, swipe along a path) can also be done with a single pointer without a path, like a tap on a button (W3C, Understanding 2.5.1).
  • 2.5.2 Pointer Cancellation (A): the action happens on release, not on touch down, so people can slide off to cancel; or it can be undone (W3C, Understanding 2.5.2).
  • 2.5.7 Dragging Movements (AA, new in 2.2): anything done by dragging can also be done with a single pointer without dragging, like Move up and Move down buttons (W3C, Understanding 2.5.7).

Apple's guidelines say the same in design words: give people more than one way to interact, and use shortcut gestures to supplement standard ones, not replace them. Their example is the Back button: there's a swipe from the edge to go back, and the button stays (Apple HIG, Gestures). That's why every row in the figure has a ⋯ button. Make it at least 24 by 24 CSS pixels, or give it enough space around it (W3C, Understanding 2.5.8 Target Size); the one in the figure is 44.


5. Don't fight the platform's gestures

People expect a swipe from the screen edge to go back, a pull at the top to refresh, a pinch to zoom. Apple advises against using a familiar gesture for something unique to your app, and against inventing a new gesture for a standard action (Apple HIG, Gestures). On the web, the CSS touch-action property tells the browser which touch gestures your element handles itself, so you only take over what you really need (MDN, touch-action).


6. Common mistakes

  • Swipe-to-delete with no visible way to delete and no undo.
  • An object that lags behind the finger, or snaps back with a fixed animation that ignores how fast it was thrown.
  • Triggering on touch down, so a slip becomes an action.
  • A custom gesture that overrides the system's "back".
  • Reorder by drag only, with no buttons.

7. Vocabulary

Term Meaning
Direct manipulation Acting on the object itself, with small, reversible moves you see at once
Path-based gesture One where the route of the finger matters, like a swipe
Multipoint gesture One that needs more than one finger, like a pinch
Threshold How far a gesture must go before letting go completes it
Scroll snapping Scrolling that settles on set positions
Single-pointer alternative The same action with one tap or click, no path, no dragging

8. Try it (no tools needed)

  1. List every gesture in an app you know: swipes, long presses, drags, pinches.
  2. For each one, find the button that does the same thing. Mark the ones with none.
  3. Try each gesture slowly and stop halfway. Does the screen tell you what will happen if you let go?

Reflection questions: Which of your product's gestures would a new user never find? What breaks if someone can only tap?


9. If you already practise

  • Spec each gesture with its threshold, its settle motion and its alternative, in the same table as the figure.
  • Test gestures with a stylus, a mouse and the keyboard, not only a finger.
  • Prefer native scrolling, snapping and system sheets before writing gesture code; they already feel right and respect the platform.

Checklist

Learn more

Books, courses and videos to go deeper

Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.

Springs and physics · Anatomy of a microinteraction · Accessibility fundamentals · WCAG and motion · The spatial model of an interface

Saved only on this device.

↑ ↓ to move · Enter to open · Esc to close