Motion and Interaction · Step 3 · 7 min read
Gestures and direct manipulation
When you drag, swipe or pinch, the interface should move with your hand, not after it. What direct manipulation means, the feedback each gesture owes, why a gesture is always a shortcut and never the only way, and what WCAG 2.2 asks for dragging and multi-finger gestures.
In a gesture, the motion isn't feedback on the action. The motion is the action. When you drag a card, it moves because your finger moves; when you let go, it settles. If the card lags behind, jumps or keeps going after you stop, the feeling of holding it is gone. Gestures are where motion design meets the hand, and the rules are stricter than anywhere else.
What you'll be able to do after reading:
- Explain direct manipulation and why it feels natural
- Say what feedback a gesture owes during, at the end of, and after the movement
- Design a gesture as a shortcut with a visible alternative
- Check a gesture against WCAG 2.5.1, 2.5.2 and 2.5.7
1. Direct manipulation
Ben Shneiderman named the idea in 1983, when most computers were still used by typing commands (Shneiderman, Direct Manipulation: A Step Beyond Programming Languages, IEEE Computer, 1983, doi:10.1109/MC.1983.1654471). NN/g's definition today: an interaction style in which people "act on displayed objects of interest using physical, incremental, and reversible actions whose effects are immediately visible on the screen" (NN/g, Direct Manipulation: Definition, 2016).
Every word of that definition is a motion rule:
- Physical: you move the thing itself, not a command about it.
- Incremental: it moves a little for every little move of yours, so you can stop at any point.
- Reversible: move back, and it goes back.
- Immediately visible: there's no gap between your hand and the screen.
2. Swipe a row, with no script
-
Ana LimaThe new icons are in the file
⋯
-
Rui CostaCan we move the review to Friday?
⋯
Swipe a row left with a finger or a trackpad, or Tab to it and use the arrow keys. The ⋯ button does the same with one tap.
scroll-snap-type, overscroll-behavior, <details>. Demo: Marcus Andrey, CC BY 4.0.The figure uses the browser's own scrolling: each row is a small sideways scroller that snaps to "content" or to "action" (MDN, CSS scroll snap). No animation is written at all, and that's the lesson. Native scrolling already follows the finger exactly, carries on with the speed you gave it, and settles at a snap point. Custom gesture code often does worse.
These are the rules that make the row:
.mo-b-row {
display: grid; grid-template-columns: 100% 6.5rem;
overflow-x: auto; scroll-snap-type: x mandatory; overscroll-behavior-x: contain;
scrollbar-width: none; border-radius: 10px; background: var(--accent-fill);
}
overscroll-behavior-x: contain stops the swipe from leaking out to the page, so a swipe at the end of the row doesn't start a "back" gesture in the browser (MDN, overscroll-behavior). Under reduced motion nothing changes, because no extra motion was added: the movement belongs to the person's hand.
3. Feedback during, at the end, after
| Gesture | Feedback while it happens | What if it goes wrong | The one-tap way |
|---|---|---|---|
| Tap | Pressed state at once | Slide off before lifting: nothing happens (WCAG 2.5.2) | It is the one-tap way |
| Long press | Something grows or lifts, so people know to keep holding | Let go early: nothing happens | The same actions in a visible menu |
| Swipe a row | The row follows the finger; the action shows underneath | Undo, or a confirm for destructive actions | A ⋯ button with the same actions (WCAG 2.5.1) |
| Drag to reorder | The item lifts, the others make room as it moves | Drop outside: it goes back where it was | Move up and Move down buttons (WCAG 2.5.7) |
| Pinch to zoom | The content scales under the fingers, in step with them | Pinch back; a reset button | + and − buttons (WCAG 2.5.1) |
| Swipe back from the edge | The screen slides with the finger, the one behind shows | Let go early: the screen springs back | A Back button, always there |
Apple's guidelines describe the job well: as people perform a gesture, give feedback "that helps them predict its results" and, if needed, "communicates the extent and type of movement required to complete the action" (Apple HIG, Gestures). In practice, three moments:
- During: the object moves with the hand, one to one. A list row slides as far as the finger, a sheet follows a downward drag. Show what will happen if they let go now: the Archive label grows, the colour fills.
- At the end: it settles. A release past the threshold completes the action; a release before it springs back. A spring is the right tool here, because it can carry on from the speed of the hand (Springs and physics).
- After: confirm, and offer a way back. Undo beats "Are you sure?" for anything that can be undone.
Apple also asks you to show when a gesture isn't available. If a locked item can't be dragged, it should resist or say so, or people will think the app froze (Apple HIG, Gestures).
4. A gesture is a shortcut, never the only way
Gestures are invisible. Nothing on a list row says "swipe me". And not everyone can perform them: a tremor, a head pointer, a switch device, a mouth stick, or just one hand busy with a coffee. WCAG 2.2 has three criteria for this:
- 2.5.1 Pointer Gestures (A): anything done with a multi-finger or path-based gesture (pinch, swipe along a path) can also be done with a single pointer without a path, like a tap on a button (W3C, Understanding 2.5.1).
- 2.5.2 Pointer Cancellation (A): the action happens on release, not on touch down, so people can slide off to cancel; or it can be undone (W3C, Understanding 2.5.2).
- 2.5.7 Dragging Movements (AA, new in 2.2): anything done by dragging can also be done with a single pointer without dragging, like Move up and Move down buttons (W3C, Understanding 2.5.7).
Apple's guidelines say the same in design words: give people more than one way to interact, and use shortcut gestures to supplement standard ones, not replace them. Their example is the Back button: there's a swipe from the edge to go back, and the button stays (Apple HIG, Gestures). That's why every row in the figure has a ⋯ button. Make it at least 24 by 24 CSS pixels, or give it enough space around it (W3C, Understanding 2.5.8 Target Size); the one in the figure is 44.
5. Don't fight the platform's gestures
People expect a swipe from the screen edge to go back, a pull at the top to refresh, a pinch to zoom. Apple advises against using a familiar gesture for something unique to your app, and against inventing a new gesture for a standard action (Apple HIG, Gestures). On the web, the CSS touch-action property tells the browser which touch gestures your element handles itself, so you only take over what you really need (MDN, touch-action).
6. Common mistakes
- Swipe-to-delete with no visible way to delete and no undo.
- An object that lags behind the finger, or snaps back with a fixed animation that ignores how fast it was thrown.
- Triggering on touch down, so a slip becomes an action.
- A custom gesture that overrides the system's "back".
- Reorder by drag only, with no buttons.
7. Vocabulary
| Term | Meaning |
|---|---|
| Direct manipulation | Acting on the object itself, with small, reversible moves you see at once |
| Path-based gesture | One where the route of the finger matters, like a swipe |
| Multipoint gesture | One that needs more than one finger, like a pinch |
| Threshold | How far a gesture must go before letting go completes it |
| Scroll snapping | Scrolling that settles on set positions |
| Single-pointer alternative | The same action with one tap or click, no path, no dragging |
8. Try it (no tools needed)
- List every gesture in an app you know: swipes, long presses, drags, pinches.
- For each one, find the button that does the same thing. Mark the ones with none.
- Try each gesture slowly and stop halfway. Does the screen tell you what will happen if you let go?
Reflection questions: Which of your product's gestures would a new user never find? What breaks if someone can only tap?
9. If you already practise
- Spec each gesture with its threshold, its settle motion and its alternative, in the same table as the figure.
- Test gestures with a stylus, a mouse and the keyboard, not only a finger.
- Prefer native scrolling, snapping and system sheets before writing gesture code; they already feel right and respect the platform.
Checklist
Learn more
- Samyukta Sherugar and Raluca Budiu, Direct Manipulation: Definition (NN/g, 2016)
- Apple, Human Interface Guidelines: Gestures
- W3C: 2.5.1 Pointer Gestures · 2.5.2 Pointer Cancellation · 2.5.7 Dragging Movements · 2.5.8 Target Size
- MDN: CSS scroll snap · overscroll-behavior · touch-action
- Ben Shneiderman, Direct Manipulation: A Step Beyond Programming Languages (IEEE Computer, 1983)
Books, courses and videos to go deeper
- MicrointeractionsDan SafferBook · O'Reilly, 2013 (Amazon)
- Animation at WorkRachel NaborsBook · A Book Apart, 2017 (Amazon)
Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.
Related
Springs and physics · Anatomy of a microinteraction · Accessibility fundamentals · WCAG and motion · The spatial model of an interface