Previous: Experimental Design

Introduction

In the previous note, we introduced the basic concept of experimental design. One key aspect was collecting data to measure the dependent variable and answer our research question. In practice, there are many ways to collect data. Here, we describe a basic technique for collecting data, discuss which events are worth logging in a study, explain how to clean and preprocess the data, and show basic visualizations for exploratory data analysis.

Interaction Logging

Interaction logging is the practice of automatically recording user actions as they interact with a system. Each click, keystroke, scroll, page transition, and pause can be timestamped and stored for later analysis. Unlike surveys or interviews, which rely on participants’ self-reports, interaction logs capture what users actually did—not what they remember or believe they did.

Common types of interaction data include:

Data Type	Example	What It Reveals
Clicks	Button presses, link selections	Which features users engage with; navigation choices
Dwell time	Time spent on a page or element	Attention, interest, confusion, or difficulty
Keystrokes	Text input events	Typing speed, error correction patterns, input effort
Errors	Failed actions, invalid inputs	Usability problems, confusing interface elements
Navigation paths	Sequences of pages visited	How users explore the interface; whether they find what they need
Scroll events	Scroll depth and direction	Whether content below the fold is seen
Hover events	Mouse movements over elements	Areas of visual attention (on desktop)

Surveys and observation are useful, but they have blind spots that logging fills:

Frequency and duration at scale. A survey might ask "Did you find the search feature useful?" but logs can tell you that 73% of users never clicked the search button at all.
Temporal patterns. Logs reveal when during a session users struggle. For example, a spike in back-button clicks after reaching a particular page.
Unconscious behavior. Users often cannot accurately report how long they spent on a task or how many errors they made. Logs capture this without recall bias.
Longitudinal tracking. Logging can passively capture behavior across days or weeks, which would be impractical with direct observation.

Interaction data generally falls into two categories:

Event-level data: individual records of discrete actions (e.g., "User 12 clicked the Submit button at 14:03:22"). This is the raw material for most analyses.
Aggregated/session-level data: summaries derived from event data (e.g., "User 12 completed 3 of 5 tasks in an average of 42 seconds each"). This is what you typically use for statistical tests.

Hands-On: Logging Clicks and Dwell Times on a Web Interface

How Data Gets Captured: A Minimal JavaScript Snippet

Before we analyze log data in R, it helps to see how logs are generated. Below is a minimal JavaScript snippet that records clicks and page dwell time, then sends the data to a server endpoint. You do not need to run this code --- it is here so you understand where log data comes from.