Generación de Heatmap de Miradas con Mediapipe y OpenCV

Eye-tracking technology has revolutionized the way we understand human interaction with digital products and content. Through heat maps generated using Gaussian algorithms, we can visualize exactly where users' visual attention is focused, providing invaluable data to improve designs, interfaces and user experiences. This detailed analysis allows us to understand not only what our customers see, but how they interact visually with what they see.

How does an eye-tracking heat map work?

The foundation of an eye-tracking heat map is based on a Gaussian distribution in a three-dimensional space. This mathematical function allows us to create a visual representation where the central point corresponds to the area where the user is focusing their gaze. Although we work in a three-dimensional space, the result is displayed in a flat two-dimensional image.

The Gaussian distribution formula includes several configurable parameters:

• Decay Factor: Determines how gradually the heat map fades as we move away from the center point.
• Sigma: Controls the size of the area affected by the gaze.
• Amplitude: Defines the intensity of the colors in the heat map.

These parameters allow us to adjust the display to accurately represent how the user's visual attention is distributed on the screen.

Technical implementation of eye tracking

To implement this solution, we continue to work with MediaPipe for face detection, using a minimum confidence threshold of 0.5. The process includes several key steps:

1. Obtain the video frame dimensions (width and height).
2. Initialize the heat map(Hitman) with those dimensions.
3. Apply exponential decay to the map so that previously observed areas fade out.
4. Convert the color format from BGR to RGB to work with MediaPipe.
5. Extract the reference points of the eyes (positions 33 and 263 for the left and right eyes).
6. Calculate the midpoint between both eyes.
7. Add the Gaussian patch centered on that midpoint.

# Get frame dimensionsheight, width = frame.shape[:2]# Initialize the heatmapheatmap = np.zeros((height, width),  dtype=np.float32)# Apply exponential decayheatmap *= decay_factor

Once the heatmap is generated, we perform a normalization to scale the values between 0 and 255, which allows us to apply a "jet" type color palette (ranging from deep red, through yellow, green and up to light blue). Finally, we superimpose this map on the original image with some transparency.

How to analyze recorded videos for eye-tracking studies?

Although real-time eye tracking is useful, many professional studies prefer to record videos of users for later analysis in greater detail. This methodology allows for more in-depth analysis without the time constraints of live processing.

To implement this solution with pre-recorded videos, we can use both CPU and GPU, the latter being significantly faster for processing. The basic procedure is:

1. Install MediaPipe in the working environment (e.g. Google Colab).
2. Load the video we want to analyze.
3. Define the output path for the processed video.
4. Configure the output video format (codec, FPS, etc.).
5. Process each frame applying the eye-tracking algorithm.
6. Save the result as a new video.

# Define output pathoutput_path = 'eye_tracking_result.mp4'# Configure output formatfourcc = cv2.VideoWriter_fourcc(*'mp4v')fps = 30video_writer = cv2.VideoWriter(output_path, fourcc, fps, (width, height))# Write each processed framevideo_writer.write(processed_frame)

Limitations of eye tracking

It is important to note that this technique has significant limitations, especially when it comes to head position relative to gaze direction. For example, if a person has his or her head tilted upward but is looking downward, the system could incorrectly interpret the direction of gaze.

This limitation is particularly relevant when the objects of interest are at different distances from the user. The system works best when the objects are relatively close and in the same plane, such as a computer screen.

Eye tracking using heat maps offers a powerful tool for understanding the visual behavior of users. Whether implemented in real time or applied to pre-recorded videos, this technology provides valuable insights into how people interact with digital interfaces and physical products. We invite you to experiment with your own videos using the techniques described and share your experiences in the comments.