Evaluation

Analysis and evaluation tools.

Performance analysis API for eye tracking systems.

Exports functions to analyze accuracy and robustness across gaze points, observer positions, and calibration quality.

pyetsimul.evaluation.accuracy_at_calibration_points(et, eye)

Computes gaze error at calibration points to assess calibration quality.

Evaluates calibration accuracy by testing gaze prediction at original calibration targets. Provides comprehensive error analysis with both spatial and angular metrics.

To visualize the results, call calib_results.interactive_plot() on the returned object.

Parameters:

• et (EyeTracker) – Eye tracker structure

• eye (Eye) – Pre-configured Eye object (required)

Return type:

CalibrationResults

Returns:

CalibrationResults object with error statistics, printing, and on-demand visualization

Calibration Analysis

Calibration analysis for eye tracking systems.

This module analyzes eye tracker calibration accuracy by testing gaze estimation at the original calibration points to assess calibration quality.

class pyetsimul.evaluation.calibration_analysis.CalibrationResults(errors, plot_data=None)

Bases: object

Calibration accuracy results with on-demand visualization.

Parameters:

• errors (dict[str, dict[str, float]])

• plot_data (dict | None)

__init__(errors, plot_data=None)

Initialize calibration results.

Parameters:

• errors (dict[str, dict[str, float]]) – Dictionary containing error statistics in different units

• plot_data (dict | None) – Internal data needed for on-demand plot creation (None if all points failed)

Return type:

None

pprint(title='Calibration Accuracy')

Print formatted calibration error statistics.

Return type:

None

Parameters:

title (str)

interactive_plot(show=True)

Create the interactive calibration plot on demand.

No figure is created until this method is called, preventing figures from lurking in matplotlib’s global figure manager and appearing unexpectedly.

Parameters:

show (bool) – If True (default), display the figure with plt.show() (blocks until closed). If False, return the figure for saving (fig.savefig()) without displaying. The figure is removed from matplotlib’s manager to prevent it from appearing unexpectedly in later plt.show() calls.

Return type:

Figure

Returns:

The matplotlib Figure.

pyetsimul.evaluation.calibration_analysis.accuracy_at_calibration_points(et, eye)

Computes gaze error at calibration points to assess calibration quality.

Evaluates calibration accuracy by testing gaze prediction at original calibration targets. Provides comprehensive error analysis with both spatial and angular metrics.

To visualize the results, call calib_results.interactive_plot() on the returned object.

Parameters:

• et (EyeTracker) – Eye tracker structure

• eye (Eye) – Pre-configured Eye object (required)

Return type:

CalibrationResults

Returns:

CalibrationResults object with error statistics, printing, and on-demand visualization

Calibration Utilities

Utility functions for calibration analysis.

This module provides utility functions for calibration analysis, including parameter printing and data formatting functions.

pyetsimul.evaluation.calibration_utils.pprint_polynomial_parameters(et)

Print polynomial parameters from calibrated eye tracker.

Displays calibration coefficients and polynomial type for analysis. Shows calibration status and coefficient values for debugging.

Return type:

None

Parameters:

et (EyeTracker)

Gaze Accuracy

Generic gaze accuracy evaluation that works with any pre-generated dataset.

class pyetsimul.evaluation.gaze_accuracy.GazeAccuracyResult(errors_3d, errors_angular, predicted_points, ground_truth_points, observer_positions, error_stats, total_measurements, successful_predictions, variation=None)

Bases: object

Results from gaze accuracy evaluation.

Parameters:

• errors_3d (list[float])

• errors_angular (list[float])

• predicted_points (list[Position3D | None])

• ground_truth_points (list[Position3D])

• observer_positions (list[Position3D])

• error_stats (dict[str, dict[str, float]])

• total_measurements (int)

• successful_predictions (int)

• variation (Any | None)

errors_3d: list[float]

errors_angular: list[float]

predicted_points: list[Position3D | None]

ground_truth_points: list[Position3D]

observer_positions: list[Position3D]

error_stats: dict[str, dict[str, float]]

total_measurements: int

successful_predictions: int

variation: Any | None = None

pprint(title='Gaze Accuracy Results')

Print formatted results matching existing style.

Return type:

None

Parameters:

title (str)

pyetsimul.evaluation.gaze_accuracy.evaluate_gaze_accuracy(eye_tracker, dataset, description='Evaluating gaze accuracy', camera_id=0, eye_id=0)

Evaluate gaze accuracy using pre-generated dataset.

This function takes a calibrated eye tracker and a dataset (in-memory dictionary) and evaluates accuracy by: 1. Reconstructing EyeMeasurement objects from stored pupil/glint data 2. Feeding them directly to the tracker’s prediction stage 3. Comparing predictions against ground truth

Parameters:

• eye_tracker (EyeTracker) – Calibrated eye tracker to evaluate with

• dataset (dict[str, Any]) – In-memory dataset from DataGenerationStrategy.execute()

• camera_id (int) – Which camera’s data to use (default: 0)

• eye_id (int) – Which eye’s data to use (default: 0)

• description (str) – Progress description

Return type:

GazeAccuracyResult

Returns:

GazeAccuracyResult with errors and statistics

Algorithm Comparison

Algorithm comparison for ranking multiple eye tracking algorithms.

class pyetsimul.evaluation.algorithm_comparison.AlgorithmRanking(rankings, error_stats, success_rates, pairwise_angular_diff, pairwise_cosine_sim, pairwise_amplitude)

Bases: object

Algorithm ranking results.

Parameters:

• rankings (dict[str, int])

• error_stats (dict[str, dict[str, dict[str, float]]])

• success_rates (dict[str, float])

• pairwise_angular_diff (dict[str, dict[str, float]])

• pairwise_cosine_sim (dict[str, dict[str, float]])

• pairwise_amplitude (dict[str, dict[str, dict[str, float]]])

rankings: dict[str, int]

error_stats: dict[str, dict[str, dict[str, float]]]

success_rates: dict[str, float]

pairwise_angular_diff: dict[str, dict[str, float]]

pairwise_cosine_sim: dict[str, dict[str, float]]

pairwise_amplitude: dict[str, dict[str, dict[str, float]]]

pprint(title='Algorithm Ranking')

Print comprehensive ranking and comparison analysis.

Return type:

None

Parameters:

title (str)

get_best_algorithm()

Return best algorithm name.

Return type:

str

get_top_n(n)

Return top N algorithm names.

Return type:

list[str]

Parameters:

n (int)

pyetsimul.evaluation.algorithm_comparison.compare_algorithms(algorithms, dataset, description='Comparing algorithms', calculate_pairwise=False)

Compare multiple algorithms on same dataset.

Return type:

AlgorithmRanking

Parameters:

• algorithms (dict[str, EyeTracker])

• dataset (dict)

• description (str)

• calculate_pairwise (bool)

pyetsimul.evaluation.algorithm_comparison.point_wise_angular_difference(predictions1, predictions2, eye_positions)

Point-wise mean angular error between two algorithms.

Return type:

float

Parameters:

• predictions1 (list[Position3D])

• predictions2 (list[Position3D])

• eye_positions (list[Position3D])

pyetsimul.evaluation.algorithm_comparison.cosine_similarity_average(predictions1, predictions2)

Average cosine similarity between prediction vectors.

Return type:

float

Parameters:

• predictions1 (list[Position3D])

• predictions2 (list[Position3D])

pyetsimul.evaluation.algorithm_comparison.amplitude_agreement(predictions1, predictions2)

Compare prediction magnitudes.

Return type:

dict[str, float]

Parameters:

• predictions1 (list[Position3D])

• predictions2 (list[Position3D])

Analysis Utilities

Utility functions for performance analysis.

This module provides helper functions used across performance analysis modules to ensure consistency and reduce code duplication.

pyetsimul.evaluation.analysis_utils.calculate_error_statistics(U, V, angular_errors)

Calculate gaze tracking error statistics.

Parameters:

• U (ndarray) – Error array in X direction (in mm)

• V (ndarray) – Error array in Y direction (in mm)

• angular_errors (ndarray) – Angular error array (in degrees)

Returns:

Error statistics with ‘mm’ and ‘deg’ keys containing mean, max, std, median

Return type:

dict