ADeep Learning based Autism Stimming Detection From Video : Skeletal Motion Analysis

Abstract

Early and objective detection of Autism Spectrum Disorder (ASD) remains a critical clinical challenge, as traditional diagnostic instruments are resource-intensive, subjective, and often administered too late to exploit peak neural plasticity. This thesis investigates the automated recognition of stereotypical self-stimulatory behaviors—commonly referred to as stimming—

from unconstrained video recordings, with the goal of providing a non-invasive, scalable screen- ing tool. To this end, a composite dataset was assembled from established public repositories

and supplementary social media recordings, re-annotated at frame-level precision across four

behavioral classes: Arm Flapping, Headbanging, Spinning, and Neutral. After an initial proto- type exposed a critical data-leakage flaw that artificially inflated accuracy, a complete method- ological redesign was undertaken based on rigorous source-video-level data partitioning. The

final approach replaces pixel-level features with a privacy-preserving skeletal representation ex- tracted by YOLOv8x-pose, from which a per-frame biomechanical feature vector is computed—

encoding joint angles, angular velocities, kinetic energy, center-of-gravity dynamics, and a pe- riodicity proxy—and feeds two dual-stream fusion architectures: BiLSTM+PoseC3D and

ST-GCN+GRU. Both architectures demonstrated strong generalization on a fully indepen- dent external test set, with the best configuration achieving over eighty percent accuracy and

an area under the ROC curve exceeding 0.95. Headbanging proved the most reliably classified behavior, while the Neutral class remained the most challenging due to kinematic overlap with low-intensity stimming. These results confirm that pose-based, biomechanically informed deep learning is a viable and principled approach to automated stimming detection, with dataset expansion and weakly-supervised learning identified as the most promising avenues for future improvement.