CADENCETemporally Coherent Whole-Body Human Mesh Recovery that Preserves Expressive Hand Motion

University of Southampton, United Kingdom

CADENCE architecture: a shared C-RADIOv4 encoder and structured decoder with keypoint feedback estimate SMPL-X per frame; a high-resolution hand pathway re-encodes hand crops and predicts finger pose; a temporal refiner uses temporal and kinematic-tree attention to mix learned zero-phase filters per joint and frame.

Smoothing a whole-body video estimate removes jitter, but a filter strong enough to steady the torso also flattens the finger strokes that carry meaning in sign language. CADENCE gives every joint its own learned temporal bandwidth, so the body is stabilized while hand motion survives.

Abstract

Whole-body SMPL-X estimators recover body, hand, and face meshes from single images, but applied frame by frame to video they produce jittery motion. Smoothing the output removes the jitter at a cost that body-only benchmarks do not measure: a temporal filter strong enough to steady the torso also flattens the fast finger strokes that carry meaning in sign language and gesture. We present CADENCE, a whole-body SMPL-X system for monocular video designed around this trade-off. A per-frame estimator built on a shared C-RADIOv4 encoder, with a high-resolution hand pathway that reuses the same encoder on hand crops, produces each frame's estimate. An offline temporal refiner then filters the SMPL-X rotation sequence: attention along the kinematic tree selects, for every joint and every frame, a mixture of learned zero-phase filters, so the torso is smoothed while fast finger strokes pass through. On 3DPW, the refiner reduces acceleration error from 17.4 to 5.4 mm/s² while lowering PA-MPJPE from 41.9 to 40.2 mm; generic smoothers applied to the same predictions reach 6.2 to 7.7 mm/s². On SignAvatars/How2Sign sign-language sequences, the refiner has the lowest hand jitter among the compared smoothers while keeping a hand-speed peak amplitude of 0.98, against 0.35 for a SmoothNet retrained on the full SMPL-X vector. The per-frame estimator also obtains the lowest full-body mesh error (25.7 mm) and hand joint error (8.3 mm) among the compared methods on EHF.

Smoothing without erasing the hands

Every smoother below takes the same per-frame CADENCE predictions on SignAvatars/How2Sign sign-language sequences. Moving left means less hand jitter; moving up means more of each hand stroke's peak speed survives (1.0 matches the reference). Generic smoothers buy stability by flattening strokes. A single-rate version of our refiner does the same. The per-joint bandwidth is what moves CADENCE up and to the left.

Hand jitter vs. hand-speed peak amplitude

  • CADENCE refiner
  • Other smoothers
  • Per-frame input (no smoothing)
Scatter plot. CADENCE refiner: jitter 1.29, peak amplitude 0.98. SmoothNet-WB: 1.59, 0.35. Gaussian: 1.43, 0.49. One-Euro: 1.89, 0.66. Moving average: 2.22, 0.58. Savitzky-Golay: 3.25, 0.76. Single-rate refiner: 1.50, 0.50. Per-frame input: 8.02, 1.18.

Data: Table 2 of the paper. Peak-A is measured against pseudo ground truth; the per-frame input overshoots it (1.18). Full numbers are in the table below.

Results

Temporal smoothing

3DPW, same per-frame input for every row (mm; Accel in mm/s²)
MethodPA-MPJPE ↓MPJPE ↓Accel ↓
Per-frame41.961.717.4
Gaussian (σ=2)41.561.17.0
Moving average41.561.16.6
One-Euro50.570.36.2
Savitzky–Golay41.661.47.6
SmoothNet-WB42.561.97.7
Single-rate refiner41.761.25.7
CADENCE refiner40.258.15.4
SignAvatars/How2Sign sign language (hand metrics)
MethodJitter ↓Peak-R ↑Peak-A ↑
Per-frame8.020.931.18
SmoothNet-WB1.590.810.35
Gaussian (σ=2)1.430.620.49
One-Euro1.890.830.66
Moving average2.220.780.58
Savitzky–Golay3.250.830.76
Single-rate refiner1.500.620.50
CADENCE refiner1.290.900.98
Sign-language clip: wrist acceleration and finger speed over time for per-frame predictions, a Savitzky-Golay smoother, and the CADENCE refiner.
On one sign-language clip, a Savitzky–Golay filter tuned to the same residual wrist jitter as the CADENCE refiner (2.5 vs. 2.6 mm/frame²) keeps 46% of the peak finger speed; CADENCE keeps 99%.

Expressive whole-body accuracy (EHF)

EHF, Procrustes-aligned errors in mm. Hand and face are wrist- and neck-aligned.
MethodP-MVE ↓P-MPJPE ↓
FullHandFaceBodyHand
PyMAF-X50.210.25.552.810.3
SMPLer-X-L37.114.15.0––
Multi-HMR28.210.85.3––
NLF-L+fit26.020.76.326.121.8
CADENCE25.712.25.120.68.3

On 3DPW, CADENCE's acceleration error (5.4) is lower than WHAM, TRAM, TCMR and MPS-Net (6.7–7.4) and slightly above the body-only GVHMR and GENMO (5.0, 5.2), while also predicting hands and face. Body-focused methods such as SAM 3D Body have lower body joint error. See the paper for the full comparison.

Qualitative results

Consecutive frames of input video and CADENCE SMPL-X overlays for athletic motion, classroom interaction, and sign language.
Consecutive frames across activities: input video above, CADENCE SMPL-X overlay below.
Hand crops with the base whole-body hand prediction, the hand pathway prediction, and the target.
The high-resolution hand pathway: input crop, whole-body hand estimate, after the pathway, and target.

BibTeX

@article{peart2026cadence,
  title   = {{CADENCE}: Temporally Coherent Whole-Body Human Mesh Recovery
             that Preserves Expressive Hand Motion},
  author  = {Peart, Keniel and Chauhan, Jagmohan and Bodala, Indu P. and
             Vishwakarma, Shelly},
  journal = {arXiv preprint},
  year    = {2026}
}

Acknowledgements

The authors acknowledge the use of the IRIDIS X High Performance Computing Facility and the Southampton-Wolfson AI Research Machine (SWARM) GPU cluster, funded by the Wolfson Foundation, at the University of Southampton. SMPL-X, 3DPW, EHF, AGORA, UBody, and SignAvatars/How2Sign are used under their respective research licenses.