BarrelIQ

Overview

BarrelIQ turns softball/baseball at-bats into measurable, reviewable swing data. A coach films a real game or practice session, uploads the video, and the app runs it through a pose-and-bat-tracking pipeline that reconstructs the swing in 3D, measures the mechanics, and produces a scouting report the coach reviews and signs off on — not one an algorithm publishes unsupervised.

It is under active development. The coach-facing Android app, the processing pipeline, and the 3D playback all work end to end today; camera calibration, an iOS build, and anything resembling a public release are still ahead.

Why I Built It

Most swing analysis happens through cage sessions, isolated clips, or subjective visual review — a coach watches a phone video a few times and calls it. None of that holds up across a season: there is no consistent measurement, and a single cherry-picked rep tells you much less than a pattern across real at-bats.

I wanted swing evaluation built around actual competitive game footage, with enough structure that a coach could compare mechanics across swings instead of re-litigating the same three clips.

How It Works

Record / Upload
      ↓
Pose + Bat Tracking
      ↓
2D → 3D Reconstruction
      ↓
Biomechanical Metrics
      ↓
Gemini Draft Analysis
      ↓
Coach Review

Record / Upload. A coach logs games, at-bats, and practice sessions, and uploads footage from the Android app.

Pose + bat tracking. YOLO11-pose finds the player and their joints; YOLOv8 with ByteTrack separately tracks the bat through the swing.

2D → 3D reconstruction. The tracked keypoints are smoothed, corrected against a rigid forward-kinematic bone-length model, and lifted from 2D to 3D with VideoPose3D. The pipeline automatically finds the swing window and detects contact inside it.

Biomechanical metrics. Body-relative measurements are computed from the 3D trajectory — hip-shoulder separation, lead-elbow angle, stride, torso and pelvis tilt, bat speed and attack angle — each carrying a confidence flag where the measurement is uncertain.

Gemini draft analysis. Gemini reads the measurements and produces an initial checklist-style score. It never sees raw video and never generates the measurements themselves — those come entirely from the CV pipeline above.

Coach review. The coach reviews the draft against the 3D playback and confirms or corrects it.

3D swing reconstruction

This is not a canned animation. The interactive model — a rigged, skinned FBX humanoid rendered in Three.js/WebGL — is driven frame by frame by the player's own tracked H36M joint data through bone retargeting. What the coach sees moving is the athlete's actual swing, reconstructed from their footage.

The coach can orbit the model, scrub to any frame, and play the motion back at speed, using the reconstruction the way they would review film — except now it is a measurable 3D object instead of a flat clip.

AI-assisted, coach-verified

Gemini produces an initial checklist score from the measured mechanics. That score is a draft — visibly marked as one — until a coach reviews it.

If the coach disagrees, both scores stay visible. The AI draft does not get silently overwritten by the coach's correction, and the coach's score does not pretend the AI never weighed in. Keeping both is a deliberate choice: losing the disagreement would lose the information about where the automated read and the coach's judgment diverge, which is exactly the case worth tracking over time.

What gets measured

• Head movement
• Hip-shoulder separation
• Lead-elbow angle
• Stride
• Torso / pelvis positioning
• Bat path and attack angle
• Contact timing
• Confidence level per measurement

Everything here comes from the pose and bat-tracking pipeline. Nothing in this list is inferred by Gemini — its job is interpreting these numbers, not producing them.

Real game vs. practice data

Game at-bats and practice reps are stored separately, and practice sessions never blend into game-only statistics. A coach can review either, but a swing taken in a practice tee session is not silently averaged into a player's in-game numbers. It is a small piece of data modeling, but it is the difference between a system that measures something real and one that just accumulates clips.

What I'm working on now

The pose pipeline reconstructs a usable 3D swing, but it does not yet have real camera calibration. World rotation is currently fixed, which means camera angle can distort certain world-space measurements — most visibly torso tilt, where a mechanically upright hitter can read as heavily tilted purely because of where the camera was set up. The display layer compensates today with per-clip auto-leveling, but the underlying stored angle metrics still need a proper normalization solution rather than a display-time patch.

I am also working through how to visualize a corrected swing without pretending there is one universal "perfect" swing to converge on. The direction I am pursuing is comparing the athlete's actual tracked motion against targeted, constrained corrections — not replacing their swing with a generic reference animation.

Tech stack

TypeScript/JavaScript · Three.js/WebGL · YOLO11-pose · YOLOv8 · ByteTrack · VideoPose3D · Gemini API · Supabase · Capacitor · GCS · Python · PyTorch/CUDA

Current Status

Working today

• Android coach app (Capacitor)
• Authenticated coach accounts, with Supabase RLS keeping each coach scoped to their own team's data
• Video upload and automated processing pipeline
• Interactive 3D swing playback
• Coach-reviewed AI scoring
• Separate practice and game session logging
• PWA offline shell

Still in progress

• Camera calibration and angle normalization
• Corrected-swing visualization
• iOS build (the wrapper is scaffolded, not yet built or tested through Xcode)
• Production Supabase deployment (development is on local Supabase today)
• Self-service onboarding, parent-facing views, billing, and public release

None of that is available publicly yet — there is no App Store or Play Store listing, no public signup, and no production hosting behind this today.

View the source on GitHub