On paper, AI seems almost limitless. It can classify images, map joints and pick out patterns with incredible speed. But step outside the lab into the spaces where people actually move — like gyms, factories, clinics and offices — and the story changes. A worker leans too far during a lift and an AI system misses it. A patient shifts weight unevenly and AI cannot tell why.
While these mistakes may seem simple, they reflect the reality of many AI systems today: They struggle to understand human movement. And that’s because while many AI systems were trained on billions of still images, human movement is never still. It unfolds over time, shaped by force, fatigue, rhythm and intent. Without understanding those elements, AI can only guess.
That gap affects safety, recovery and performance across industries. And it explains why so many teams are now racing to teach AI something humans learn as toddlers: How to make sense of how people move. As more industries lean on AI to monitor, guide, or automate physical work, the limits of today’s systems are becoming harder to ignore.
Why AI Misreads the Physical World
Most vision systems excel at recognizing objects. They can tell a shoe from a chair or a person from a wall. But the moment they are asked to evaluate how a body moves, the real cracks begin to show. A single frame can capture a pose, but not whether the person is stable, compensating, or about to lose form.
Movement also carries layers of meaning that AI struggles to interpret. A knee moving inward might signal fatigue, limited mobility, or simply a change in stance. A shoulder rising during a lift could come from habit or pain. Humans read these cues instinctively, but machines often can’t.
Environment adds another layer of complexity. In gyms, clinics, factories and homes, lighting shifts, angles vary and other people walk through the frame. Computer vision research has shown that even relatively small changes in lighting, occlusion and orientation can significantly reduce model accuracy, even when those models perform well on standard benchmarks.
Researchers across the industry are running into the same wall. Teams working on movement analysis for robotics, rehabilitation and workplace ergonomics — from projects like Meta’s Ego4D dataset to Google’s MotionLM research for embodied AI — have all highlighted how unpredictable real movement can be and how easily today’s models get confused outside controlled environments.
As Amol Gharat, cofounder of FlexAI, told me, “Teaching an AI to see form is different from teaching it to recognize objects. We’re not looking for a cat in a photo. We’re tracking kinetic chains across time, understanding how joints should move relative to each other under load.” That difference matters because many systems in use today were built for general vision tasks and were never designed to understand how bodies behave under real-world constraints.
For industries that depend on precise movement, this shortfall could have devastating consequences. If models misinterpret how workers lift, how patients walk, or how athletes land from a jump, they miss the very patterns that contribute to injury, lost performance and, in some cases, fatal incidents.
The Data Gap Slowing Movement Intelligence
One of the biggest obstacles is the lack of realistic, annotated movement data. Motion capture labs have long collected high-quality datasets, but they rely on controlled conditions: Specialized cameras, marker suits, fixed lighting and choreographed movements. These datasets are valuable for science, but they do not resemble daily life in a warehouse, clinic, or gym.
Recent biomechanics research also underlines how much human movement varies across people, fatigue levels and injury histories. That variability is exactly what AI needs to see to make reliable judgments. Yet most consumer videos don’t include the kind of biomechanical labels these systems rely on. And without that expert guidance, AI can’t tell whether a movement is harmless or a sign that something is going wrong.
Because of this, many teams across the field have been forced to build their own datasets. FlexAI is one example. When FlexAI’s founders went looking for real gym movement data, they found almost nothing suitable. “Minimal funds forced us to build a large-scale dataset from scratch — in the form of watching thousands of videotaped samples — under the direction of fitness trainers,” said CEO Amin Niri. Every frame needed labels for hip position, knee tracking, spinal alignment and other markers that matter in strength training.
Similar challenges also show up in other fields. Rehab researchers build custom datasets for joint instability. Sports-tech companies record athletes across skill levels to capture real variation. Workplace safety teams gather footage from actual job sites to understand how fatigue and repetition affect posture. Everyone is discovering that generic pose-estimation datasets simply don’t reflect the complexity of human movement in the wild.
Even with the right data, speed and privacy still get in the way. Feedback has to arrive instantly to help someone adjust mid-movement. But sending video to the cloud often creates delays and raises new concerns about storage and access. That is one reason companies working on movement guidance are pushing more computation onto the device itself rather than remote servers. “Every millisecond of latency matters when you’re trying to give feedback mid-rep,” Gharat noted.
A New Push To Understand How People Move
If AI can learn to understand movement like humans, the impact could stretch far beyond fitness apps on a phone. Physical therapy teams could track recovery remotely and adjust plans based on how patients actually move at home. Workplace safety programs could identify risky lifting patterns or awkward postures before they turn into recordable injuries. Sports organizations could offer motion analysis to far more athletes than a biomechanics lab can reach.
The hardest remaining challenge is understanding the human state behind what the camera sees. FlexAI’s models can detect form breakdowns and compensation patterns during a lift, but they cannot yet explain whether those patterns come from fatigue, limited mobility, an old injury, or simple confusion about technique. The same is true in rehab or workplace settings. A change in movement might mean someone is tired, in pain, stressed, or simply adjusting to a new task.
Bridging that gap will likely require more than computer vision. It will mean combining movement data with self-reported input, wearable signals and context from the environment. It also raises questions about transparency and trust. Workers, patients and athletes will want to know how their movement data is used and what decisions it feeds.
For now, the teams working on movement intelligence are less interested in replacing human experts and more focused on scaling their reach. As Niri put it, “We’re not replacing trainers. We’re making expertise accessible.” That mindset may be what keeps this next wave of AI grounded. The goal is not to hand over control to machines, but to give people clearer, earlier insight into how they move and what that means for their health, safety and performance.
Human movement is turning into one of the hardest tests for AI and solving it will not come from larger models alone. It will come from better data, better context and a deeper respect for how complex the human body really is.
For Gharat, the path forward is clear: AI must learn to keep pace with people, not the other way around. “The future will belong to systems that move fast and understand deeply enough, to guide the body in real time.”
