What Powers Peloton Guide?

You’ve probably heard that Peloton Guide, our dynamic TV-connected strength device, uses artificial intelligence to help Members level up their fitness game with personalized metrics, rep tracking, recommended classes, and more. The cool thing about machine learning is that it continually learns over time, meaning that as you improve, Guide is improving with you.

Still, you may be wondering how exactly Peloton Guide knows what you’re doing and how artificial intelligence is driving it. That’s why we talked to the people that know it best: Sanjay Nichani, vice president of artificial intelligence and computer vision, and Ben Schultz, director of product management, both on the Peloton Guide team. Read on to learn more about the technology that helps take your strength workouts to the next level.

What is machine learning?

Machine learning is complex, but in its simplest terms, it’s a form of artificial intelligence that uses data to learn an outcome. It continuously gets better by identifying patterns in the data and adjusting to what it learns.

So in the case of Peloton Guide, the data that’s being looked at is images of people working out, Nichani says. A machine sees a certain pose or movement and it understands it’s the image of the pose. This feeds into the data set.

“The program is figuring out the mapping of an input to an output based on the data,” Nichani adds. “You are relying on learning from the data.”

Once Peloton Guide sees enough image examples, it learns the mapping and is able to track Member movements.

The beauty of this type of technology is that because it’s constantly learning, it has the ability to grow into other modalities (think barre or pilates!) in the future.

“With Peloton Guide, we are focused on providing a great strength experience and acknowledging the hard work our members do,” Schultz says.

How does the machine get its data?

“First and foremost, Peloton Guide is not watching you and collecting your data,” says Schultz.

Instead, Peloton Guide is informed by a number of different data sources, all unrelated to Peloton Members. “We videotape people in a controlled environment from different angles—think head on or to the side—so the machine can get a more holistic view,” he explains. This helps the team train the Movement Tracker algorithms to understand the distance people may be from their camera, the height of the camera, and their positioning.

They also look at a ton of environmental factors like lighting, various workout environments, and equipment. Finally they think about Member attributes, body composition, clothing, and fitness levels to help teach the machine. And they do pull from Member feedback (so make sure to give those thumbs up or thumbs downs after class!) and investigate error messages (such as if a body isn’t in the frame or if a pet runs through the background while you’re working out).

There are lots of factors to consider, but the more diverse data Peloton Guide receives, the better it will do in the future.

If I modify a movement, does the machine understand that?

Think about, say, a push-up. Many people modify it by doing s on their knees, while others might have to adjust the movement in other ways based on their physical abilities. The Peloton Guide team wants to make sure they still get credit for that. That’s why they’ve been teaching the machine both the full expression of a movement as well as variations on that movement.

Plus, the team is working on additional ways to help Members who have different workout abilities and fitness levels find the right classes.

Is there anything Members can do to help Peloton Guide get smarter?

Setting your space up for success will help your Peloton Guide recognize and give you credit for all of that hard strength work. If possible, try to ensure the room is well lit with the lighting coming from the same direction as the camera. It also helps if your workout clothes aren’t the same color as your walls, so there’s some contrast. The camera should be level with a neutral angle view (below the upper chest). And if at all possible, set Peloton Guide up in a space where you won’t have people moving in and out of the room while you complete your workout.

Stay tuned for more Peloton Guide updates as it continues to improve! And in the meantime, read on to learn more about how Guide has helped our Members grow stronger.