Text-Based AI Motion Capture Solutions
September 12, 2024
Snapchat’s Game Changing Move in AI Video and Portrait Creation
September 17, 2024Text-Based AI Motion Capture Solutions
September 12, 2024Snapchat’s Game Changing Move in AI Video and Portrait Creation
September 17, 2024Beta Testing Cartwheel AI Text-to-Animation
I recently had the chance to try out the private beta of Cartwheel, an AI tool for motion capture. I decided to give it a go and see how well it could handle different animation requests.
Cartwheel is an AI text-to-animation solution where you can type in the kind of animation you want, and it will create it for a biped character. It’s a great example of how you can bring your imagination to life.
Path to
Creating a Multi-Action Animation Sequence Using Cartwheel
Prompting a Complex Animation
At first, I used basic prompts like "walk" and "jump," and while the results were fine, I felt the need to push the limits. So, I tried a more complex sequence where the character walks forward 10 steps, jumps, rotates, and then runs in the opposite direction. The AI understood the walk and jump parts, but the full sequence didn’t come out quite right.
The character's movements were a bit off, and sometimes one character would execute the action well while another would struggle. Occasionally, the animation results were also biased, like the character holding a rifle while walking.
Varitation Selection
The variation selection popup is useful, however, I wish I could preview the motions more clearly in each video without having to select them. I was initially concerned that if I chose one variation, I might lose access to the others. At first, I was very particular about selecting the one that looked best. Later, I realized I could always return to the variations without losing them.
I believe this concern stemmed from other platforms' pricing systems, where losing the variations could mean needing to re-enter the prompt, with each prompt potentially costing credits.
Break the animation sequence down into individual actions.
To fix the issue with the complex animation sequence, I decided to take a different approach and break the sequence down into individual actions: walk, jump, rotate, and run.
I tried to write each prompt in detail:
- Walk forward 10 steps.
- Jump high and forward.
- Turn around to face the opposite direction.
- Run for 20 steps.
This worked better, although step 2, "the jump," wasn't great at first. After tweaking the prompt and adding more detail, the jump improved, and the final results were closer to what I wanted.
Revised prompt for number 2:
2. Perform a forward long jump, leading with the right leg, like in a sporting event.
The Issues I Encountered
While exporting the animations into the 3D software to combine, a few thoughts came to mind.
Can I access all my previous animation prompts again?
As I was prompting and exporting the animations, one concern was whether my previous prompts and animations were saved. After looking around, I found a "Favorites" button, which made me wonder: if I favorite an animation variation, does that mean I can re-access it?
However, it wasn’t clear how to access past prompts, and I wondered, what if I forget to favorite an animation? Does that mean I lose it forever?
These concerns arose with the rise of prompt engineering, where each prompt might cost users credits, adding pressure to not lose past work.
Character Polygons / Vertices Are Not Connected
Cartwheel's brand characters look nice and smooth, but when I exported the animation with the mesh into the 3D software, I realized the character's polygon meshes were not connected. This resulted in a low-poly, hard-edged appearance, which is great if that's the look you're after, but it didn’t appear as smooth as it did on the Cartwheel platform. While this is something that can be easily fixed in 3D software, it might be confusing for users who expect the same smooth look they see in Cartwheel.
Feature Request
Allow Combining Animations on the Cartwheel Platform
Overall, I was able to complete the animation work and combine the actions into one sequence where the character walks, jumps, rotates, and runs. While doing so, a feature request came to mind. Cartwheel's AI is impressive but struggles with complex animations. One improvement would be to guide or allow users to break down complicated animations into smaller actions directly on the platform.
It may be challenging for the AI team to improve the model for complex animations, but introducing a feature that lets users combine individual animations and export them as a sequence could be much easier. This would simplify creating a single animation without needing external 3D software, which many users may not be familiar with.
In conclusion, Cartwheel is a powerful text-to-animation tool, but it could be even better with features that make handling complex sequences easier and more flexible.
Written by
