Markerless face motion capture from a video file with Deepmotion and Blender

Опубликовано: 12 Октябрь 2024
на канале: GarageFarm Academy

11,057

196

How to turn your video into facial motion capture data automatically using no markers at all? In this video DJ tests the new feature introduced by Deepmotion in their Animate 3d cloud service (link: https://www.deepmotion.com/animate-3d)
This expands the already impressive features we covered in this video:    • Automatic motion capture with Deepmot...

Creating a simple alien abduction scene from raw video recording, DJ shows how you can create a 3d avatar straight on Deepmotion's webpage (that can even use a photo for creating a likeness of your 3d character). With such a custom 3d character or a dummy model, you can easily create a full-rigged animation that you can download and use in Blender or any other 3d software like 3dsMax, Cinema 4D, or Maya. The facial animations are made using shape keys so if you'd like to use your own character with the service you need to make sure it’s properly set up with a rig and appropriate shape keys for the face features.

This new exciting AI technology still has a lot of flaws and is far from perfect, but with some small tweaks, it is already quite usable. DJ shows how you can simplify the animation curves to reduce the number of keyframes. This makes tweaking the animations a lot easier. Finding places that feel off in the animations is quite easy - after all, you have the reference video footage from the start. Compare it and tweak the curves at the right keyframes and with a little patience and hard work you can polish the animations to look presentable.
After all, creating them by hand from scratch would take much more time.
With the vTubers, metaverse, and other trends of that kind - this can be a real lifesaver !

Once the animation is ready, there’s nothing more enjoyable than putting it in an environment and rendering a final shot. DJ uses cycles GPU render engine to do that. To speed up the renders even more, he sends them straight from Blender to Garagefarm.NET’s GPU cloud render farm using the renderbeamer addon and app.
Using paid render farms can be tricky and sometimes generate unnecessary costs but at Garagefarm we are committed to providing you with useful tools that make your work smooth and avoid unpleasant surprises. A really useful technique is rendering test frames of your animation by setting up a step - which means only some of the frames will be rendered and you can detect any possible issues before running the full sequence render.
This might seem like we don’t want you to pay us more - but we truly believe your renders should only cost as much as they have to. Our satisfied customers confirm that by using our service again on their next projects.

Timestamps:
0:00 - DJ's alien abduction scene
0:31 - what this video is about
0:52 - Deepmotion cloud service
1:14 - creating the 3d avatar
2:06 - creating motion capture animations from video file with Deepmotion's Animate 3d
3:39 - importing the mocap animation to Blender
4:34 - simplifying animation curves
5:23 - how to make the animation fit the video more?
5:37 - rendering the scene with cycles using GarageFarm.NET cloud renderfarm
7:08 - compositing
7:24 - final animation (again) - see you in another galaxy!

Grasswald - https://www.graswald3d.com/
Short face mocap -    • Face motion capture with lip sync usi...
My other video about mocap -    • Automatic motion capture with Deepmot...

___________________________________________________________________

👉 Facebook Group 👈
Join our group, a friendly space for Blender 3d enthusiasts and pros, to share tips, ask questions, and learn, learn learn!   / 666797333683403

GREAT DEAL! 💸
Use the code 🏷️ GFACDMY🏷️ while registering to get 💰 FREE 💰 $50 credits and render your projects - http://bit.ly/GarageFarmNET

If you're new to cloud rendering and would like to learn more, here's a great resource for you https://garagefarm.net/blog/how-to-us...
If you're ready to get started, register at https://garagefarm.net/ and get $25 trial credits to test the service.