Lip Sync, Clothes Generator, Mocap, & Reallusion
I finally have all the basic functionality developed in my new collection of custom-made HDA assets and I can finally start production on that Short Film I’ve been bragging about for months! I really wanted to do a Short Film! But all I’ve been doing for the past few months is talk about it, and never really got my hands dirty on the storyboard, nor have you seen me post anything related to the concept art. I’ve always been busy working on a custom HDA tool in Houdini or trying to get Houdini to work with Blender and other miscellaneous stuff having no relation to my Short Film. But if you look more closely and try to put all the pieces together, you’ll see where the yellow brick road will take me. All the HDA’s have a purpose and contribute to 3D character creation.
It All Comes Together…
Every piece of update I make to my Lip Sync HDA’s or Clothing Generators are related to the character design and character creation pipeline. The development of all these tools are a necessary step that I think needs to be finished before jumping to storyboards and concept design. Here’s a breakdown of the purpose of each tool that you’ve seen me release in the past few months:
Facial Expressions HDA combined with Lip Sync HDA (in Houdini)
Facial Expressions HDA
Facial Expressions bring life into a character and facial expressions really sells the character. No matter how much animation we have, the life in the character will mostly come from the face. So I knew from the start that I needed a Facial Expressions HDA somewhere along my journey. Once I got started onto it, I came up with more ideas on updating the Lip Sync HDA tools as well. At the time of development, the Lip Sync v3 HDA needed manual selecting mouth landmarks, (and it still does, because I haven’t released my updates to the new v4).
I admit the setup process to get my Lip Sync v3 HDA working requires manual selection of 12 points on the character’s mouth defining the landmarks of the mouth. This gets quite tedious, but in the big picture after selecting 12 points on your mesh, it starts talking with lip animation instantly after feeding in some audio files. Not too bad for just clicking 12 points on a face mesh!
Facial Expression Generator HDA (in Houdini)
I came up with an idea to make the 12 point selection process even easier and this was all thanks to working on the Facial Expressions HDA, which inspired this update. The Facial Expressions HDA requires manipulating the mouth points the same was as the Lip Sync HDA with some differences in functionality, you don’t need to displace the mesh with lip shapes. Smiling, evil grins, or sad puffy face, they all require the manipulation of the mouth and eyebrows to help sell the look.
This tool is currently still in the works and has been a great pleasure to test! You can imagine testing my Facial Expressions HDA tool on different characters’ faces to make them smile, sad, happy, and laugh is extremely fun!
Lip Sync HDA
Using Houdini for character lip animation was huge for me! When I first discovered Voice tools inside Houdini, I knew this was a game changer for me. I made wrappers around the Voice Tools in Houdini and manually defined 41 different phoneme lip shapes to produce assets that creates a super easy Lip Sync workflow for character animation. The Lip Sync HDA is one of my favorite tools that I developed, because in my eyes, it’s the most useful one of all.
Lip Sync animation Generator HDA (made in Houdini)
Lip Sync animation Generator HDA (made in Houdini)
Clothing Generator HDA (made in Houdini) Pants, Long Sleeve Shirt, Tank Top, and Vest
Clothing Generator HDA
The Clothing Generator HDA was mostly inspired by my small budget. I don’t have much money and I definitely don’t have the time nor talent to design every piece of clothing for each character in my soon to be Short Film. I needed a solution that would generate pieces of clothing that is unique enough to fit different body types and quick to use. And the Clothing Generator HDA was born!
I currently have a small selection of clothing at the time of writing this, but I believe that I can easily expand and update this tool to create more complex pieces of clothing in the future.
Character & Face Generator HDA
Head Generator HDA in Houdini
I created the Face Generator HDA first and afterwards I decided to make a full body generator like a Character Generator to complete the set. This was never meant to be high quality and never meant to be used for production, because I knew it would be too hard to achieve results of that caliber in the short time I had available. It was always meant for concept design or prototyping tools.
The Face Generator HDA turned out really good in the end, thanks to a lot of suggestions made by viewers during the Live Stream where I first demonstrated the prototype of the Face Generator HDA. The best parts of the Face Generator HDA is that it comes fully compatible with the Lip Sync tools and the new upcoming Facial Expressions HDA, making it even easier to make it talk and animate facial expressions.
Head Generator HDA with proper head topology automatically created in Houdini
Head Generator HDA (in Houdini) previewing the basic shape of the head before applying proper topology
Character Generator HDA in Houdini
Because the Character Generator HDA was never meant to be production ready quality, I needed to find an alternative to create characters and I also needed a hair generator. That’s where I started looking into Reallusion’s CC3 and discovered they had a ton of different addon’s.
I was actually going to start on a Hair Generator tool for Houdini, but I soon came across Reallusion’s new Hair & Beard Builder addon for CC3. I already bought the CC3 application, so the Hair & Beard Builder was very appealing when I first discovered the addon. (Reallusion’s Character Creator 3 is an application that allows you to procedural generator HQ characters very easily, check out a review of the software I did here.) This was perfect timing, because it would save me time from developing the hair generator and I could advance a bit faster into the schedule.
test my #mocap with ipisoft & #houdini & #Blender3d with my new Facial Expression HDA#blender #blendercommunity #indiegamedev #indiedev #goprocedural #gamedevelopment #gamedev #indiegame #gametools #animation3d #cg #CGI #VFX #3dartist #gameart #3dmodeling #MocapForAll pic.twitter.com/JESAq6t9pv
— Blue Bursting Bubble (@BubblePins) September 9, 2021
Mocap (Motion Capture), the Last Piece to the Puzzle
Motion capture was the last piece to the puzzle, but surprisingly it was one of the first things I already had a solution for as well. Many of my fans may know I own a Kinect, but what you may not have known is that I bought another Kinect to get a Dual Kinect Mocap setup. This was purchased near late last year and I had been doing a few simple tests with it. The original software I was using was Ni-Mate which worked beautifully with a single Kinect, but after I upgraded to the paid version of Ni-Mate, that’s when I realized it doesn’t support Dual Kinects. Highly disappointed and quite a blow to my budget as well.
Eric character from SideFX animated with Mocap & Dual Kinect setup captured with ipisoft and manually keyframed finger poses. Mouth Smiling animation is done with my Facial Expressions HDA
Dual Kinect with Ipisoft
I went on trying to find another solution and came across ipisoft, which at the time was having a sale! What great timing! It was quite a big sale too, at 30% discount, the money I saved from this discount made up for the money I spent on the Ni-Mate upgrade, which didn’t work with my Dual Kinect setup. So it was breakeven for me. Lucky save!
Unfortunately ipisoft only allows you to purchase a subscription plan and they don’t have anything for you to buy to own or at least that I’m not aware of anything of that sort. Anyways, I bought a 2 year subscription plan that was on sale at the time and did some simple testing. Amazing results! I’m not sure if it was the Dual Kinect setup but it caught my 360 turns and almost all the motions I threw at it.
This was the last piece I needed to do a full character animation pipeline, which will become my main backbone in my hopefully soon to come Short Film project.