Photogrammetry: 3D Facial Scanning

What I noticed the most during this week’s working with facial scanning using photographs was the importance of having good quality photographs, and lots of them. I could not get the program to construct the 3D model correctly, not even by going in and correcting manually. Perhaps this was due to not having enough photos of good enough quality, maybe it is because my lovely skin is too smooth for the program to find good reference points, although that is not very likely.

The first thing when doing a facial scan is to place markers on the face of the model being photographed. This is because the program needs reference points to calculate the angles and directions of the photographs, you need at least four of these reference points for the program to be able to calculate.

Here you can see the four marked points

  

I did hear from one of my classmates that birthmarks worked very well as reference points, I was not able to test this myself as I do not have any though. Other possible landmarks to use are nostrils and corners of the eyes. I think that the most important thing is to use reference points placed on areas that cannot be moved very much by the muscles of the face since it can be very difficult to stay completely still during a photography session.

Next thing you do is to take pictures of your model from all angles, all around the head – straight on, from beneath and above.

Examples of pictures from different angles

 

When you have your photos and made sure that they are of good quality (I deem it very possible that one problem with my pictures was that some were a bit blurry) you transfer them into a computer and then run them through the program that you are going to use, we used Autodesk’s 123D Catch.

Manual stichting in 123D Catch

You follow the steps prompted by the program and it will hopefully process the images into a 3D model, you might need to use the tools provided inside the program to adjust things. I tried to manually stich and check and add reference points, although it did help it did not work in the end and I had to use an example head provided by our teacher.

Messed up scan in 123D Catch

When the model is done you can export it as a 3D file that your 3D software can import, for instance .obj or .fbx, and then import it into the program – we used a trial version of 3D Coat for this. Once it was loaded in 3D Coat I started to draw on the typology, starting around the eyes as that is one of the most important areas where it is very important to use continuous loops to create good, believable animations, and then going on to the mouth which is very similar.

Screenshot from 3D Coat showing typology around eye and mouth areas

It would have been a fun experience to have worked on my own face, I wish it would have worked. I think it could also have been a good learning experience when doing the typology since I am (obviously) very familiar with my own face, which could have made it easier to understand where to draw the typology. I do not think it was necessary though, but it would have been fun.

I think Photogrammetry is truly amazing; the level of realism that this provides is astonishing. I think that you might need to question the necessity of it though; do you want to create that kind of realism just because you can? I think it may be risky as it could take away the artistic expression if you are not careful. It is very important to think through very thoroughly what kind of style you want to have in your game, and WHY.

I absolutely love realistic art and like to experiment on making it as realistic as I possibly can, the feeling of every new drawing looking more like a photograph than the last one. But during my time at the University I have come to question the why, is it really important to make realistic things? As an old art teacher once told me, is there need for it when the camera can do it better?

If this is something that can optimize the workflow when creating game art, which I felt it could although it was very tedious and much work, it can be an absolutely fantastic tool. I just think that we need to keep in mind the artistic expression so that it is not lost in a chase of absolute realism.