DFNRMVS Tutorial, make your own high quality 3d face mesh from pictures

Here is a detailed tutorial on how to make the code DFNRMVS to work on your pc.

on the link you have attached a txt file with this explanation, might be better to follow.

Install miniconda, get it from:

https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe

Download the ZIP from the sites bellow

https://github.com/zqbai-jeremy/DFNRMVS

https://github.com/zqbai-jeremy/face3d

Unzip them and you will have 2 new folders, called: "DFNRMVS-master" and "face3d-master"

go inside "DFNRMVS-master" and create a folder called "external"

rename "face3d-master" to "face3d"

Move this folder to be inside "DFNRMVS-master/external"

you will end up having this folders/files

DFNRMVS-master\external\face3d\examples

DFNRMVS-master\external\face3d\face3d

DFNRMVS-master\external\face3d\3D Face Papers.md

DFNRMVS-master\external\face3d\README.md

#Open Conda CMD

#Type

conda create -n DFNRMVS python=3.9

activate dfnrmvs

#Get inside the unzipped DFNRMVS folder and install

pip install open3d trimesh pyrender pytube pillow scikit-image matplotlib Cython

conda install -c conda-forge tinyobjloader

pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116

pip install face-alignment

go inside

"DFNRMVS-master\external\face3d\face3d\mesh\cython"

and execute

python setup.py build_ext -i

You can give a test to see if its working, getting inside the examples folder

"\DFNRMVS-master\external\face3d\examples"

and typing

python 1_pipeline.py

A image called rendering might show up on the folder with current date:

"DFNRMVS-master\external\face3d\examples\results\pipeline\rendering.jpg"

Create the folder Out inside the path

"DFNRMVS-master\external\face3d\examples\Data\BFM"

you end up with the path

"DFNRMVS-master\external\face3d\examples\Data\BFM\Out\"

Download the file https://github.com/Juyong/3DFace/blob/master/Exp_Pca.bin

and put it in

"DFNRMVS-master\external\face3d\examples\Data\BFM\Out\"

Download the file https://raw.githubusercontent.com/microsoft/Deep3DFaceReconstruction/master/BFM/std_exp.txt

And save it at

"DFNRMVS-master\external\face3d\examples\Data\BFM\Out\"

Create the folder "net_weights" on the DFNRMVS folder, the path will be:

"DFNRMVS-master\net_weights"

Download the file https://drive.google.com/file/d/1HvsJ8IztdUS8VUNSLlkdWsOOJ3AgZ2Pe/view?usp=sharing

and

https://drive.google.com/file/d/1GITjxOq_QzJqY3KTfB3UmAYt8ZRaUDO8/view?usp=sharing

and put it in

"DFNRMVS-master\net_weights"

##### Getting the BFM.mat, BFM_info.mat and BFM_UV.mat

Got to the site:

https://faces.dmi.unibas.ch/bfm/main.php?nav=1-2&id=downloads

you must agree with all the Terms and fill the form to receiva an email with the link

On the link:

Get Face Profiling: http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3DDFA/Code/FaceProfilingRelease_v1.1.zip

and 3DDFA: http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3DDFA/Code/3DDFA.zip

On the link http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/HPEN/main.htm

Get the file: http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/HPEN/Code/CVPR2015_HPEN.zip

From the link https://github.com/anilbas/3DMMasSTN/blob/master/util/BFM_UV.mat

Download the "BFM_UV.mat" file and save it at "DFNRMVS-master\external\face3d\examples\Data\BFM\stn"

Extracting

Extract BaselFaceModel.tgz, get "01_MorphableModel.mat" and save at

"DFNRMVS-master\external\face3d\examples\Data\BFM\raw"

From FaceProfilingRelease_v1.1.zip, export the files "Model_tri_mouth.mat,Model_face_contour_trimed.mat" to

"DFNRMVS-master\external\face3d\examples\Data\BFM\3ddfa"

From 3DDFA.zip, go tothe folder "3DDFA_Release\Matlab" export the files "Model_Expression.mat,vertex_code.mat" to

"DFNRMVS-master\external\face3d\examples\Data\BFM\3ddfa"

From CVPR2015_HPEN.zip, go tothe folder "CVPR2015_HPEN\High-Fidelity_PEN" export the files "Modelplus_nose_hole.mat,Modelplus_parallel.mat " to

"DFNRMVS-master\external\face3d\examples\Data\BFM\3ddfa"

From CVPR2015_HPEN.zip, go tothe folder "CVPR2015_HPEN\High-Fidelity_PEN\ModelGeneration" export the files "model_info.mat" to

"DFNRMVS-master\external\face3d\examples\Data\BFM\3ddfa"

If you dont have Matlab, register and get a trial version at: https://www.mathworks.com/

load the "D:\FACE\DFNRMVS-master\external\face3d\examples\Data\BFM\generate.m" file on matlab

Execute.

If you have problem executing saying that it could not find the file, change the code to have the full path for the files that it loads

in my case, they were:

DFNRMVS-master/external/face3d/examples/Data/BFM/raw/01_MorphableModel.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/3ddfa/Model_Expression.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/3ddfa/Model_tri_mouth

DFNRMVS-master/external/face3d/examples/Data/BFM/3ddfa/Model_face_contour_trimed.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/3ddfa/Modelplus_nose_hole.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/3ddfa/Modelplus_parallel

DFNRMVS-master/external/face3d/examples/Data/BFM/3ddfa/vertex_code.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/Out/pncc_code.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/stn/BFM_UV.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/Out/BFM.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/Out/BFM_info.mat

DFNRMVS-master/external/face3d/examples/Data/BFM/Out/BFM_UV.mat

It will generate

BFM.mat

BFM_info.mat

BFM_UV.mat

on the folder: "DFNRMVS-master\external\face3d\examples\Data\BFM\Out"

You can try to run it, going inside the DFNRMVS root folder "DFNRMVS-master" and type

"python demo.py"

To be sure that everything is working, I suggest to erase all the folder inside "DFNRMVS-master\out_dir" because the code already comes with those files generated.

Rmoving the folders inde it, you will be sure that everything is working.

If everything went well, it meas that everything is setyup correctly, and you can start making your own 3d face meshes.

Get at least 2 pictures of the same head, try to get pictures with an angle of at least 30º for a better performance.

Leave just the head on the image, and resize both pictures to 240x240.

Put the images on the folder "DFNRMVS-master\out_dir\case0" and run

"python demo.py"

And Have fun making your own high quality 3d face meshes 😊