Using Machine Learning to Develop an Artificial Intelligence Algorithm that Guides Nasolabial Repair
Lohrasb Ross Sayadi 1, James B. Hu 1, Andrew K. Guan 1, QiLong Zhangli 1, Jeffrey Zhang 1, Eldrick Millares 2, Raj Vyas MD FACS 1. 1 University Of California Irvine, Department of Plastic Surgery 2 Stanford University, Department of Engineering
Background: Cleft lip and nasal repair restores oral function and strives to achieve normal nasolabial aesthetics. Anthropometry of the lip and nose guides operative design, yet mastering cleft operative markings and surgical sequencing can be challenging. To accelerate this learning curve, we developed an artificial intelligence (AI) based platform that uses a novel machine learning algorithm to reliably detect cleft nasolabial anthropometry.
Methods: We utilized High-Resolution Net (HRNet), a recent family of deep learning models that has achieved state of the art results in many computer-vision tasks, including facial landmark detection. HRNet follows the current trend in computer vision of stacking multiple convolutional layers, but differs in one key area. Whereas previous models generally downsample the dimensionality of the input at each layer, HRNet performs this downsampling in parallel with a series of convolutional layers that preserves dimensionality and allows for intermediate representations with higher dimensionality while simultaneously extracting lower dimension features. To adapt the facial landmark detection HRNet for our task, we employed transfer learning, a technique in machine learning to transfer knowledge gained from a source task to a target task. Transfer learning reduces training time, increases accuracy on target task, and reduces required training examples in the target task. Here, a craniofacial plastic surgeon manually marked the key anthropometric landmarks of 460 two-dimensional photographs of infants and children with unilateral cleft lip. These images are compared against the detected markings assigned by our algorithm. For model evaluation, we calculated error using the Normalized Mean Error (NME), an evaluation metric in facial landmark detection.
Results: After training on our dataset, we obtained NMEs for each point finding that all points were found between 0.02929 (rala) and 0.05544 (rcphi). In comparison, NME in state-of-the-art facial recognition datasets is in the range of 0.0385 (300W) to 0.0460 (WFLW). Our training dataset is about 1% the size of these benchmarks, illustrating ability to leverage relatively small quantities of data to achieve surprisingly accurate marking of cleft lip/nose anthropometry.
Conclusion: In the present study, we developed a deep learning model that accurately identifies the nasolabial anthropometry of a unilateral cleft lip deformity and uses this information to mark a unilateral cleft lip/nose repair on a preoperative photograph. Using light-based three dimensional surface projection technology developed by our team, we plan to harness our cleft facial recognition algorithm to project cleft repair markings onto the three dimensional surface anatomy of nasolabial clefts. Combining this AI-based platform with augmented reality (AR) based platforms optimizes remote guidance and facilitates knowledge and skill transfer. Such technology can impact and accelerate both domestic surgical teaching and overseas cleft outreach.
Back to 2021 Abstracts