HoloTrauma 3X Triadic AI Co reasoning for robot assisted emergency maxillofacial reconstruction

References

• Joachim, D. J. & Miloro, M. The evolution of virtual surgical planning in craniomaxillofacial surgery: A comprehensive review. J. Oral. Maxillofac. Surg. 83, 294–306 (2025).

Google Scholar

• Karnatz, N. et al. Development and application of digital maxillofacial surgery system based on mixed reality technology. Front. Surg. 8, 719985 (2022).

Google Scholar

• Miragall, M. F. et al. Face the future—artificial intelligence in oral and maxillofacial surgery. J. Clin. Med. 12, 6843 (2023).

Google Scholar

• Chen, Z. et al. A meta-analysis and systematic review comparing the effectiveness of traditional and virtual surgical planning for orthognathic surgery: Based on randomized clinical trials. J. Oral. Maxillofac. Surg. 79, 471.e1–471.e9 (2021).

Google Scholar

• Vyas, K., Gibreel, W. & Mardini, S. Virtual surgical planning (vsp) in craniomaxillofacial reconstruction. Facial Plast. Surg. Clin. North Am. 30, 239–252 (2022).

Google Scholar

• Lin, H.-H. & Lo, L.-J. Three-dimensional computer-assisted surgical simulation and intraoperative navigation in orthognathic surgery: A literature review. J. Formos. Med. Assoc. 114, 300–307 (2015).

Google Scholar

• Motamedian, S. R., Khojasteh, A. & Sadeghi, M. Application of artificial intelligence in orthognathic surgery: A scoping review. J. Stomatol., Oral. Maxillofac. Surg. 126, 101234 (2025).

Google Scholar

• Hua, J., Aziz, S. & Shum, J. W. Virtual surgical planning in oral and maxillofacial surgery. Oral. Maxillofac. Surg. Clin. North Am. 31, 519–530 (2019).

Google Scholar

• Keyser, B., Afzal, Z. & Warburton, G. Computer-assisted planning and intraoperative navigation in the management of temporomandibular joint ankylosis. Atlas Oral. Maxillofac. Surg. Clin. North Am. 28, 111–120 (2020).

Google Scholar

• Khadka, A. et al. Changes in upper airway after mandibular setback surgery in obstructive sleep apnea patients and non-obstructive sleep apnea patients: A systematic review. Int. J. Oral. Maxillofac. Surg. 40, 1135–1142 (2011).

Google Scholar

• Wilkat, M. et al. Enhancing surgical occlusion setting in orthognathic surgery planning using mixed reality technology: A comparative study. Clin. Oral. Investig. 28, 547 (2024).

Google Scholar

• Tian, S. et al. Deep learning in medical image analysis for craniofacial deformity: A review. Bioengineering 10, 1268 (2023).

Google Scholar

• Swennen, G. R. J., Mollemans, W. & Schutyser, F. Three-dimensional treatment planning of orthognathic surgery in the era of virtual imaging. J. Oral. Maxillofac. Surg. 67, 2080–2092 (2009).

Google Scholar

• Hsu, S. S.-S. et al. Accuracy of a computer-aided surgical simulation protocol for orthognathic surgery: A prospective multicenter study. J. Oral. Maxillofac. Surg. 71, 128–142 (2013).

Google Scholar

• Gateno, J. et al. Clinical feasibility of computer-aided surgical simulation (cass) in the treatment of complex cranio-maxillofacial deformities. J. Oral. Maxillofac. Surg. 79, 1103–1120 (2021).

Google Scholar

• Simon, B. D., Ozyoruk, K. B., Gelikman, D. G., Harmon, S. A. & Türkbey, B. The future of multimodal artificial intelligence models for integrating imaging and clinical metadata: A narrative review. Diagnostic Interventional Radiol. 31, 303–312 (2025).

Google Scholar

• Lee, J. O. et al. Multimodal generative ai for interpreting 3d medical images and videos. npj Digital Med. 8, 273 (2025).

Google Scholar

• Liu, H., Li, C., Wu, Q. & Lee, Y. J. Visual instruction tuning. In Advances in Neural Information Processing Systems (NeurIPS), 36 (2023). 2304.08485.
• Zhang, S., Xu, Y. & Usuyama, N. Vision-language models in medical imaging: A survey. IEEE Rev. Biomed. Eng. 17, 234–257 (2024).

Google Scholar

• Acosta, J. N., Falcone, G. J., Rajpurkar, P. & Topol, E. J. Multimodal biomedical ai. Nat. Med. 28, 1773–1784 (2022).

Google Scholar

• Achiam, J. et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. 2303.08774 (2023).
• Tu, T. et al. Towards generalist biomedical ai. NEJM AI 1, AIoa2300138 (2024).

Google Scholar

• Moor, M., Banerjee, O. & Abad, Z. S. H. Foundation models for generalist medical artificial intelligence. Nature 616, 259–265 (2023).

Google Scholar

• Li, B., Wang, R. & Wang, G. Llava-med: Training a large language-and-vision assistant for biomedicine. Nat. Commun. 15, 1735 (2024).

Google Scholar

• Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023).

Google Scholar

• Liu, H., Li, C., Li, Y. & Lee, Y. J. Improved baselines with visualinstruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 26296–26306 (2024).
• Zhao, W. X., Zhou, K. & Li, J. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).
• Wang, S., Zhao, Z. & Ouyang, X. Large language models for healthcare: A comprehensive benchmark. npj Digital Med. 7, 45 (2024).

Google Scholar

• Brown, T. B. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).

Google Scholar

• Kitaguchi, D., Takeshita, N., Hasegawa, H. & Ito, M. Artificial intelligence-based computer vision in surgery: Recent advances and future perspectives. Ann. Gastroenterological Surg. 6, 29–36 (2022).

Google Scholar

• Mascagni, P. et al. Artificial intelligence for surgical safety: Automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann. Surg. 275, 955–961 (2022).

Google Scholar

• Xu, B. et al. Multimodal machine learning in image-based and clinical biomedicine: Survey and prospects. Int. J. Computer Vis. 132, 3753–3786 (2024).

Google Scholar

• Haidegger, T. Robot-assisted minimally invasive surgery—surgical robotics in the data age. Proc. IEEE 110, 835–864 (2022).

Google Scholar

• Aung, Y. M., Al-Jumaily, A. & Ong, S. K. Robotic system for orthognathic surgery: Concept and design. Int. J. Computer Assist. Radiol. Surg. 11, 2169–2177 (2016).

Google Scholar

• Fuchs, A. et al. Robotic-assisted surgery in oral and maxillofacial surgery: A review. Clin. Oral. Investig. 28, 60 (2024).

Google Scholar

• Moglia, A., Georgiou, K., Georgiou, E., Satava, R. M. & Cuschieri, A. A systematic review on artificial intelligence in robot-assisted surgery. Int. J. Surg. 95, 106151 (2021).

Google Scholar

• Fiorini, P., Goldberg, K. Y., Liu, Y. & Taylor, R. H. Concepts and trends in autonomy for robot-assisted surgery. Proc. IEEE 110, 993–1011 (2022).

Google Scholar

• Datta, S. et al. Reinforcement learning in surgery. Surgery 170, 329–332 (2021).

Google Scholar

• Qi, C. R., Yi, L., Su, H. & Guibas, L. J. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Adv. Neural Inf. Process. Syst. 30, 5099–5108 (2017).

Google Scholar

• He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778 (IEEE, 2016).
• Radford, A. et al. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML), 139, 8748–8763 (PMLR, 2021).
• Efron, B., Tibshirani, R. J. An Introduction to the Bootstrap. Chapman and Hall/CRC: New York, 1994.

Google Scholar

• Austin, P. C. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46, 399–424 (2011).

Google Scholar

• On, S. W., Cho, S. W., Park, S. Y., Jung, Y. S. & Kim, H. M. Advancements in computer-assisted orthognathic surgery: A comprehensive review and clinical application in south korea. J. Dent. 146, 105061 (2024).

Google Scholar

• Tang, Y. et al. Self-supervised pre-training of Swin Transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20730–20740 (IEEE, 2022).
• Nori, H. et al. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. arXiv preprint arXiv:2311.16452 (2023).
• Bai, W. et al. M3d-medical: A multi-modal 3d medical image analysis framework with large language models. arXiv preprint arXiv:2404.15537 (2024).
• Kiyasseh, D. et al. A vision transformer for decoding surgeon activity from surgical videos. Nat. Biomed. Eng. 7, 780–796 (2023).

Google Scholar

• Dosovitskiy, A., Beyer, L. & Kolesnikov, A. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations (2021).
• Liao, S., Ma, L., Lu, F. & Tian, Y. Deep learning-based automated cephalometric landmark detection for orthognathic surgery planning. Int. J. Computer Assist. Radiol. Surg. 17, 371–380 (2022).

Google Scholar

• Hatamizadeh, A., Tang, Y. & Nath, V. Unetr: Transformers for 3d medical image segmentation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 574–584 (2022).
• Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthc. 3, 1–23 (2022).

Google Scholar

• Karaman, S. & Frazzoli, E. Sampling-based algorithms for optimal motion planning. Int. J. Robot. Res. 30, 846–894 (2011).

Google Scholar

Download references