Advancing sex estimation from amelogenin: Applications to archaeological, deciduous, and fragmentary dental enamel

A significant limitation of bioarchaeological and forensic research is inaccuracy of sex estimation techniques for skeletal remains of pre-pubertal individuals. This means that while age estimation is relatively accurate for individuals during growth and development, the inability to estimate sex widens the error ranges of age estimates and has limited the questions that can be asked about cultural and biological sex differences in past populations. From the forensic standpoint, it restricts an examiner's ability to estimate a key variable without expensive DNA testing. Dental enamel represents one of the hardest biological substances, and so is resilient to post-mortem degradation processes. It is frequently recovered from archaeological and forensic contexts even when other skeletal remains are too degraded for extensive recovery. One of the main proteins in dental enamel, amelogenin, is isomorphic based on sex, since it is coded by the AMELX and AMELY genes on the X- and Y-chromosomes (respectively). Recent work has demonstrated the effectiveness of proteomics in estimating sex based on the amelogenin protein using both modern and archaeological samples. The present research presents a validation study and further methodological development of the use of mass spectrometry-based peptide analysis for sex estimation. It also presents the first large scale study of peptide extraction from naturally produced enamel fragments (in addition to whole teeth), demonstrating the efficacy of enamel fragment use for sex estimation from archaeological human remains. We develop a straightforward and simple pipeline for processing the complex data for non-tryptic peptides in a timely manner, while providing a quantitative measure without the need for expensive commercial tools. Relative intensities of AMELX- and AMELY-unique peptides in individual samples were computed and an Amelogenin Sex Estimation score was derived as the difference of the log2 of these intensities. We validated our approach using samples from both modern individuals with known sex (n = 28) and archaeological individuals with osteologically-estimated sex (n = 40), yielding highly confident sex estimation rate of all samples with AMELX signals recovered, and peptide recovery unsuccessful in only one archaeological instance. This corresponds to a probability > 99 % for consistent assignment. We further analyzed samples from 86 individuals (representing 90 samples) of unknown sex, yielding a highly confident result in 83 individuals. Finally, we validated our method using data from an external study based on a different method and found a full agreement on all tested samples. Our results represent a significant methodological development with implications for archaeological and forensic human remains. Successful isolation of peptides from naturally derived fragments suggests a way forward for ethical practice and conservation, as it involves testing on samples which otherwise would provide little information.