Unilever PLC · LSE:ULVR
Unilever — Technical Presentation (September 2022)
24 slides from Unilever's technical presentation, Sep 26, 2022
Slide 1: Using automated machine learning for the prediction of developmental and reproductive toxicity
Using automated machine learning for the prediction of developmental and reproductive toxicity Showcase week (Theory) Marcus Wang 26 Sep 2022
Slide 2: Developmental and reproductive toxicity (DART)
Developmental and reproductive toxicity (DART) • Reproductive • Effects on fertility (male/female), childbirth, lactation • Developmental • Related to the fetus • Mortality, structural alterations, growth, functional deficits 2
Slide 3: Developmental and reproductive toxicity (DART)
Developmental and reproductive toxicity (DART) • Limited quality data • Expensive to measure • Long duration to get results • Ethical issues • Regulatory requirements 3
Slide 4: Data sources
Data sources • Data was compiled from a total of 12 different sources • includes both in vitro and in vivo data • Includes data from publicly available sources as well as commercial software eg. DEREK • Includes drug-like chemicals as well as industrial chemicals (ToxCast) • Database includes data c
Slide 5: How the data is processed
How the data is processed • Toxicity value recorded for each compound by source • Entries with missing or unclear data was removed • Salts and metals were removed • Database was checked for duplicates entries based on InChI Key, SMILES, CAS columns • Enantiomers were merged • Entries for the same co
Slide 6: Database statistics
Database statistics • 3255 compounds (no salts/metals) • 1672 positives • 1583 negatives • Largest known database for DART that has been compiled 6
Slide 7: Automated machine learning (AutoML)
Automated machine learning (AutoML) 7
Slide 8: Automated machine learning (AutoML)
Automated machine learning (AutoML) • Automating the machine learning process • Optimise model hyperparameters automatically • Automated feature selection/processing also possible 8
Slide 9: Automated machine learning (AutoML)
Automated machine learning (AutoML) • More efficient than manually specifying and adjusting hyperparameters • Increasing popularity in recent years 9
Slide 10: Automated machine learning (AutoML)
Automated machine learning (AutoML) • AutoGluon package used • Includes common machine learning models eg. Random forest, ANN 10
Slide 11: Model metrics
Model metrics • Proportion of true positives 𝑇𝑃 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 (𝑆𝐸) = 𝑇𝑃 + 𝐹𝑁 • Proportion of true negatives 𝑇𝑁 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 (𝑆𝑃) = 𝑇𝑁 + 𝐹𝑃 11
Slide 12: Model metrics
Model metrics • Correct classification by model 𝑇𝑁 + 𝑇𝑃 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 (𝑄) = 𝑇𝑁 + 𝐹𝑃 + 𝑇𝑃 + 𝐹𝑁 • Matthews correlation coefficient (MCC) 𝑇𝑃 × 𝑇𝑁 − (𝐹𝑃 × 𝐹𝑁) 𝑀𝐶𝐶 = (𝑇𝑃 + 𝐹𝑃)(𝑇𝑃 + 𝐹𝑁)(𝑇𝑁 + 𝐹𝑃)(𝑇𝑁 + 𝐹𝑁) 12
Slide 13: Preparing the data for input
Preparing the data for input • Per run for a total of 5 runs Database (ca. 3250 compounds) Random split Remaining data (80%) Test (20%) Random split 5-folds (64% training, 16% validation) 13
Slide 14: Benchmarking results
Model SE (%) SP (%) Accuracy (%) MCC CatBoost_BAG_L1 77.9 ± 4.9 87.8 ± 1.8 83.1 ± 2.2 0.662 ± 0.044 CatBoost_BAG_L2 ExtraTreesEntr_BAG_L1 78.9 ± 5.0 79.5 ± 4.6 90.7 ± 3.8 89.7 ± 3.7 85.1 ± 3.2 84.9 ± 3.2 0.704 ± 0.067 0.698 ± 0.067 Benchmarking ExtraTreesEntr_BAG_L2 79.0 ± 4.9 90.8 ± 3.5 85.3 ± 2.8
Slide 15: Benchmarking results
Benchmarking results • Models benchmarked against literature results • Better accuracy than all results so far on this dataset used for benchmarking Model SE (%) SP (%) Accuracy (%) MCC Jiang et al. 78.5 88.1 83.6 - 2019 Feng et al. 77.3 90.7 84.4 - 2021 WeightedEnsemble_L3 79.1 ± 4.4 91.4 ± 3.6 85.
Slide 16: Global models
Model SE (%) SP (%) Accuracy (%) MCC CatBoost_BAG_L1 68.9 ± 2.6 73.5 ± 1.1 71.0 ± 1.0 0.423 ± 0.019 CatBoost_BAG_L2 69.1 ± 5.0 74.8 ± 4.2 71.7 ± 1.3 0.44 ± 0.025 ExtraTreesEntr_BAG_L1 ExtraTreesEntr_BAG_L2 73.0 ± 1.5 71.8 ± 2.9 70.6 ± 1.9 73.6 ± 2.9 71.8 ± 0.6 72.6 ± 0.7 0.436 ± 0.012 0.453 ± 0.014
Slide 17: In vivo/in vitro results
In vivo/in vitro results • Models built for in vivo and in vitro datasets extracted from database • Models built with in vivo data are much better Test type Model SE (%) SP (%) Accuracy (%) MCC RandomForestGi in vivo ni_BAG_L1 82.6 ± 1.1 80.1 ± 2.9 81.5 ± 1.6 0.625 ± 0.032 RandomForestGi in vitro ni
Slide 18: In vivo/in vitro results
In vivo/in vitro results • Principal component analysis (PCA) plots of in vivo/in vitro data • Two different regions in feature space indicating chemicals are structurally different 18
Slide 19: Additional results
Additional results Comparison/Visualisation of other datasets extracted from database • ML models and feature plots • Developmental toxicity vs. reproductive toxicity • By data source 19
Slide 20: Work in progress
Work in progress Structural alerts (SAs) • Associate structural fragment with mechanism of action for toxicity • Constructed with Wedlake et al. 2020 KNIME workflow • Comparison with SAs from DART scheme profile (Wu et al. 2013) in OECD QSAR Toolbox, DEREK SAs and other sources of SAs in the literat
Slide 21: Conclusion
Conclusion • We report the largest known database (ca. 3255 chemicals) for the general prediction of DART. • We have used AutoML to quickly produce a best-performing model with 73% accuracy for predicting general DART. • These results can be used for screening purposes or as part of next-generation
Slide 22: Acknowledgements
Acknowledgements Goodman group 22
Slide 23: References
References (1) Challa, A. P.; Beam, A. L.; Shen, M.; Peryea, T.; Lavieri, R. R.; Lippmann, E. S.; Aronoff, D. M. Machine learning on drug-specific data to predict small molecule teratogenicity. Reprod Toxicol 2020, 95, 148-158. DOI: 10.1016/j.reprotox.2020.05.004 (2) Ciallella, H. L.; Russo, D. P.;
Slide 24: References
References (11) Gilmour, N.; Kimber, I.; Williams, J.; Maxwell, G. Skin sensitization: Uncertainties, challenges, and opportunities for improved risk assessment. Contact Dermatitis 2019, 80 (3), 195-200. DOI: 10.1111/cod.13167 (12) Baltazar, M. T.; Cable, S.; Carmichael, P. L.; Cubberley, R.; Cull,