Comparison of various methods for validity evaluation of QSAR models

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Quantitative structure–activity relationship (QSAR) modeling is one of the most important computational tools employed in drug discovery and development. The external validation of QSAR models is the main point to check the reliability of developed models for the prediction activity of not yet synthesized compounds. It was performed by different criteria in the literature.

Methods

In this study, 44 reported QSAR models for biologically active compounds reported in scientific papers were collected. Various statistical parameters of external validation of a QSAR model were calculated, and the results were discussed.

Results

The findings revealed that employing the coefficient of determination (r ²) alone could not indicate the validity of a QSAR model. The established criteria for external validation have some advantages and disadvantages which should be considered in QSAR studies.

Conclusion

This study showed that these methods alone are not only enough to indicate the validity/invalidity of a QSAR model.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13065-022-00856-4.

Related collections

Most cited references 60

Record: found
Abstract: found
Article: not found

QSAR modeling: where have you been? Where are you going to?

Artem Cherkasov, Eugene N Muratov, Denis Fourches … (2014)

Quantitative structure-activity relationship modeling is one of the major computational tools employed in medicinal chemistry. However, throughout its entire history it has drawn both praise and criticism concerning its reliability, limitations, successes, and failures. In this paper, we discuss (i) the development and evolution of QSAR; (ii) the current trends, unsolved problems, and pressing challenges; and (iii) several novel and emerging applications of QSAR modeling. Throughout this discussion, we provide guidelines for QSAR development, validation, and application, which are summarized in best practices for building rigorously validated and externally predictive QSAR models. We hope that this Perspective will help communications between computational and experimental chemists toward collaborative development and use of QSAR models. We also believe that the guidelines presented here will help journal editors and reviewers apply more stringent scientific standards to manuscripts reporting new QSAR studies, as well as encourage the use of high quality, validated QSARs for regulatory decision making.

0 comments Cited 280 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Is Open Access

QSAR without borders

Eugene N Muratov, Jürgen Bajorath, Robert P Sheridan … (2020)

Word cloud summary of diverse topics associated with QSAR modeling that are discussed in this review. Prediction of chemical bioactivity and physical properties has been one of the most important applications of statistical and more recently, machine learning and artificial intelligence methods in chemical sciences. This field of research, broadly known as quantitative structure–activity relationships (QSAR) modeling, has developed many important algorithms and has found a broad range of applications in physical organic and medicinal chemistry in the past 55+ years. This Perspective summarizes recent technological advances in QSAR modeling but it also highlights the applicability of algorithms, modeling methods, and validation practices developed in QSAR to a wide range of research areas outside of traditional QSAR boundaries including synthesis planning, nanotechnology, materials science, biomaterials, and clinical informatics. As modern research methods generate rapidly increasing amounts of data, the knowledge of robust data-driven modelling methods professed within the QSAR field can become essential for scientists working both within and outside of chemical research. We hope that this contribution highlighting the generalizable components of QSAR modeling will serve to address this challenge.

0 comments Cited 194 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient.

Paola Gramatica, Nicola Chirico (2011)

The main utility of QSAR models is their ability to predict activities/properties for new chemicals, and this external prediction ability is evaluated by means of various validation criteria. As a measure for such evaluation the OECD guidelines have proposed the predictive squared correlation coefficient Q(2)(F1) (Shi et al.). However, other validation criteria have been proposed by other authors: the Golbraikh-Tropsha method, r(2)(m) (Roy), Q(2)(F2) (Schüürmann et al.), Q(2)(F3) (Consonni et al.). In QSAR studies these measures are usually in accordance, though this is not always the case, thus doubts can arise when contradictory results are obtained. It is likely that none of the aforementioned criteria is the best in every situation, so a comparative study using simulated data sets is proposed here, using threshold values suggested by the proponents or those widely used in QSAR modeling. In addition, a different and simple external validation measure, the concordance correlation coefficient (CCC), is proposed and compared with other criteria. Huge data sets were used to study the general behavior of validation measures, and the concordance correlation coefficient was shown to be the most restrictive. On using simulated data sets of a more realistic size, it was found that CCC was broadly in agreement, about 96% of the time, with other validation measures in accepting models as predictive, and in almost all the examples it was the most precautionary. The proposed concordance correlation coefficient also works well on real data sets, where it seems to be more stable, and helps in making decisions when the validation measures are in conflict. Since it is conceptually simple, and given its stability and restrictiveness, we propose the concordance correlation coefficient as a complementary, or alternative, more prudent measure of a QSAR model to be externally predictive.

0 comments Cited 128 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Ali Shayanfar: shayanfara@tbzmed.ac.ir

Journal

Journal ID (nlm-ta): BMC Chem

Journal ID (iso-abbrev): BMC Chem

Title: BMC Chemistry

Publisher: Springer International Publishing (Cham )

ISSN (Electronic): 2661-801X

Publication date (Electronic): 23 August 2022

Publication date PMC-release: 23 August 2022

Publication date Collection: December 2022

Volume: 16

Issue: 1

Electronic Location Identifier: 63

Affiliations

[1 ]GRID grid.412888.f, ISNI 0000 0001 2174 8913, Student Research Committee, Faculty of Pharmacy, , Tabriz University of Medical Sciences, ; Tabriz, Iran

[2 ]GRID grid.412888.f, ISNI 0000 0001 2174 8913, Pharmaceutical Analysis Research Center, , Tabriz University of Medical Sciences, ; Tabriz, Iran

[3 ]GRID grid.412888.f, ISNI 0000 0001 2174 8913, Editorial Office of Pharmaceutical Sciences Journal, Faculty of Pharmacy, , Tabriz University of Medical Sciences, ; Tabriz, Iran

Article

Publisher ID: 856

DOI: 10.1186/s13065-022-00856-4

PMC ID: 9396839

PubMed ID: 35999611

SO-VID: 17a0a133-6275-4b50-8cb5-d5741ff7b212

License:

Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

History

Date received : 22 April 2022

Date accepted : 9 August 2022

Funding

Funded by: FundRef http://dx.doi.org/10.13039/501100004366, Tabriz University of Medical Sciences;

Award ID: 65369

Award Recipient : Ali Shayanfar

Custom metadata

Keywords: biological activity,external validation,qsar,statistical parameters

Data availability:

Keywords: biological activity, external validation, qsar, statistical parameters

Comparison of various methods for validity evaluation of QSAR models

Read this article at

Abstract

Background

Methods

Results

Conclusion

Supplementary Information

Related collections

NeuroImaging Methods

Most cited references 60

QSAR modeling: where have you been? Where are you going to?

QSAR without borders

Real external predictivity of QSAR models: how to evaluate it? Comparison of different validation criteria and proposal of using the concordance correlation coefficient.

Author and article information

Contributors

Journal

Affiliations

Article

History

Funding

Categories

Custom metadata

Comments

Comment on this article

Similar content 856

Cited by 4

Most referenced authors 425