QSAR studies of new pyrido[3,4-b]indole derivatives as inhibitors of colon and pancreatic cancer cell proliferation


We have discovered a new class of pyrido[b]bindole derivatives that show potent and broad spectrum anticancer activity with IC50 values down to submicromolar levels. Structure–activity relationship data acquired with the compounds as antiproliferative agents against several cancer cell lines, i.e., human HCT116 colon cancer cell line, HPAC and Mia-PaCa2 pancreatic cancer cell lines, were subjected to two different QSAR modeling methods. A kernel-based partial least squares (KPLS) regression analysis with chemical 2D fingerprint descriptors, and a PHASE pharmacophore alignment with 3D-QSAR study. The KPLS method afforded successful predictive QSAR models for antiproliferative activity of the HCT116 colon cell line and on two of the pancreatic cancer cell lines HPAC and Mia-PaCa2, with the following statistics: R2s of 0.99, 0.99, and 0.98, for training set coefficients of determination, and external test set predictive r2s of 0.70, 0.58, and 0.70, respectively. The best 2D fingerprint descriptor for both the HCT116 and HPAC data out of the eight finger prints utilized was the atom triplet fingerprint; whereas the one that worked best for the Mia-PaCa2 data was the linear fingerprint descriptor. The PHASE pharmacophore based 3D-QSAR study afforded a four-point pharmacophore model comprising one hydrogen bond donor (D) and three ring (R) elements, which yielded a successful 3D-QSAR model only with the HCT116 cell line data with training set R2 of 0.683, and an external test set predictive r2 of 0.562. With the PHASE 3D-QSAR, the influence of electronic effects and hydrophobicity were visualized, and were in agreement with the observed SAR of substitutions, while the KPLS method the relative extent of contribution of each atom in a compound to the activity. These models will foster the lead optimization process for this potent series of anticancer pyrido [3,4-b]indole compounds.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  1. An Y, Sherman W, Dixon SL (2013) Kernel-based partial least squares: application to fingerprint-based QSAR with model visualization. J Chem Inf Model 53:2312–2321

    CAS  Article  Google Scholar 

  2. Canvas User Manual Version 1.7 (2015) Schrödinger, LLC, New York, NY

  3. Canvas User Manual, Version 1.7 (2015) Schrödinger, LLC, New York, NY

  4. Chang G, Guida WC, Still WC (1989) An internal-coordinate Monte Carlo method for searching conformational space. J Am Chem Soc 111:4379–4386

    CAS  Article  Google Scholar 

  5. Dixon SL, Smondyrev AM, Rao SN (2006) PHASE: a novel approach to pharmacophore modeling and 3D databasesearching. Chem Biol Drug Des 67:370–372

    CAS  Article  Google Scholar 

  6. Duan J, Dixon SL, Lowrie JF, Sherman W (2010) Analysis and comparison of 2D fingerprints: Insights into database screening performance using eight fingerprint methods. J Mol Graph Model 29:157–170

    CAS  Article  Google Scholar 

  7. Golbraikh A, Shen M, Xiao Z, Xiao YD, Lee KH, Tropsha A (2003) Rational selection of training and test sets for the development of validated QSAR models. J Comput Aided Mol Des 17:241–253

    CAS  Article  Google Scholar 

  8. Hert J, Willett P, Wilton DJ, Acklin P, Azzaoui K, Jacoby E, Schuffenhauer A (2004) Comparison of fingerprint-based methods for virtual screening using multiple bioactive reference structures. J Chem Inf Comput Sci 44:1177–1185

    CAS  Article  Google Scholar 

  9. Hert J, Willett P, Wilton DJ, Acklin P, Azzaoui K, Jacoby E, Schuffenhauer A (2005) Enhancing the effectiveness of similarity-based virtual screening using nearest-neighbor information. J Med Chem 48:7049–7054

    CAS  Article  Google Scholar 

  10. Jorgensen WL, Maxwell DS, Tirado-Rives J (1996) Development and testing of the OPLS all-atom force field on conformational energetics and properties of organic liquids. J Am Chem Soc 118:11225–11236

    CAS  Article  Google Scholar 

  11. Kolossvary I, Guida WC (1996) Low mode search: an efficient, automated computational method for conformational analysis—application to cyclic and acyclic alkanes and cyclic peptides. J Am Chem Soc 118:5011–5019

    CAS  Article  Google Scholar 

  12. Marriott DP, Dougall IG, Meghani P, Liu YJ, Flower DR (1999) Lead generation using pharmacophore mapping and three-dimensional database searching: application to muscarinic M(3) receptor antagonists. J Med Chem 42:3210–3216

    CAS  Article  Google Scholar 

  13. Patil SA, Addo JK, Deokar H, Sun S, Wang J, Li W, Suttle DP, Wang W, Zhang R, Buolamwini JK (2017) Synthesis, Biological Evaluation and Modeling Studies of New Pyrido[3,4-b]indole Derivatives as Broad-Spectrum Potent Anticancer Agents. Drug Des. 6:143

    Article  Google Scholar 

  14. Rosipal R (2003) Kernel partial least squares for nonlinear regression and discrimination. Neural Netw World 13:291–300

    Google Scholar 

  15. Sastry M, Lowrie JF, Dixon SL, Sherman W (2010) Large-scale systematic analysis of 2D fingerprint methods and parameters to improve virtual screening enrichments. J Chem Inf Model 50:771–784

    CAS  Article  Google Scholar 

  16. Schrödinger Suite 2015-4, Maestro, version 10.4, Schrödinger, LLC, New York, NY

  17. Wang W, Qin JJ, Voruganti S, Wang MH, Sharma H, Patil S, Zhou J, Wang H, Mukhopadhyay D, Buolamwini JK, Zhang R (2014a) Identification of a new class of MDM2 inhibitor that inhibits growth of orthotopic pancreatic tumors in mice. Gastroenterology 147:893–902

    CAS  Article  Google Scholar 

  18. Wang W, Qin JJ, Voruganti S, Srivenugopal KS, Nag S, Patil S, Sharma H, Wang MH, Wang H, Buolamwini JK, Zhang R (2014b) The pyrido[b]indole MDM2 inhibitor SP-141 exerts potent therapeutic effects in breast cancer models. Nat Commun 5:5086

    CAS  Article  Google Scholar 

  19. Willett P (2006) Similarity-based virtual screening using 2D fingerprints. Drug Discov Today 11:1046–1053

    CAS  Article  Google Scholar 

Download references


Financial support from the Colleges of Pharmacy at Rosalind Franklin University of Medicine and Science and the University of Tennessee Health Science Center is acknowledged, and so is financial support from National Institutes of Health (NIH)/ National Cancer Institute (NCI) grants CA100102 and RO1CA186662 subcontract (Zhang, PI) that partly supported the work in the Buolamwini laboratory. RZ was also supported by NIH/NCI grants R01 CA186662 and R01CA214019. The content is solely the responsibility of the authors, and does not necessarily represent the official views of the National Institutes of Health.

Author information



Corresponding author

Correspondence to John K. Buolamwini.

Ethics declarations

Conflict of interest

JKB, RZ, and WW are co-inventors on patents pertaining to the compounds. The remaining authors declare that they have no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Deokar, H., Deokar, M., Wang, W. et al. QSAR studies of new pyrido[3,4-b]indole derivatives as inhibitors of colon and pancreatic cancer cell proliferation. Med Chem Res 27, 2466–2481 (2018). https://doi.org/10.1007/s00044-018-2250-5

Download citation


  • KPLS
  • Fingerprints
  • 3D-QSAR
  • Anticancer activity
  • Beta-carboline
  • Pharmacophore