Background

Protein structure prediction is one of the most important problems in structural bioinformatics [1–3]. Comparative (or homology) modeling is currently the most accurate and practical structure prediction method [4–19].

In general comparative modeling involves four steps [11, 20, 21]: (1) identify a homologous template protein for a target protein; (2) generate an alignment between the template and the target; (3) create a model based on the alignment and the template structure; (4) evaluate and refine the model. The two key factors determining the quality of comparative modeling are the template structure and the alignment accuracy [22]. Traditionally, automated comparative modeling methods use the top-ranked template and its alignment with the target protein to model its structure. This approach cannot always achieve the best results because it may not be able to select the best template and to generate the optimal alignment [23]. Previous research [6, 24–28], particularly the human prediction [23, 29–31] in the six edition of Critical Assessment of Techniques of Protein Structure Prediction (CASP6) [32–36], has shown that using multiple templates can often improve the quality of comparative modeling over a single template. Although human experts commonly select multiple templates and combine them manually to predict structure in their practice, multiple-template combination has not been widely used by automated servers until the latest CASP7 experiment, 2006. In CASP7, several servers including FOLDpro and 3Dpro [37, 38], HHSearch [39], 3D-JIGSAW-POPULUS [13], MetaTasser [30, 40, 41], Zhang-Server [29], FAMS [42], used multiple templates to improve template-based structure prediction. Some of these methods such as Zhang-Server, FOLDpro, and 3Dpro show the good performance on the comparative modeling targets, particularly on the high-accuracy modeling targets. However, few of the automated multi-template methods are published so far.

A published multiple-template algorithm [29–31] tries to extract distance (or contact) restraints from multiple templates. The consistent distance restraints from multiple templates are used to guide structure modeling. This method is currently coupled with the in-house model assembly tool TASSER [29, 31] and cannot be used with the popular, publicly available, comparative model generation tools such as Modeller [8, 11, 20, 43, 44], nest [17], SEGMOD-ENCAD [45], SWISS-Model [18, 46, 47], 3D-JIGSAW [13], and Builder [9]. Most of these tools take as inputs the alignments between templates and a target to generate structure models, instead of directly accepting distance restraints.

Thus, instead of extracting distance restraints from multiple templates as in [29–31], we develop a different, parametric algorithm to select templates and to combine their alignments directly. The multiple alignments between the target and the templates can be directly fed into the widely used, standard comparative modeling tools such as Modeller [8] with the built-in multi-template modeling function, to generate models. The preliminary implementation of the method was ranked second in the automated high-accuracy structure prediction during the CASP7 community-wide experiment.

Furthermore, we systematically and rigorously compare the performance of the multiple- and single-template methods on the 45 comparative modeling targets of the CASP7 experiment. On average the multiple-template combination algorithm improves the GDT-TS score [48] of the predicted models by 6.8% over the single template approach. The pairwise statistical analysis shows that the improvement is significant. Thus, our experiment strictly demonstrates that the combination of multiple templates and their alignments can significantly improve comparative modeling over the single top template approach. Moreover, we compare the multi-template method against the ideal approach, which always uses the best, possible template in the Protein Data Bank [49]. The analysis shows that the multi-template combination algorithm can even achieve slightly better performance than the ideal approach on the 27 CASP7 comparative modeling targets. However, the improvement is not statistically significant.

Results and Discussions

We develop a pipeline for multi-template protein comparative modeling as shown in Figure 1. Given an input target (or query) protein, the pipeline uses PSI-BLAST [50] to search homologous structure templates. The target-template alignments ranked by PSI-BLAST e-values are combined with respect to the target protein. Briefly speaking, the algorithm always uses the most significant template-target alignment. The other significant alignments relative to the most significant one are also automatically included. The less significant template-target alignments are chosen only if they can align with a continuous region of the target that is not covered by the previously selected template-target alignments. And only the alignment fragments that align with the uncovered regions are used. The combined alignments and template structures are fed into Modeller [8] to generate structure models for the target protein. The details of the algorithm are described in the Methods section.

Figure 1
figure 1

An automated multi-template comparative modeling pipeline.

Full size image

Our multi-template combination method was first blindly tested in the CASP7 experiment, 2006. Since then we systematically evaluate the algorithm on the CASP7 comparative modeling targets. A target is classified as a comparative modeling target if a structure template covering all the domains of the target can be found by PSI-BLAST search as in [51]. Here, we firstly compare the multi-template combination algorithm with a single-top-template algorithm on all the comparative modeling targets. Secondly, we compare two approaches on the high-accuracy comparative modeling targets. Thirdly, we compare the multi-template combination algorithm against the ideal approach that always uses the best template. Fourthly, we compare the multi-template combination method against the other automated methods and the human predictors in the CASP7 experiment. Finally, we discuss why the multi-template approach improves model quality.

The models of the multi-template combination algorithm were generated during the CASP7 period when the structures of the targets were not known. For the comparision we use Modeller [8] to generate 3D structure models for the single top template and the best template respectively, based on the alignments generated by PSI-BLAST during the CASP7 experiment. We use LGA [48], a sequence-dependent structure alignment tool, to evaluate the models against the experimental structures to get GDT-TS scores.

Comparison with the Single Top Template Approach on the CASP7 Comparative Modeling Targets

We compare the multi-template combination algorithm with the single-template algorithm on the 45 CASP7 comparative modeling targets, for which PSI-BLAST can identify at least two significant templates. The other two comparative modeling targets (T0326 and T0328) that have only one template found by PSI-BLAST are excluded. The single-template algorithm always uses the most significant template with the lowest e-value of PSI-BLAST.

Table 1 shows the GDT-TS scores of 45 targets using the single- and multi-template methods, respectively. The number of templates used by the multi-template combination algorithm ranges from 2 to 39. The average number of templates used is 12.4. According to the results, the multi-template combination improves GDT-TS score for the majority of cases (38 out of 45 targets) as shown in Figure 2, consistent with the previous human prediction experiment [23, 31]. The average score of using multiple templates is 71.15 versus 66.59 of using the single most significant template. The average improvement of GDT-TS score is 6.8% (raw score increase = 4.56).

Table 1 The results of the multiple- and single-template methods on the 45 comparative modeling targets of CASP7.
Full size table
Figure 2
figure 2

GDT-TS scores of 45 comparative modeling targets (multi-template versus single-template). For 38 out of 45 targets, the multi-template approach yields higher GDT-TS scores than the single-template approach. The dots above the line represent the targets where the multi-template method yields higher scores, on the line where two methods yields the same scores, and below the line where the single-template method yields higher scores.

Full size image

We conduct a paired t-test (t-value = 4.39, the degree of freedom = 44) on the GDT-TS scores of 45 targets. The p-value of getting an average score difference >= 4.56 is 3.5 × 10-5 under the hypothesis that the difference is 0. So combining multiple templates and their alignments significantly improves the quality of comparative modeling over the single-template approach.

Comparison with the Single Top Template Approach on the CASP7 High-Accuracy Modeling Targets

A special category of template-based modeling (i.e. high-accuracy modeling), where models have GDT-TS scores > 80 compared to experimental structures, is particularly useful for understanding protein function at the atomic level. To emphasize its importance, CASP7 dedicated a category of high-accuracy template-based modeling to evaluate methods on the targets for which there is at least one template with LGA-S score > 80 and at least one method produced a model with GDT-TS > 80. Using this criteria, CASP7 classifies 28 domains from 24 protein targets into the high-accuracy modeling category. Among them T0326 and T0328 have only one template. The templates for T0311 and T0367 cannot be found by PSI-BLAST. The structure of T0334 is not released at the time of writing the paper. So we exclude these five targets and select the remaining 23 high-accuracy domains to compare the multiple-template combination algorithm with the single-template approach.

Table 2 reports the results of the multi- and single-template methods on the high-accuracy domains. The average GDT-TS score for the multi- and single-template approaches is 86.7 and 81.0 respectively. The average difference is 5.7. We conduct a paired t-test on the scores (t-value = 2.51, the degree of freedom = 22). The p-value of getting an average difference >= 5.7 is 0.01 under the hypothesis that there is no difference between the multi- and single-template methods. According to the standard 0.05 threshold, the difference is significant. Among 20 out of 23 high-accuracy targets, the multi-template combination method generates better models than the single-template method as shown in Figure 3.

Table 2 The results of the multiple- and single-template methods on 23 CASP7 high-accuracy domains.
Full size table
Figure 3
figure 3

GDT-TS scores of 23 high-accuracy targets (multi-template versus single-template). For 20 out of 23 domains (dots above the line), the multi-template approach yields higher GDT-TS scores than the single-template approach.

Full size image

Comparison with the Best Template Approach on the CASP7 Comparative Modeling Targets

A very challenging problem of comparative modeling is to improve the model accuracy over the best templates [14, 52, 53]. The series of the community-wide experiments from CASP1 to CASP6 show that few methods can consistently improve the model accuracy over the best templates or even the top ranked templates. A couple of recent methods [29, 54, 55] and the latest CASP7 experiment show that in some cases comparative modeling and refinement can improve model quality over the best templates. Thus, here we test if our multiple template combination algorithm can improve structure modeling over the best templates on the comparative modeling targets.

We use the best templates for the comparative modeling targets provided at the CASP7 web site. We select 27 targets whose best templates can be identified by PSI-BLAST to compare the multi- and best template methods. We also use the PSI-BLAST alignment between the best templates and the targets to generate structure models.

As shown in Table 3 and Figure 4, among 27 targets, the multi-template combination method produces better models for 16 targets, worse models for 10 targets, and the same quality model for 1 target. Thus, the multi-template algorithm produces better models for 62% of the targets. However, on average, the GDT-TS score is only slightly increased by .5.

Table 3 The GDT-TS scores of the multi- and best template methods on the 27 CASP7 comparative modeling targets.
Full size table
Figure 4
figure 4

GDT-TS scores of the 27 comparative modeling targets (multi-template versus best-template). For 16 out of 27 targets (points above the line), the multi-template approach yields higher GDT-TS scores than the best-template approach.

Full size image

We conduct a pairwise t-test on the GDT-TS scores of two approaches (t-value = 0.6, degree of freedom = 26). The p-value of the statistical analysis is 0.28. Thus, on average, the GDT-TS score increase of using multiple templates over the best possible template is not significant.

Comparison with the other Servers and Human Predictors on the CASP7 High-Accuracy Modeling Targets

We compare the accuracy of our multi-template combination algorithm against the other servers that participated in the CASP7 community-wide experiment. Two of our servers, FOLDpro and 3Dpro, along with 65 other servers were evaluated in the category of high-accuracy structure prediction in CASP7. FOLDpro and 3Dpro used the same multi-template combination algorithm, but run on different versions of non-redundant sequence databases. Table 4 reports the official total GDT-TS scores of the first models of top 20 (out of 67) servers on the 28 high-accuracy domains in CASP7. The GDT-TS scores are directly taken from the official CASP7 evaluation. The data is kindly provided by Dr. Yang Zhang at his web site [56]. The results show that our methods FOLDpro and 3Dpro using multiple-template combination algorithm were ranked second and third respectively. The performance of our methods that use the simple combination of PSI-BLAST alignments is very close to the best method (Zhang-Server) that extracts distance restraints from multiple templates used in conjunction with a more sophisticated and complicated model generation tool TASSER.

Table 4 The total GDT-TS scores of the top 20 out of 67 servers on the 28 high-accuracy comparative modeling domains in CASP7.
Full size table

Furthermore, using the more strict measure GDT-HA specially designed for high-accuracy models, FOLDpro and 3Dpro are ranked third and fifth according to the official CASP assessment [57, 58]. We also compare the performance of automated servers with the human predictors. The comparision is not fair because human predictions started from the server predictions and took much longer time (about three weeks of human versus two days of server). However, it is still interesting to see what values human predictions can add on the high accuracy targets.

Table 5 reports the top 10 predictors among the 116 human and 67 server predictors in the high-accuracy structure modeling in CASP7. The data is kindly provided by Dr. Yang Zhang at his website [59]. The results show that three automated servers Zhang-Server, FOLDpro, and 3Dpro yielded the performance comparable to the best human predictors that used much longer time and took a pool of server predictions as inputs. Our servers FOLDpro and 3Dpro were ranked fifth and sixth, respectively. Zhang-Server is better than all human predictors except for Zhang human predictor from the same group. FOLDpro is better than 113 out of 116 human predictors.

Table 5 The total GDT-TS scores of the top 10 out of 183 predictors (67 servers + 116 human predictors) in the category of the high-accuracy structure modeling in CASP7.
Full size table

Since the main goal of this paper is to demonstrate the effectiveness of using multiple templates instead of evaluating different predictors in CASP7, readers are advised to check out the CASP7 assessment papers published in the Proteins journal for the thorough evaluations using different measures such as GDT-TS, GDT-HA and AL0.

Good and Bad Examples of Using Multiple Templates

The correct usage of multiple homologous templates in general but, not always can improve comparative modeling [60, 61]. As the reviewer point out, the effectiveness of multi-template modeling may correlate with the number of templates, the structural consistency amongst templates, and query-template similarity. Clarifying their relation can help decide when to use multiple templates. However, currently no quantitative measure of the relationship can be derived. Thus, here we discuss a few good and bad examples to illustrate the advantages and disadvantages of using multiple templates.

Figure 5 shows a good example (T0315, length = 257). The best template for the target is protein 1J6O in the PDB, whose Root Mean Square Distance (RMSD) with the experimental structure (2GCX) is 1.33 Å for 240-residue long aligned region. The other good template is 1YIX having RMSD 1.44 Å for 244-residue long aligned region. However, both templates have some bad regions that do not align well with the experimental structure. Figure 5(1) and Figure 5(2) show two different bad loop regions for the two templates, respectively. Interestingly, the two bad regions are corrected in the model generated by using multiple templates as in Figure 5(3). In addition to the obvious improvement in the two regions, the backbone of most other regions of the model are also closer to the experimental structure than the two templates. The RMSD between the model and the experimental structure is 0.88 Å for 248-residue long aligned regions. This example clearly shows that combining the complementary good templates can improve model quality.

Figure 5
figure 5

An good example (CASP7 target T0315) of using multiple templates to improve model quality. (1) The superimposition of the experimental structure (PDB code: 2GZX) and the best template (PDB code: 1J6O). Blue and red lines represent the backbone of the experimental and template structures, respectively. One bad region is identified. (2) The superimposition of the experimental structure and a good template (PDB code: 1YIX). One bad region is identified. (3) The superimposition of the experimental structure and the model generated by 3Dpro during CASP7, based on multiple templates). Two bad regions in (1) and (2) are corrected in the model (3). Most other regions of the model are also closer to the experimental structure than the two templates.

Full size image

The other good example is a two-domain target T0324, where the improvement of using multiple templates on the second domain (4-helix bundle) is drastic. The GDT-TS score of the second domain is increased by 48 (Table 1). A close examination reveals that the top template does not provide the right orientations for the helices, which are corrected by the other templates.

However, multi-template combination may occasionally decrease the model quality when there is a very good template that is much closer to the target than all other templates. One such an example is T0291 (length = 310). The most significant and best template is 1JPA, whose RMSD with the experimental structure (2GSF) is 0.72 Å for 264-residue long aligned region. The sequence identity and PSI-BLAST e-value is 81% and 10-153. The RMSD between other three significant templates (2SRC, 1Y57, 1KSW) and the experimental structure is 3.22 Å for 250-residue aligned region, 2.33 Å for 262-residue aligned region, and 3.20 Å for 250-residue aligned region respectively. These three templates are much more different from the target structure than the best template. However, because the alignments between these three templates and the experimental structure are very significant (e-value < 10-142 and sequence identity > 40%), these three templates together with the best one are combined to generate models for the target.

The RMSD between the model and the experimental structure is 2.63 Å for 271-residue long alignment, which is better than two templates (2SRC and 1KSW), but worse than the best template (1JPA). This example shows that combining multiple templates may not help if one template is much closer to the true structure than all other templates.

Why does the Multi-Template Approach Work in General?

We consider the following factors contributing to the improvement. First, statistically, the average model generated from multiple templates is better than the single top template on average. This is due to the ability of Modeller extracting spatial restraints from multiple templates and weighting them effectively. The weighting scheme can weight the most likely distance restraints more, resulting in picking correct aligned regions from different templates in most cases [8]. The effective combination of good aligned regions of different templates can improve comparative modeling, particulary in the cases where multiple templates provide the complementary good coverage of the target.

Second, multiple templates contain sequence and structure conservation and variation information (e.g. conserved distance restraints), which is not available in a single template. The evolutionary information is often useful to improve both secondary and tertiary structure prediction [58, 61–63]

Third, PSI-BLAST can generate good local alignments for homologous proteins (comparative modeling), particulary for close homologs (easy comparative modeling). In fact, PSI-BLAST, a profile-sequence local alignment method, generates better alignments for the comparative modeling (or easy) targets than the profile-profile alignment methods (e.g., ClustalW [64], T-Coffee [65], COACH [66], and Palign [67]) we tested, which is consistent with the previous observations (Dr. Kimmen Sjölander, personal communication). However, profile-profile alignment methods are shown to produce better alignments on hard targets (<= 20% identity) [39] and to have stronger fold recognition power than profile-sequence alignment methods [37, 39, 66–74].

Fourth, the ranking of homologous templates by PSI-BLAST e-values for a target protein is also reasonably good, although not perfect. The greedy combination of PSI-BLAST templates and alignments into a multiple alignment centered on the target protein is effective for comparative modeling as shown in our experiments.

Conclusion

In this study, we have developed a novel and effective algorithm of selecting and combining multiple templates and their alignments generated by PSI-BLAST, which significantly improves the quality of comparative modeling over the traditional single-template approach on the CASP7 benchmark. The alignment files of both the single-template approach and the multi-template approach are available [see Additional files 1 and 2]. The comparative modeling web server of using multiple templates is accessible at the MULTICOM website [75].

Methods

Multi-Template Combination Algorithm

We develop a novel and effective multi-template combination algorithm to select and combine template-target alignments for comparative modeling (Figure 1). The algorithm uses as inputs the template proteins identified by the PDB-BLAST approach [50, 68], similar as ISS [76] and DOUBLE-BLAST approaches [77].

The PDB-BLAST approach searches for structure templates for a target protein in two steps. First, it uses PSI-BLAST [50] to search the target protein against the NCBI non-redundant sequence database [78] to build a profile. The e-value threshold (-h option) for building iterative profiles is set to 10-10; the number of iterations (-j option) is set to 3; and the e-value threshold (-e option) for inclusion in the final profile is set to 0.001. Second, it uses PSI-BLAST to search the profile against a template library compiled from the proteins in the Protein Data Bank [49] to identify structure templates homologous to the target protein. The number of iterations in this step is set to 5 and all other parameters to the default values. We select the template-target alignments with e-value < 0.001 for the combination.

Each returned template protein may have one or more local alignments with the target protein. Each alignment is associated with an e-value that measures its similarity significance. We use the logarithm of the e-value as the significance score. The smaller the score, the more significant is the alignment.

The only inputs to the algorithm are the template-target alignments and the associated significance scores. Thus, with some modification, the algorithm can be applied to other template-based structure prediction methods with different scoring schemes (e.g. z-score in threading). Figure 6 briefly describes the algorithm. The algorithm always uses the most significant template-target alignment. The other significant alignments whose significance score is less than σ and close to the score of the most significant template-target alignment within the threshold δ are also automatically included. The less significant template-target alignments are used only when they can align with a continuous region of the target whose size is bigger than ρ and which is not covered by the previously selected template-target alignments. And only the alignment fragments that align with the uncovered regions are excised and included. ρ controls the size of the selected fragments, which can be used to avoid selecting very small fragments.

Figure 6
figure 6

Multi-Template Selection and Combination Algorithm.

Full size image

One template-target alignment may provide several fragments that align with disjoint, uncovered regions of the target. The alignment fragments can be extended at both ends up to τ residues if possible, which may create a linker to overlap with the other fragments from the same template or the alignments from other templates. After the extension, the overlapped or non-gapped alignment fragments excised from the same template-target alignment are combined into one alignment. The join of fragments can introduce long-range constraints, which is useful for structure modeling.

The alignments between the target and the selected templates are combined into a multiple alignment using the target sequence as an anchor, the same as the central star multiple alignment approach [79, 80] employed by the construction of position specific scoring matrix (PSSM) [81, 82] in PSI-BLAST [50]. The multiple alignment together with the template structures are fed into comparative modeling tools such as Modeller to generate structure models for the target protein. Modeller weights the spatial restraints extracted from multiple templates according to sequence identity appropriately [8], resulting in generating a better average model than a single template in most situations.

The algorithm is fully parametrized (Figure 6). All parameters can be tuned. In our experiments, we set σ to -20, δ to 12, ρ to 5, and τ to 5. The parameters were tuned on the CASP6 targets and blindly tested on the CASP7 targets. The most important parameters are σ and δ because the combination of the whole significant templates contributes most to the improvement of structure modeling. ρ and τ usually play a less important role because short un-covered regions of a target protein are usually well handled by the loop building module of Modeller.

Limitation and Future Work

The multi-template combination algorithm developed here is very effective for comparative modeling where target and template proteins have strong homologous relationship and their alignments are rather accurate. But the method does not always produce good results for hard template-based structure prediction, i.e. protein fold recognition.

We had applied the similar algorithm to the hard fold recognition targets during CASP7 using the global alignments generated by COACH [66] (results not shown here). The algorithm works well when the structure templates and their alignments with the target are consistent, but performs poorly when structure templates or sequence-structure alignments have a lot of spatial inconsistency (particularly in unalignable regions). The models generated from multiple inconsistent target-template alignments usually contain a lot of atom-atom clashes – a quantitative indicator of spatial inconsistency. One possible reason is that the current version of Modeller cannot well resolve a large amount of conflicting distance restraints within multiple templates as also observed in [60].

Thus, although multiple templates are useful in general, a more sophisticated way of combining them and removing inconsistency is required to achieve better performance for fold recognition and threading, where the template-target relatedness and alignment are uncertain and less reliable than comparative modeling. The spatial inconsistency may be removed either in model reconstruction phase as in TASSER or in alignment optimization phase as in [83].

One possible direction is to use structure alignment tools such as DALI [84], SSAP [85], VAST [86], CE [87], and TM-align [88] to check the structure consistency between templates and to remove inconsistent templates and alignments (work in progress).