Background

At Seattle Biomedical Research Institute (SBRI), we are involved, as part of the Leishmania Genome Network (LGN), in the sequencing and annotation of the trypanosomatid protozoan species L. major Friedlin (LmjF). Following DNA sequence determination, putative protein-coding regions within the sequence are predicted and functionally classified. Although trypanosomatids are eukaryotes, their gene structure is more similar to that of prokaryotes; they have essentially no introns and small intergenic regions. Two small LmjF chromosomes (chr1 and chr3) have been completely sequenced and annotated. The 79 protein-coding genes predicted from chr1 are organized in two large divergent polycistronic gene clusters of 29 and 50 genes, on the "bottom" and "top" DNA strains, respectively [1]; while chr3 contains two convergent polycistronic clusters of 65 and 29 genes, with a single divergent gene at one telomere and a single tRNA between the two large clusters [2].

Presently, a large number of methods exist for in silico prediction of coding regions [3–7]. These computational methods use a range of underlying statistical properties of the coding regions and can be generally classified as consensus (signal sensors) and non-consensus (content sensors) [8, 9]. The non-consensus methods can be further classified as trained, which require unbiased sets of coding regions, and untrained, which use statistical properties to discriminate between coding and non-coding regions. Although non-consensus methods have been very successful in identifying genes in most of the sequencing projects, currently none have 100% specificity and sensitivity. In the absence of such a method, the use of a combination of methods is next best option [10–13]. Since LmjF genes do not contain introns, and the signal sequences for trans-splicing and polyadenylation are poorly defined, consensus methods have little utility for Leishmania gene prediction. In addition, ~70% of the genes have no significant homology to existing genes in sequence databases, so extrinsic content sensing methods are of limited use; leaving only intrinsic content sensing methods for possible use in gene prediction. Given that the number of experimentally confirmed gene prediction in Leishmania is currently small, and many methods use similar statistical approaches [4], the choice of two trained methods (GLIMMER[14] and CODON USAGE[15]) and two untrained methods (TESTCODE[16], and GENESCAN[17]) which rely on unrelated statistical measures should provide substantial power for gene prediction in LmjF.

The freely available JAVA-based software package ARTEMIS[18] was designed specifically as an annotation platform and has a user-friendly graphical interface. It simplifies time-consuming processes such as inter-file format conversion, BLAST analysis [19], and provides a convenient environment for viewing the gene structure and organization of large DNA segments. Here we describe a method for importing data from GLIMMER, TESTCODE, and into GENESCAN into ARTEMIS, to enhance gene prediction and annotation.

Results and Discussion

We have developed a partially automated process for prediction and annotation of LmjF protein-coding genes in which the gene predictions from GLIMMER and the statistical outputs from TESTCODE and GENESCAN are imported into ARTEMIS (see additional file 1), where they can be viewed graphically alongside the CODON USAGE statistics already built into ARTEMIS. Figure 1 shows a panel containing results from each of the four gene-prediction methods for a typical LmjF sequence. The predictions from GLIMMER are imported as CDS features and displayed as colored rectangles in the panel showing ORFs (the vertical bars are the stop codons) in all six reading frames. The window scans from TESTCODE, GENESCAN and CODON USAGE are displayed graphically in panels above the GLIMMER predictions. The thresholds used to indicate likely protein-coding ORFs for TESTCODE and GENESCAN are 4.0 and 9.7, respectively. This allows visual comparison of the four gene prediction methods and manual alteration of the GLIMMER-predicted CDS features if necessary. The reliance on multiple gene prediction methods increases confidence in the predictions.

Figure 1
figure 1

This panel of ARTEMIS shows the comparison of four different methods used at SBRI for sequence annotation: a) CODON USAGE b) GENESCAN c) TESTCODE and d) GLIMMER. The CODON USAGE panel shows results for the three reading frames (shown by different colors) of the top strand; those from the bottom strand are not shown. The panel immediately following the TESTCODE panel displays the position of all stop codons (with vertical lines) in all six reading frames. The vertical scales in the top three panels refer to the value of the statistic calculated by the corresponding algorithm. The predictions of GLIMMER appear as blue boxes in this panel. The horizontal scale in the center of this panel indicates the nucleotide coordinates of the sequence for this and the three upper panels (and is adjustable on the right hand scroll bar). The bottom panel displays the translated amino acids in six different reading frames. The horizontal scale refers to the nucleotide coordinates for the sequence within this panel.

Full size image

In Table 1, we show a comparison of the results of automated gene prediction using the four different programs with the manual annotations for three completely sequenced chromosomes (chr1, chr3 and chr4) from LmjF. The False Positive rate for each individual method was quite high, with GLIMMER being significantly worse than the others. Most of the False Positives were due to prediction of genes on the wrong coding strand. All methods, with the exception of TESTCODE, showed a low number of False Negatives. The poor performance of TESTCODE was largely due to use of a high cut-off value (9.7) for the average Fickett statistic of the whole ORF, rather than smaller windows. Thus, individually, each of the automated programs had high Error Discovery Rates (fraction of incorrect predictions made for expected predictions, Table 1), ranging from 0.77 for GENESCAN to 1.96 for GLIMMER.

Table 1 Automated gene predictiona in Leishmania major
Full size table

Combination of the programs improved the Error Discovery Rate, especially in terms of false positives (Table 2). When only ORFs predicted by all four programs are considered, the false positive rate was <1%, but the false negative rate was almost 50%. By including ORFs predicted by only three of the four programs, the false negative rate was dramatically lowered to 10%, but the false positive rate rose to >10%. Further relaxation of stringency (two of four programs) resulted in a substantial increase in false positives (78%), with only modest decrease in false negatives (~5%). Thus, the Error Discovery Rate is least (21%) by considering the consensus prediction of three out of four programs. The use of two trained (GLIMMER and CODON USAGE), and two non-trained (TESTCODE and GENESCAN) algorithms reduced false positives and false negatives.

Table 2 Automated gene prediction by combination of different methods.
Full size table

Conclusions

The semi-automated comparative analysis clear shows that some degree of manual annotation is still necessary in projects where there is large proportion of novel genes. The manual annotation is time consuming and labor intensive. The ARTEMIS desktop environment, with importation of trained and non-trained non-consensus gene-prediction algorithms, facilitates easy comparison of the results and allows the user to make more-informed decisions for calling protein-coding genes. Thus, this improvised and powerful software, developed using already existing gene identification methods and annotation platform, is extremely helpful for whole genome sequencing projects.

Methods

GLIMMER 2.0 http://www.tigr.org/software/glimmer/[14] was trained using predicted protein-coding genes from LmjF chr1 [1] (manual annotations based on TESTCODE and CODON USAGE) and chr4 (manual annotations using HEXAMER and CODON USAGE: A. Ivens, personal communication) using the default settings. The trained GLIMMER was run on LmjF sequence using the default setting with a minimum gene length of 75 amino acids and output was parsed into an EMBL-formatted feature table file. This data were imported into ARTEMIS 4.0 (installed on Intel-based Linux or Windows 2000 machines) using the "Read Features Into" option of the "File" menu. This allows the GLIMMER-predicted genes to be displayed as CDS Features. The TESTCODE[16], GENESCANhttp://202.41.10.146/public_htmlnew/gs.htm[17] and CODON USAGE[15] algorithms were re-coded in C++ and the statistical results collected in text files with single value for each sliding window (100 nt windows, sliding by onent increments). These TESTCODE and GENESCAN data were imported into ARTEMIShttp://www.sanger.ac.uk/Software/Artemis/[18] using the "Add User Plot" option of the "Display" menu, and displayed graphically. This procedure can be used to import other sliding window methods. The CODON USAGE bias statistics, which has been coded as part of ARTEMIS, is calculated for the three reading frames of each DNA strand and displayed in different colors using the "Add Usage Plot" option of the "Display" menu to import Leishmania CODON USAGE tables. Figure 1 shows a panel containing results from each of the four gene-prediction methods for a typical LmjF sequence.

For automated GENESCAN, TESTCODE and CODON USAGE predictions, genes were called only for those ORFs larger than 100 amino acids with mean scores (over the entire ORF) above thresholds of 4.0, 9.7, and 0, respectively. For overlapping ORFs (on the same or opposite strands), the one with the highest signal was used.