Efficient and Robust Path Openings Using the ScaleInvariant Rank Operator
 1.7k Downloads
 1 Citations
Abstract
Some basic properties of a slightly generalized version of the scaleinvariant rank operator are given, and it is shown how this operator can be used to create a nearly scaleinvariant generalization of path openings that is robust to noise. Efficient algorithms are given for sequences and directed acyclic graphs with binary values, as well as sequences with real (greyscale) values. An algorithm is also given for directed acyclic graphs with real weights. It is shown that the given algorithms might be extended even further by allowing for scores based on a totally ordered semigroup.
Keywords
Scale invariance Rank operator Path opening1 Introduction
Rank operators are often used for filtering images, and can be considered in the framework of mathematical morphology. They are generalizations of the minimum, maximum and median filters: at each position selecting the rth value in a sorted list of all values within a certain neighbourhood. Previously, we introduced the scaleinvariant rank operator (SIR operator) on binary masks, with an application in radio astronomy [22]. This is a highly efficient operator that identifies all intervals (of any size) that contain a sufficient fraction of ones in the input mask. The name stems from the fact that this operator can be interpreted as combining the results of rank operators with linear structuring elements of all possible lengths.
The SIR operator is an interesting option for anyone who needs to robustly “grow” regions or close gaps, without knowing in advance what size these regions have. Indeed, we showed [22] that this can work quite well in practice. However, the operator was limited to 1D signals (rows/columns of images were processed independently). In this work, we analyse the SIR operator in more detail, exposing a link to path openings [13, 14, 19]. In Sect. 3, we show how the SIR operator can be generalized to higherdimensional signals by considering paths, leading to a straightforward generalization of path openings that allows for gaps in paths. We also briefly discuss the generalization to greyscale. In the current context, paths are understood to be approximately, but not necessarily perfectly, straight curves, formalized as paths in certain directed acyclic graphs (see Fig. 1). When a long path is broken up into smaller segments because of noise, the “missing” pixels can be said to constitute gaps. In many practical applications, it is crucial to be able to deal with gaps.
Section 4 presents algorithms for the four cases we treat: binary sequences, greyscale sequences, binary directed acyclic graphs, and greyscale directed acyclic graphs. With the exception of greyscale graphs, all cases allow an algorithm that is at most a logarithmic factor worse than linear in the size of the problem. The results are roughly in line with known results for path openings [12, 14].
Section 5 compares generalized path openings with two existing schemes for making path openings robust to noise: incomplete path openings [14, 29] and robust path openings [8]. Incomplete path openings allow for finegrained tuning of how many pixels may be missing from a path, but are inefficient when this number becomes high. Robust path openings put a cap on the maximum size of each individual gap, allowing (at the strictest setting) up to 50 % of a path to be empty, while still requiring more passes over the data than a normal path opening. The generalized path openings we introduce are nearly as efficient as normal path openings, while also allowing for a finegrained tuning of the minimum fill fraction of a path.
2 Related Work
3 The New SIR Operator
Below we first introduce a slightly more general SIR operator on sequences (1D signals), and then present a further generalization to graphs (or nD signals/images) that completes the unification of the SIR operator and path openings. Note that we will mostly discuss binary signals, only briefly discussing the generalization to greyscale towards the end of the current section. The implementations do support greyscale though.
Definition 1
The parameter s gives the minimum fraction of the interval that should be set in f (at least in the limit for \(p\rightarrow \infty \)), while l gives the minimum measure of intervals in the output. Also note that instead of just counting elements, we could also use a general measure \(\mu \) on \(\mathbb {Z}\) [4] such that \(\mu (\{x\})\) is positive for all \(x\in \mathbb {Z}\). Using a general measure would allow using nonconstant weights for positions, for example when dealing with nonuniform sampling. Theorem 1 below shows that \(\alpha _{s,l}\) is indeed an opening. Also, it is possible to see that \(\alpha _{1,l}(f)\) equals \(\rho _{1,l}(f)\) (see Corollary 1) and that it recovers a traditional path opening. Combined with the fact that \(\alpha _{s,l}\) can be implemented in pretty much the same way as normal path openings, these facts motivate the name.
For \(l=0\), \(0<s<1\), and the graph \((\mathbb {Z},\{(x,x+1)\mid x\in \mathbb {Z}\})\), the new formulation is equivalent to the old formulation. The main advantage of this new formulation is that it allows us to have a continuum of operators all the way up to \(s=1\), in which case we get a path opening as the limit case. At first glance, it might look like the new formulation is actually less suited for use with \(s=1\) because of the division by \(1s\), but in practice dealing with the resulting infinities is not a problem. In particular, we maintain the scores using floating point numbers, and those allow for infinity in a way that works just fine with our algorithm. If one wanted to use integers/fixed point, it would certainly be possible to do this, but some care would need to be taken to avoid overflow (or, rather, underflow).
3.1 Properties
We will now state some properties of the operator \(\rho =\rho _{s,l}\). Note that the subscripts (s and l) are used only when needed for clarification.
Lemma 1
The operator \(\rho \) is increasing: \(f\subseteq g\implies \rho (f)\subseteq \rho (g)\).
Proof
Lemma 2
The operator \(\rho _{s,0}\) is extensive: \(\rho _{s,0}(f)\supseteq f\).
Proof
The reason the SIR operator is called scale invariant is that it considers paths of all sizes, and (for \(l=0\)) only looks at the fraction of the path that is present to determine whether or not to include it in the output. It thus does not depend on the scale of the signal. For \(l>0\), we lose some of this scale invariance in a controlled manner (although it is still approximately scale invariant for large intervals), which is relevant for being able to construct openings [30, Prop. 2].
3.1.1 SIROperatorBased Openings
Theorem 1 below shows how the SIR operator can be used to create openings, while Corollary 1 shows that some SIR operators are in fact openings.
Lemma 3
The operator \(\rho \) is an infoverfilter [15, ch. 12]: \(\rho (f{{\mathrm{\cap }}}\rho (f))=\rho (f)\).
Proof
Finally, as \(\rho \) is increasing (Lemma 1), we have
\(\rho (f{{\mathrm{\cap }}}\rho (f))\subseteq \rho (f)\). This leads us to conclude that \(\rho \) is an infoverfilter. \(\square \)
Theorem 1
The operator \(\alpha _{s,l}\) defined by \(\alpha _{s,l}(f)=f{{\mathrm{\cap }}}\rho _{s,l}(f)\) is an algebraic opening: it is increasing, antiextensive—\(\alpha _{s,l}(f)\subseteq f\)—and idempotent—\(\alpha _{s,l}(\alpha _{s,l}(f))=\alpha _{s,l}(f)\).
Proof
That \(\alpha _{s,l}\) is increasing follows from the fact that \(\rho _{s,l}\) is increasing (Lemma 1). That it is antiextensive follows trivially from the definition. Its idempotency follows from the fact that \(\rho _{s,l}\) is an infoverfilter (Lemma 3), as shown in the proof of Theorem 6.26 by Heijmans [15] (alternatively: [25, Prop. 4.1]). \(\square \)
Corollary 1
The operator \(\rho _{1,l}\) is equal to \(\alpha _{1,l}\), and is thus an algebraic opening.
Proof
3.2 Greyscale
We now briefly discuss how to extend the problem to the greyscale case. In the greyscale case, the output is as if the binary algorithm is run with different thresholds, and each output position is set to the maximum corresponding value of the binary application (see Fig. 3). It can be checked that this recovers the interpretation of the SIR operator as a scaleinvariant rank(max) operator when the minimum path length \(l=0\). An example application can be the detection of radiofrequency interference (RFI) in radio astronomy: when the initial detection results in confidence levels, a greyscale SIR transform can be used directly on the confidence levels.
4 Algorithms

O(n) for the binary operator on sequences,

\(O(V+E)\) for binary on (general) graphs,

\(O(n\,\log (n))\) for greyscale on sequences, and

\(O(\min (V,L)\,V)\) for greyscale on (sparse) graphs.
One important implementation note that concerns all algorithms is what to make of the fraction \(s/(1s)\) when \(s=1\). As mentioned earlier, our algorithms simply use floating point values, which support an actual infinity. Although this type of infinity does not support the convention \(\pm \infty \times 0=0\), this turns out not be an issue, since the algorithms build up the scores incrementally using addition (which is supported in an appropriate manner).
4.1 Binary Sequences
The SIR operator can be implemented very efficiently for binary sequences. We already used this in earlier work [22], but will now provide a bit more detail. We will also present a second method that requires less temporary storage. It is also easy to implement and slightly more amenable to generalization/adaptation. Additionally, it shows how the problem is closely related to the maximum contiguous subsequence sum problem, as essentially the same algorithm can be used to solve both.
We can now also compute \(\lambda _(x)\) on f as \(\lambda _+(n+1x)\) on \(f'\), with \(f'(x')=f(n+1x')\), and then \(\rho (f)\) using \(\lambda _+(x)+\lambda _(x)w(f(x))\) as prescribed by Eq. (10). This leads to Algorithm 2, which produces the same result as the earlier algorithm based on Eq. (9). As it sums everything twice, it might be a bit more expensive. However, if it is advantageous to trade memory for a bit of arithmetic, we can merge the computation of \(\lambda _+\) or \(\lambda _\) with the computation of \(\rho (f)\), giving an algorithm that only needs the \(\lambda _+\) array for temporary storage.
4.2 Binary Graphs
The new pathbased SIR operator can be computed using an algorithm that is virtually identical to Algorithm 2, except that \(\lambda _+\) and \(\lambda _\) would be computed using Algorithm 3. This can be compared directly to the algorithm for computing a path opening transform introduced by Heijmans et al. [14, Alg. 2]. It follows that \(\alpha _{1,l}(f)\) and \(\rho _{1,l}(f)\) are just as efficient as traditional (binary) path openings, with a time complexity in \(O(V+E)\), and amenable to many of the same tweaks and adaptations [6, 16, 20].
4.3 Greyscale Sequences
We will now present the greyscale version of Algorithm 1 for sequences, which has a lower time complexity than applying the general algorithm presented below to sequences. For 1D greyscale path openings (equivalent to attribute openings on length in 1D), a very efficient algorithm running in O(1) (amortized) cost per pixel was presented by Morard et al. [19]. This was recently adapted to compute path openings on graphs as well [12] (albeit with a worse time complexity). In a nutshell, this algorithm uses a stack of path lengths, which get updated using an algorithm quite similar to Algorithm 1, except that elements are pushed onto and popped off the stack to keep track of the currently relevant grey levels. Unfortunately, this algorithm depends on only having to access the top of the stack; for the SIR operator this is not enough, as we may need to keep track of the scores of grey levels above the current grey level of the input.
To make the above efficient, binary search trees can be used to represent \(\Lambda _+\) and \(\Lambda _\) (instead of the stacks used by Morard et al. [19]). This allows \(O(\log (n))\) insertion, lookup, and deletion, with n the length of the sequence. So if we ignore having to update any scores, this would allow for an algorithm whose time complexity is in \(O(n\,\log (n))\), only a logarithmic factor worse than for path openings. The rest of this section shows how to keep track of the correct scores, without sacrificing this time complexity. The overall algorithm is detailed in Algorithm 4.
4.3.1 Building \(\Lambda _+\) and \(\Lambda _\)
We will build a balanced binary search tree, representing \(\Lambda _+(x1)\) (or \(\Lambda _(x1)\)), on all values encountered so far for every position x. Given these trees, we can effectively reconstruct \(\lambda (x,v)\) using Eqs. (13) and (11). Section 4.3.2 discusses how to do this reconstruction efficiently, while this section describes how to efficiently build all those trees. In particular, we will build these trees by scanning through the positions, and building \(\Lambda _\pm (x)\) from \(\Lambda _+(x1)\) or \(\Lambda _(x+1)\). By deferring updates of scores until we access nodes anyway, and by using functional data structures, we keep the overall number of operations (as well as the space requirements) limited to \(O(n\,\log (n))\).
In more detail, to be able to efficiently keep track of the correct scores, in addition to storing a score, let every node keep track of how much score it (and its descendants) “missed” (in a separate missed field that is stored for each node), in the sense that those nodes should have had (an accumulated total of) so much added to their scores in previous steps (which we defer until we actually access those nodes). As soon as a node is accessed, we propagate the missed score to its direct children, we update its own score to the sum of its score and what it missed, and we set its missed score to zero (see Algorithm 5). Note that when propagating the missed score to its direct children, we do not recursively update any further descendants. When inserting/updating a value f(x), we follow Algorithm 6. The process is illustrated in Fig. 5.
 1.
Prune the tree by removing any nodes with nonpositive scores and
 2.
Insert the current value into the tree with a score of w(1) plus the score of its immediate successor (if any), or add w(1) to the old score of the current value if it was already in the tree (see Algorithm 6).
It is important to notice that the extra bookkeeping only requires a constant number of operations extra per node accessed in the traditional insert/update algorithm, even if we include rotations or other manipulations that might be necessary to keep the search tree balanced. This is because it is sufficient to “access” (using Algorithm 5) the nodes involved in the rotation before performing the rotation. As a result, one can see that all operations on the tree keep their original time complexities. Our implementation uses a scapegoat tree [9]. Being a tree with amortized rather than worstcase logarithmic bounds for insertion/deletion, it is good enough for our purposes, while being simpler to implement than most other selfbalancing search trees. Also, in contrast to splay trees [27], scapegoat trees still feature a worstcase logarithmic bound for lookups. This is useful for the final step: using \(\Lambda _+\) and \(\Lambda _\) to get the output. Note that the scapegoat tree was modified to allow storing every intermediate version, without affecting the time complexity (it was turned into a functional data structure). We expect that virtually any selfbalancing search tree or skip list [21, 23] can be adapted to work with our algorithm.
4.3.2 The Final Output
We still need to combine \(\Lambda _+\) and \(\Lambda _\) to compute the output, and preferably without affecting the time complexity. One way to do this is to simply keep track of the trees encoding \(\Lambda _+\) and \(\Lambda _\) at each position, and then to search these trees simultaneously for the right output value. Assuming that the trees are balanced (their height is in \(O(\log (n))\)), this can be done in \(O(\log (n))\) time per pixel.
Lemma 4
Proof
A demonstration of the differences between the various types of path openings
Path pattern  Fill fraction  Generalized  Incomplete  Robust  

\(l=3,\,s=5/7\)  \(L=7,\,K=2\)  \(L=7,\,G=1\)  
X...XXX  4/7  No  No  No  
X.X.X.X  4/7  No  No  Yes  
X..XXXX  5/7  No  Yes  No  
X.X.XXX  5/7  No  (\(5\not \ge \frac{5}{2}\times 2+3=8\))  Yes  Yes 
XXXXX...XXXXXX  11/14  Yes  (\(11\ge \frac{5}{2}\times 3+3=10.5\))  No  No 
X.X.X.XXXXXXXX  11/14  Yes  No  Yes  
XXX..XXX  6/8  Yes  Yes  No  
X.XXXXX  6/7  Yes  Yes  Yes 
The procedure described in the previous paragraph allows us to search both \(\Lambda _(x)\) and \(\Lambda _+(x)\) in parallel, in time linearly dependent on the sum of the heights of the trees representing \(\Lambda _(x)\) and \(\Lambda _+(x)\). If we assume that the trees are balanced (as in our implementation), the final computation of \(\rho _{s,l}(f)\) from \(\Lambda _\) and \(\Lambda _+\) takes \(O(n\,\log (n))\) time, resulting in the same overall time complexity for the entire filter. Turning the SIR operator \(\rho _{s,l}\) into the generalized path opening \(\alpha _{s,l}\) takes linear time, as it only requires computing the meet of the input and the result of the SIR operator.
Note that in addition to using the algorithm above on sequences, one should also be able to use the above algorithm with the technique introduced by Morard et al. [20] to compute the socalled “parsimonious path openings” on 2D images (and directed acyclic graphs in general). The idea here is to apply a 1D filter to a cleverly selected [3] subset of paths, giving an approximation to the full pathbased filter.
4.4 Greyscale Graphs
Our implementation of the greyscale SIR operator on 2D images (and the associated generalized path opening) is based on the algorithm for traditional greyscale path openings developed by [2, 29]. This algorithm starts by computing \(\lambda _+\) and \(\lambda _\) for the lowest grey level, and then proceeds to update it for the next higher grey level, and so on and so forth, keeping track for each pixel of the highest level at which \(\lambda \) is large enough. The main differences compared to our implementation are that we chose to iterate over the grey values from high to low, and that we cannot assume that pixels outside the current upper level set do not participate. Algorithm 8 gives an overview of the most critical section of the algorithm, based on the version of Appleton and Talbot’s algorithm presented by [12, 26].
It should be noted that the time complexity of the greyscale algorithm on graphs is typically superlinear, and potentially quadratic (depending on the image content). This can be derived in more or less the same way as for normal path openings [12, 26], except that now path weights do not need to be integers, resulting in an upper bound of \(O(\min (V,L)\,V)\) (assuming that the grey levels can be sorted in time linear in L and that the graph is sparse). Here L is the set of grey levels in the image. Note that it remains to be seen whether this bound is tight. In this light, it is interesting to see, as is shown in the previous section, that the onedimensional case can be solved in a slightly different way, resulting in a lower (for high bitdepth images at least) time complexity of \(O(V\,\log (V))\). It remains to be seen whether a similar approach can also be applied to general graphs, but it looks like this would at the very least severely complicate matters, as we would somehow need to (lazily) merge binary search trees very efficiently.
5 Comparison to Robust and Incomplete Path Openings
An additional advantage of generalized path openings over incomplete path openings is that if we wish to allow for arbitrary noninteger vertex/edge weights, incomplete path openings probably become intractable. This is because they rely on a dynamic programming solution that relies on the path lengths and gap sizes being integers.
Compared to robust path openings, SIR path openings allow one to be much more strict in terms of the fraction of the path that can be missing, at the expense of having no explicit upper bound on the size of a single gap. Incomplete path openings to some extent allow both (in that a certain number of pixels can be missing for a fixed path length), but SIR path openings have the potential to be much faster. In principle, SIR path openings should hardly be slower to compute than normal path openings, while the running times of incomplete path openings scale linearly with the number of allowed missing pixels [29].
6 Examples
6.1 Binary Graphs/2D Images
Figure 6 shows the results of applying the binary generalized path opening to a test image. The parameters were selected by trial and error to give reasonable results, see our previous work [22] for an example of how suitable parameters can be found in an actual application (for the 1D SIR operator).
6.2 Greyscale Sequences
Figure 7 shows some examples of the SIR operator on sequences. The operator was applied to interferometric astronomical radio observations recorded with the Westerbork Synthesis Radio Telescope at low frequency (150 MHz). The image is a dynamic spectrum, where brightness represents the amplitude of the complex correlation between two antennas, with time on the xaxis and frequency on the yaxis. Before sky maps can be made from these data, automated RFI detectors have to remove the RFI, which are linelike features with increased brightness. As was shown earlier [22], it is advantageous to scaleinvariantly extend and fill the holes in these features. Ideally, the greyscale operator should be applied on detection confidence levels, but available RFI detectors do not output these. Therefore, we use the raw data before RFI detection.
Note that even though the data are 2D in nature, it is beneficial to apply the 1D operator on each row rather than the 2D operator, as the 2D operator (in this case) suffers from what can only be described as “leakage” even for fairly high values of s (see Fig. 8), while the 1D operator can be useful even for much lower values of s, which our previous work on the SIR operator [22] showed to be useful in this setting. Most likely the 2D results could probably be improved using the constraints introduced by Luengo Hendriks [16] or by integrating orientation scores [10, 11], but we have not (yet) integrated these options into our implementation.
6.3 Greyscale Graphs/2D images
Figure 9 shows an application to 2D slices from a transmission electron microscopy dataset. From the original greyscale data, another greyscale image is created by first computing a secondorder tensorial orientation score [10], and then encoding the fractional anisotropies of the tensors as grey levels. The fractional anisotropy of a 2by2 symmetric matrix equals \(\sqrt{(\lambda _1  \lambda _2)^2}/\sqrt{\lambda _1^2 + \lambda _2^2}\), where \(\lambda _1\) and \(\lambda _2\) are eigenvalues. Typically, this measure is only computed for (symmetric) positive semidefinite matrices, but it is still meaningful for matrices with negative eigenvalues, and the overwhelming majority of the matrices (over 98 %) in this example are in fact positive semidefinite. This preprocessing makes the linear features stand out more from the background, as in the original dataset the range of grey levels is roughly similar both in the background and along the features, making it very difficult for a (generalized) path opening to preserve the linear features while suppressing the background.
The “Ranking Orientation Responses of Path Operators” (RORPO) modification [17, 18] was incorporated into our implementation of generalized path openings to further reduce the background response, since this modification responds only to long and thin structures. This is accomplished by applying the path opening separately to the different graphs corresponding to different directions (see Fig. 1), and then computing the final result as the difference between the highest and the fourthhighest (in this case the lowest) response for each pixel.
It is clear that both results suppress a lot of background noise, while retaining much of the original linelike features. The generalized path opening can be seen to preserve a bit more of some of the features (especially in the right half of the image, below the circle), but interestingly it also suppresses more of the background (above the circle for example). This is because by itself it will tend to preserve more of the background in each direction, making the RORPO modification more effective.
7 Generalized Scoring Functions
We first present Algorithm 9: a generalized equivalent of combining Algorithm 3 and Algorithm 2. Here \(\mathcal {S}\) is a totally ordered set, and ‘\(\bullet \)’ is an associative binary operator on this totally ordered set. The overall time complexity is still \(O(V+E)\), with V being the number of vertices and E the number of edges in the input graph. Requiring ‘\(\bullet \)’ to be associative guarantees that we can assign an unambiguous score to any path, regardless of the traversal order. Therefore, we will assume that ‘\(\bullet \)’ is associative, and the score \(S(abc\cdots )\) of the sequence of weights \(abc\cdots \) will be equated to \(a\bullet b\bullet c\bullet \cdots \). Note that the computation of \(\lambda _\) has been modified (compared to that for \(\lambda _+\)) to give the same score as \(\lambda _+\) for the same path, using the associativity of ‘\(\bullet \)’.
Lemma 5
Proof
To see that this condition is indeed sufficient, we first show that right before computing \(\rho \), \(\lambda _+(x)\) contains the highest possible score of any path ending at x. To this end, imagine that there would be some y for which \(\lambda _+(y)\) is too low (it can never be too high, as the algorithm effectively computes the overall score for a particular path ending at y). There must then be some \(x\ne y\) from which y can be reached, such that at x, choosing a lower than maximal score would have been better for the score at y. However, this cannot be, since if we equate c with the score of the path from x to y (not including x), Eq. (18) guarantees that taking the maximum score at x results in the highest score at y as well. In the same way, \(\lambda _\) can be shown to contain the highest possible scores for all paths beginning in each vertex. We are now in a position to show that \(\rho \) is computed correctly.
Again, \(\rho \) cannot be too high, so imagine it is too low. This means that we either should have taken a different value for \(\lambda _(x)\), and/or for one of the \(\lambda _+(y)\) (\((y,x)\in E\)). Since \(\lambda _+\) and \(\lambda _\) contain the highest possible scores for paths ending/beginning in each vertex, it is immediately clear from the condition above that using different values (corresponding to different paths) can only decrease the final result, so \(\rho \) cannot possibly be too low. This concludes the proof. \(\square \)
Summarizing, our algorithm works for any scheme based on an associative operator ‘\(\bullet \)’ operating on a totally ordered set \(\mathcal {S}\), such that Eq. (18) holds; in other words, if \(\mathcal {S}\) is a totally ordered semigroup with operator ‘\(\bullet \)’ [7]. Note that there might be other schemes that also work, but they would still need to satisfy something similar to Eq. (18). Also, it might be possible to make further changes to the algorithm to allow for a wider range of scoring schemes, or even to allow for a slightly milder condition on ‘\(\bullet \)’ than associativity, but this is beyond the current scope.
This just leaves the question of whether anything interesting is possible, beyond what we have shown here. Defining \(a\bullet b=\max (a,b)\) would definitely work, and so would a scheme using a lexicographical order on tuples of numbers with perposition addition (vectors). Whether these options will find an application, and what other options exist, remains to be seen.
8 Conclusion
We have presented a generalized version of the SIR operator we previously introduced, and determined that although the SIR operator itself is not an opening or closing, it is a socalled infoverfilter. This allowed us to construct openings (and closings) based on the SIR operator that are approximately scale invariant and generalize path openings.
Interestingly, our generalized path openings give an efficient and scaleinvariant method for finding long paths with gaps. This method has a lower time complexity than at least incomplete path openings [14, 29], and likely robust path openings [8] as well, while having the same kind of flexibility as incomplete path openings. The main thing that sets our method apart from both earlier methods is that it is approximately scale invariant, and thus does not put any cap on the maximum gap size, as long as the gap is a small enough fraction of the complete path. Whether and when this is an advantage or a disadvantage remains to be seen.
We also looked at generalizing the SIR operator to greyscale. Compared to the binary algorithm, the greyscale algorithm on sequences is fairly complex, but its time complexity of \(O(n\,\log (n))\) is still quite reasonable, and we have provided a prototype implementation. This algorithm is heavily inspired by the algorithm given by [19] for 1D (path) openings. Like for path openings, the greyscalegraphcase appears to be much more difficult to implement efficiently than both the greyscalesequencecase and the binarygraphcase. We provide an implementation for greyscale images (/graphs) that is roughly of equal cost as the existing algorithms for traditional path openings [16, 29].
In our examples, we have shown a very simple way to combine greyscale generalized path openings with orientation scores [10, 11]. This approach is not ideal, but provides a simple way of incorporating at least some information on (the local “strength” of the) orientation. In future work, it would be interesting to incorporate orientation scores more fully into the algorithm, for example by computing the path openings in different directions on different orientations in the orientation score. What makes this particularly interesting is that both orientation scores and generalized path openings are (approximately) scale invariant, and that the two algorithms complement each other: orientation scores provide local information on orientation, while generalized path openings find global structures.
Since especially the binary algorithms presented here are very attractive in terms of simplicity and speed, we have examined how much further we can generalize the scoring functions. It appears that there is still some room for other types of scoring functions, as long as they are based on associative operators satisfying an orderpreservation condition. It will be interesting to see if any applications can be found where these other types of scoring functions will prove useful. Finally, we wonder whether it is possible to give a statistical argument for using the scoring function given here, or find other path scoring functions which are somehow optimal from a statistical point of view, while still allowing for efficient computation.
Footnotes
 1.
It is perhaps more common to consider the rankmax opening. This is the meet of the identity operator and the rankmax operator.
 2.
Note that we originally defined the SIR operator with a parameter \(\mu =1s\).
 3.
That we write \(\alpha _{1,l}\) rather than \(\alpha _l\) is to maintain consistency with the generalized operator \(\alpha _{s,l}\) introduced later.
 4.
To be precise: given the specified constraints we can conclude that \(\sigma (p){{\mathrm{\cap }}}f\ge \lim _{s\uparrow 1} c(s)\,\sigma (p)\setminus f+l\iff \sigma (p)\subseteq f{{\mathrm{\text { and }}}}p\ge l\), noting that neither f, nor p, nor l depends on s. This corresponds to, for our purposes, equating \(\infty \times 0\) with 0.
 5.
By analogy with opening transforms.
Notes
Acknowledgments
The authors would like to thank Robert van Liere for making available some interesting and highquality datasets. This research is (partially) funded by The Netherlands Organisation for Scientific Research (NWO), Project No. 612.001.001.
References
 1.Aliprantis, C.D., Burkinshaw, O.: Principles of Real Analysis. Academic Press, San Diego (1998)MATHGoogle Scholar
 2.Appleton, B., Talbot, H.: Efficient Path Openings and Closings. In: Ronse, C., Najman, L., Decencière, E. (eds.) Mathematical Morphology: 40 Years On Computational Imaging and Vision, vol. 30, pp. 33–42. Springer, The Netherlands (2005). doi: 10.1007/1402034431_4
 3.Asplund, T.: Improved Path Opening by Preselection of Paths. Master’s thesis, Uppsala Universitet (2015)Google Scholar
 4.Bartle, R.G.: The Elements of Integration. Wiley, New York (1966)MATHGoogle Scholar
 5.Bentley, J.: Programming pearls: algorithm design techniques. Commun. ACM 27, 865–873 (1984). doi: 10.1145/358234.381162 CrossRefGoogle Scholar
 6.Bismuth, V., Vaillant, R., Talbot, H., Najman, L.: Curvilinear Structure Enhancement with the Polygonal Path Image: Application to GuideWire Segmentation in XRay Fluoroscopy. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) Medical Image Computing and Computer Assisted Intervention. LNCS, vol. 7511, pp. 9–16. Springer, Berlin (2012). doi: 10.1007/9783642334184_2
 7.Clifford, A.H.: Totally ordered commutative semigroups. Bull. Am. Math. Soc. 64(6), 305–316 (1958). doi: 10.1090/S000299041958102219 MathSciNetCrossRefMATHGoogle Scholar
 8.Cokelaer, F., Talbot, H., Chanussot, J.: Efficient robust ddimensional path operators. IEEE J. Sel. Top. Signal Process. 6(7), 830–839 (2012). doi: 10.1109/jstsp.2012.2213578 CrossRefGoogle Scholar
 9.Galperin, I., Rivest, R.L.: Scapegoat trees. In: Proceedings of the fourth annual ACMSIAM Symposium on Discrete algorithms. SODA ’93, pp. 165–174. Society for Industrial and Applied Mathematics, Philadelphia (1993)Google Scholar
 10.van de Gronde, J.J.: Tensorial Orientation Scores. In: Azzopardi, G., Petkov, N. (eds.) Computer Analysis of Images and Patterns. LNCS, vol. 9257, pp. 783–794. Springer International Publishing, Switzerland (2015). doi: 10.1007/9783319231174_67
 11.van de Gronde, J.J., Lysenko, M., Roerdink, J.B.T.M.: PathBased Mathematical Morphology on Tensor Fields. In: Hotz, I., Schultz, T. (eds.) Visualization and Processing of Higher Order Descriptors for MultiValued Data, Mathematics and Visualization, pp. 109–127. Springer International Publishing, Switzerland (2015a). doi: 10.1007/9783319150901_6
 12.van de Gronde, J.J., Schubert, H.R., Roerdink, J.B.T.M.: Fast Computation of Greyscale Path Openings. In: Benediktsson, J.A., Chanussot, J., Najman, L., Talbot, H. (eds.) Mathematical Morphology and Its Applications to Signal and Image Processing. LNCS, vol. 9082, pp. 621–632. Springer International Publishing, Switzerland (2015). doi: 10.1007/9783319187204_52
 13.Heijmans, H., Buckley, M., Talbot, H.: Pathbased morphological openings. In: IEEE International Conference on Image Processing, vol. 5, pp. 3085–3088. (2004). doi: 10.1109/icip.2004.1421765
 14.Heijmans, H., Buckley, M., Talbot, H.: Path openings and closings. J. Math. Imaging Vis. 22(2), 107–119 (2005). doi: 10.1007/s1085100548853 MathSciNetCrossRefGoogle Scholar
 15.Heijmans, H.J.A.M.: Morphological Image Operators. Academic Press, San Diego (1994)MATHGoogle Scholar
 16.LuengoHendriks, C.L.: Constrained and dimensionalityindependent path openings. IEEE Trans. Image Process. 19(6), 1587–1595 (2010). doi: 10.1109/tip.2010.2044959 MathSciNetCrossRefGoogle Scholar
 17.Merveille, O., Talbot, H., Najman, L., Passat, N.: Tubular Structure Filtering by Ranking Orientation Responses of Path Operators. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision—ECCV 2014. LNCS, pp. 203–218. Springer International Publishing, Switzerland (2014). doi: 10.1007/9783319106052_14
 18.Merveille, O., Talbot, H., Najman, L., Passat, N.: Ranking Orientation Responses of Path Operators: Motivations, Choices and Algorithmics. In: Benediktsson, J.A., Chanussot, J., Najman, L., Talbot, H. (eds.) Mathematical Morphology and Its Applications to Signal and Image Processing. LNCS, vol. 9082, pp. 633–644. Springer International Publishing, Switzerland (2015). doi: 10.1007/9783319187204_53
 19.Morard, V., Dokládal, P., Decencière, E.: Onedimensional openings, granulometries and component trees in O(1) per pixel. IEEE J. Sel. Top. Signal. Process. 6(7), 840–848 (2012). doi: 10.1109/jstsp.2012.2201694 CrossRefGoogle Scholar
 20.Morard, V., Dokládal, P., Decencière, E.: Parsimonious path openings and closings. IEEE Trans. Image Process. 23(4), 1543–1555 (2014). doi: 10.1109/tip.2014.2303647 MathSciNetCrossRefGoogle Scholar
 21.Munro, J.I., Papadakis, T., Sedgewick, R.: Deterministic Skip Lists. In: Proceedings of the Third Annual ACMSIAM Symposium on Discrete Algorithms. SODA ’92, pp. 367–375. Society for Industrial and Applied Mathematics, Philadelphia (1992)Google Scholar
 22.Offringa, A.R., van de Gronde, J.J., Roerdink, J.B.T.M.: A morphological algorithm for improving radiofrequency interference detection. Astron. Astrophys. 539, A95+ (2012). doi: 10.1051/00046361/201118497
 23.Pugh, W.: Skip lists: a probabilistic alternative to balanced trees. Commun. ACM 33(6), 668–676 (1990). doi: 10.1145/78973.78977 MathSciNetCrossRefGoogle Scholar
 24.Ronse, C.: Orderconfiguration functions: mathematical characterizations and applications to digital signal and image processing. Inf. Sci. 50(3), 275–327 (1990). doi: 10.1016/00200255(90)900142 MathSciNetCrossRefMATHGoogle Scholar
 25.Ronse, C., Heijmans, H.J.A.M.: The algebraic basis of mathematical morphology. CVGIP: Image Underst. 54(1), 74–97 (1991). doi: 10.1016/10499660(91)900762 CrossRefMATHGoogle Scholar
 26.Schubert, H., van de Gronde, J.J., Roerdink, J.B.T.M.: Efficient computation of greyscale path openings. Math. Morphol. Theory Appl. (2015). AcceptedGoogle Scholar
 27.Sleator, D.D., Tarjan, R.E.: Selfadjusting binary search trees. J. ACM 32(3), 652–686 (1985). doi: 10.1145/3828.3835 MathSciNetCrossRefMATHGoogle Scholar
 28.Soille, P.: On morphological operators based on rank filters. Pattern Recognit. 35(2), 527–535 (2002). doi: 10.1016/s00313203(01)000474 CrossRefMATHGoogle Scholar
 29.Talbot, H., Appleton, B.: Efficient complete and incomplete path openings and closings. Image Vis. Comput. 25(4), 416–425 (2007). doi: 10.1016/j.imavis.2006.07.021 CrossRefGoogle Scholar
 30.Urbach, E.R., Roerdink, J.B.T.M., Wilkinson, M.H.F.: Connected shapesize pattern spectra for rotation and scaleinvariant classification of grayscale images. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 272–285 (2007). doi: 10.1109/tpami.2007.28 CrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.