Separating bichromatic point sets in the plane by restricted orientation convex hulls

Abstract

We explore the separability of point sets in the plane by a restricted-orientation convex hull, which is an orientation-dependent, possibly disconnected, and non-convex enclosing shape that generalizes the convex hull. Let R and B be two disjoint sets of red and blue points in the plane, and $\mathcal {O}$ be a set of $k\ge 2$ lines passing through the origin. We study the problem of computing the set of orientations of the lines of $\mathcal {O}$ for which the $\mathcal {O}$-convex hull of R contains no points of B. For $k=2$ orthogonal lines we have the rectilinear convex hull. In optimal $O(n\log n)$ time and O(n) space, $n = \vert R \vert + \vert B \vert$, we compute the set of rotation angles such that, after simultaneously rotating the lines of $\mathcal {O}$ around the origin in the same direction, the rectilinear convex hull of R contains no points of B. We generalize this result to the case where $\mathcal {O}$ is formed by $k \ge 2$ lines with arbitrary orientations. In the counter-clockwise circular order of the lines of $\mathcal {O}$, let $\alpha _i$ be the angle required to clockwise rotate the ith line so it coincides with its successor. We solve the problem in this case in $O({1}/{\Theta }\cdot N \log N)$ time and $O({1}/{\Theta }\cdot N)$ space, where $\Theta = \min \{ \alpha _1,\ldots ,\alpha _k \}$ and $N=\max \{k,\vert R \vert + \vert B \vert \}$. We finally consider the case in which $\mathcal {O}$ is formed by $k=2$ lines, one of the lines is fixed, and the second line rotates by an angle that goes from 0 to $\pi$. We show that this last case can also be solved in optimal $O(n\log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

1 Introduction

A classic topic in computational geometry is designing efficient algorithms to separate sets of red and blue points. Several separability criteria have been considered in the literature, as well as separators of different complexities. Well-known constant-complexity separators include a line or a hyperplane [10, 24, 27, 32], a wedge or a double-wedge [1, 3, 25, 26, 44], a circle [7, 8, 17, 37], and one or two boxes [2, 16, 33, 50]. Typical separators of linear complexity include different types of polygonal chains (e.g. monotone or with alternating constant turn) [25, 39], different types of enclosing shapes (e.g. a polygon or a non-traditional convex hull) [5, 19], and sets of geometric objects of the same type, such as a hyperplanes [35] and triangles [34]. These choices of separators have been used not only on points, but also on segments, circles, simple polygons, etc.

Separability problems are closely related to clustering applications, where separating/discriminating is a necessary task. Consider for example a damaged region modeled by a set of points that needs to be separated from the rest. In this context, we want to extract the region with minimum area bounded by an enclosing shape that is easy to cut and compute; see [5, 6, 12, 15, 20, 21, 49] for references on these types of shapes.

In this paper we extend the previous work on separability of two-colored point sets in the Euclidean plane. Let P be a finite set of points. The convex hull of P, that we denote with $\mathcal {C}{\mathcal {H}}({P})$, is the closed region obtained by removing from the plane all the open halfplanes which are empty of points of P. We explore the separability by orientation-dependent, possibly disconnected, and non-convex enclosing shapes that generalize this definition by using open wedges instead of open halfplanes. We first study the rectilinear convex hull. The rectilinear convex hull of P, that we denote with $\mathcal {R}{\mathcal {H}}({P})$, is the closed region obtained by removing from the plane all the open axis-aligned wedges of aperture angle $\frac{\pi }{2}$, which are empty of points of P (see Sect. 2 for a formal definition). Observe in Fig. 1 that $\mathcal {R}{\mathcal {H}}({P})$ might be a simply connected set, yielding an intuitive and appealing structure. However, in other cases $\mathcal {R}{\mathcal {H}}({P})$ can have several connected components, some of which might be single points of P.

Fig. 1

The rectilinear convex hull of a finite point set P. The (standard) convex hull of P is shown in dashed lines

The rectilinear convex hull introduces two important differences with respect to the convex hull. On one hand we have that $\mathcal {R}{\mathcal {H}}({P})\subset \mathcal {C}{\mathcal {H}}({P})$ [40, Theorem 4.7], a property that provides more flexibility to better classify a subset of points. On the other hand we have that $\mathcal {R}{\mathcal {H}}({P})$ is orientation-dependent, which introduces the orientation of the empty wedges as a search space for several optimization criteria; e.g., minimum area or boundary points. To illustrate these differences, consider two disjoint sets R and B of red and blue points in the plane. Using the standard convex hull, the relative positions of R and B may lead to situations as in Fig. 2.

Fig. 2

Relative positions of B and $\mathcal {C}{\mathcal {H}}({R})$. The hull $\mathcal {C}{\mathcal {H}}({B})$ is shown in dashed lines

Using instead the rectilinear convex hull with an arbitrary orientation, we can achieve further goals such as completely separating R and B, or minimizing the number of misclassified points; i.e., points of one color inside the hull of the other color. See Fig. 3.

Fig. 3

The orientation of the coordinate axis is shown in the bottom left corner

The main contribution of this paper is a time-optimal algorithm to compute a rectilinear convex hull with arbitrary orientation that is monochromatic, i.e., that has no misclassified points. We also provide similar results for generalizations of the rectilinear convex hull that stem from a variation of convexity known as restricted orientation convexity [20, 41] or $\mathcal {O}$-convexity¹. As we show, despite the separability problem seems harder in the context of $\mathcal {O}$-convexity than for standard convexity, under certain assumptions both cases can be solved within the same time and space complexities.

1.1 Background and related work

Restricted-orientation convexity in the Euclidean plane is a generalization of orthogonal convexity, and at the same time a restriction of standard convexity. The orientation of a line is the smallest of the two possible angles it makes with the $X^+$ positive semiaxis. A set of orientations $\mathcal {O}$ is a set of lines with different orientations passing through some fixed point. A region of the plane is called $\mathcal {O}$-convex if its intersection with any line parallel to a line of $\mathcal {O}$ is either empty, a point, or a line segment. Since this notion of convexity was defined in the early eighties, several results of topological and combinatorial flavors can be found in the literature, as well as computational problems that are usually adaptations of well-known problems related to standard convexity [20, 31].

The $\mathcal {O}$-convex hull of a finite point set is an $\mathcal {O}$-convex superset of such point set that generalizes both the standard and the rectilinear convex hull; refer to Sect. 3.1 for a formal definition. The $\mathcal {O}$-convex hull is relevant for research fields that require restricted-orientation enclosing shapes [18]. In the particular case where $\mathcal {O}$ is formed by two orthogonal lines, $\mathcal {O}$-convexity is known as orthogonal convexity² and the $\mathcal {O}$-convex hull is known as the rectilinear convex hull. The rectilinear convex hull has been extensively studied in the context of fields as diverse as polyhedra reconstruction [14], facility location [47], and geometric optimization [29]; as well as in practical research fields such as pattern recognition [28], shape analysis [15], and VLSI circuit layout design [48].

As far as we are aware, there are no previous results on the problem of separating bichromatic point sets by an $\mathcal {O}$-convex hull while the orientations of the lines of $\mathcal {O}$ are changing. Nevertheless, if the lines are fixed, then the problem can be trivially solved by combining the algorithm from Alegría et al. [6] to compute the $\mathcal {O}$-convex hull of a finite set of n points in $O(n \log n)$ time, and a straightforward extension of the so-called staircase structure used to store the vertices of the rectilinear convex hull [40, Section 4.1.3]. With this approach we obtain an $O(n \log n)$ time and O(n) space algorithm to decide if there is a monochromatic $\mathcal {O}$-convex hull for any fixed orientations of the lines of $\mathcal {O}$.

The problem of separating a bichromatic point set using an $\mathcal {O}$-convex separator has already been studied for the particular case of orthogonal convexity. In this case the problem consists in computing, if any, an orthogonally-convex geometric separator for R and B among all possible orientations of the coordinate axes. The most popular separator is the axis-aligned rectangle. For $n = \vert R \vert + \vert B \vert$, an arbitrarily-oriented separating rectangle can be found in $O(n\log n)$ time and O(n) space [49]. Several variations have also been solved including separability by two disjoint rectangles [33], bichromatic sets of imprecise points [46], maximizing the area of the separating rectangle [2, 9], and an extension where the separator is a box in three dimensions [27]. Along with the axis-aligned rectangle, two more ortho-convex separators can be found in the literature. In [45] the authors use as separator an axis-aligned L-shaped region and solve the problem in $O(n^2)$ time. In [39] the authors use as separator an alternating orthogonal polygonal chain, and also solve the problem in $O(n^2)$ time.

Our separability problem can also be considered as an instance of a general class of problems which consist in computing the orientations where an orientation-dependent geometric object satisfies some optimization criteria. Our problem can then be stated as the problem of computing the orientations of the lines of $\mathcal {O}$ for which the $\mathcal {O}$-convex hull of R has the minimum number of misclassified points. If such a number is different from zero, then the given point sets cannot be separated by the particular $\mathcal {O}$-convex hull. In this context, the $\mathcal {O}$-convex hull is called a weak separator for R and B. The concept of weak separability was introduced by Houle [23, 24]. Separability results in this direction have been explored using $\mathcal {O}$-convex separators such as hyperplanes, strips, and rectangles [10, 16, 24, 30].

Besides geometric separability, other similar types of problems can also be found in the literature. Given a set P of n points in the plane, in [6] the authors compute the angle by which the lines of $\mathcal {O}$ have to be simultaneously rotated around the origin for the $\mathcal {O}$-convex hull of P to have minimum area. A similar problem is solved in [5], where the authors compute the values of $\beta$ for which the $\mathcal {O}_\beta$-convex hull of P has maximum area, among other optimization criteria (refer to Sect. 3.2 for a formal definition of the $\mathcal {O}_\beta$-convex hull). More recently, in [13] the authors solved the problem of computing the set of empty squares with arbitrary orientations among a set of points. From this result they derive an algorithm to compute the square annulus with arbitrary orientation of optimal width or area that encloses P, among other algorithmic results.

1.2 Results

In this paper we contribute with the following results:

• An optimal $O(n \log n)$ time and O(n) space algorithm to compute a monochromatic rectilinear convex hull with arbitrary orientation, where $n = \vert R \vert + \vert B \vert$.

• An algorithm to compute a monochromatic $\mathcal {O}$-convex hull with arbitrary orientation for a set $\mathcal {O}$ of $k \ge 2$ lines. In the counter-clockwise circular order of the lines of $\mathcal {O}$, let $\alpha _i$ be the angle required to clockwise rotate the ith line around the origin so it coincides with its successor. The algorithm runs in $O({1}/{\Theta } \cdot N \log N)$ time and $O({1}/{\Theta }\cdot N)$ space, where $\Theta = \min \{ \alpha _1,\ldots ,\alpha _k \}$ and $N=\max \{ k, \vert R \vert + \vert B \vert \}$.

• An optimal $O(n \log n)$ time and O(n) space algorithm to compute the values of $\beta$ for which there is a monochromatic $\mathcal {O}_\beta$-convex hull.

• In all the cases, if there is no orientation of separability, the algorithms can be easily adapted to compute the hull that minimizes the number of misclassified points.

1.3 Adopted conventions

Throughout the rest of the paper, we denote with R and B two disjoint sets of red and blue points in the plane and denote $n = \vert R \vert + \vert B \vert$. For the sake of simplicity, we assume that the set $R \cup B$ contains no three points on a line. Although all our algorithms can be extended to appropriately handle point sets with three points on a line, such extensions will require a tedious case analysis. Regarding the set of orientations, we assume for the sake of simplicity that all the lines of $\mathcal {O}$ have different orientations and pass through the origin. We also assume that $\mathcal {O}$ contains a finite number of lines, and denote $k = \vert \mathcal {O}\vert$. We remark that sets of orientations with an infinite number of lines have been considered in the literature [20, 41]. Finally, in our algorithms we adopt the real RAM model of computation [40], which is customary in computational geometry and allows us to perform standard arithmetic and trigonometric operations in constant time.

1.4 Outline of the paper

In Sect. 2 we solve the separability problem using a rectilinear convex hull with arbitrary orientation. In Sect. 3 we solve the separability problem using an $\mathcal {O}_\beta$-convex hull, and an $\mathcal {O}$-convex hull with arbitrary orientation where the set $\mathcal {O}$ contains $k \ge 2$ lines. Finally, we dedicate Sect. 4 to prove lower bounds.

2 The rectilinear convex hull

In this section we solve the following problem.

Problem 1

Given a set of orientations $\mathcal {O}$ formed by $k=2$ orthogonal lines, compute the set of rotation angles for which the lines of $\mathcal {O}$ have to be simultaneously rotated around the origin in the counterclockwise direction, so the rectilinear convex hull of R contains no points of B.

We start with a formal definition of the rectilinear convex hull. For the sake of completeness, we also briefly describe the properties of the rectilinear convex hull that are relevant to solve Problem 1. More details on these and other properties can be found in [20, 38].

Let $\rho _1$ and $\rho _2$ be two rays leaving a point $x\in \mathbb {R}^{2}$ such that, after rotating $\rho _1$ around x by an angle of $\theta \in [0, 2\pi )$, we obtain $\rho _2$. We refer to the two open regions in the set $\mathbb {R}^{2}\setminus (\rho _1 \, \cup \, \rho _2)$ as wedges. We say that both wedges have vertex x and sizes $\theta$ and $2\pi -\theta$, respectively. Throughout this section assume that the orientation set $\mathcal {O}$ is formed by two orthogonal lines. A quadrant is a wedge of size $\frac{\pi }{2}$ whose rays are parallel to the lines of $\mathcal {O}$. Let P denote a finite set of points in the plane. We say a region of the plane is free of points of P, or P-free for short, if there are no points of P in its interior. The rectilinear convex hull of P, denoted with $\mathcal {R}{\mathcal {H}}({P})$, is the set

$$\begin{aligned} \mathcal {R}{\mathcal {H}}({P})= \mathbb {R}^{2} \setminus \bigcup _{q\in \mathcal {Q}}q, \end{aligned}$$

where $\mathcal {Q}$ denotes the set of all P-free open quadrants of the plane. See Fig. 4.

Fig. 4

A finite point set P and $\mathcal {R}{\mathcal {H}}({P})$ for two different rotation angles of the lines of $\mathcal {O}$

Note that $\mathcal {R}{\mathcal {H}}({P})$ is not convex if at least one edge of the standard convex hull of P is not parallel to a line of $\mathcal {O}$. Moreover, $\mathcal {R}{\mathcal {H}}({P})$ may be disconnected. Each connected component is either a single point of P, or a closed orthogonal polygon whose edges are parallel to a line of $\mathcal {O}$. The rectilinear convex hull has also at most four “degenerate edges”, which are orthogonal polygonal chains connecting either two extremal vertices, or a connected component to an extremal vertex. Of special relevance is the property we call orientation dependency: except for some particular cases, like rotating the orientations by $\frac{\pi }{2}$, the $\mathcal{R}\mathcal{H}(P)$ at different orientations of the lines of $\mathcal {O}$ are non-congruent to each other.

Let $\mathcal {O}_\theta$ denote the set of lines obtained after simultaneously rotating the lines of $\mathcal {O}$ around the origin in the counter-clockwise direction by an angle of $\theta$. We denote with $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ the rectilinear convex hull of P computed with respect to $\mathcal {O}_\theta$. We solve Problem 1 by describing an algorithm to compute the (possibly empty) set of angular intervals of $\theta$ for which $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is B-free. Note that we are considering strict containment, so a blue point lying on the boundary of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is not contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$; see for example the blue point labeled x in Fig. 3. Our algorithm runs in $O(n\log n)$ time and O(n) space. These are the same complexities required to compute the rectilinear convex hull of a set of n points for a fixed orientation of the lines of $\mathcal {O}$ [38].

2.1 Maximal wedges and maximal arcs

Before describing our algorithm, we need some auxiliary results. We start by characterizing the points of the plane strictly contained in $\mathcal {R}{\mathcal {H}}_{\theta }({P})$. We omit the proof of the following proposition, since it derives directly from the definition of the rectilinear convex hull.

Proposition 1

A point $x \in \mathbb {R}^{2}$ is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ if, and only if, every quadrant with vertex on x contains at least one point of P.

Let $w_x$ be a P-free wedge with vertex at a point $x\in \mathbb {R}^{2}$. We say that $w_x$ is maximal, if no other P-free wedge with vertex on x intersects $w_x$. Assume that $w_x$ is maximal. Let $w_o$ be the wedge resulting from translating $w_x$ so that its vertex lies on the origin. The maximal arc of x induced by $w_x$ is the circular arc that results from the intersection of $w_o$ and $\mathbb {S}^1$ (the unit circle centered at the origin). Note that, since wedges (and hence, quadrants) are open regions, Proposition 1 excludes points on the boundary of $\mathcal {R}{\mathcal {H}}_{\theta }({P})$, and the endpoints of a maximal arc do not belong to the maximal arc itself. See Fig. 5.

Fig. 5

A maximal arc of a point $x\in \mathbb {R}^{2}$

A maximal arc is feasible if it is induced by a maximal wedge with size at least $\frac{\pi }{2}$. Hereafter, we consider $\mathcal {O}_\theta$ to be not only a set of two orthogonal lines, but also the set of four rays in which the orthogonal lines are split by the origin.

Lemma 2

For any fixed value of $\theta$, a point $x \in \mathbb {R}^{2}$ is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ if, and only if, every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\theta$.

Proof

We show that every quadrant with vertex on x contains at least one point of P if, and only if, every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\theta$. The lemma follows from this fact and Proposition 1. In the following, we assume without loss of generality that $\theta =0$ and x lies on the origin, so the lines of $\mathcal {O}_\theta$ coincide with the coordinate axes and every quadrant with vertex on x is bounded by two coordinate semi-axes.

$(\Longrightarrow )$ Using Proposition 1, assume that every quadrant with vertex on x contains at least one point of P. We show that every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\theta$. Let w be a maximal wedge with vertex at x and size at least $\frac{\pi }{2}$, and let a be the feasible maximal arc of x induced by w. Since the size of w is at least $\frac{\pi }{2}$, then w contains at least one coordinate semi-axis. On the other hand, w cannot contain two coordinate semi-axis, since otherwise w would contain a P-free quadrant. This would be a contradiction, since we assumed that every quadrant with vertex on x contains at least one point of P. Hence w contains exactly one coordinate semi-axis, and thus, a is intersected by a single ray of $\mathcal {O}_\theta$.

$(\Longleftarrow )$ Assume that every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\theta$. We show that every quadrant with vertex on x contains at least one point of P, which is enough by Proposition 1. For the sake of contradiction, suppose there is a P-free quadrant q with vertex on x. Then there is a maximal wedge w with vertex on x that contains q. Since wedges are open regions, if the size of w is equal to $\frac{\pi }{2}$ then w contains no coordinate semi-axis, and thus, it induces a feasible maximal arc intersected by no ray of $\mathcal {O}_\theta$. On the other hand, if the size of w is greater than $\frac{\pi }{2}$, then w contains at least two coordinate semi-axes, and thus, it induces a feasible maximal arc of x intersected by at least two rays of $\mathcal {O}_\theta$. Either case is a contradiction, since we assumed that every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\theta$. $\square$

An illustration of Lemma 2 is shown in Fig. 6. In the figure we can see a set P of four points, $\mathcal {R}{\mathcal {H}}_{\theta }({P})$, a point $x\in \mathbb {R}^{2}$, the feasible maximal arcs of x, and the lines of $\mathcal {O}_\theta$. The maximal arcs are drawn with thick circular arcs. Instead of drawing the arcs on a single circle representing $\mathbb {S}^1$, we draw them separately on concentric circles for the sake of clarity. In Fig. 6a the point x is not contained in $\mathcal {R}{\mathcal {H}}_{\theta }({P})$; hence, there is at least one feasible maximal arc of x that is not intersected by a single ray of $\mathcal {O}_\theta$. Note that the maximal wedge w induces a feasible maximal arc a that is intersected by two rays of $\mathcal {O}_\theta$. In Fig. 6b the point x is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({P})$; hence, all the feasible maximal arcs of x are intersected by a single ray of $\mathcal {O}_\theta$.

Fig. 6

Containment of a point $x\in \mathbb {R}^{2}$ in $\mathcal {R}{\mathcal {H}}_{\theta }({P})$

The adaptation of Lemma 2 to a bichromatic setting is straightforward. A blue maximal wedge is an R-free maximal wedge with vertex on a blue point. A blue maximal arc is a maximal arc induced by a blue maximal wedge. A blue maximal arc is feasible if it is induced by a blue maximal wedge with size at least $\frac{\pi }{2}$.

Lemma 3

A blue point $b \in B$ is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ if, and only if, every blue maximal arc of b that is feasible is intersected by a single ray of $\mathcal {O}_\theta$.

Let ${\widehat{D}}$ be a direction in $\mathbb {S}^1$ and let w be a P-free maximal wedge with vertex on a point $x \in \mathbb {R}^{2}$. We say that w is constrained to ${\widehat{D}}$ if w contains the ray leaving x with direction $\widehat{D}$. We compute the set of blue maximal arcs that are feasible by means of a procedure to compute the set of blue maximal wedges constrained to a given direction. This procedure is an adaptation for bichromatic point sets of the restricted unoriented maximum approach from Avis et al. [11]. Given a set P of n points in the plane and an angle $\Theta \ge {\pi }/{2}$, the authors compute, in $O(n \log n)$ time and O(n) space, the set of P-free wedges with size at least $\Theta$ and vertex on a point of P.

The adapted procedure is as follows. Let $\widehat{D}$ denote the direction given as input. Without loss of generality, assume that $\widehat{D}$ is equal to the $Y^+$ semiaxis. We first sort the points of the set $R \cup B$ in a direction orthogonal to ${\widehat{D}}$ (along the X axis in our assumption). We then perform two sweeps on the sorted set of points. In the first sweep we traverse the points from left to right. A red point is processed using an on-line algorithm to construct the convex hull of the red visited points, one point at a time. To process a blue point b, we compute the R-free wedge with vertex on b that is bounded by a ray leaving b with direction ${\widehat{D}}$, and the tangent from b to the red convex hull. In the second sweep we traverse the sorted set of points from right to left and process points in a symmetric way. Let $w_l(b)$ and $w_r(b)$ denote, respectively, the R-free wedges obtained after processing a blue point b in the sweeps from left-to-right and from right-to-left. After performing both sweeps, we report $w_l(b) \cup w_r(b)$ as a blue maximal wedge constrained to ${\widehat{D}}$, for all $b \in B$. See Fig. 7.

Fig. 7

The R-free maximal wedge with vertex on b constrained to ${\widehat{D}}$

In the procedure described above, we first sort the points in the direction orthogonal to ${\widehat{D}}$ in $O(n \log n)$ time. Using standard techniques [40], during each sweep we process a point in $O(\log \vert R \vert ) = O(\log n)$ time: If a red point, we are updating the convex hull of a point set by inserting a new point. If a blue point, we are computing the tangent from a point to a convex polygon described by the sorted list of its vertices. Since each blue point is the vertex of a single R-free maximal wedge constrained to ${\widehat{D}}$, the whole procedure takes $O(n \log n)$ time and O(n) space. We obtain the following lemma.

Lemma 4

Given a direction ${\widehat{D}}$ in $\mathbb {S}^1$ and two disjoint sets R and B of red and blue points in the plane, the set of $\vert B \vert$ blue maximal wedges constrained to ${\widehat{D}}$ can be computed in $O(n \log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

And we obtain the following result.

Lemma 5

There are O(n) blue maximal arcs that are feasible. The set of blue maximal arcs that are feasible can be computed in $O(n \log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

Proof

A maximal arc is induced by a blue maximal wedge with size at least $\frac{\pi }{2}$. Since a blue point is the vertex of at most four of such wedges, then each blue point has at most four blue maximal arcs that are feasible. Hence, there are $O(\vert B \vert ) = O(n)$ arcs.

We compute the set of blue maximal arcs that are feasible as follows. Note that a maximal wedge with size at least $\frac{\pi }{2}$ is constrained to one of the $X^+$, $X^-$, $Y^+$, or $Y^-$ coordinate semiaxis. In $O(n \log n)$ time and O(n) space, we compute the set of blue maximal wedges constrained to each coordinate semiaxis, by means of the algorithm used to prove Lemma 4. Then, we traverse the resulting set of $O(\vert B \vert ) = O(n)$ blue maximal wedges, and keep those with size at least $\frac{\pi }{2}$. Finally, we transform each maximal wedge into a maximal arc in O(1) time per wedge. Since the most expensive step is the computation of the set of blue maximal wedges, the whole procedure takes $O(n \log n)$ time and O(n) space. $\square$

2.2 The algorithm

We are now ready to describe the algorithm to compute the set of angular intervals of $\theta \in (0,2\pi ]$ for which $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is B-free. Our strategy is to perform an angular sweep on the set of blue maximal arcs that are feasible, while we maintain the set $B_\theta$ of blue points in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$. To perform the angular sweep we increment $\theta$ from 0 to $\pi /2$, so the four rays of $\mathcal {O}_\theta$ sweep all the directions of $\mathbb {S}^1$. By Lemma 3, for a particular value of $\theta$ during the sweep process, a blue point b is contained in $B_\theta$ if all the blue maximal arcs of b that are feasible are intersected by a single ray of $\mathcal {O}_\theta$. Hence, $B_\theta$ only changes at the values of $\theta$ where a ray of $\mathcal {O}_\theta$ passes over an endpoint of a maximal arc. We call these rotation angles intersection events. By means of a set of $\vert B \vert$ auxiliary variables, we update $B_\theta$ at each intersection event in constant time. The algorithm is described in detail next.

Step 1. Computing the set of feasible maximal arcs.

The first step of the algorithm is to compute the set $\mathcal {A}$ of $O(\vert B \vert ) = O(n)$ blue maximal arcs that are feasible. We compute this set by means of the procedure used to prove Lemma 5. Hence, this step takes $O(n \log n)$ time and O(n) space.

Step 2. Computing the list of intersection events.

The second step is to transform the set of blue maximal arcs that are feasible into a sorted circular list $\mathcal {L}$ of intersection events. Since intersection events are given by the endpoints of maximal arcs, each maximal arc is transformed into two intersection events, hence there are $O(\vert B \vert ) = O(n)$ intersection events. Let a be a blue maximal arc that is feasible, and let p and q be the endpoints of a. We transform a into a pair of intersection events by computing, in O(1) time, the directions in $\mathbb {S}^1$ of the rays leaving the origin that pass through p and q, see Fig. 8. We can thus transform the set of blue maximal arcs that are feasible into the set of intersection events in O(n) time. We store the set of O(n) intersection events in $\mathcal {L}$, sorted as the endpoints of the maximal arcs appear while traversing $\mathbb {S}^1$ in the counter-clockwise direction. Since the most expensive task is to sort the set of intersection events, this step takes $O(n \log n)$ time and O(n) space.

Fig. 8

The endpoints p and q of a can be transformed in O(1) time into two intersection events $\theta _p$ and $\theta _q$, respectively

Step 3. Performing the angular sweep.

The final step is to perform an angular sweep on the set of blue maximal arcs that are feasible. Let $b_1,\ldots ,b_{\vert B \vert }$ be the set of blue points labeled with no particular order. Let $N_i$, $0 \le N_i \le 4$, denote the number of blue maximal arcs of the point $b_i$ that are feasible, and let $n_i$, $0 \le n_i \le N_i$, denote the number of blue maximal arcs of $b_i$ that are intersected by a single ray of $\mathcal {O}_\theta$. We use an array of $\vert B \vert$ Boolean flags to represent if a blue point belongs to $B_\theta$, so the status of a blue point can be changed in O(1) time. Following the condition from Lemma 3, we set the i-th flag of the array to True if $n_i = N_i$ ($b_i$ belongs to $B_\theta$), and to False if $n_i < N_i$ ($b_i$ does not belong to $B_\theta$).

To process intersection events during the angular sweep we use the following auxiliary structures. For each blue maximal arc a that is feasible, we define a variable $\rho (a)$ that contains the number of rays of $\mathcal {O}_\theta$ currently intersecting a. We use a min-priority queue $\mathcal {Q}$ to predict the next intersection event, among the events induced by all the blue maximal arcs. Let $r_1,\ldots ,r_4$ be the rays of $\mathcal {O}_\theta$ sorted in counter-clockwise circular order around the origin, and let $\theta _i$ be the smallest rotation angle for which $r_i$ passes over an endpoint of a maximal arc. The queue contains the angles $\theta _1,\ldots ,\theta _4$ that are less than $\frac{\pi }{2}$. The next intersection event is thus given by the minimum element in $\mathcal {Q}$. Since $\mathcal {Q}$ contains at most four elements, both update and query operations on $\mathcal {Q}$ take O(1) time. See Fig. 9.

Fig. 9

Illustration of the angular sweep. For the sake of clarity, we consider a set B with a single blue point b

We now describe how to perform the angular sweep. First, we initialize the auxiliary data structures described above at an initial value of $\theta$, say $\theta =0$. Consider the four rays of $\mathcal {O}_\theta$ sorted in counter-clockwise circular order around the origin. We first merge, in O(n) time, the angles in $\mathcal {L}$ with the orientation angles given by the sorted set of rays of $\mathcal {O}_\theta$. After merging, we can say which blue maximal arcs are intersected by each ray of $\mathcal {O}_\theta$, as well as the smallest rotation angle for which each ray of $\mathcal {O}_\theta$ passes over the endpoint of a maximal arc. Using this information, in O(n) time we compute the values of the variables $n_i$, $1 \le i \le \vert B \vert$, and $\rho (a)$ for all $a \in \mathcal {A}$, and initialize the set of $\vert B \vert$ Boolean flags we use to represent $B_\theta$. We finally initialize $\mathcal {Q}$ in O(1) time. Hence, the whole initialization step takes O(n) time.

We perform the angular sweep by incrementing $\theta$ from 0 to ${\pi }/{2}$. The next intersection event is obtained by extracting the minimum angle from $\mathcal {Q}$. Consider an intersection event $\theta$ for which a ray $r\in \mathcal {O}_\theta$ is passing over the endpoint of a maximal arc a of a blue point $b_i$. We process the event as follows:

• If r starts intersecting a, then we increase $\rho (a)$ by one. If, instead, r stops intersecting a, then we decrease $\rho (a)$ by one.

• If $\rho (a)$ was changed, then we update $n_i$. If $\rho (a)$ is equal to one, then we increase $n_i$ by one. If $\rho (a)$ is instead different from one, then we decrease $n_i$ by one.

• If $n_i$ was changed, then we update the Boolean flags that represent $B_\theta$. If $n_i = N_i$, then we set the i-th flag of $B_\theta$ to True. If instead $n_i < N_i$, then we set the i-th flag to False.

• Finally, we obtain from $\mathcal {L}$ the successor $\theta ^{\prime }$ of $\theta$, and insert the angle $\theta ^{\prime } - \theta$ into $\mathcal {Q}$.

Lemma 6

The set $B_\theta$ can be computed and maintained while $\theta$ is increased from 0 to ${\pi }/{2}$ in $O(n \log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

Proof

Steps 1 and 2 take $O(n \log n)$ time and O(n) space. Since we have O(n) intersection events and each event is processed in O(1) time, the sweep process of Step 3 takes O(n) time and O(n) space. The lemma follows. $\square$

By keeping track of the changes of $B_\theta$ we can construct the angular intervals for which all the flags of $B_\theta$ are False. Hence, from Lemma 6 we obtain the main result of this section.

Theorem 7

Given two disjoint sets R and B of points in the plane, the (possibly empty) set of angular intervals of $\theta \in [0,2\pi )$ for which $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is B-free (i.e., $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is a separator of R and B) can be computed in $O(n\log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

The algorithm we described to prove Theorem 7 is time-optimal. A proof of the $\Omega (n\log n)$-time lower bound is presented in Sect. 4. There are a couple of additional facts worth mentioning. First, the algorithm only computes a set of angular intervals. To actually compute a monochromatic rectilinear convex hull we need to first choose an angle in one of these intervals, and then spend additional $O(n\log n)$ time [6, 29, 38]. Second, the reported angular intervals are maximal in the sense that no two of them intersect each other. The intervals are also open since they are bounded by intersection events and, at such events, a blue point lies on the boundary of some R-free quadrant. Hence, the point lies on the boundary of the rectilinear convex hull of R. Finally, since there is at most one change in $B_\theta$ per intersection event and there are O(n) intersection events, then there are O(n) angular intervals of $\theta$ where $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is B-free. A matching lower bound is achieved by the point set we describe next.

2.3 Lower bound for the number of intervals of separability

In this subsection we describe a bichromatic point set with $\Omega (n)$ angular intervals of $\theta$ for which $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is B-free. The first ingredient of the construction is the fact that $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ may be disconnected. As previously mentioned, a connected component is either a single point of P, an orthogonal polygonal chain, or a closed orthogonal polygon. The polygonal chain connects two extremal points of P and contains exactly two segments. The orthogonal polygon may have at most two “degenerate edges” in each direction, which are horizontal or vertical segments connecting its vertices with extremal points of P. The segments of the polygonal chains and the edges of the orthogonal polygons are called the edges of $\mathcal {R}{\mathcal {H}}_{\theta }({P})$. Each edge is contained in a ray of some P-free quadrant. Such a P-free quadrant is said to be stabbing P. Note that each of the rays of a P-free quadrant stabbing P contains an edge of $\mathcal{R}\mathcal{H}_{\theta }(P)$. See Fig. 10a.

Let $r_1,\ldots ,r_4$ be the rays of $\mathcal {O}_\theta$ labeled in counter-clockwise circular order around the origin. For the sake of simplicity, in the following we assume an index $i \in \{ 1,\ldots ,4\}$ is such that $i + 4 := i$. Let $Q_i$ denote the quadrant bounded by $r_i$ and $r_{i+1}$. A $Q_i$-quadrant is a translation of $Q_i$. We say that a $Q_i$-quadrant and a $Q_{i+2}$-quadrant are opposite to each other. The following lemma states the conditions in which $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ is disconnected. See Fig. 10.

Lemma 8

(Alegría et al. [6], Lemma 1) Let i and j, $i \ne j$, be two indices in $\{1,\ldots ,4\}$. Let $q_i$ be a $Q_i$-quadrant and $q_j$ be a $Q_j$-quadrant. If both $q_i$ and $q_j$ are stabbing P and $q_i \, \cap \, q_j \, \cap \, \mathcal {C}{\mathcal {H}}({P})\ne \emptyset$, then the following statements hold true:

• (a) The quadrants $q_i$ and $q_j$ are opposite to each other, that is $\vert j - i \vert = 2$.

• (b) For all $k\in \{1,\ldots ,4\}$, $k\ne i$, $k\ne j$, every $Q_k$-quadrant $q_k$ and every $Q_{k+2}$-quadrant $q_{k+2}$ are such that $q_k \, \cap \, q_{k+2} \, \cap \, \mathcal {C}{\mathcal {H}}({P})= \emptyset$.

• (c) $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ is disconnected.

Fig. 10

A set P of six points for which $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ is formed by four connected components (two of which are points of P), and an orientation set $\mathcal {O}_\theta$, both for some value of $\theta$. Figure 10a illustrates Lemma 8, and Fig. 10b illustrates Lemma 9

It is known that $\mathcal {R}{\mathcal {H}}_{\theta }({P})$ is contained in $\mathcal {C}{\mathcal {H}}({P})$ regardless of the value of $\theta$ [40, Theorem 4.7]. Hence, a quadrant stabbing P is necessarily intersecting $\mathcal {C}{\mathcal {H}}({P})$. Let u and v be two vertices of $\mathcal {C}{\mathcal {H}}({P})$ such that u precedes v in the clockwise circular order of the vertices of $\mathcal {C}{\mathcal {H}}({P})$. Let r be the ray leaving u passing through v. The direction of the edge $\overline{u v}$ is the translation of r so that u lies on the origin. The following lemma is used in our construction to identify which of the four families of $Q_i$-quadrants can stab P. Refer again to Fig. 10.

Lemma 9

(Alegría et al. [6], Observation 2) If a $Q_i$-quadrant is stabbing P, then there is at least one edge of $\mathcal {C}{\mathcal {H}}({P})$ whose direction is contained in $Q_i$.

We are now ready to describe the construction. The convex hull of R is a rhombus whose diagonals are parallel to the coordinate axes. Let $v_1,\ldots ,v_4$ be the vertices of $\mathcal {C}{\mathcal {H}}({R})$ labeled in clockwise circular order starting at the left-most vertex. The vertices $v_2$ and $v_4$ lie outside the circle C that has the line segment $\overline{v_1 v_3}$ as diameter; thus, the interior angles of the rhombus at $v_2$ and $v_4$ are smaller than $\frac{\pi }{2}$, as well as the orientation $\alpha$ of the line through $v_3$ and $v_4$; see Fig. 11a.

Fig. 11

A bichromatic point set with $\Omega (n)$ angular intervals of separability: The set of red points

Let $d_i$ be the direction of the edge $\overline{v_i v_{i+1}}$. Let $l_\alpha = d_1 \cup d_3$ and $l_{\pi -\alpha } = d_2 \cup d_4$ be respectively, the lines through the origin with orientations $\alpha$ and $\pi -\alpha$ formed by the orientations of the edges of $\mathcal {C}{\mathcal {H}}({R})$; see Fig. 11b, left. Let $\ell _i$ denote the line of $\mathcal {O}_\theta$ that contains the rays $r_i$ and $r_{i+2}$. While incrementing $\theta$ from 0 to $\frac{\pi }{2}$, the lines of $\mathcal {O}_\theta$ counter-clockwise rotate around the origin while the lines $l_\alpha$ and $l_{\pi -\alpha }$ remain fixed. The rotation angles that are relevant for the lower bound are those in the interval $\varphi =[\frac{\pi }{2}-\alpha , \alpha ]$. At the angle $\theta =\frac{\pi }{2}-\alpha$ the line $\ell _2$ coincides with $l_{\pi -\alpha }$. At the angle $\theta =\alpha$ the line $\ell _1$ coincides with $l_\alpha$. For any other rotation angle in $\varphi$, the directions $d_1$ and $d_4$ lie in $Q_1$, whereas $d_2$ and $d_3$ lie in $Q_3$. Hence, by Lemma 9 the set R is stabbed only by $Q_2$- and $Q_4$-quadrants for all $\theta \in \varphi$. Note that R is stabbed on all the edges of the rhombus, and the vertices of the stabbing quadrants lie on semicircles in the interior of the rhombus whose diameters are the edges of the rhombus. Therefore, every point in the dashed regions lies in the intersection of a stabbing $Q_2$-quadrant and a stabbing $Q_4$-quadrant. Since these quadrants are opposite to each other, we have by Lemma 8 that $\mathcal {R}{\mathcal {H}}_{\theta }({\{v_1,\ldots ,v_4\}})$ is disconnected for all $\theta \in \varphi$. As shown in Fig. 11b, right, $\mathcal {R}{\mathcal {H}}_{\theta }({\{v_1,\ldots ,v_4\}})$ is actually formed by three connected components: the point $v_2$, the point $v_4$, and a rectangle inscribed in C whose sides are parallel to the lines of $\mathcal {O}_\theta$. By intersecting all such rectangles for all the rotation angles in $\varphi$, we obtain the rhombus highlighted in Fig. 11c. Note that any red point lying in this region is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ for all $\theta \in \varphi$. Hence, we may add as many red points as desired without affecting the construction.

The set of blue points is shown in Fig. 12a. Let d(p, q) denote the Euclidean distance between two given points p and q. The blue points lie in the interior of $\mathcal {C}{\mathcal {H}}({R})$, on a circle with center on the middle point of the segment $\overline{v_1 v_3}$, and radius ${d(v_1,v_3)}/{2} - \varepsilon$ for $0< \varepsilon < {d(v_1,v_3)}/{2}$. The points are spread so that, at every $\theta \in \varphi$, at most a single blue point is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$. As shown in Fig. 12b (see the figures from left to right), while incrementing $\theta$ from $\frac{\pi }{2}-\alpha$ to $\alpha$, two of the vertices of the rectangle inscribed in C remain anchored at the red points, while the other two traverse the red circular arcs in the counter-clockwise direction. Hence $\mathcal {R}{\mathcal {H}}_{\theta }({B})$ captures one blue point at a time, generating $\Omega (n)$ disjoint angular intervals of separability.

Fig. 12

A bichromatic point set with $\Omega (n)$ angular intervals of separability: The set of blue points

2.4 Inclusion detection

An elementary property of the standard convex hull is the following: $\mathcal {C}{\mathcal {H}}({B})$ is in the interior of $\mathcal {C}{\mathcal {H}}({R})$ if all the blue points are in the interior of $\mathcal {C}{\mathcal {H}}({R})$. This property translates to the rectilinear convex hull, regardless of the slopes of the lines of $\mathcal {O}_\theta$, and the connected components of both $\mathcal {R}{\mathcal {H}}_{\theta }({B})$ and $\mathcal {R}{\mathcal {H}}_{\theta }({R})$.

Lemma 10

If all the points of B are in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$, then $\mathcal {R}{\mathcal {H}}_{\theta }({B})$ is in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$.

Proof

Suppose a fixed value of $\theta$ and that all the points of B are in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$. Let x be a point in the plane in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({B})$. By Proposition 1, every $Q_i$-quadrant with vertex at x contains at least one blue point. Let $w_x$ be a $Q_i$-quadrant with vertex at x, and b denote one of the blue points contained in $w_x$. Let $w_b$ be the $Q_i$-quadrant resulting from translating $w_x$ so its vertex lies on b. Since we assumed all the blue points being in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$, then b is in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ and by Proposition 1, $w_b$ contains at least one red point r. Note that $w_x$ contains r since $w_b\subset w_x$. Thus, every $Q_i$-quadrant with vertex at a point in the interior of $\mathcal {R}{\mathcal {H}}_{\theta }({B})$ contains at least one red point. $\square$

By Lemma 10, if there is a value of $\theta$ for which all the Boolean flags of the set that encodes $B_\theta$ are True, then $\mathcal {R}{\mathcal {H}}_{\theta }({B})$ is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$. We obtain the following theorem as a consequence of Lemma 6.

Theorem 11

Given two disjoint sets R and B of points in the plane, the (possibly empty) set of angular intervals of $\theta \in [0,2\pi )$ for which $\mathcal {R}{\mathcal {H}}_{\theta }({B})$ is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ can be computed in $O(n \log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

It is not hard to see that, as in the separability problem, there are O(n) angular intervals of containment. We next adapt the bichromatic point set from Sect. 2.3 to obtain a matching lower bound. Consider a rhombus whose diagonals are parallel to the coordinate axes, and a circle C whose diameter is the diagonal of the rhombus that is parallel to the X axis. The convex hull of the set R is now formed by the five points shown in Fig. 13a. The points $v_1$, $v_2$, and $v_5$ lie on vertices of the rhombus, and the points $v_3$ and $v_4$ on intersection points between the rhombus and the circle C.

Fig. 13

A bichromatic point set with $\Omega (n)$ angular intervals of containment: The set of red points

The relevant rotation angles are again those in the interval $\varphi =[\frac{\pi }{2}-\alpha , \alpha ]$. Note that, since the direction of the edge $\overline{v_3 v_4}$ is parallel to the Y-axis, the observations we made about the construction described in Sect. 2.3 still hold: for any $\theta \in \varphi$ we have that $\mathcal {C}{\mathcal {H}}({R})$ is stabbed only by $Q_2$- and $Q_4$-quadrants, and $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is formed by three connected components. The relevant difference is the central component, which instead of a rectangle inscribed in C, is now an L-shaped region whose sides are parallel to the sides of $\mathcal {O}_\theta$; see Fig. 13b. While incrementing $\theta$ from $\frac{\pi }{2}-\alpha$ to $\alpha$, three of the vertices of this region are anchored at $v_1$, $v_3$, and $v_4$, while the remaining three vertices traverse the dashed semicircles in the counter-clockwise direction. By intersecting the L-shaped regions for all the rotation angles in $\varphi$, we obtain the region highlighted in Fig. 13c. Note that any red point lying in this region is contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ for all $\theta \in \varphi$. Hence, we may add as many red points as desired without affecting the construction.

The set of blue points is shown in Fig. 14a. The points lie in the interior of the triangle with vertices $v_1, v_3, v_4$, on a circle with center on the middle point of the segment $\overline{v_3 v_4}$, and radius ${d(v_3,v_4)}/{2} - \varepsilon$, for $0< \varepsilon < {d(v_3,v_4)}/{2}$. The points are spread so at every $\theta \in \varphi$, at most a single blue point is not contained in $\mathcal {R}{\mathcal {H}}_{\theta }({R})$. As shown in Fig. 14b (see the figures from left to right), while rotating the lines of $\mathcal {O}_\theta$ around the origin by incrementing $\theta$ from $\frac{\pi }{2}-\alpha$ to $\alpha$, the reflex vertex of the L-shaped region traverses the red circular arc in the clockwise direction. Hence $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ loses one blue point at a time, generating $\Omega (n)$ intervals of containment.

Fig. 14

A bichromatic point set with $\Omega (n)$ angular intervals of containment: The set of blue points

We summarize the lower bounds discussions of Sects. 2.4, 2.3 in the following proposition.

Proposition 12

There exist disjoint sets R and B of red and blue points in the plane that induce $\Omega (n)$ intervals of $\theta$ in which either i) $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ is B-free or ii) $\mathcal {R}{\mathcal {H}}_{\theta }({R})$ contains $\mathcal {R}{\mathcal {H}}_{\theta }({B})$, where $n = \vert R \vert + \vert B \vert$ and R may have O(1) points.

3 Generalizations

In this section we generalize the results from Sect. 2. First, in Sect. 3.1, we consider the case in which the set $\mathcal {O}$ contains not only two lines, but $k \ge 2$ lines with arbitrary orientations. In this setting the corresponding convex hull is known as the $\mathcal {O}$-convex hull [41]. Then, in Sect. 3.2, we consider the case in which the set $\mathcal {O}$ of two orthogonal lines is substituted by a set $\mathcal {O}_\beta$ of two lines that are not necessarily orthogonal to each other, but form an angle $\beta \in (0,\pi )$. In this setting the corresponding convex hull is known as the $\mathcal {O}_\beta$-convex hull [5]. We split the description of each generalization in three parts. In the first part, we adapt the needed results from Sect. 2.1 to characterize the conditions in which a blue point is contained in the hull of the set of red points. In the second part, we adapt the algorithm from Sect. 2.2 to compute and maintain the set of blue points contained in the hull of the set of red points while we change the orientation of the lines of $\mathcal {O}$. Finally, in the third part, we generalize the results from Sects. 2.3 and 2.4 to bound the number of angular intervals of separability and containment between the hulls of the red and the blue point sets.

3.1 The $\varvec{\mathcal {O}}$-convex hull

In this subsection we solve the following problem.

Problem 2

Given a set $\mathcal {O}$ of orientations formed by $k \ge 2$ lines, compute the set of rotation angles for which the lines of $\mathcal {O}$ have to be simultaneously rotated counterclockwise around the origin, so the $\mathcal {O}$-convex hull of R contains no points of B.

For the sake of simplicity, throughout this subsection we consider indices i to be modulo 2k. We also assume that the $k \ge 2$ lines of $\mathcal {O}$ are labeled with $\ell _1,\dots ,\ell _k$ so that $i<j$ implies that the orientation of $\ell _i$ is smaller than the orientation of $\ell _j$. Let $r_i$ and $r_{i+k}$ denote the rays into which $\ell _i$ is split by the origin. Given two indexes i and j, we denote with $W_{i,j}$ the wedge spanned as we counterclockwise rotate $r_i$ anchored at the origin until we obtain $r_j$. A $W_i^j$-wedge is a translation of $W_{i,j}$. We say that a $W_{i+1}^{i+k}$-wedge is an $\mathcal {O}$-wedge, see Fig. 15a. The $\mathcal {O}$-convex hull of a finite point set P, denoted with $\mathcal {O}\mathcal {H}({P})$, is the set

$$\begin{aligned} \mathcal {O}\mathcal {H}({P})=\mathbb {R}^{2} \setminus \bigcup _{i=1}^{2k}\mathcal {W}^i, \end{aligned}$$

where $\mathcal {W}^i$ denotes the union of all the P-free $W_{i+1}^{i+k}$-wedges, see Fig. 15b. Note that, as the rectilinear convex hull, the $\mathcal {O}$-convex hull of a finite point set is typically not convex, may be disconnected, and is orientation-dependent. More details on these and other properties can be found in [20].

Fig. 15

A set $\mathcal {O}$ of orientations with $k=3$ lines in (a), and the $\mathcal {O}$-convex hull of a finite point set P in (b)

Let $\mathcal {O}_\theta$ denote the set of lines obtained after simultaneously rotating the lines of $\mathcal {O}$ counterclockwise around the origin by an angle of $\theta$. We solve Problem 2 by describing an algorithm to compute the (possibly empty) set of angular intervals of $\theta \in [0, 2\pi )$ for which the $\mathcal {O}_\theta$-convex hull of R is B-free. See Fig. 16.

Fig. 16

The sets R and B, and the $\mathcal {O}_\theta$-convex hull of R for three different values of $\theta$. In each figure, the set $\mathcal {O}$ is shown at the bottom left corner along with the coordinate axes, which are shown with dashed lines

We start with the following generalization of Proposition 1, which derives directly from the definition of $\mathcal {O}$-convex hull.

Proposition 13

A point $x \in \mathbb {R}^{2}$ is contained in $\mathcal {O}\mathcal {H}({P})$ if, and only if, every $\mathcal {O}$-wedge with vertex on x contains at least one point of P.

As in Sect. 2, we consider $\mathcal {O}$ to be not only a set of k lines, but also the set of 2k rays in which the lines of $\mathcal {O}$ are split by the origin. We generalize the definition of feasible maximal arc as follows. Let $\alpha _i$ denote the size of the wedge $W_{i+1,i+k}$. We denote with $\Theta = {\text {min}}\{ \alpha _1, \ldots , \alpha _k\}$ the smallest angle among the sizes of the $\mathcal {O}$-wedges defined by the lines of $\mathcal {O}$. We say that a maximal arc is feasible, if it is induced by a maximal wedge with size at least $\Theta$. See Figs. 17, 18.

Fig. 17

At the top, a set $\mathcal {O}$ with $k=5$ lines. At the bottom and from left to right, the wedges $W_{i+1,i+k}$ for $i = 1,\ldots ,2k$. The first two figures show the wedges with the smallest size among all

Fig. 18

A feasible arc for the set $\mathcal {O}$ of Fig. 17

We generalize Lemma 2 as follows.

Lemma 14

For any fixed value of $\theta$, a point $x \in \mathbb {R}^{2}$ is contained in the $\mathcal {O}_\theta$-convex hull of P if, and only if, every feasible maximal arc of x is intersected by a set $r_i,r_{i+1},\ldots ,r_j$ of rays of $\mathcal {O}_\theta$ such that $j < i + k$.

Proof

By a straightforward adaptation of the proof of Lemma 2, we can show that every $\mathcal {O}_\theta$-wedge with vertex on x contains at least one point of P if, and only if, every feasible maximal arc of x is intersected by a set $r_i,r_{i+1},\ldots ,r_j$ of rays such that $j < i + k$. The key observation for this adaptation is that, if two rays $r_i$ and $r_j$ with $j = i + k$ intersect a maximal arc a that is feasible, then a is induced by a maximal wedge that contains an $\mathcal {O}_\theta$-wedge bounded by rays parallel to $r_i$ and $r_j$. The lemma follows from this fact and Proposition 13. $\square$

In the following lemma we rephrase Lemma 14 to a bichromatic setting.

Lemma 15

For every fixed value of $\theta$, a blue point $b\in B$ is contained in the $\mathcal {O}_\theta$-convex hull of R if, and only if, every blue maximal arc of b that is feasible is intersected by a set $r_i,r_{i+1},\ldots ,r_j$ of rays of $\mathcal {O}_\theta$ such that $j < i + k$.

We now adapt the algorithm from Sect. 2.2. The adaptation consists of four steps. The first step is an additional preprocessing step in which we compute the angle $\Theta$. The remaining steps are adaptations of those of the original algorithm.

Step 0. Computing the angle $\varvec{\Theta }$.

To compute the angle $\Theta$, we first sort the lines of $\mathcal {O}$ by orientation in increasing order in $O(k \log k)$ time and O(k) space. Then, we compute in O(k) time the set of angles $\{ \alpha _1,\ldots ,\alpha _k \}$. We finally obtain $\Theta$ by keeping the smallest angle in the set. Clearly, this step takes in $O(k \log k)$ time and O(k) space.

Step 1. Computing the set of feasible maximal arcs.

In this step we generalize the Step 1 of the original algorithm to compute the set of blue maximal arcs that are feasible.

We start by computing the set ${\mathcal {W}}$ of blue maximal wedges with size at least $\Theta$. We proceed as follows. A blue maximal wedge with size at least $\Theta$ is constrained to either the $X^+$ semiaxis, or to one of the ${2\pi }/{\Theta }$ directions defined by counterclockwise rotating $X^+$ by an integer multiple of $\Theta$. By means of the algorithm we described in Sect. 2.1, we compute the set of blue maximal wedges constrained to each one of these ${2\pi }/{\Theta } + 1$ directions. From the resulting set of wedges, we obtain ${\mathcal {W}}$ by keeping those wedges whose size is at least $\Theta$. By Lemma 4, we have computed ${\mathcal {W}}$ in $O({1}/{\Theta } \cdot n \log n)$ time and $O({1}/{\Theta } \cdot n)$ space. Moreover, note that ${\mathcal {W}}$ contains $O({1}/{\Theta } \cdot n)$ wedges.

We now traverse ${\mathcal {W}}$, and process each wedge $w\in \mathcal {W}$ by transforming w into a blue maximal arc that is feasible in O(1) time. Since each wedge is transformed into a single arc, there are $O({1}/{\Theta } \cdot n)$ blue maximal arcs that are feasible. Clearly, the time complexity of the whole step is $O({1}/{\Theta } \cdot n \log n)$ time and $O({1}/{\Theta } \cdot n)$ space.

Step 2. Computing the list of intersection events.

In this step we generalize the Step 2 of the original algorithm to compute the sorted list of intersection events. This step does not need to be modified; nevertheless, since we now have $O({1}/{\Theta } \cdot n)$ intersection events, the original complexity is replaced by $O({1}/{\Theta } \cdot n \log n)$ time and $O({1}/{\Theta } \cdot n)$ space.

Step 3. Performing the angular sweep.

Finally, in this step we generalize the Step 3 of the original algorithm to perform the angular sweep on the set of blue maximal arcs that are feasible.

The required adaptations are the following. The set $B_\theta$ now denotes the set of blue points contained in the $\mathcal {O}_\theta$-convex hull of R. The upper bound on $N_i$ is increased from four to ${2\pi }/{\Theta }$. The variable $n_i$ now denotes the number of blue maximal arcs of $b_i$ that are intersected either by one ray of $\mathcal {O}_\theta$, or by a set $r_u,\ldots ,r_v$ of rays of $\mathcal {O}_\theta$ such that $v < u + k$. Following the condition from Lemma 14, the array of $\vert B \vert$ Boolean flags used to encode the set $B_\theta$ has the i-th flag set to True if $n_i=N_i$, and to False if $n_i < N_i$.

The variable $\rho (a)$ now denotes the range of subindices of the rays of $\mathcal {O}_\theta$ intersecting the arc a. Observe that $\rho (a)$ cannot be empty. Suppose that $\rho (a) = (u,v)$, $u \le v$, and, at an intersection event, a ray starts intersecting a. Since the lines of $\mathcal {O}_\theta$ are labeled by increasing orientation and are rotated in the counter-clockwise direction, the range is thus increased to $(u-1,v)$. If instead the ray stops intersecting a, then the range is reduced to $(u,v-1)$. Finally, the queue $\mathcal {Q}$ now contains at most 2k angles instead of four. Let $\theta _i$ denote the smallest counter clockwise rotation angle for which the ray $r_i\in \mathcal {O}_\theta$ passes over an endpoint of a blue maximal arc. The queue contains the angles $\theta _i$, $1\le i \le 2k$, that are less than $\pi$. Hence update operations on $\mathcal {Q}$ take $O(\log k)$ time.

The sweep is essentially performed in the same way as explained in the algorithm from Sect. 2.2. There are slight modifications to the algorithm and an increment in the time and space complexities, consequence of having $O({1}/{\Theta } \cdot n)$ intersection events and O(k) angles in $\mathcal {Q}$. Since the lines of $\mathcal {O}$ are already sorted by slope (refer to Step 0), the O(n) time complexity of the initialization step is replaced by $O({1}/{\Theta } \cdot n)$ time. On the other hand, since $\mathcal {O}$ may not be symmetric, to perform the angular sweep we increment $\theta$ from 0 to $\pi$ so the rays of $\mathcal {O}_\theta$ sweep all the directions of $\mathbb {S}^1$, and the endpoints of each blue maximal arc are touched by all the lines of $\mathcal {O}_\theta$. Consider an intersection event $\theta$ for which a ray $r_j\in \mathcal {O}_\theta$ is passing over the endpoint of a maximal arc a of a blue point $b_i$. Assume that $\rho (a) = (u,v)$. We process the intersection event as follows:

• If $r_j$ starts intersecting a then $j = u-1$, so we set $\rho (a) = (u-1,v)$ to add j to $\rho (a)$. If instead $r_j$ stops intersecting a then $j = v$, so we set $\rho (a) = (u,v-1)$ to remove j from $\rho (a)$.

• If $\rho (a)$ was changed then we update $n_i$ as follows. If j was added to $\rho (a)$ and $v - j = k$ then we decrease $n_i$ by one. If instead j was removed from $\rho (a)$ and $v - j = k - 1$ then we increase $n_i$ by one.

• Finally, we update the set of Boolean flags that encode $B_\theta$, obtain the next intersection event, and update $\mathcal {Q}$ as explained in the algorithm of Sect. 2.2.

Lemma 16

The subset of blue points contained in the $\mathcal {O}_\theta$-convex hull of R can be computed and maintained, while $\theta$ is increased from 0 to $\pi$, in $O({1}/{\Theta } \cdot N \log N)$ time and $O({1}/{\Theta } \cdot N)$ space, where $N = \max \{ k, \vert R \vert + \vert B \vert \}$.

Proof

As in the algorithm from Sect. 2.2, the most expensive step is the execution of the angular sweep (Step 3). Since each intersection event is processed in O(1) time, the queue $\mathcal {Q}$ is updated in $O(\log k)$ time, and there are $O({1}/{\Theta } \cdot n)$ intersection events, then the time and space complexities of Step 3 are $O({1}/{\Theta } \cdot n \cdot \log k) = O({1}/{\Theta } \cdot N \log N)$ time and $O({1}/{\Theta } \cdot n) = O({1}/{\Theta } \cdot N)$ space, where $N = \max \{ k, n = \vert R \vert + \vert B \vert \}$. $\square$

From Lemma 16 we obtain the main result of this subsection.

Theorem 17

Given two disjoint sets R and B of points in the plane and a set $\mathcal {O}$ of $k \ge 2$ lines with different orientations, the (possibly empty) set of angular intervals of $\theta \in [0,2\pi )$ for which the $\mathcal {O}_\theta$-convex hull of R is B-free (i.e., the $\mathcal {O}_\theta$-convex hull of R is a separator of R and B) can be computed in $O({1}/{\Theta } \cdot N \log N)$ time and $O({1}/{\Theta } \cdot N)$ space, where $N = \max \{ k, \vert R \vert + \vert B \vert \}$.

There are a couple of remarks regarding the algorithm we described to prove Theorem 17. First note that the time and space complexities are parametrized by both $\Theta$ and k. If ${1}/{\Theta }$ is a constant value and k is of the same order of magnitude than $\vert R \vert$ and $\vert B \vert$, then the complexities become $O(n \log n)$ time and O(n) space. These are the same complexities reported in Theorem 7 for the problem of separability by a rectilinear convex hull. Second, as in Theorem 7, the reported angular intervals are maximal and open, and the algorithm does not compute a separating $\mathcal {O}$-convex hull. To actually compute an $\mathcal {O}$-convex hull separating R from B, we first choose a rotation angle in an interval of separability, and then spend additional $O({1}/{\Theta } \cdot N \log N)$ time [6]. Finally, we have an observation regarding the value of k, which derives from Observation 2 of [6]. To state the observation we first need to generalize the notion of stabbing quadrant we introduced in Sect. 2.3.

A connected component of the $\mathcal {O}$-hull of a finite point set P is either (i) a single point of P, (ii) a polygonal chain of two segments parallel to the lines of $\mathcal {O}$ that connect two extremal points, or (iii) a closed polygon whose edges are parallel to the lines of $\mathcal {O}$. The polygon may have “degenerate edges”, which are line segments connecting its vertices with extremal points of P. As with the rectilinear convex hull, the segments of the polygonal chains and the edges of the polygons are called the edges of the $\mathcal {O}$-convex hull. Each edge is contained in a ray of some P-free $\mathcal {O}$-wedge. We say that such $\mathcal {O}$-wedge is stabbing P.

Observation 1

Let h be the number of edges of $\mathcal {C}{\mathcal {H}}({P})$. If k is greater than h, then for any fixed value of $\theta$ there are $k - h$ lines in $\mathcal {O}_\theta$ that induce $\mathcal {O}_\theta$-wedges that do not stab P.

Observation 1 implies that, if k is greater than the number h of edges of $\mathcal {C}{\mathcal {H}}({R})$, then a separating $\mathcal {O}$-convex hull can be constructed using only $k-h$ of the lines of $\mathcal {O}$. In Fig. 10 for example, any line added to the set $\mathcal {O}$ lying on the blue region (hence having an orientation greater than the orientation of $\ell _1$ and smaller than the orientation of $\ell _2$), induce $\mathcal {O}_\theta$-wedges that do not stab the point set P.

3.1.1 Lower bound on the number of intervals of separability

We now adapt the construction from Sect. 2.3 to obtain a $\Omega (n)$ bound on the number of intervals of separability. For the sake of simplicity, we first describe the construction using a set $\mathcal {O}$ with $k=3$ lines with orientations 0, $\frac{\pi }{3}$, and $\frac{2}{3}\pi$. We later show how the construction can be extended to a set with more than three lines with arbitrary orientations.

Given two points p and q, let $\ell _{pq}$ be the line through p and q directed from p to q. Let C(p, q) be the circular arc spanned by p, q, and the angle $\frac{\pi }{3}$, that is contained in the right halfplane supported by $\ell _{pq}$. We denote with r(p, q) the radius of C(p, q). The convex hull of R is again a rhombus whose diagonals are parallel to the coordinate axes; see Fig. 19a. The points $v_2$ and $v_4$ lie outside the region bounded by $C(v_1,v_3) \cup C(v_3,v_1)$, so the interior angles of the rhombus at $v_2$ and $v_4$ are less than $\frac{\pi }{3}$. Let $\alpha$ be the orientation of the line through $v_3$ and $v_4$. For all $\theta$ in the interval $\varphi = [\frac{\pi }{3}-\alpha ,\alpha ]$, the direction of each edge of the rhombus lies in either $W_{2,3}$ or $W_{5,6}$; see Fig. 19b, left. From this fact and a straightforward generalization of the arguments of Sect. 2.3 we have that, for all $\theta \in \varphi$, the $\mathcal {O}_\theta$-convex hull of $\{ v_1,\ldots ,v_4 \}$ is formed by three connected components: the point $v_2$, the point $v_4$, and a rhombus inscribed in $C(v_1,v_3) \cup C(v_3,v_1)$ whose sides are parallel to $\ell _2$ and $\ell _3$; see Fig. 19b, right. By intersecting all such rhombi for all the rotation angles in $\varphi$, we obtain the rhombus highlighted in Fig. 19c. Note that any red point lying in this region is contained in the $\mathcal {O}_\theta$-convex of R for all $\theta \in \varphi$. Hence, we may add as many red points as desired without affecting the construction.

Fig. 19

Adapting the construction of Sect. 2.3: The set of red points

The set of blue points is shown in Figs. 20a, b. The points lie in the interior of $\mathcal {C}{\mathcal {H}}({R})$, on a circle concentric to $C(v_1,v_3)$ with radius $r(v_1,v_3) - \varepsilon$, for $0< \varepsilon < r(v_1,v_3)$. Note that the observations and lemmas from Sect. 2.3 can be applied to this construction with minor and straightforward modifications. Therefore, the bichromatic point set has $\Omega (n)$ angular intervals of separability.

Fig. 20

Adapting the construction of Sect. 2.3: The set of blue points is described in (a) and (b), and a generalization to a set $\mathcal {O}$ with more than three lines in (c)

Consider now a set $\mathcal {O}$ of $k>3$ lines with arbitrary orientations. To extend the previous construction to this case, first choose any pair of consecutive lines $\ell _i$ and $\ell _{i+1}$ in $\mathcal {O}$. Then create the point sets as described above, using a rhombus $\mathcal {R}$ such that the internal angles at two opposite vertices are less than the size of the wedge $W_{i,i+1}$. As shown in Fig. 20c, the directions of the edges of $\mathcal {R}$ lie either in $W_{i,i+1}$ or $W_{i+k,i+k+1}$. It is thus not hard to see that the arguments above still hold, so the construction has $\Omega (n)$ angular intervals of separability.

3.1.2 Inclusion detection

It is not hard to see that, with minor modifications, the statement and proof of Lemma 10 can be generalized to $\mathcal {O}$-convexity. Hence, in the algorithm we described to prove Theorem 17, if there is a value of $\theta$ for which all the Boolean flags of the set that encodes $B_\theta$ are True, then the $\mathcal {O}_\theta$-convex hull of B is contained in the $\mathcal {O}_\theta$-convex hull of R. We obtain the following theorem as a consequence of Lemma 16.

Theorem 18

Given two disjoint sets R and B of points in the plane, the (possibly empty) set of angular intervals of $\theta \in (0,2\pi ]$ for which the $\mathcal {O}_\theta$-convex hull of B is contained in the $\mathcal {O}_\theta$-convex hull of R can be computed in $O({1}/{\Theta } \cdot N \log N)$ time and $O({1}/{\Theta } \cdot N)$ space, where $N = \max \{ k, \vert R \vert + \vert B \vert \}$.

We now combine the constructions we described in Sects. 2.4 and 3.1.1 to obtain a point set with $\Omega (n)$ angular intervals of containment. We assume that $\mathcal {O}$ contains $k=3$ lines with orientations 0, $\frac{\pi }{3}$, and $\frac{2}{3}\pi$. The construction can be extended to a set with more than three lines with arbitrary orientations, in a similar way as we described in Sect. 3.1.1. The point set is illustrated in Fig. 21. The adaptation is based on the rhombus we used in the construction from Sect. 3.1.1; refer again to Fig. 19. Three red points lie on vertices of the rhombus, and two red points on the intersections between the rhombus and a vertical line. The line is chosen such that the region bounded by $C(v_3,v_4) \cup \overline{v_3 v_4}$ does not contain $v_1$. Note that any red point lying in this region highlighted in red is contained in the $\mathcal {O}_\theta$-convex of R. Hence, we may add as many red points as desired without affecting the construction. The blue points lie in the interior of the triangle with vertices $v_1$,$v_3$, and $v_4$, on a circle concentric to $C(v_3,v_4)$ with radius $r(v_3,v_4) - \varepsilon$, for $0< \varepsilon < r(v_3,v_4)$. As in Sect. 2.4 the points are spread so that, while rotating the lines of $\mathcal {O}$ around the origin, the $\mathcal {O}$-convex hull of R loses one blue point at a time. From similar arguments as those made on Sects. 2.4 and 3.1.1 , the bichromatic point set has $\Omega (n)$ angular intervals of containment.

Fig. 21

A bichromatic point set with $\Omega (n)$ angular intervals of containment. The red points lie on the boundary of a rhombus, and in the interior of the red highlighted region. The blue points lie on a circle concentric to $C(v_3,v_4)$ with radius $r(v_3,v_4) - \varepsilon$

We summarize the lower bounds discussions of Sect. 3.1.2, 3.1.1 in the following proposition.

Proposition 19

There exist disjoint sets R and B of red and blue points in the plane that induce $\Omega (n)$ intervals of $\theta$ in which either i) the $\mathcal {O}$-convex hull of R is B-free or ii) the $\mathcal {O}$-convex hull of R contains the $\mathcal {O}$-convex hull of B, where $n = \vert R \vert + \vert B \vert$ and R may have O(1) points.

3.2 The $\varvec{\mathcal {O}_\beta }$-convex hull

Let $\mathcal {O}_\beta$ denote a set of orientations formed by two lines with orientations 0 and $\beta$. An $\mathcal {O}_\beta$-quadrant is one of the four open wedges that result from subtracting the lines of $\mathcal {O}_\beta$ from the plane. The $\mathcal {O}_\beta$-convex hull of a finite point set P, denoted with $\mathcal {O}_\beta \mathcal {H}({P})$, is the set

$$\begin{aligned} \mathcal {O}_\beta \mathcal {H}({P})= \mathbb {R}^{2} \setminus \bigcup _{q\in \mathcal {Q}}q, \end{aligned}$$

where $\mathcal {Q}$ denotes the set of all P-free $\mathcal {O}_\beta$-quadrants of the plane [5]. In this subsection we solve the following problem.

Problem 3

Compute the set of values of $\beta \in (0,\pi )$ for which the $\mathcal {O}_\beta$-convex hull of R contains no points of B.

For the sake of simplicity, throughout this section we assume the set $R \cup B$ not only contains no three points on a line, but also no pair of points on a horizontal line. To solve Problem 3, we adapt the results from Sect. 2 to find the values of $\beta \in (0,\pi )$ for which $\mathcal {O}_\beta \mathcal {H}({R})$ is B-free; see Fig. 22. We start with the adaptation of Proposition 1, which derives directly from the definition of $\mathcal {O}_\beta$-convex hull.

Fig. 22

The sets R, B, and $\mathcal {O}_\beta \mathcal {H}({R})$ for three different values of $\beta \in (0,\pi )$. The set $\mathcal {O}_\beta$ and the coordinate axes are shown in the bottom-left corner of each figure

Proposition 20

A point $x\in \mathbb {R}^{2}$ is contained in $\mathcal {O}_\beta \mathcal {H}({P})$ if, and only if, every $\mathcal {O}_\beta$-quadrant with vertex on x contains at least one point of P.

As in Sect. 2, we consider $\mathcal {O}_\beta$ to be not only a set of two lines, but also the set of four rays in which the lines are split by the origin. We adapt the definition of feasible maximal arc as follows: we say that a maximal arc is feasible if it is induced by a P-free maximal wedge constrained to either the $X^+$ or the $X^-$ semiaxis. See Fig. 23.

Fig. 23

Maximal arcs of a point $x\in \mathbb {R}^{2}$

We now adapt Lemma 2 as follows.

Lemma 21

For any fixed value of $\beta$, a point $x \in \mathbb {R}^{2}$ is contained in $\mathcal {O}_\beta \mathcal {H}({P})$ if, and only if, every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\beta$.

Proof

We show that every $\mathcal {O}_\beta$-quadrant with vertex on x contains at least one point of P if, and only if, every feasible maximal arc of x is intersected by a single ray of $\mathcal {O}_\beta$. The lemma follows from this fact and Proposition 20.

For any fixed value of $\beta$ there is an affine transformation that maps horizontal lines to horizontal lines, and lines with orientation $\beta$ to vertical lines [41, Section 2.5]. Let $\mathcal {O}^\prime _\beta$ and $x^\prime$ denote the set and the point obtained after applying the transformation to $\mathcal {O}_\beta$ and x, respectively. Assume without loss of generality that $x^\prime$ lies on the origin. The proof follows by observing that (i) the lines of $\mathcal {O}^\prime _\beta$ coincide with the coordinate axes, (ii) every maximal wedge with vertex on $x^\prime$ that induces a feasible maximal arc contains either the $X^+$ or the $X^-$ semiaxes, and (iii) by similar arguments to those we used to prove Lemma 2, such wedge contains a second semiaxis if, and only if, $x^\prime$ is contained in $\mathcal {O}_\beta \mathcal {H}({P})$. $\square$

We rephrase Lemma 21 to a bichromatic setting as follows.

Lemma 22

A blue point $b\in B$ is contained in $\mathcal {O}_\beta \mathcal {H}({R})$ if, and only if, every blue maximal arc of b that is feasible is intersected by a single ray of $\mathcal {O}_\beta$.

Let $\mathcal {H}(R)$ denote the narrowest horizontal corridor enclosing R. Consider a blue point b lying outside $\mathcal {H}(R)$. Note that b has a single maximal arc that is feasible, since b is the vertex of a single R-free maximal wedge constrained to both the $X^+$ and the $X^-$ semiaxis. Such arc is intersected by both lines of $\mathcal {O}_\beta$ for all $\beta \in (0,\pi )$. Hence, by Lemma 22, the point b is not contained in $\mathcal {O}_\beta \mathcal {H}({R})$ for all $\beta \in (0,\pi )$. See Fig. 24.

Fig. 24

A blue point lying outside $\mathcal {H}(R)$ has a single feasible maximal arc, which is induced by a wedge constrained to both semiaxes of X. The point is not contained in $\mathcal {O}_\beta \mathcal {H}({R})$ for all $\beta \in (0,\pi )$

Consider now that b is contained in $\mathcal {H}(R)$. In this case b has two maximal arcs that are feasible, since b is the vertex of a single R-free maximal wedge constrained to the $X^+$ semiaxis, and a single R-free maximal wedge constrained to the $X^-$ semiaxis. Such arcs are intersected by both lines of $\mathcal {O}_\beta$ for values of $\beta$ in two angular intervals $(0,\beta _1)$ and $(\beta _2,\pi )$ for some angles $\beta _1,\beta _2\in (0,\pi )$ such that $0<\beta _1<\beta _2 <\pi$. Hence, by Lemma 22, the point b is not contained in $\mathcal {O}_\beta \mathcal {H}({R})$ for values of $\beta$ in the intervals $(0,\beta _1)$ and $(\beta _2,\pi )$, and it is contained in $\mathcal {O}_\beta \mathcal {H}({R})$ values of $\beta$ in the interval $(\beta _1,\beta _2)$. See Fig. 25.

Fig. 25

A blue point lying inside $\mathcal {H}(R)$ has two feasible maximal arcs induced by two maximal wedges: one constrained to the $X^+$ semiaxis, and the second one constrained to the $X^-$ semiaxis

Let $b_1,\ldots ,b_{\vert B \vert }$ be the points of B, and $S_i$ denote the set of angular intervals of $\beta$ for which $b_i$ is not contained in $\mathcal {O}_\beta \mathcal {H}({R})$. From the discussion above we have that $S_i$ contains either one or two angular intervals. For some small enough $\varepsilon$, the intervals contain either the angle $\varepsilon$, the angle $\pi - \varepsilon$, or both. Hence, the set $\underset{i=1}{\overset{\vert B \vert }{\bigcap }} \, S_i$ of angular intervals of $\beta$ for which $\mathcal {O}_\beta \mathcal {H}({R})$ is B-free consists of at most two intervals.

Theorem 23

Given two disjoint sets R and B of points in the plane, there are at most two open angular intervals of $\beta \in (0,\pi )$ where $\mathcal {O}_\beta \mathcal {H}({R})$ is B-free. These intervals can be computed in $O(n\log n)$ time and O(n) space, where $n=\vert R\vert +\vert B\vert$.

Proof

By means of the algorithm we described in the proof of Lemma 4, we compute the set of blue maximal wedges that are constrained to either the $X^+$ or the $X^-$ semiaxis in $O( n \log n)$ time and O(n) space. We then transform in O(n) time the resulting set of maximal wedges into a set of maximal arcs that are feasible, as described in the proof of Lemma 5. Next, we transform each of such arcs into an angular interval in O(1) time, as we described in Step 2 of the algorithm from Sect. 2.2. In O(n) time, we use these intervals to compute the set $S_i$ of angular intervals for which a blue point $b_i$ is not contained in $\mathcal {O}_\beta \mathcal {H}({R})$, for all $1 \le i \le \vert B \vert$. We finally compute the angular intervals where $\mathcal {O}_\beta \mathcal {H}({R})$ is B-free in O(n) time, by computing the set $\underset{i=1}{\overset{\vert B \vert }{\bigcap }} \, S_i$. $\square$

As discussed above, the values of $\beta$ where a blue point is contained in $\mathcal {O}_\beta \mathcal {H}({R})$ form at most a single angular interval. We obtain the following result from this fact and similar arguments to those we use to prove Theorem 23.

Theorem 24

Given two disjoint sets R and B of points in the plane, there is at most one open angular interval of $\beta \in (0,\pi )$ where $\mathcal {O}_\beta \mathcal {H}({B})$ is contained in $\mathcal {O}_\beta \mathcal {H}({R})$. This interval can be computed in $O(n \log n)$ time and O(n) space, where $n = \vert R \vert + \vert B \vert$.

We finish this section with the following two remarks regarding Theorem 23, 24. First, for a fixed value of $\beta$, the $\mathcal {O}_\beta$-convex hull of a finite point set can be computed in $O(n \log n)$ time and O(n) space [5]. Therefore, we need to spend an additional $O(n \log n)$ time and O(n) space to compute the actual monochromatic $\mathcal {O}_\beta$-convex hull in Theorem 23, or the $\mathcal {O}_\beta$-convex hull of R or B in Theorem 24. Second, since there is a constant number of angular intervals of $\beta$ where $\mathcal {O}_\beta \mathcal {H}({R})$ is B-free or $\mathcal {O}_\beta \mathcal {H}({B})$ is contained in $\mathcal {O}_\beta \mathcal {H}({R})$, one may think that these intervals can be computed in O(n) time. Nevertheless, we show in Sect. 4 that the best possible time bound is actually $\Omega (n \log n)$.

4 Lower bounds separation and inclusion detection problems

In this section we consider the lines of $\mathcal {O}$ to be fixed, and we prove an $\Omega (n\log n)$ time lower bound in the algebraic computation tree model for the following problems:

Problem

(Rectilinear Convex Hull Separability Detection, RH-SD) Given two disjoint sets of n red and n blue points in the plane, decide if no blue point is contained in the rectilinear convex hull of the red point set.

Problem

(Rectilinear Convex Hull Containment Detection, RH-CD) Given two disjoint sets of n red and n blue points in the plane, decide if all the blue points are contained in the rectilinear convex hull of the red point set.

Problem

(Rectilinear Convex Hull Point Inclusion, RH-PI) Given two disjoint sets of n red and n blue points in the plane, compute the subset of blue points contained in the rectilinear convex hull of the red point set.

Note that these problems are particular cases of those we studied in Sects. 2, 3.1, and 3.2. For the problems related to the rectilinear convex hull (Sect. 2) and the $\mathcal {O}$-convex hull (Sect. 3.1), the set $\mathcal {O}_\theta$ is fixed to the case where $\theta$ is a constant value and contains $k=2$ orthogonal lines. For the problem related to the $\mathcal {O}_\beta$-convex hull (Sect. 3.2), the set $\mathcal {O}_\beta$ is fixed to the case where $\beta =\frac{\pi }{2}$. The $\Omega (n\log n)$ time lower bounds thus imply that the time complexities reported in Lemma 6, Theorem 7, 11, 23, 24 are the best possible.

We first prove the lower bounds for the RH-SD and the RH-CD problems. The proofs are by reduction from the following auxiliary problems.

Problem

($\varepsilon$-Closeness) Given a set $x_1,\ldots ,x_n$ and $\varepsilon > 0$ of $n+1$ real numbers, decide whether any two numbers $x_i$ and $x_j$ ($i \ne j$) are at distance less than $\varepsilon$ from each other.

Given a set $x_1,\ldots ,x_n$ of real numbers, we say that two numbers $x_i$ and $x_j$ are consecutive if $x_i < x_j$, and there is no k such that $x_i< x_k < x_j$.

Problem

(Complement-Greater-or-Equal, CGE) Given a set $x_1,\ldots ,x_n$ and $\varepsilon > 0$ of $n+1$ real numbers, decide whether the maximum distance between consecutive numbers is less than $\varepsilon$.

The problems $\varepsilon$-Closeness and CGE have an $\Omega (n\log n)$ time lower bound in the algebraic computation tree model [8, 40]. The reductions from these problems are based on a construction we describe next. We transform a set $x_1,\ldots ,x_n$ and $\varepsilon > 0$ of $n+1$ real numbers, into two disjoint sets R and B of $2n+2$ red and $n-1$ blue points, such that some blue point is contained in the rectilinear convex hull of the red point set if, and only if, the distance between a pair of consecutive numbers is less than $\varepsilon$.

Let $x_{min} = {\text {min}} \{x_1,\ldots ,x_n\}$, $x_{max} = {\text {max}} \{x_1,\ldots ,x_n\}$, and m, M be two real numbers such that $m \ll x_{min}$ and $M \gg x_{max}$. The set B is produced by transforming the set $\{ x_1,\ldots ,x_n \} \setminus \{ x_{max} \}$ of real numbers into the set

$$\begin{aligned} \{ b_i = (x_i, x_i) \; \vert \; 1 \le i \le n \} \setminus \{ b_{max} = (x_{max}, x_{max}) \} \end{aligned}$$

of $n-1$ blue points on the line $\ell$ with equation $y = x$. The set R is produced by transforming the set $x_1,\ldots ,x_n$ of real numbers into the set

$$\begin{aligned} \{ r^{\tiny {+}}_i = (x_i - \varepsilon , x_i) \; \vert \; 1 \le i \le n \} \end{aligned}$$

of n red points on the line $\ell ^{\tiny {+}}$ with equation $y = x + \varepsilon$, the set

$$\begin{aligned} \{ r^{\tiny {-}}_i = (x_i, x_i - \varepsilon ) \; \vert \; 1 \le i \le n \} \end{aligned}$$

of n red points on the line $\ell ^{\tiny {-}}$ with equation $y = x - \varepsilon$, and the points $r_m = (m, m), r_M = (M, M)$ on $\ell$. See Fig. 26.

Fig. 26

Transforming the set $x_1,\ldots ,x_n$ and $\varepsilon >0$ of real numbers into two disjoint sets of red and blue points. The blue points lie on the line $\ell$ ($y=x$). The red points lie on the lines $\ell ^{\tiny {+}}$ ($y=x+\varepsilon$) and $\ell ^{\tiny {-}}$ ($y=x-\varepsilon$)

Let $x_i$ and $x_j$ be two numbers in the set $x_1,\ldots ,x_n$ such that $x_i < x_j$. Consider the four different quadrants whose vertices lie on the blue point $b_i$. Remember that a quadrant is an open region. Note that the $Q_1$-quadrant contains the red point $r_M$ and the $Q_3$-quadrant contains the red point $r_m$. If the distance between $x_i$ and $x_j$ is less than $\varepsilon$, then the $Q_2$-quadrant contains the red point $r^{\tiny {+}}_j$ and the $Q_4$-quadrant contains the red point $r^{\tiny {-}}_j$. By Proposition 1, in such case the blue point $b_i$ is strictly contained in $\mathcal {R}{\mathcal {H}}({R})$. If instead the distance between $x_i$ and $x_j$ is at least $\varepsilon$, then both the $Q_2$-quadrant and the $Q_4$-quadrant are R-free, hence $b_i$ is not strictly contained in $\mathcal {R}{\mathcal {H}}({R})$. See Fig. 27.

Fig. 27

All the points of R are vertices of $\mathcal {R}{\mathcal {H}}({R})$. The points $r_m$ and $r_M$ are two singleton connected components, whereas any other connected component of $\mathcal {R}{\mathcal {H}}({R})$ is an orthogonal polygon whose sides are parallel to the coordinate axes. Since the distance between $x_{i}$ and $x_j$ is less than $\varepsilon$, then $b_i$ is strictly contained in $\mathcal {R}{\mathcal {H}}({R})$

Lemma 25

The construction described above transforms the set $x_1,\ldots ,x_n$ and $\varepsilon >0$ of $n+1$ real numbers into two disjoint sets R and B of $2n+2$ red and $n-1$ blue points in O(n) time and space. A pair of numbers $x_i$ and $x_j$, $x_i < x_j$, are at distance less than $\varepsilon$ if, and only if, the blue point $b_i$ is strictly contained in $\mathcal {R}{\mathcal {H}}({R})$.

We now prove the lower bound for the RH-SD and the RH-CD problems.

Theorem 26

The RH-SD problem requires $\Omega (n\log n)$ time under the algebraic computation tree model.

Proof

By reduction from the $\varepsilon$-Closeness problem. Consider an instance of the $\varepsilon$-Closeness problem given by a set $x_1,\ldots ,x_n$ and $\varepsilon >0$ of $n+1$ real numbers. Using the construction we described above, we create the disjoint sets R and B of $2n+2$ red and $n-1$ blue points. We add additional $n+3$ points to B by placing blue points on the line $\ell$ for values of x in the interval $(m,x_{min}-\varepsilon )\cup (x_{max},M)$. By Proposition 1, these additional points are not contained in the rectilinear convex hull of R, regardless of the distances between consecutive numbers in the set $x_1,\ldots ,x_n$. See Fig. 28a.

We use an algorithm to solve the RH-SD problem on the sets R and B of $2n +2$ red and $2n + 2$ blue points. If the algorithm returns true, we reject the instance of the $\varepsilon$-Closeness problem, otherwise we accept the instance. By Lemma 25, there is at least one blue point contained in the rectilinear convex hull of R if, and only if, there is a pair of consecutive numbers in $x_1,\ldots ,x_n$ at distance less than $\varepsilon$. Therefore, we have correctly solved the $\varepsilon$-Closeness problem in O(n) time plus the time required to solve the RH-SD problem. $\square$

Fig. 28

Illustrations for (a) Theorem 26 and 28(b) Theorem 27

Theorem 27

The RH-CD problem requires $\Omega (n\log n)$ time under the algebraic computation tree model.

Proof

By reduction from the CGE problem. Consider an instance of the CGE problem given by a set $x_1,\ldots ,x_n$ and $\varepsilon >0$ of $n+1$ real numbers. Using the construction we described above, we create the disjoint sets R and B of $2n+2$ red and $n-1$ blue points. We add additional $n+3$ points to B by placing on the line $\ell$ three blue points for values of x in the interval $(x_{min} -\varepsilon ,x_{min})$, and a point $b_i^{\textsc {c}}=(x_i -\frac{\varepsilon }{2}, x_i - \frac{\varepsilon }{2})$ for $1\le i\le n$. By Proposition 1, these additional points are contained in the rectilinear convex hull of R, regardless of the distances between consecutive numbers in the set $x_1,\ldots ,x_n$. See Fig. 28b.

We use an algorithm to solve the RH-CD problem on the sets R and B of $2n+2$ red and $2n+2$ blue points. If the algorithm returns true we accept the instance of the CGE problem, otherwise we reject the instance. By Lemma 25, there is at least one blue point not contained in the rectilinear convex hull of R if, and only if, the maximum distance between consecutive numbers in $x_1,\ldots ,x_n$ is at least $\varepsilon$. Therefore, we have correctly solved the CGE problem in O(n) time plus the time required to solve the RH-CD problem. $\square$

A solution of the RH-PI problem can be trivially transformed into a solution of the problems RH-SD and RH-CD in O(1) and O(n) time, respectively: An instance of the RH-SD problem is positive if the subset of blue points contained in the rectilinear convex hull of the red point set is empty, whereas an instance of the RH-CD is positive if the subset contains n points. Hence, as a consequence of Theorem 26, 27 we obtain the following theorem.

Theorem 28

The RH-PI problem requires $\Omega (n\log n)$ time in the algebraic computation tree model.

5 Concluding remarks

We described efficient algorithms to compute the orientations of the lines of $\mathcal {O}$ for which there is an $\mathcal {O}$-convex hull separating R from B. If $\mathcal {O}$ is formed by two lines we considered two cases. In the first case we simultaneously rotate both lines around the origin. In the second case we rotate one of the lines while the second one remains fixed. In both cases our algorithms run in optimal $O(n \log n)$ time and O(n) space. The optimality is shown by providing a matching lower bound for the problem. If instead $\mathcal {O}$ is formed by $k\ge 2$ lines, we simultaneously rotate all the lines of $\mathcal {O}$ around the origin. Our algorithm runs in this case in $O({1}/{\Theta } \cdot N \log N)$ time and $O({1}/{\Theta } \cdot N)$ space, where $N = \max \{ k, \vert R \vert + \vert B \vert \}$ and $\Theta$ is the smallest among the sizes of the $\mathcal {O}$-wedges induced by the set of orientations.

The central strategy of all our algorithms is to perform an angular sweep in which, while we change the orientations of the lines of $\mathcal {O}$, we keep the number of blue points contained in the $\mathcal {O}$-convex hull of R. Note that, without increasing the time and space complexities, the angular sweep can be easily adapted to compute the set of angular intervals for which the $\mathcal {O}$-convex hull of R contains the minimum number of blue points. Using the terminology from Houle [23, 24], if such number is equal to zero, then the particular $\mathcal {O}$-convex hull is a strong separator for R and B, otherwise is a weak separator for R and B. Hence, our algorithm can be used to solve a variation of the so-called weak separability problem in which a given bichromatic point set is separated by an $\mathcal {O}$-convex hull.

Finally, we remark that the sweeping process can be also modified to add further optimizations. By applying the techniques from [5, 6] for example, we can obtain the $\mathcal {O}$-convex hull with the minimum (or maximum) area, perimeter, or number of vertices, that is either a strong or a weak separator for R and B. These additional optimizations do not increase neither the time nor the space complexities of the original algorithms.