Simultaneous Approximation of Multi-criteria Submodular Function Maximization

Abstract

Recently intensive interest has been raised on approximation of the NP-hard submodular maximization problem due to their theoretical and practical significance. In this work, we extend this line of research by focusing on the simultaneous approximation of multiple submodular function maximization. We address the existence and nonexistence results for both deterministic and randomized approximation when the submodular functions are symmetric and asymmetric, respectively, along with algorithmic corollaries. We offer complete characterization of the symmetric case and partial results on the asymmetric case.

1 Introduction

Given a ground set X, let $2^{X}$ be its power set. A set function $f:\,2^{X}\rightarrow \mathbb {R}$ is nonnegative if $f(A)\geqslant 0,$ $\forall A\subseteq X.$ It is symmetric if $f(A)=f(X-A),\,\forall A\subseteq X,$ or simply $f(A)=f(\overline{A})$ whenever the ground set is obvious. It is submodular if

$$\begin{aligned} f(A)+f(B)\geqslant f(A\cup B)+f(A\cap B),\quad \forall A,\quad B\subseteq X. \end{aligned}$$

Let $f_{j}:\,2^{X}\rightarrow \mathbb {R}^{+}$ ($j=1,\cdots ,k$) be k nonnegative submodular functions. The main focus of this work is on maximizing multiple nonnegative submodular functions, namely, solving the following k-criteria submodular function maximization problem:

$$\begin{aligned} (P):\,\max \limits _{S\subseteq X}\left\{ f_{1}(S),\cdots ,f_{k}(S)\right\} . \end{aligned}$$

Sometimes we also need to refer to the jth ($j\in \{1,\cdots ,k\}$) mono-criterion problem:

$$\begin{aligned} \left( P_{j}\right) :\,\max \limits _{S\subseteq X} f_{j}(S). \end{aligned}$$

The main issue of solving any multi-criteria optimization is that a solution simultaneously maximizing all objectives may not exist in general. Therefore, many solution concepts, such as Pareto-optimality (e.g., [23]), goal programming (e.g., [18]) and budgeted-constraint approach (e.g., [22, 25]), etc., have been proposed in the literature to address this issue. The standard solution concept is Pareto optimality: a solution such that no criterion could be made better off without making some other criteria worse off. However, every solution concept has its pros and cons and hence may be appropriate only for some of the real-life applications, resulting in many different solution concepts still being proposed in the multi-criteria optimization area. For a detailed account of various solution concepts and their applications, please refer to [9].

In this work, we adopt a fairly recent solution concept of simultaneous approximation, first proposed by Stein and Wein [27], initially defined for scheduling problems, and later extended to some other problems such as Max-Cut [2]. For our purpose, we cast this solution concept under the framework of a game setting. While the game can be defined analogously for any multi-criteria optimization problem, we will focus on the k-criteria submodular function maximization problem from now on.

1.1 The $k$-criteria Submodular Maximum Game

Given $k$ nonnegative submodular functions $f_j: 2^X\rightarrow \mathbb {R}^{+}$ ($j=1,\cdots, k$) over a common ground set $X$, there are $k$ players. Each player tries to maximizes its payoff $f_j(S)$ over its strategy space $S\subseteq X$. Moreover, all players are only allowed to adopt the same strategy (pure or randomized), namely we focus on the symmetric game setting (the asymmetric setting where players can use different strategies is trivial because of the inter-independence of the payoffs of the players).

The usual approach for dealing with this game is via various concepts of equilibrium (such as the most famous one: Nash-equilibrium), whose main consideration is on solution stability. However, this work will focus on a different aspect of this game, namely, fairness. As explained shortly, the simultaneous approximation approach can offer some insights on this aspect. Due to the correspondence between the original $k$-criteria submodular function maximization problem (P) and the game defined above, we call any pure strategy of the game as a deterministic solution to (P), and any mixed strategy of the game as a randomized solution of the (P). Now, we formally define the most important concepts in the simultaneous approximation approach. Let $S^*_j$ ($j=1,\cdots , k$) be the optimal solution for the $j$th mono-criterion problem $(P_j)$.

Definition 1.1

For any $\alpha \in [0,1]$,

• (i) A subset $S\subseteq X$ is an $\alpha$-deterministic solution for the problem (P) if

  $$\begin{aligned} f_j(S)\geqslant \alpha f_j(S_j^*),\ \forall j=1,\cdots , k. \end{aligned}$$

• (ii) A mixed strategy $T$ is an $\alpha$-randomized solution for the problem (P) if

  $$\begin{aligned}{\mathbf{E}}[f_j(T)]\geqslant \alpha f_j(S_j^*),\ \forall j=1,\cdots , k. \end{aligned}$$

The ratio $\alpha$ stipulates that how close each payoff function is to its ideal optimum, and hence indeed can be used as a measure on how fair the eventual solution is to all the players. Actually there is an independent body of research who used this ratio to characterize fairness in resource allocation problems and this ratio is called the $\alpha$ -fairness ratio (e.g., [13]) because “there is a concrete connection between simultaneous optimization and fairness”. This will be discussed in detail in Sect. 1.4 when we review existing literature.

We note that the focus on symmetric strategies is not unnatural because this concept is well accepted in game theory and has been extensively investigated in some game applications, such as the search and rendezvous games (e.g., [1]). Furthermore, simultaneous approximation has direct applications in quantifying the fairness in resource allocation problems (e.g., bandwidth allocation in computer network) as mentioned earlier. Another application potential in terms of pure strategies is in the design of approximation algorithms when two or more objectives needed to be balanced to achieve a good approximation and these ideas have been implicitly or explicitly employed in almost all approximation algorithm design methods. However, the simultaneous mixed strategies have no direct connection with the standard randomized algorithm in the mon-criteria optimization problems.

For this type of solution concept, there are two fundamental issues to be addressed:

• (I) characterize $\alpha$ such that an $\alpha$-deterministic (or randomized) solution exist; and

• (II) find a good simultaneous approximation solution in polynomial-time for the deterministic case and randomized polynomial-time for the randomized case?

Note that the first issue (I), trivial for the mono-criterion problem, turns out to be highly nontrivial for the multi-criteria problem studied in this work, which is precisely the interest of this work.

1.2 Our Contributions

For the aforementioned issue (I), when the submodular functions are symmetric, we provide complete characterizations of the existence and non-existence for both deterministic and randomized solutions, respectively. When the submodular functions are not necessarily symmetric, we provide an example to show that there may exist no simultaneously bounded deterministic solution for more than one criterion, and we present an existence bound for the randomized solutions. The detailed existence and non-existence results are summarized below (See Table 1 for small $k$).

• 1. For symmetric submodular functions,

  • there exists a $\frac{1}{2}$-deterministic solution for $k=2$, and this is the best possible; however, no simultaneously bounded deterministic solution exists for $k\geqslant 3$;

  • there exists a $\frac{2^{k-1}}{2^k-1}$-randomized solution for any $k\geqslant 2$, and this bound is the best possible, namely, there exist symmetric submodular functions $(f_1,\cdots , f_k)$ for any $k\geqslant 2$ such that no $\alpha$-randomized solution exists for any $\alpha >\frac{2^{k-1}}{2^k-1}$.

• 2. For asymmetric submodular functions,

  • no simultaneously bounded deterministic solution exists for $k\geqslant 2$;

  • there exists a $\max \left\{ \frac{1}{k},\frac{2^{k-2}}{2^k-1}\right\}$-randomized solution for any $k\geqslant 2$, and this bound is the best possible for $k=2$ and $k=3$.

Table 1 Existence bounds for small $k$, where the asterisk ones are tight bounds for the asymmetric case

On the algorithmic consequence, these existence results also lead to poly-nomial-time deterministic or randomized solutions, addressing the aforementioned issue (II). Note that the simple (polynomial-time) randomized algorithm which equi-probably select or reject any given element into the solution can be easily proved to yield a $\frac{1}{2}$-randomized solution for the symmetric submodular function and a $\frac{1}{4}$-randomized solution for the asymmetric case, and hence matching the existence results shown above asymptotically when $k\rightarrow \infty$. Later on, we will discuss how these existence results can lead to improved polynomial time approximation algorithms for the multi-criteria problems compared to the aforementioned simple random algorithm for small criteria number $k$, when equipped with existing approximation algorithms for the mono-criterion problem.

1.3 Background and Applications

Submodular optimization is one of the most important subclasses of problems in combinatorial optimization with great practical and theoretical significance, including many well-known combinatorial problems as special cases, such as the cut problem in direct/undirected/hypergraphs, certain constraint satisfaction problems, entropy sampling, and the facility location problem, among others. Moreover, submodular optimization problems are ubiquitous in almost all research disciplines, both science and social science, in particular in economics and management [28]. Both submodular minimization and maximization in the mono-criteria case have been extensively investigated in the literature. While submodular minimization problem admits strongly polynomial-time algorithm [26], the problem of maximizing a single submodular function is strongly NP-hard in general as it includes the famous Max-Cut problem as a special case. Since the existence of optimal solution for mono-criterion submodular maximization problem is out of question, the main interest lies in the design of good approximation algorithm and proof of inapproximability for the problem. This line of research has attracted a lot of attentions recently (e.g., see [4, 12] and references therein).

Our work here extends this line of research to the multi-criteria domain, which calls for extra attention on the existence, a non-trivial issue here, contrary to the mono-criteria case. Moreover, we believe, the extension to the multi-criteria problem can better meet the practitioners’ needs in almost any application of the mono-criteria version, and hence of greater practical significance in such problems, originally only dealt with from a single objective point of view. With ever increasing global competition, corporations have been rapidly adapting their business models to be competitive not only on one aspect, but on several grounds. This new requirement provides a fruitful application filed for multi-objective optimization techniques to be deployed in facilitating business decision-making.

The results developed in this work, both the existence and algorithmic consequences, can be particularly insightful in those decisions-making situations, where different decision-makers have its own objective estimation of the objective functions, trying to collaboratively reach a solution that is equally close to everyone; or a single-decision maker has several similar but potentially conflicting objective functions, trying to find a solution that is equally close to all its estimation. Historically, the $\alpha$-fairness ratio has been used in addressing fair resource allocation [13, 15], a fundamental problem in economics. Along this same line, this fairness ratio has been applied to many other settings in computer network, scheduling, routing and load balancing etc. (See Sect. 1.4 for more details). To further appreciate the application potentials of the results developed in this work, below we use another concrete example from cluster analysis to serve as one of the motivations in pursuing this research.

Cluster analysis (or data classification), the partition of observed data into clusters of “closely related” observations, has extensive applications in business management, economics, health care system, information retrieval, bioinformatics, machine learning, data mining, pattern recognition, image analysis, etc. While numerous clustering methods exit in the literature, we focus one method based on the Max-Cut problem. In the simplest bi-partition case, the desired clusters can be obtained by solving an Max-Cut problem on a complete graph such that each observation is a node, along with the edge weight being the “closeness” of the observations, measured usually by some distance measures like the Euclidean distance, the Manhattan distance (or taxicab norm or 1-norm), the maximum norm (or infinity norm), or the Hamming distance, etc. In the mono-criteria case, we have to deal with each distance measure separately, and the resultant clustering from one distance measure may be dramatically different from the one based on another distance measure. Suppose we now want to find a clustering that is equally close to all distance measures, then the results developed in this work (particularly Theorems 3.1–3.3) are perfect fits as the cut function is a symmetric submodular function.

1.4 Prior and Related Results

There are several extant results that are closely related to our work. The most relevant previous work is the recent result on bi-criteria Max-Cut problem by Angel [2], which is a special symmetric submodualr function. They prove the following results for this bi-criteria problem: (1) there exists a $\frac{1}{2}$-deterministic solution and this is the best possible; however, no simultaneously bounded deterministic solution exists for $k\geqslant 3$; (2) there exists a $\frac{\sqrt{5}-1}{2}\approx 0.618$-randomized solution, and no $\alpha$-randomized exists for any $\alpha >\frac{2}{3}\approx 0.667$.

Our work here therefore greatly extends the above results in two directions: (1) we consider any number of criteria $k\geqslant 2$; and (2) we consider any submodular function (asymmetric or symmetric). Furthermore, as direct consequences of some of the results obtained in this work, we resolved several open questions left therein, when our results are specialized to the Max-Cut problem.

The second most related result is by Goel and Meyerson [13], who studied simultaneous approximation for maximizing symmetric concave functions or minimizing convex function. Their main result is that for any resource allocation problem where the set of feasible solutions is a convex set, there exists an $\alpha$-approximation solution with $\alpha$ being logarithmic in some natural problem parameters. For example, for the special case of the multicommodity flow problem, we prove the existence of a solution with $\alpha =O(\log n)$ where $n$ is the number of nodes in the network. Along the same line, more simultaneous approximation results for specific problems include the satisfiability problem, spanning tree, $q$-hop spanning tree problem, cut complement problem [10], routing and load balancing [14, 19], online fair resource allocation [5, 6], scheduling problems [3], set-cover problem, $k$-facility-location, TSP, and scheduling [17], network bandwidth allocation problem [8, 20]. However, our model is different from theirs as this body of work as they are based on linear or integer programming, while we focus on set function optimization.

The third related result is given recently by Chekuri et al [7] on monotonic submodular functions: there exists a $\left( 1-e^{-1}\right)$-deterministic solution for any number of monotonic submodular functions and it can be found in polynomial time.

After some preliminary results in Sect. 2, we first consider the existence issue for the symmetric and asymmetric cases in Sects. 3 and 4, respectively. We then present the algorithmic corollaries in Sect. 5. Finally, we provide some concluding remarks in Sect. 6.

2 Preliminaries

We present some useful facts which will be essential in the analysis of the existence and non-existence results later on.

2.1 Facts Useful for Existence Analysis

We establish the following properties for any nonnegative submodular function.

Lemma 2.1

Assume that $f: 2^X\rightarrow \mathbb {R}^{+}$ is any nonnegative submodular function. Given two sets $A, B\subseteq X$, let $A\Delta B=(A\backslash B)\cup (B\backslash A)$ be the symmetric difference. Then

$$\begin{aligned} f(A\Delta B)+f(B)+f(\overline{A\Delta B})+f(\overline{B})\geqslant f(A)+f(\overline{A}). \end{aligned}$$

Proof

By the submodularity of $f$:

$$\begin{aligned} f(A\Delta B)+f(B)&\geqslant f((A\Delta B)\cap B)+f((A\Delta B)\cup B)=f(B-A)+f(A\cup B),\\ f(\overline{A\Delta B})+f(\overline{B})&\geqslant f((\overline{A\Delta B})\cap \overline{B})+f((\overline{A\Delta B})\cup \overline{B})=f(\overline{B-A})+f(\overline{A\cup B}), \end{aligned}$$

implying that

$$ \begin{aligned} f(A\Delta B)+f(B)+f(\overline{A\Delta B})+f(\overline{B})&\geqslant (f(B-A)+f(\overline{A\cup B})) \\&\quad+(f(\overline{B-A}+f(A\cup B))\\&\geqslant (f(\emptyset )+f(\overline{A}))+(f(A)+f(X)) \\&\geqslant f(A)+f(\overline{A}). \end{aligned} $$

where, the second inequality follows from the submodularity, and the last inequality follows from the nonnegativity. $\square$

Assuming symmetry in the above leads to the result below for symmetric submodular function.

Lemma 2.2

Assume that $f: 2^X\rightarrow \mathbb {R}^{+}$ is any nonnegative symmetric submodular function. Given two sets $A, B\subset X$, let $A\Delta B$ be the symmetric difference. Then

$$\begin{aligned} f(A\Delta B)+f(B)\geqslant f(A). \end{aligned}$$

2.2 Facts Useful for Non-existence Analysis

We introduce some notational conventions for easy presentation. Consider a complete graph $K_{2^k}$ with vertex set $V_k=\{0,1,\cdots , 2^k-1\}$. Vertices in $V$ are embedded into the $k$-dimensional space $\mathbb {R}^k$ such that each $i\in V_k$ is associated with the (row) vector $\mathop {\bf {i}}=(i_1,\cdots , i_k)\in \mathbb {R}^k$, where $i_{\ell }\in \{0,1\}$ for $\ell =1,\cdots , k$. Geometrically, the vertices in $V$ can be viewed as the corners of the $k$-dimensional hypercube (see Fig. 1 for an illustration when $k=3$). For any vector, we define its 1-norm $||\bullet ||_1$ as the sum of its coordinates in the vector. From now on, we will use boldface letters to represent vectors. For example, given $V_k$, we denote $\mathbf{V}_{\mathbf {k}} =\{{\mathbf{i}}: i \in V_{k}\}$ as the corresponding set of vertices represented with the vectors.

Fig. 1

Geometric representation of $V_{3}=\{0,1,2,3,4,5,6,7\}$ and the corresponding set of vector-represented vertices $\mathbf{V}_3=\{(0,0,0), (0,0,1), (0,1,0), (0,1,1), (1,0,0), (1,0,1), (1,1,0),(1,1,1)\}$

Remark 2.3

An alternative (algebraic) way of looking at the vector $\mathbf{i}={(i_1,\cdots , i_k)}$ is that the coordinates can be obtained as follows: First, each integer $i\in V_k$ can be converted to its binary number representation (keep $k$-digits by adding leading zeros if necessary), which then can be viewed as a vector with components being the binary digits of integer $i$. For example, when $k=3$, $i=2\in V_3$ can be converted to a binary number $010$, which can be viewed as a (row) vector ${\mathbf {2}} =(i_1,i_2,i_3)=(0,1,0)\in {\mathbb {R}}^3$.

For any $\mathbf {i}, \mathbf {j}\in \mathbf {V}_{\mathbf {k}}$, we define the component-wise modulo operation:

$$ \mathbf{q}_{\mathbf {{ij}}}=\mathbf{i}+\mathbf{j}\quad ({\text{mod }{2}})=\left( i_1+j_1 \quad({{\text{mod }{2}}}),\cdots, i_k+j_k\quad({{\text{mod }{2}}})\right). $$

Let ${\mathcal{Q}_{k}}=\{\mathbf{q}_{\mathbf{ij}}: {\mathbf{i}}, {\mathbf{j}}\in {\mathbf{V}}_{\mathbf{k}}\}$. Then we have the following fact about $\mathcal{Q}_k$.

Fact 2.4

$\mathcal{Q}_k$ is a multiset consisting of $2^k$ copies of each element from $\mathbf{V}_\mathbf{k}$.

Proof

It is easy to see that $\mathbf{V}_\mathbf{k}$ is closed under this modulo operation, namely for any $\mathbf{i}, \mathbf{j}\in \mathbf{V_k}$, $\mathbf{q}_\mathbf{{ij}}\in \mathbf{V}_{\mathbf{k}}$. Moreover, each element in $\mathbf{i}\in \mathbf{V}_{\mathbf{k}}$, after module operation with $\mathbf{0}$, belongs to $\mathcal{Q}_k$. Therefore, $\mathcal{Q}_k$ is a multiset consisting of only elements from $\mathbf{V}_{\mathbf{k}}$. We now prove that each element occurs exactly $2^k$ times. Imagine all elements in $\mathbf{V}_{\mathbf{k}}$ as a sequence: $\pi =(\mathbf {0},\cdots ,\mathbf{2}^\mathbf{k}{-1})$. For any $\mathbf{i}\in {\mathbf{V}}_\mathbf{k}$, consider the following sequence generated after the modulo operation: $\pi ^{\mathbf{i}}=(\mathbf{0}+\mathbf{i}\quad({{\text{mod }{2}}}),\cdots ,\mathbf{2}^{\mathbf{k}}{-}{\mathbf{1}}+{\mathbf{i}\quad({\text{mod }{2}}}))$. Then it is easy to see that $\pi ^{\mathbf{i}}$ is a permutation of $\pi$, and moreover, different $\mathbf{i}$’s will generate different permutations, implying the desired multiplicity result since there are $2^k$ such different $\mathbf{i}$’s in $\mathbf {V}_{\mathbf{k}}$. $\square$

3 Symmetric Submodular Functions

We provide a complete characterization for both deterministic and randomized cases for the symmetric submodular functions in Sects. 3.1 and 3.2, respectively.

3.1 Existence and Non-existence of Deterministic Solution

We consider the bi-criteria problem first, and then show that, by an example, no simultaneously bounded deterministic solution exists for three and more criteria.

Theorem 3.1

Assume that there are two criteria, namely, $k=2$, and assume the submodular functions in problem (P) are nonnegative and symmetric. Then we have:

• (i) For any instance of problem (P), there exists a $\frac{1}{2}$-deterministic solution for the problem (P).

• (ii) Moreover, there exist two nonnegative and symmetric submodular functions such that no $\alpha$-deterministic solution exists such that $\alpha >\frac{1}{2}$.

Proof

• (1) Let $A_1$ and $A_2$ be the optimal solutions for the two mono-criterion problems $(P_1)$ and $(P_2)$, respectively. We construct the desired deterministic solution $A$ as follows:

  $$ A= {\left\{ \begin{array}{ll} A_1,& \text { if } \,\,2f_2(A_1)\geqslant f_2(A_2);\\ A_2,&{} \text { if } \,\,2f_1(A_2)\geqslant f_1(A_1);\\ A_1\Delta A_2, &\text { otherwise}. \end{array}\right. } $$

  Evidently, we obtain a $\frac{1}{2}$-deterministic solution in the first case, and a $\frac{1}{2}$-deterministic solution in the second case. In the third case, we have $2f_2(A_1)<f_2(A_2)$ and $2f_1(A_2)<f_1(A_1)$, together with Lemma 2.2 from Sect. 2.1, implying that, for the first objective,

  $$\begin{aligned} f_1(A_1\Delta A_2)+f_1(A_2)\geqslant f_1(A_1), \end{aligned}$$

  or equivalently

  $$\begin{aligned} f_1(A_1\Delta A_2)\geqslant f_1(A_1)-f_1(A_2)\geqslant f_1(A_1)-\frac{1}{2}f_1(A_1)=\frac{1}{2}f_1(A_1). \end{aligned}$$

  Analogously, for the second objective, we can prove

  $$\begin{aligned} f_2(A_1\Delta A_2)\geqslant \frac{1}{2}f_2(A_2). \end{aligned}$$

  Therefore in the third case we obtain a $\frac{1}{2}$-deterministic solution.

• (2) To prove the non-existence results, we consider the cut function in undirected graph. The following example belongs to [3]. Consider the complete graph $K_3$ in Fig. 2a. The optimal cut value for both objectives are two while no feasible cut has values strictly better than one on both objectives.

$\square$

For more than two criteria (i.e., $k\geqslant 3$), the following example, an extension of the above example, shows that in general no simultaneously bounded deterministic solution exists for symmetric submodular functions.

Example 3.2

(Unbounded deterministic solution for the symmetric case when $k\geqslant 3$ ) Consider the Max-Cut problem on an undirected cycle graph $C_k=(1,\cdots , k)$, whose cut functions are symmetric and submodular. The weights for these $k$ criteria are specified by a $k$-dimensional vector. Assign respectively weight vector $e_i$ $(i=1,\cdots , k$) to the $k$ edges, where $e_i$ is the all-zero vector except on the $i$th coordinate (See Fig. 2b for the case of $k=3$). The optimum cut value for each criterion is one while any deterministic cut achieves zero on at least one coordinate. Thus, no simultaneously bounded deterministic solution exists.

Fig. 2

Non-existence example for the symmetric cases a Two criteria b Three criteria

3.2 Existence and Non-existence of Randomized Solution

First, we prove the existence result in Theorem 3.3. Then, in Theorem 3.4, we prove that the result in Theorem 3.3 is the best possible via the Max-Cut problem as the cut function is a special symmetric submodular function. The readers may find it beneficial to refer to the illustrative examples for the case of $k=3$ in Sects. 3.2.3 and 3.2.4 while reading the proofs of these two theorems, as these proofs are highly non-trivial from an ex ante point of view.

Theorem 3.3

Assume the submodular functions are nonnegative, symmetric and submodular in problem (P). Then there exists an $\alpha$-randomized solution such that

$$\begin{aligned} \alpha =\frac{2^{k}{-1}}{2^k}{-1}, \end{aligned}$$

and this quantity approaches to 0.5 when $k\rightarrow \infty$.

Proof

Let $\mathcal{A}=\{A_1,\cdots , A_k\}$, where each $A_j$ ($j=1,\cdots , k$) is the optimal solution for the mono-criterion problem ($P_j$). W.l.o.g., assume the optimal objective values are all equal to one, namely $f_{\ell }(A_{\ell })=1$ ($\ell =1,\cdots , k$). For any nonempty subset $X=\{A_{i_1},\cdots ,A_{i_{\ell }}\}\subseteq \mathcal{A}$ ($1\leqslant i_1\leqslant \dots \leqslant i_{\ell }\leqslant k$ and $1\leqslant \ell \leqslant k$), define the symmetric difference solution

$$\begin{aligned} S_{i_1\dots i_{\ell }}=A_{i_1}\Delta \cdots \Delta A_{i_{\ell }}. \end{aligned}$$

Let $\mathcal{S}$ be the set of all such symmetric difference sets corresponding to the $2^k-1$ nonempty sets of $\mathcal{A}$ and obviously $|\mathcal{S}|=2^k-1$. The desired randomized solution $S$ is obtained by equiv-probably selecting all the sets in $\mathcal{S}$. The desired expected value of $S$ follows from the submodularity of $f_j$ ($j=1,\cdots , k$) after appropriate pairing and harnessing the properties of symmetric difference. Due to symmetry, we only prove the desired result for $f_1$.

$$\begin{aligned} (2^k-1){\mathbf {E}}{[f_1(S)]}&= \sum _{i=1}^nf_1(S_i)+\sum _{i,j\,:\,1 \leqslant i < j \leqslant n}f_1(S_{ij})\\&\quad+\sum _{i,j,k\,:\,1 \leqslant i < j < k \leqslant n}f_1(S_{ijk})\ +\ \cdots \ + f_1(S_{1\cdots k}). \end{aligned} $$

Any term $S_{i_1\cdots i_{\ell }}$ above, satisfying that $i_1=1$ and $\ell \geqslant 2$, can be equivalently written as $S_{i_1\cdots i_{\ell }}=A_1\Delta S_{i_2\cdots i_{\ell }}:=A_1\Delta B$. Considering the pair of cuts $A_1\Delta B$ and $B$, Lemma 2.2 from Sect. 2.1 implies that

$$f_1(A_1\Delta B)+f_1(B)\geqslant f_1(A_1)=1. $$

Note that we have in total $2^{k-1}-1$ such pairs, together with $f_1(A_1)=1$, implying the desired result

$$ {{(2^{k}{-1})}{\mathbf {E}}{[f_1(S)]}\geqslant {2^{k-1}.} }$$

$\square$

Theorem 3.4

For the $k$-criteria Max-Cut problem, there exists an instance such that no $\beta$-randomized solution exists such that $\beta >\alpha$.

Proof

Consider the complete graph $K_{2^k}$ with vertex set $V$ from Sect. 2.2. Each edge will be assigned with a weight (row) vector in ${\mathbb{R}}^{k}$ such that its $\ell$th coordinate is the weight for the $\ell$th mono-criterion problem. We then show that for this weighted complete graph, no random solution has a ratio strictly better than $\alpha$ with respect to the cut function.

For any edge $(i,j)$ such that $i<j\in V$, define the weight vector for edge $(i,j)$ as follows:

$$ \mathbf{w}_{\mathbf{{ij}}}=\frac{\mathbf{q}_{\mathbf {{ij}}}}{||\mathbf{q}_{\mathbf{{ij}}}||_1}. $$

Note that the sum of weights $||\mathbf{w}_{\mathbf{{ij}}}||_1$ on each edge $(i,j)$ is equal to one, implying that the max cut value in $K_{2^k}$ with respect to unit weights is equal to $2^{k-1}2^{k-1}=2^{2(k-1)}$. Let $S$ be any random cut in this weighted graph. And denote ${\mathbf{E}}[\text {cut}_j(S)]$ to be the expected cut value of $S$ for criterion $\ell$. Then

$$\begin{aligned}{\mathbf {E}}\left[ \sum _{\ell =1}^k\text {cut}_{\ell }(S)\right] \leqslant 2^{2(k-1)}. \end{aligned}$$

(3.1)

We devote the rest of the argument to the proof of the next claim.

Claim 3.5

The max cut value for each criteria is at least $(2^k-1)2^{k-1}/k$.

This claim, together with (3.1), implies the desired result

$$\begin{aligned} \alpha \leqslant \frac{1}{k}\sum \limits _{j=1}^k\frac{{\mathbf {E}}[\text {cut}_j(S)]}{\text {cut}_j(S^*)}=\frac{\frac{1}{k}{\mathbf {E}}\left[ \sum \limits _{j=1}^k\text {cut}_j(S)\right] }{\text {cut}_j(S^*)}\leqslant \frac{2^{k-1}2^{k-1}/k}{(2^k-1)2^{k-1}/k}=\frac{2^{k-1}}{2^k-1}. \end{aligned}$$

We prove Claim 3.5 for the $\ell$th criterion by demonstrating a cut $(\mathcal{S}_{\ell },\bar{\mathcal{S}}_{\ell })$ with the value equal to $(2^k-1)2^{k-1}/k$ as follows:

For each vertex $i\in V$, $i\in \mathcal{S}_{\ell }$ iff ${\mathbf{i}_{{\ell }}}=1$, that is, the $\ell$ th coordinate of the corresponding vector is one.

Evidently $|\mathcal{S}_{\ell }|=2^{k-1}$. Let $\mathcal{Q}_{\ell }$ be the set of all $q_{ij}$’s such that $\mathbf{q}_{\mathbf{{ij}}}=\mathbf{i}+\mathbf{j}\quad({{\text{mod }{2}})}$ for any $i\in \mathcal{S}_{\ell }$ and any $j\in \bar{\mathcal{S}_{\ell }}$. We need the following claim for $\mathcal{Q}_{\ell }$ to calculate the cut value of $(\mathcal{S}_{\ell },\bar{\mathcal{S}_{\ell }})$.

Claim 3.6

$\mathcal{Q}_{\ell }$ is a multiset consisting of $2^{k-1}$ copies of distinct elements from $\mathcal{S}_{{\ell }}$.

Due to symmetry, we only prove Claim 3.6 for $\ell =1$. Let $V^{\prime }=\{0, 1,\cdots , 2^{k-1} -1\}$. Then for each $q^{\prime }\in V^{\prime }$, $\mathbf{q}^{{\prime }}\in \mathbb{R}^{k-1}$, let $\mathcal{Q}^{\prime }=\{q^{\prime }_{ij}: \mathbf{q}^{{\prime }}_\mathbf{{ij}}=\mathbf{i}^{{\prime }} +\mathbf{j}^{{\prime }} {\text{mod }{2}},\ \forall i^{{\prime }}, j^{{\prime }} \in V^{{\prime }}\}$. Then $\mathcal{Q}^{\prime }$ is a multiset consisting of $2^{k-1}$ copies of distinct elements from $V^{\prime }$ from Fact 2.4 in Sec. 2.2. Claim 3.6 now follows by noting that (1) for any $i^{\prime }, j^{\prime }\in V^{\prime }$, we can obtain an $i\in \mathcal{S}_{\ell }=\mathcal{S}_1$ as $\mathbf{i}=(1, \mathbf{i}^{{\prime }})\in \mathbb{R}^{k}$ and a $j\in \bar{\mathcal{S}}_{\ell }= \bar{\mathcal{S}}_1$ as $\mathbf{j}=(0, \mathbf{j}^{{\prime }})\in \mathbb{R}^{k}$, and vice versa. And (2) $\mathbf{i}+\mathbf{j} \quad({{\text{mod }{2}})}=(1,\, \mathbf{i}^{{\prime }}+\mathbf{j}^{{\prime }} \quad({{\text{mod }{2}})})$.

We now calculate the cut value for the $\ell$ criterion:

$$ \sum \limits_{i\in \mathcal{S},j\in \bar{\mathcal{S}}}(\mathbf{w}_{\mathbf {ij}})_{\ell }=\sum_{q\in \mathcal{Q}_{\ell}}\frac{\mathbf{q}_{\ell }}{||\mathbf{q}||_{1}}=2^{k-1}\sum \limits_{c=1}^{k} \frac{C_{k-1}^{c-1}}{c}=\frac{2^{k-1}(2^k-1)}{k}.$$

In the above, the second equality follows from Claim 3.6 that each $q$ has a multiplicity of $2^{k-1}$, and a simple counting argument that the number of different $q\in \mathcal{Q}$ whose $\ell$th coordinator is equal to one and the corresponding edge is assigned value $1=c$, equal to $C_{k-1}^{c-1}$, namely $\left| \{q\in {\mathcal{Q}}_{\ell }: {\mathbf{q}}_{\ell }={1}, || {\mathbf{q}}||_{1}=c\}\}\right| =C_{k-1}^{c-1}$. $\square$

3.2.1 Discussion on the Results

Note that the results in Theorems 3.3 and 3.4 improve/extend the result in [2] for the special bi-criteria maximum cut problem. First, for the upper bound, our result specialized to the bicriteria maximum cut problem given an improved bound $2/3\approx 0.667$ over $(\sqrt{5}-1)/2\approx 0.618$, and the new bound is the best possible considering the lower bound example given in [2]. The latter lower bound is also extended in this work to any number of criteria, and hence answering affirmatively the main open question left in [2] (See Corollary 5.2 in Sect. 5.1 for more details).

3.2.2 Remarks on the Proofs of Theorems 3.3 and 3.4

One may wonder why we choose the symmetric difference operator $\Delta$ in the construction of existence proof (Theorem 3.3) or how we came up with the instance in the non-existence proof (Theorem 3.4). These constructions are highly non-trivial ex ante—at least to us when we started the investigation.

We initially attempted to use computer programs to find the exact ratio for small $k$. For example we found that the exact ratio for $k=2$ is $\frac{2}{3}$, and the lower bound instance we used can be decomposed into a sequence of special 2-factors in graph theory. This led us to the theory of cycle decomposition, an active research area in graph theory. By applying some existing results in the cycle decomposition theory, we were able to prove a non-existence result that $\alpha \leqslant \frac{k}{2k-1}$, which turns out to be the tight bound for $k=2$. At the same time, our extensive calculation also helped us finding some patterns in the existence proof, which pointed the direction towards the balanced cuts in an auxiliary graph. Now the case of $k=2$ is completely resolved since the two bounds coincide. However, when we tried to extend the existence proof to larger $k$, the ratio we obtained did not agree with the non-existence bound $\frac{k}{2k-1}$, which prompted us to do more computations and try to locate the right instance. Only after numerous attempts, luck struck us and offered us the instance we have been desperately looking for.

From an ex post point of view, the proofs in their present forms seem natural (though still non-trivial). However, our luck did not go that far to allow us to solve the asymmetric case in Sect. 4, which is very intriguing from an ex ante point of view as of now, and we have to leave it as one of the main open questions from this work (See Sect. 6 for more discussions).

3.2.3 An Illustration of the Proof of Theorem 3.3 for $k=3$

For Theorem 3.3, when $k=3$, we have

$$\begin{aligned} \mathcal{S}=\{A_1,A_2,A_3,A_1\Delta A_2, A_1\Delta A_3, A_2\Delta A_3, A_1\Delta A_2\Delta A_3\}. \end{aligned}$$

To show, for $j=1$, that $(2^k-1)\mathbb {E}[f_1(S)]\geqslant 2^{k-1}$, or, when $k=3$, ${\mathbf {E}}{[f_1(S)]}\geqslant \frac{4}{7}$, we form the following pairs: $(A_1\Delta A_2, A_2)$, $(A_1\Delta A_3, A_3)$, and $(A_1\Delta A_2\Delta A_3, A_2\Delta A_3)$. Then the desired inequality follows from the following results:

$$\begin{aligned} f_1(A_1)&= 1,\\ f_1(A_1\Delta A_2)+f_1(A_2)&\geqslant f_1(A_1)=1,\\ f_1(A_1\Delta A_3)+f_1(A_3)&\geqslant f_1(A_1)=1,\\ f_1(A_1\Delta A_2\Delta A_3)+f_1(A_2\Delta A_3)&\geqslant f_1(A_1)=1, \end{aligned}$$

where the first one follows from the assumption that all optimal objective values are one, and the last three follow from Lemma 2.2.

3.2.4 An Illustration of the Proof of Theorem 3.4 for $k=3$

For Theorem 3.4, when $k=3$, Table 2 shows the results from the module operation: ${\mathbf{q}}_{\mathbf{{ij}}}={\mathbf{i}}+{\mathbf{j}}\quad({{\text{mod }{2}})}$, along with edge weights table in Table 3.

Table 2 Module operation on $V_3$ in Theorem 3.3

Table 3 Edge weights derived from Table 2

Since the sum of weights on each edge is one (Table 3), we know that the max cut value in $K_{2^k}$ with respect to unit weights is equal to $2^{k-1}2^{k-1}=2^{2(k-1)}$, which is equal to 16 when $k=3$. The desired result follows if the max cut value for each criteria is at least $(2^k-1)2^{k-1}/k$, which is $28/3$ for $k=3$. Note that $\mathcal{S}=\{4,5,6,7\}$ for the first criterion, $\{2,3,6,7\}$ for the second, and $T=\{1,3,5,7\}$ for the third are such cuts, generated respectively by $\{\underline{1}00,\underline{1}01,\underline{1}10, \underline{1}11\}$, $\{0\underline{1}0,0\underline{1}1,1\underline{1}0, 1\underline{1}1\}$, and $\{00\underline{1},01\underline{1},10\underline{1}, 11\underline{1}\}$. Table 4 illustrates Claim 3.6.

Table 4 Module operation on $V^{\prime }_2$ (left) and the resultant Module operation on $V_3$ (right) in the proof of Claim 3.6 for the first criterion in Theorem 3.4

4 Asymmetric Submodular Functions

We consider deterministic and randomized cases for the asymmetric submodular functions.

4.1 Existence and Non-existence of Deterministic Solution

The following example shows that in general, no simultaneously bounded deterministic solution exists for asymmetric submodular functions when $k\geqslant 2$.

Example 4.1

(Unbounded deterministic solution for asymmetric case for any $k\geqslant 2$ ) Consider the Max-Cut problem on a directed cycle graph $C_k=(1,\cdots , k)$, whose cut functions are asymmetric and submodular. Assign respectively weight vector $e_i$ $(i=1,\cdots , k$) to the $k$ edges, where $e_i$ is the all-zero vector except on the $i$th coordinate. The optimum cut value for each criterion is one while any deterministic cut achieves zero on at least one coordinate. Thus, no simultaneously bounded deterministic solution exists.

4.2 Existence and Non-existence of Randomized Solution

The algorithm that evenly choose all the $k$ optimal solutions for the mono-criterion problems offers a $\left( \frac{1}{k},\cdots , \frac{1}{k}\right)$-randomized solution for any nonnegative set function. And the following example shows that they are the best possible for $k\leqslant 3$.

Example 4.2

For $k=2$, consider the two nodes graphs with two arcs going in opposite directions between them. One with weight $(1,0)$ and another $(0,1)$. For $k=3$, consider the directed cycle $C_3$ with weights $(1,0,0)$, $(0,1,0)$ and $(0,0,1)$.

The above simple random algorithm will not offer us the best bound for $k\geqslant 4$, as shown in the theorem below.

Theorem 4.3

Assume the submodular functions are nonnegative and submodular in problem (P). Then there exists an $\alpha$-randomized solution such that

$$\begin{aligned} \alpha =\frac{2^{k-2}}{2^k-1}, \end{aligned}$$

and this quantity approaches to 0.25 when $k\rightarrow \infty$.

Proof

Let $\mathcal{A}=\{A_1,\cdots , A_k\}$ be the maximum cut for problem (P). W.l.o.g., assume the optimal cut values are all equal to one. For any nonempty subset $X=\{A_{i_1},\cdots ,A_{i_{\ell }}\}\subseteq \mathcal{A}$ ($1\leqslant i_1\leqslant \dots \leqslant i_{\ell }\leqslant k$ and $1\leqslant \ell \leqslant k$), we define the symmetric difference cut along with its complement

$$\begin{aligned} {\left\{ \begin{array}{ll} S_{i_1\dots i_{\ell }}=&{}A_{i_1}\Delta \cdots \Delta A_{i_{\ell }},\\ \overline{S}_{i_1\dots i_{\ell }}=&{}\overline{A_{i_1}\Delta \cdots \Delta A_{i_{\ell }}}. \end{array}\right. } \end{aligned}$$

In total we have $2\big (2^k-1\big )$ such symmetric difference cuts corresponding to the $2^k-1$ nonempty sets of $A$ and their complements. The desired randomized cut $S$ is obtained by equiv-probably selecting these $2\big (2^k-1\big )$ cuts. The desired expected value of $S$ follows from the submodularity of $f_j$ ($j=1,\cdots , k$) after appropriate pairing and harnessing the properties of symmetric difference. Due to symmetry, we only prove the desired result for $f_1$.

$$\begin{aligned} (2^k-1)\mathbb {E}[f_1(S)]&= \sum _{i=1}^n(f_1(S_i)+f_1(\overline{S}_i))+\sum _{i,j\,:\,1 \leqslant i < j \leqslant n}(f_1(S_{ij})+f_1(\overline{S}_{ij}))\\&\quad+\sum _{i,j,k\,:\,1 \leqslant i < j < k \leqslant n}(f_1(S_{ijk})+f_1(\overline{S}_{ijk}))\ +\ \cdots \ \\&\quad+ (f_1(S_{1\cdots n}) +f_1(\overline{S}_{1\cdots k})). \end{aligned} $$

Any terms $S_{i_1\cdots i_{\ell }}$ and $\overline{S}_{i_1\cdots i_{\ell }}$ above, satisfying that $i_1=1$ and $\ell \geqslant 2$, can be equivalently written as $S_{i_1\cdots i_{\ell }}=A_1\Delta S_{i_2\cdots i_{\ell }}:=A_1\Delta B$, and $\overline{S}_{i_1\cdots i_{\ell }}=\overline{A_1\Delta B}$, respectively. Considering the pair of cuts $A_1\Delta B$ and $B$, and the pair of cuts $\overline{A_1\Delta B}$ and $\overline{B}$, Lemma 2.1 from Sect. 2.1 implies that

$$\begin{aligned} f_1(A_1\Delta B)+f_1(B)+f_1(\overline{A_1\Delta B})+f_1(\overline{B})\geqslant f_1(A_1)+f_1(\overline{A}_1)\geqslant 1. \end{aligned}$$

Note that we have in total $2^{k-1}-1$ such pairs, together with $f_1(A_1)+f_1(\overline{A}_1)\geqslant 1$, implying the desired result

$$\begin{aligned} 2(2^k-1){\mathbf {E}}[f_1(S)]\geqslant 2^{k-1}. \end{aligned}$$

$\square$

5 Algorithmic Corollaries

We now discuss the issue of how to find a deterministic or randomized feasible solution for the multi-criteria problem in polynomial-time. We first consider the deterministic solution case, and then followed with the randomized solution.

5.1 Polynomial-time Algorithm for Deterministic Solution

We only consider the bi-criteria symmetric case as there is no simultaneously bounded deterministic solution for symmetric case with more than three objectives and the asymmetric case. Theorem 3.1 implies that there exists a polynomial time algorithm which can find an $\frac{\alpha }{2}$-deterministic solution by calling any given $\alpha$-approximation algorithm for the mono-criterion problem. In particular, under the value-oracle model, the best possible algorithm for symmetric submodular maximization has an approximation ratio $\frac{1}{2}$ ([12]), implying that:

Corollary 5.1

In the bi-criteria symmetric submodular maximization problem, assume that the submodular function is given under the oracle model. Then there exists a polynomial-time algorithm which can find a $\frac{1}{4}$-deterministic solution.

We may get better bound if the submodular function is given under succinct representation, where the submodular functions depend only on a constant number of elements, such as the cut function in graph and hypergraphs. In particular, for the bi-criteria weighted max-cut problem, it was already proved in [3] that there exists a polynomial-time algorithm which can find a $\left( 0.439,0.439\right)$-deterministic solution by calling the $0.879$-approximation algorithm by Goemans and Williamson [16].

5.2 Polynomial-time Algorithm for Randomized Solution

It is well-known that the randomized algorithm which equi-probably selects or rejects any given element into the solution is a $\frac{1}{2}$-approximation algorithm for the mono-criterion symmetric submodular function maximization problem and a $\frac{1}{4}$-approximation for the asymmetric case. Moreover, the $\frac{1}{2}$-approximation cannot be improved under the value oracle model ([12]).

It is not difficult to see that, for the multi-criteria problem, the same simple randomized algorithm yields a $\left( \frac{1}{2},\cdots ,\frac{1}{2}\right)$-randomized solution for the symmetric case and a $\frac{1}{4}$-randomized solution for asymmetric case.

Note that the existence results proved here cannot yield better bounds than the aforementioned simple random algorithm under the oracle model. But it can offer better bounds under the succinct representation model for smaller $k$. In particular, for the symmetric case, Theorem 3.3 together with the 0.879-approximation algorithm in [16] imply that

Corollary 5.2

For the $k$-criteria Max-Cut problem, there exists a polynomial-time algorithm which can find a 0.586 0-randomized solution for $k=2$, and a 0.502 3-randomized solution for $k=3$.

The last bound actually answered affirmatively the open question posted in [3] for three criteria: “the existence of a polynomial time randomized algorithm that has a performance ratio strictly better than $\frac{1}{2}$ for three and more criteria is open.”.

For the asymmetric case, Theorem 4.3 together with the $0.874$-approximation algorithm in [11, 21] imply that

Corollary 5.3

For the $k$-criteria Max-Di-Cut problem, there exists a polynomial-time algorithm which can find a 0.291 3-randomized solution for k = 2.

6 Concluding Remarks

An obvious open question left here is to find the tight randomized ratio for the asymmetric submodular function when $k\geqslant 4$. To offer a taste of how challenging this open question might be, we report a tight result for the special 4-criteria Max-Di-Cut problem:

• There exists a $\frac{185}{674}\approx 0.274\,48$-randomized solution for the 4-criteria Max-Di-Cut problem, and this bound is tight as shown by the instance in Fig. 3.

This is proved with the aid of symbolic calculation via Maple and Matlab and the same code goes out of memory for $k\geqslant 5$.

Fig. 3

Non-existence proof for $k=4$: where only cross-layer edges are shown and each of the three vertical layers is a clique and the edges are omitted here. Namely nodes (0111,1011, 1101, 1110) form the left-clique, nodes (0011, 0101, 0110, 1001, 1010, 110) form the middle-clique and nodes (0001, 0010, 0100, 1000) form the right-clique. Moreover, the numbers indicate the sum of weight on each edge, which is evenly distributed to the four criteria. Namely 3 is the edge weight within the left-clique and the right-clique, 7 is the edge weight within the middle-clique, 7.5 is the weight between the middle-clique and left-cliques (right-clique), and 2 is the edge weight between the left-clique and the right-clique

Note that the last bound 0.274 4 implies that general bound earlier for any asymmetric submodular function in Theorem 4.3 is not tight, as it only offers $\max \{\frac{1}{4}, \frac{4}{15}\}\approx 0.266\,7$. Therefore one possible first step in resolving the aforementioned open question is to find such a tight bound for the special directed cut function when $k\geqslant 5$.