Skip to main content

Consolidation for compact constraints and Kendall tau LP decodable permutation codes


Invented in the 1960s, permutation codes have reemerged in recent years as a topic of great interest because of properties making them attractive for certain modern technological applications, especially flash memory. In 2011 a polynomial time algorithm called linear programming (LP) decoding was introduced for a class of permutation codes where the feasible set of codewords was a subset of the vertex set of a code polytope. In this paper we investigate a new class of linear constraints for matrix polytopes with no fractional vertices through a new concept called “consolidation.” We then introduce a necessary and sufficient condition for which LP decoding methods, originally designed for the Euclidean metric, may be extended to provide an efficient decoding algorithm for permutation codes with the Kendall tau metric.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    In some references, G is chosen as a generalized permutation group, e.g., a signed permutation group.

  2. 2.

    The original problem is to maximize \(\mathrm {Trace}( \varvec{\mu } \varvec{\lambda }^T X )\). It is directly obtained that \( \mathrm {Trace}( \varvec{\mu } \varvec{\lambda }^T X ) = \mathrm {Trace}( \varvec{\lambda }^T X \varvec{\mu } ) = \varvec{\lambda }^T X \varvec{\mu }\).

  3. 3.

    Televis: a toy consisting of two balls and a string which connects the balls.

  4. 4.

    In some references, a dihedral group of degree n is denoted by \(D_n\).

  5. 5.

    This notation is defined in Sect. 5.1, Definition 15.


  1. 1.

    Andrews G.E.: The Theory of Partitions. Addison-Wesley, Reading (1976).

    MATH  Google Scholar 

  2. 2.

    Barg A., Mazumdar A.: Codes in permutations and error correction for rank modulation. In: Proceedings of the IEEE ISIT, pp. 854–858 (2010).

  3. 3.

    Berger T., Jelinek F., Wolf J.: Permutation codes for sources. IEEE Trans. Inf. Theory I8, 160–169 (1972).

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Birkhoff G.: Three observations on linear algebra. Univ. Nac. Tacuman Rev. Ser. A 5, 147–151 (1946).

    MathSciNet  MATH  Google Scholar 

  5. 5.

    Blake I.F., Cohen G., Deza M.: Coding with permutations. Inf. Control 43, 1–19 (1979).

    MathSciNet  Article  MATH  Google Scholar 

  6. 6.

    Buchheim C., Cameron P.J., Wu T.: On the subgroup distance problem. Discret. Math. 309, 962–968 (2009).

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Deza M., Huang T.: Metrics on permutations, a survey. JCISS (1998).

  8. 8.

    Fultion W., Harris J.: Representation Theory: A First Course. Springer, New York (1991).

    Google Scholar 

  9. 9.

    Garey M., Johnson D.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, New York. ISBN 978-0-7167-1045-5, OCLC 11745039 (1979).

  10. 10.

    Hagiwara M.: On ML-certificate linear constraints for rank modulation with linear programming decoding and its application to compact graphs. In: Proceedings of the IEEE ISIT, pp. 2993–2997 (2012).

  11. 11.

    Hill R.: A First Course in Coding Theory. Oxford Applied Mathematics and Computing Science Series. Oxford University Press, Oxford (1986).

    Google Scholar 

  12. 12.

    Humphreys J.E.: Reflection Groups and Coxeter Groups. Cambridge University Press, Cambridge (1992).

    MATH  Google Scholar 

  13. 13.

    Jiang A., Schwartz M., Bruck J.: Error-correcting codes for rank modulation. In: Proceedings of the IEEE ISIT, pp. 1736–1740 (2008).

  14. 14.

    Jiang A., Mateescu R., Schwartz M., Bruck J.: Rank modulation for flash memories. IEEE Trans. Inf. Theory 55, 2659–2673 (2009).

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Kane R.: Reflection Groups and Invariant Theory. Springer, New York (2001).

    Book  MATH  Google Scholar 

  16. 16.

    Karmarkar N.: A new polynomial time algorithm for linear programming. Combinatorica 4, 373–395 (1984).

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Kong J., Hagiwara M.: Comparing Euclidean, Kendall tau metrics toward extending LP decoding. In: Proceedings of the ISITA, pp. 91–95 (2012).

  18. 18.

    Knuth D.: The Art of Computer Programming, vol. 3. Addison-Wesley, Reading (1998).

    MATH  Google Scholar 

  19. 19.

    Mazumdar A., Barg A., Zemor G.: Constructions of rank modulation codes. IEEE Trans. Inf. Theory 59, 1018–1029 (2012).

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    Papandreou N., Pozidis H., Mittelholzer T., Close G.F., Breitwisch M., Lam C., Eleftheriou E.: Drift-tolerant multilevel phase-change memory. In: Proceedings of the IEEE IMW, pp. 1–4 (2011).

  21. 21.

    Peterson W.W., Nation J.B., Fossorier M.P.: Reflection group codes and their decoding. IEEE Trans. Inf. Theory 56, 6273–6293 (2010).

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Schreck H., Tinhofer G.: A note on certain subpolytopes of the assignment polytope associated with circulant graphs. Linear Algebra Appl. 111, 125–134 (1988).

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Slepian D.: Permutation modulation. Proc. IEEE 53, 228–236 (1965).

    Article  Google Scholar 

  24. 24.

    Tinhofer G.: Graph isomorphism and theorems of Birkhoff type. Computing 36(4), 285–300 (1986).

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Tinhofer G.: A note on compact graphs. Discret. Appl. Math. 30(2–3), 253–264 (1991).

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Vinck A.J.H.: Coded modulation for powerline communications. AEÜ Int. J. Electron. Commun. 54, 45–49 (2009).

    Google Scholar 

  27. 27.

    Von Neumann J.: A certain zero-sum two-person game equivalent to an optimal assignment problem. Ann. Math. Stud. 28, 5–12 (1953).

    MathSciNet  Google Scholar 

  28. 28.

    Wadayama T., Hagiwara M.: LP decodable permutation codes based on linearly constrained permutation matrices. In: Proceedings of the IEEE ISIT, pp. 139–143 (2011).

Download references


The authors would like to thank Dr. JB Nation for his help proofreading. We are also grateful to the anonymous reviewers of this journal for many valuable insights that have significantly improved the quality of this paper. This paper is partially supported by KAKENHI(B) 16K12391, 16K06336, and 26289116.

Author information



Corresponding author

Correspondence to Justin Kong.

Additional information

Communicated by P. Charpin.



Definition 22

(Linear constraints, satisfy, \(\models \)) A linear constraint l(X) for an n-by-n matrix is defined as either a linear equation or linear inequality on entries of a matrix.

Formally speaking, by regarding an entry \(X_{i,j}\) as a variable (\(0 \le i,j < n\)), we state either

$$\begin{aligned} l(X) : \sum _{0 \le i,j < n} c_{i,j} X_{i,j} = c_0, \end{aligned}$$


$$\begin{aligned} l(X) : \sum _{0 \le i,j < n} c_{i,j} X_{i,j} \ge c_0, \end{aligned}$$

for some \(c_0, c_{i,j} \in {\mathbb {R}}\). The relation \(=\) or \(\ge \) is uniquely determined by l(X). Instead of the symbols \(=\) and \(\ge \), we may use \(\trianglerighteq _l\) (or simply \(\trianglerighteq \)), e.g.,

$$\begin{aligned} l(X) : \sum _{0 \le i,j < n} c_{i,j} X_{i,j} \trianglerighteq _l c_0. \end{aligned}$$

If we do not need to clarify the variable X of a linear constant l(X), we denote it simply by l.

For a linear constraint \(l \in {\mathcal {L}}\) and a matrix \(X \in \mathrm {M}_n ({\mathbb {R}})\), if X satisfies l, we write \(X \models l\). If \(X \models l\) for every \(l \in {\mathcal {L}}\), we write \(X \models {\mathcal {L}}\).

Definition 23

(Birkhoff polytope, doubly stochastic polytope) Let \({\mathcal {L}} \) be a doubly stochastic constraint for an n-by-n matrix. The collection of n-by-n matrices which satisfy all linear constraints in \({\mathcal {L}}\) is denoted by \({\mathcal {D}}_n [{\mathcal {L}}]\). We call \({\mathcal {D}}_n [ {\mathcal {L}}]\) a doubly stochastic polytope of \({\mathcal {L}}\).

For a subset \({\mathcal {D}} \subset \mathrm {M}_n ( {\mathbb {R}} )\), \({\mathcal {D}}\) is said to be a doubly stochastic polytope if there exists a doubly stochastic constraint \({\mathcal {L}}\) such that \({\mathcal {D}} = {\mathcal {D}}_n [ {\mathcal {L}} ]\). The polytope \(\mathrm {DSM}_n\) is an example of doubly stochastic polytope. \(\mathrm {DSM}_n\) is said to be a Birkhoff polytope.

Definition 24

(Vertex) Let \({\mathcal {D}}\) be a doubly stochastic polytope.

An element \(X \in {\mathcal {D}}\) is said to be a vertex if there are neither elements \(X_0, X_1 \in {\mathcal {D}}\) with \(X_0 \ne X_1\) nor positive numbers \(c_0, c_1 \in {\mathbb {R}},\) \(c_0 + c_1 = 1\), such that

$$\begin{aligned} X = c_0 X_0 + c_1 X_1. \end{aligned}$$

We denote the set of vertices for \({\mathcal {D}}\) by \(\mathrm {Ver}( {\mathcal {D}} )\).

Definition 25

(Kendall tau distance) Given \(\sigma ,\tau \in S_n\), the Kendall tau distance \(\mathrm {d}_K(\sigma ,\tau )\) between \(\sigma \) and \(\tau \) is defined as

$$\begin{aligned} \mathrm {d}_K(\sigma ,\tau ):= \left| \left\{ (i,j) \;\vert \; 0\le i < j \le n-1, \big (\sigma ^{-1}\tau \big )_i > \big (\sigma ^{-1}\tau \big )_j\right\} \right| , \end{aligned}$$

where |A| denotes the cardinality of the set A.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hagiwara, M., Kong, J. Consolidation for compact constraints and Kendall tau LP decodable permutation codes. Des. Codes Cryptogr. 85, 483–521 (2017).

Download citation


  • Coding and information theory
  • Graphs and abstract algebra
  • Linear programming
  • Permutation groups
  • Decoding

Mathematics Subject Classification

  • 68P30
  • 05C25
  • 90C05
  • 20B
  • 94B35