## Abstract

The Burrows-Wheeler Transform (BWT) is among the most influential discoveries in text compression and DNA storage. It is a reversible preprocessing step that rearranges an n-letter string into runs of identical characters (by exploiting context regularities), resulting in highly compressible strings, and is the basis of the bzip compression program. Alas, the decoding process of BWT is inherently sequential and requires Ω(n) time even to retrieve a single character. We study the succinct data structure problem of locally decoding short substrings of a given text under its compressed BWT, i.e., with small additive redundancy r over the Move-To-Front (bzip) compression. The celebrated BWT-based FM-index (FOCS’00), as well as other related literature, yield a trade-off of r = Õ(n/t) bits, when a single character is to be decoded in O(t) time. We give a near-quadratic improvement r = Õ(n lg(t)/t). As a by-product, we obtain an exponential (in t) improvement on the redundancy of the FM-index for counting pattern-matches on compressed text. In the interesting regime where the text compresses to o(n) (say, n/polylg(n)) bits, these results provide an exp(t) overall space reduction. For the local decoding problem of BWT, we also prove an Ω(n/t^{2}) cell-probe lower bound for “symmetric" data structures. We achieve our main result by designing a compressed partial-sums (Rank) data structure over BWT. The key component is a locally-decodable Move-to-Front (MTF) code: with only O(1) extra bits per block of length n^{Ω}(1^{)}, the decoding time of a single character can be decreased from Ω(n) to O(lg n). This result is of independent interest in algorithmic information theory.

Original language | American English |
---|---|

Title of host publication | STOC 2019 - Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing |

Editors | Moses Charikar, Edith Cohen |

Publisher | Association for Computing Machinery |

Pages | 744-755 |

Number of pages | 12 |

ISBN (Electronic) | 9781450367059 |

DOIs | |

State | Published - 23 Jun 2019 |

Externally published | Yes |

Event | 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019 - Phoenix, United States Duration: 23 Jun 2019 → 26 Jun 2019 |

### Publication series

Name | Proceedings of the Annual ACM Symposium on Theory of Computing |
---|---|

ISSN (Print) | 0737-8017 |

### Conference

Conference | 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019 |
---|---|

Country/Territory | United States |

City | Phoenix |

Period | 23/06/19 → 26/06/19 |

### Bibliographical note

Publisher Copyright:© 2019 Association for Computing Machinery.

## Keywords

- Burrows-wheeler transform
- Cell-probe model
- Local decoding
- Pattern matching
- Succinct data structures
- Text compression