On Pruning State-Space LLMs

Tamer Ghattas; Michael Hassid; Roy Schwartz

doi:10.18653/v1/2025.emnlp-main.950

On Pruning State-Space LLMs

Tamer Ghattas
, Michael Hassid
, Roy Schwartz

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g., WANDA), while using other methods lead to fast performance degradation.

Original language	English
Title of host publication	EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
Editors	Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Publisher	Association for Computational Linguistics (ACL)
Pages	18800-18814
Number of pages	15
ISBN (Electronic)	9798891763326
DOIs	https://doi.org/10.18653/v1/2025.emnlp-main.950
State	Published - 2025
Event	30th Conference on Empirical Methods in Natural Language Processing, EMNLP 2025 - Suzhou, China Duration: 4 Nov 2025 → 9 Nov 2025

Publication series

Name	EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Conference

Conference	30th Conference on Empirical Methods in Natural Language Processing, EMNLP 2025
Country/Territory	China
City	Suzhou
Period	4/11/25 → 9/11/25

Bibliographical note

Publisher Copyright:
© 2025 Association for Computational Linguistics.

Access to Document

10.18653/v1/2025.emnlp-main.950

https://aclanthology.org/2025.emnlp-main.950.pdfLicense: CC BY

Cite this

Ghattas, T., Hassid, M., & Schwartz, R. (2025). On Pruning State-Space LLMs. In C. Christodoulopoulos, T. Chakraborty, C. Rose, & V. Peng (Eds.), EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (pp. 18800-18814). (EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2025.emnlp-main.950

Ghattas, Tamer ; Hassid, Michael ; Schwartz, Roy. / On Pruning State-Space LLMs. EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. editor / Christos Christodoulopoulos ; Tanmoy Chakraborty ; Carolyn Rose ; Violet Peng. Association for Computational Linguistics (ACL), 2025. pp. 18800-18814 (EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference).

@inproceedings{69c3d8573892419b8c5e472d8e0deb00,

title = "On Pruning State-Space LLMs",

abstract = "Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g., WANDA), while using other methods lead to fast performance degradation.",

author = "Tamer Ghattas and Michael Hassid and Roy Schwartz",

note = "Publisher Copyright: {\textcopyright} 2025 Association for Computational Linguistics.; 30th Conference on Empirical Methods in Natural Language Processing, EMNLP 2025 ; Conference date: 04-11-2025 Through 09-11-2025",

year = "2025",

doi = "10.18653/v1/2025.emnlp-main.950",

language = "אנגלית",

series = "EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "18800--18814",

editor = "Christos Christodoulopoulos and Tanmoy Chakraborty and Carolyn Rose and Violet Peng",

booktitle = "EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference",

address = "ארצות הברית",

}

Ghattas, T, Hassid, M & Schwartz, R 2025, On Pruning State-Space LLMs. in C Christodoulopoulos, T Chakraborty, C Rose & V Peng (eds), EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, Association for Computational Linguistics (ACL), pp. 18800-18814, 30th Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, 4/11/25. https://doi.org/10.18653/v1/2025.emnlp-main.950

On Pruning State-Space LLMs. / Ghattas, Tamer; Hassid, Michael; Schwartz, Roy.
EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. ed. / Christos Christodoulopoulos; Tanmoy Chakraborty; Carolyn Rose; Violet Peng. Association for Computational Linguistics (ACL), 2025. p. 18800-18814 (EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On Pruning State-Space LLMs

AU - Ghattas, Tamer

AU - Hassid, Michael

AU - Schwartz, Roy

PY - 2025

Y1 - 2025

N2 - Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g., WANDA), while using other methods lead to fast performance degradation.

AB - Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g., WANDA), while using other methods lead to fast performance degradation.

UR - https://www.scopus.com/pages/publications/105040242695

U2 - 10.18653/v1/2025.emnlp-main.950

DO - 10.18653/v1/2025.emnlp-main.950

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:105040242695

T3 - EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

SP - 18800

EP - 18814

BT - EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

A2 - Christodoulopoulos, Christos

A2 - Chakraborty, Tanmoy

A2 - Rose, Carolyn

A2 - Peng, Violet

PB - Association for Computational Linguistics (ACL)

T2 - 30th Conference on Empirical Methods in Natural Language Processing, EMNLP 2025

Y2 - 4 November 2025 through 9 November 2025

ER -

Ghattas T, Hassid M, Schwartz R. On Pruning State-Space LLMs. In Christodoulopoulos C, Chakraborty T, Rose C, Peng V, editors, EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. Association for Computational Linguistics (ACL). 2025. p. 18800-18814. (EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference). doi: 10.18653/v1/2025.emnlp-main.950

On Pruning State-Space LLMs

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this