Slamming: Training a Speech Language Model on One GPU in a Day

Gallil Maimon; Avishai Elmakies; Yossi Adi

doi:10.18653/v1/2025.findings-acl.631

Slamming: Training a Speech Language Model on One GPU in a Day

Gallil Maimon
, Avishai Elmakies
, Yossi Adi

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples - https://pages.cs.huji.ac.il/adiyosslab/slamming.

Original language	English
Title of host publication	Findings of the Association for Computational Linguistics
Subtitle of host publication	ACL 2025
Editors	Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Publisher	Association for Computational Linguistics (ACL)
Pages	12201-12216
Number of pages	16
ISBN (Electronic)	9798891762565
DOIs	https://doi.org/10.18653/v1/2025.findings-acl.631
State	Published - 2025
Event	63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025 - Vienna, Austria Duration: 27 Jul 2025 → 1 Aug 2025

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
ISSN (Print)	0736-587X

Conference

Conference	63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025
Country/Territory	Austria
City	Vienna
Period	27/07/25 → 1/08/25

Bibliographical note

Publisher Copyright:
© 2025 Association for Computational Linguistics.

Access to Document

10.18653/v1/2025.findings-acl.631

Cite this

Maimon, G., Elmakies, A., & Adi, Y. (2025). Slamming: Training a Speech Language Model on One GPU in a Day. In W. Che, J. Nabende, E. Shutova, & M. T. Pilehvar (Eds.), Findings of the Association for Computational Linguistics: ACL 2025 (pp. 12201-12216). (Proceedings of the Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2025.findings-acl.631

Maimon, Gallil ; Elmakies, Avishai ; Adi, Yossi. / Slamming : Training a Speech Language Model on One GPU in a Day. Findings of the Association for Computational Linguistics: ACL 2025. editor / Wanxiang Che ; Joyce Nabende ; Ekaterina Shutova ; Mohammad Taher Pilehvar. Association for Computational Linguistics (ACL), 2025. pp. 12201-12216 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{279c6de15cf74bfc9a2da2604fb848f4,

title = "Slamming: Training a Speech Language Model on One GPU in a Day",

abstract = "We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples - https://pages.cs.huji.ac.il/adiyosslab/slamming.",

author = "Gallil Maimon and Avishai Elmakies and Yossi Adi",

note = "Publisher Copyright: {\textcopyright} 2025 Association for Computational Linguistics.; 63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025 ; Conference date: 27-07-2025 Through 01-08-2025",

year = "2025",

doi = "10.18653/v1/2025.findings-acl.631",

language = "אנגלית",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "12201--12216",

editor = "Wanxiang Che and Joyce Nabende and Ekaterina Shutova and Pilehvar, \{Mohammad Taher\}",

booktitle = "Findings of the Association for Computational Linguistics",

address = "ארצות הברית",

}

Maimon, G, Elmakies, A & Adi, Y 2025, Slamming: Training a Speech Language Model on One GPU in a Day. in W Che, J Nabende, E Shutova & MT Pilehvar (eds), Findings of the Association for Computational Linguistics: ACL 2025. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics (ACL), pp. 12201-12216, 63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, 27/07/25. https://doi.org/10.18653/v1/2025.findings-acl.631

Slamming: Training a Speech Language Model on One GPU in a Day. / Maimon, Gallil; Elmakies, Avishai; Adi, Yossi.
Findings of the Association for Computational Linguistics: ACL 2025. ed. / Wanxiang Che; Joyce Nabende; Ekaterina Shutova; Mohammad Taher Pilehvar. Association for Computational Linguistics (ACL), 2025. p. 12201-12216 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Slamming

T2 - 63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025

AU - Maimon, Gallil

AU - Elmakies, Avishai

AU - Adi, Yossi

PY - 2025

Y1 - 2025

N2 - We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples - https://pages.cs.huji.ac.il/adiyosslab/slamming.

AB - We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples - https://pages.cs.huji.ac.il/adiyosslab/slamming.

UR - https://www.scopus.com/pages/publications/105028630334

U2 - 10.18653/v1/2025.findings-acl.631

DO - 10.18653/v1/2025.findings-acl.631

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:105028630334

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 12201

EP - 12216

BT - Findings of the Association for Computational Linguistics

A2 - Che, Wanxiang

A2 - Nabende, Joyce

A2 - Shutova, Ekaterina

A2 - Pilehvar, Mohammad Taher

PB - Association for Computational Linguistics (ACL)

Y2 - 27 July 2025 through 1 August 2025

ER -

Maimon G, Elmakies A, Adi Y. Slamming: Training a Speech Language Model on One GPU in a Day. In Che W, Nabende J, Shutova E, Pilehvar MT, editors, Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics (ACL). 2025. p. 12201-12216. (Proceedings of the Annual Meeting of the Association for Computational Linguistics). doi: 10.18653/v1/2025.findings-acl.631

Slamming: Training a Speech Language Model on One GPU in a Day

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this